Clipping from Twitter

Hi there,

I’d like to be able to save tweets and tweetorials (twitter threads) to DevonThink for later review, but can’t figure out a good way to do it. When I try to clip from Twitter I either get HTML pages that say “something went wrong” or text only notes that show me only a fraction of the tweet (exclude replies, images, etc.). It seems like the only way I can get anything close to exporting an entire twitter thread is to just print to PDF and save that, which is less than ideal.

Has anyone found a good way to do this?

Thanks!

Twitter.com is a dynamic website and most of its content is only loaded & displayed after user interaction (e.g. scrolling), even printing is difficult. You could try to use the classic website (mobile.twitter.com) instead but it doesn’t include pictures & videos, only text & links.

1 Like

Twitter’s a bit of a nightmare of ‘web 2.0’ dynamic javascript injections, as @cgrunenberg points out. One thing I’ve found to be useful is to toggle JavaScript off, then capture the page from ‘legacy’ twitter, and send that to DEVON as a web archive - there’s a KeyboardMaestro macro on Github for this very purpose.

As a side-effect, it’s a godsend when pages start complaining about operating adblockers. I have it bound to ⌃J in Safari.

It works fine for me when I Clip as a Web Archive

Another option would be for DT to provide direct access via the Twitter API, like MaxQDA does…see https://www.maxqda.com/help-max18/import/importing-twitter-data

This still doesn’t give access to historical tweets though.

1 Like

I have Twitter feed for certain users.(text only+URL to the tweet) Via right click>new>twitter feed.
But you want Twitter threads that might not be possible with Twitter feed.

I tend to simply “clip as bookmark” Twitter and other forum threads which are by their very nature live contents: you cannot know wether they will expand or not between the time you clip them and the time you review them.

Have you tried capturing the threads with Instapaper then importing them into DEVONthink?

FWIW I use the Devonthink sorter (displayed as ‘menu extra’) to clip via section screenshot - the main problem there is that it won’t mark the URL or give note space to let me paste it in before clipping the screenshot. I tend not to do it en masse, so to pop into the ‘recently added’ smart group and paste the URL in the given space under ‘generic info’ isn’t too much of a hassle at all, but it could be for another user.

It obviously depends on the length of the thread, too. I’ll zoom out the browser enough that I can fit the whole thread into a single page (no scrolling), but that only goes so far until the content is no longer legible.

I also set up an email account for my devonthink inbox, so if I’m on my android I can copy the URL, easily do a scrolling screenshot (!!!), & send it via email (URL in the message box) without too much trouble. This is actually my favorite option, because (1) scrolling screenshot and (2) once the email is sent, I’m bumped right back to where I was on Twitter and can keep scrolling through my feed, no need to open up Devonthink before I lose track of the URL.

I finally found an expedient way to save twitter threads.

  1. open thread on twitter, copy url
  2. paste url into search box at https://threadreaderapp.com/
  3. Click the icon for “Clip to Devonthink 3” extension (installed on Safari and Google Apple>System Preferences>Extensions>Devonthink 3, share menu)
  4. Choose Format PDF (One Page), modify title as desired, add the original twitter thread url, add tags as desired.
  5. go to DT# inbox or wherever you clip to
  6. confirm that thread is saved as PDF+Text (I have PDFs made searchable on import)
    Done!
    Note - need to wait for images to load on threadreader. You can’t click through to embedded video clips using this format.
4 Likes

Interesting find. Thanks for sharing. :slight_smile:

I wanted to save a Twitter post as web archive, but got an error when viewing them in DT. So I clipped as bookmark, headed over to DT and converted the bookmark to a web archive. This way I can view the post without error.

HTH others :slight_smile:

2 Likes

MaxQDA looks nice but it doesn’t seem to capture images and videos.

Thank you for the idea and it has some cool analysis features, for sure.

I’ve been using tweetdeck.twitter.com.
I found that the only way to capture the tweets there is by using Formatted Note.
The other formats have the nasty problem of refreshing when you open them…
But formatted note keeps most everything nicely and tweetdeck.twitter.com gives a pretty nice way of viewing a few tweets.

Not all tweets are shown, but if you are needing to save the most recent and you can follow the subject closely, it will allow you to capture everything nicely.

What would be nice is to have a way in DT3 to capture a search of any kind and have it go through and create documents for each tweet in the thread capturing an archive quality version.
I’d like to have the image or video attached to the tweet included.

For now, I’ll stick with Formatted Note and use tweetdeck.

Clipping with this new version of DEVONthink 3.8.4 has introduced a crashing issue for me.
But even before it has a tiny issue I should probably explain here… just in case it can be fixed…

The Problem / procedure

  • click on tweetdeck url
  • click on a tweet so you have a back button and you can see the replies to the tweet
  • ctrl+f saves a formatted text version of the tweetdeck in the DT3 browser to the current group
  • be sure to have download manager visible and set to “start” in the bottom left
  • click back to return to the tweetdeck stream to get your next tweet
  • you will see the download manager get a new entry
    CleanShot 2022-07-02 at 09.21.17
  • also, you will have a new window popup. It is blank in version 3.8.4, but in the last version I was using it was a different version of tweetdeck… perhaps an image of one of the saved tweetdeck captures I had made. I didn’t investigate. I simply close it and move on. But it’s a step I have to do EVERY capture.
  • sometimes, at this point, DT3 will crash (latest version I started experiencing this).

What would I like?

  • Ideally a smarter capture system. There are tools like FireShot: Webpage Screenshots + Annotations 1.10.05 updated! that have an understanding of the browser DOM and allow you to capture scrolling areas. I would like to be able to capture just one scrolling column in the tweetdeck and have it saved with images and text and links with a single hotkey and nothing put into my downloads window and no extra windows and no crashes.
  • DT3 needs to offer a capture web page as image + OCR + capture links to annotation + capture all images to annotation + capture all media (zips, pdfs, xls, txt, etc.) to download manager. That would sort of solve lots of issues for me I think. It would give me exactly what I see. I would give me the text, it would give me an all-in one capture. We already have scripts that capture all media and the OCR exists, but I don’t know how to capture the whole page as an image.
  • Download manager to be attached to inspector (yes, I know that may seem odd since inspector is for the current file… Perhaps just let me have a double dock at right - download manager/logs/etc - and inspector.
  • Ability to drag and drop from download manager without needing to open the downloads folder group.
  • A way to visualize the paginated PDF at the time of capture (print preview) with a visual means of adjusting the margins and zoom ands then saving a profile that can be selected quickly the next capture

Additional Notes/Clarifications

  • I use a hot key ctrl+f to capture formatted text version of tweetdeck.
  • I have a url bookmark I open in DT3.
  • So with tweetdeck you have columns and you can add several if you wish. I use just one in order to do a capture of the tweet and the surrounding tweets. It does have some limits, but it works okay for me.
  • I tried all the capture formats and found that only formatted text gives me a good capture.
  • Except for videos. For videos, I have to do it differently because the formatted text version removes the video - sort of like the iPhone if you try to capture a screenshot of a video. They make the screen black because they want to protect the content, so in those causes you have to use a second phone to capture the video. Go figure. Coming from somebody that shares all my work, I will never understand patent and copyright law, but these folks think they have a right to protect their ideas… Anyhow…
  • To capture the video tweets, I use PDF Paginated. It doesn’t capture multiple pages like the formatted text, so the bottom of the tweetdeck column is cut off (I suppose this could be fixed if I had a way to preview the PDF Pagination and fix it by zooming but for not that process is too difficult to do if you have 20 tweets to do in a few seconds). The PDF capture shows me the tweet with the image of the video which the API didn’t protect in this case.
  • Then I capture the video by right-click > Download Video and it appears in my Downloads Window.
  • By the way, I found a nice tool that lets me quickly open menu commands in DT3 and other macOS Paletro — Command Palette in any applications and I find it excellent for getting the download manager to show. I do wish the download manager could be part of the inspector view at the right of the application because I use it a lot and I also wish I could drag the items downloaded from the manager to the folder where I am working. For now, I have to open the download manager and see if the download is done or activate the downloader if I have stopped it and then open another window of DT3 and drag the file from there.
  • Note that the capture to PDF doesn’t have the same problem as the capture to formatted text. Hitting the back button after a PDF paginated capture work exactly as it should, but since it cuts off the text at the bottom, it is still a problem and only useful if you are happy with the first page of the tweetdeck.

Any luck with this problem with DT3 and tweetdeck.twitter.com?

If you open tweetdeck, open a tweet and then use formatted note to capture it, that works,
but when you hit the back button, DT3 opens a new, blank window and if you do this more than a few times, DT3 crashes. It also adds a download entry to the downloads window

Can we at least

  1. not popup the extra window
  2. not add to downloads

Then perhaps we can fix the crashing problem.

It wasn’t crashing in version 3.8.2 and earlier.

Thank you

Which version do you use? Version 3.8.5 should fix the crash.

3.8.5

Still crashing for me.

Temporary solution found…

Fireshot now has their beta available on Mac.
I’ve been using it on PC for a while and kept bugging them to make it work on MacOS.
They’ve done it. It’s not yet as powerful, but it will let you quickly generate PDF screenshots that inclde text and hyperlinks.

Works for Tweetdeck from safari and then I can drag all the PDFs into DT3…

Please choose Help > Report Bug while pressing the Alt modifier key and send the result and a web archive causing the issue to cgrunenberg - at - devon-technologies.com - thanks!

1 Like