I’d like to be able to save tweets and tweetorials (twitter threads) to DevonThink for later review, but can’t figure out a good way to do it. When I try to clip from Twitter I either get HTML pages that say “something went wrong” or text only notes that show me only a fraction of the tweet (exclude replies, images, etc.). It seems like the only way I can get anything close to exporting an entire twitter thread is to just print to PDF and save that, which is less than ideal.
Twitter.com is a dynamic website and most of its content is only loaded & displayed after user interaction (e.g. scrolling), even printing is difficult. You could try to use the classic website (mobile.twitter.com) instead but it doesn’t include pictures & videos, only text & links.
Twitter’s a bit of a nightmare of ‘web 2.0’ dynamic javascript injections, as @cgrunenberg points out. One thing I’ve found to be useful is to toggle JavaScript off, then capture the page from ‘legacy’ twitter, and send that to DEVON as a web archive - there’s a KeyboardMaestro macro on Github for this very purpose.
As a side-effect, it’s a godsend when pages start complaining about operating adblockers. I have it bound to ⌃J in Safari.
I have Twitter feed for certain users.(text only+URL to the tweet) Via right click>new>twitter feed.
But you want Twitter threads that might not be possible with Twitter feed.
I tend to simply “clip as bookmark” Twitter and other forum threads which are by their very nature live contents: you cannot know wether they will expand or not between the time you clip them and the time you review them.
FWIW I use the Devonthink sorter (displayed as ‘menu extra’) to clip via section screenshot - the main problem there is that it won’t mark the URL or give note space to let me paste it in before clipping the screenshot. I tend not to do it en masse, so to pop into the ‘recently added’ smart group and paste the URL in the given space under ‘generic info’ isn’t too much of a hassle at all, but it could be for another user.
It obviously depends on the length of the thread, too. I’ll zoom out the browser enough that I can fit the whole thread into a single page (no scrolling), but that only goes so far until the content is no longer legible.
I also set up an email account for my devonthink inbox, so if I’m on my android I can copy the URL, easily do a scrolling screenshot (!!!), & send it via email (URL in the message box) without too much trouble. This is actually my favorite option, because (1) scrolling screenshot and (2) once the email is sent, I’m bumped right back to where I was on Twitter and can keep scrolling through my feed, no need to open up Devonthink before I lose track of the URL.
Click the icon for “Clip to Devonthink 3” extension (installed on Safari and Google Apple>System Preferences>Extensions>Devonthink 3, share menu)
Choose Format PDF (One Page), modify title as desired, add the original twitter thread url, add tags as desired.
go to DT# inbox or wherever you clip to
confirm that thread is saved as PDF+Text (I have PDFs made searchable on import)
Done!
Note - need to wait for images to load on threadreader. You can’t click through to embedded video clips using this format.
I wanted to save a Twitter post as web archive, but got an error when viewing them in DT. So I clipped as bookmark, headed over to DT and converted the bookmark to a web archive. This way I can view the post without error.
MaxQDA looks nice but it doesn’t seem to capture images and videos.
Thank you for the idea and it has some cool analysis features, for sure.
I’ve been using tweetdeck.twitter.com.
I found that the only way to capture the tweets there is by using Formatted Note.
The other formats have the nasty problem of refreshing when you open them…
But formatted note keeps most everything nicely and tweetdeck.twitter.com gives a pretty nice way of viewing a few tweets.
Not all tweets are shown, but if you are needing to save the most recent and you can follow the subject closely, it will allow you to capture everything nicely.
What would be nice is to have a way in DT3 to capture a search of any kind and have it go through and create documents for each tweet in the thread capturing an archive quality version.
I’d like to have the image or video attached to the tweet included.
For now, I’ll stick with Formatted Note and use tweetdeck.
Clipping with this new version of DEVONthink 3.8.4 has introduced a crashing issue for me.
But even before it has a tiny issue I should probably explain here… just in case it can be fixed…
The Problem / procedure
click on tweetdeck url
click on a tweet so you have a back button and you can see the replies to the tweet
ctrl+f saves a formatted text version of the tweetdeck in the DT3 browser to the current group
be sure to have download manager visible and set to “start” in the bottom left
click back to return to the tweetdeck stream to get your next tweet
you will see the download manager get a new entry
also, you will have a new window popup. It is blank in version 3.8.4, but in the last version I was using it was a different version of tweetdeck… perhaps an image of one of the saved tweetdeck captures I had made. I didn’t investigate. I simply close it and move on. But it’s a step I have to do EVERY capture.
sometimes, at this point, DT3 will crash (latest version I started experiencing this).
What would I like?
Ideally a smarter capture system. There are tools like FireShot: Webpage Screenshots + Annotations 1.10.05 updated! that have an understanding of the browser DOM and allow you to capture scrolling areas. I would like to be able to capture just one scrolling column in the tweetdeck and have it saved with images and text and links with a single hotkey and nothing put into my downloads window and no extra windows and no crashes.
DT3 needs to offer a capture web page as image + OCR + capture links to annotation + capture all images to annotation + capture all media (zips, pdfs, xls, txt, etc.) to download manager. That would sort of solve lots of issues for me I think. It would give me exactly what I see. I would give me the text, it would give me an all-in one capture. We already have scripts that capture all media and the OCR exists, but I don’t know how to capture the whole page as an image.
Download manager to be attached to inspector (yes, I know that may seem odd since inspector is for the current file… Perhaps just let me have a double dock at right - download manager/logs/etc - and inspector.
Ability to drag and drop from download manager without needing to open the downloads folder group.
A way to visualize the paginated PDF at the time of capture (print preview) with a visual means of adjusting the margins and zoom ands then saving a profile that can be selected quickly the next capture
Additional Notes/Clarifications
I use a hot key ctrl+f to capture formatted text version of tweetdeck.
I have a url bookmark I open in DT3.
So with tweetdeck you have columns and you can add several if you wish. I use just one in order to do a capture of the tweet and the surrounding tweets. It does have some limits, but it works okay for me.
I tried all the capture formats and found that only formatted text gives me a good capture.
Except for videos. For videos, I have to do it differently because the formatted text version removes the video - sort of like the iPhone if you try to capture a screenshot of a video. They make the screen black because they want to protect the content, so in those causes you have to use a second phone to capture the video. Go figure. Coming from somebody that shares all my work, I will never understand patent and copyright law, but these folks think they have a right to protect their ideas… Anyhow…
To capture the video tweets, I use PDF Paginated. It doesn’t capture multiple pages like the formatted text, so the bottom of the tweetdeck column is cut off (I suppose this could be fixed if I had a way to preview the PDF Pagination and fix it by zooming but for not that process is too difficult to do if you have 20 tweets to do in a few seconds). The PDF capture shows me the tweet with the image of the video which the API didn’t protect in this case.
Then I capture the video by right-click > Download Video and it appears in my Downloads Window.
By the way, I found a nice tool that lets me quickly open menu commands in DT3 and other macOS Paletro — Command Palette in any applications and I find it excellent for getting the download manager to show. I do wish the download manager could be part of the inspector view at the right of the application because I use it a lot and I also wish I could drag the items downloaded from the manager to the folder where I am working. For now, I have to open the download manager and see if the download is done or activate the downloader if I have stopped it and then open another window of DT3 and drag the file from there.
Note that the capture to PDF doesn’t have the same problem as the capture to formatted text. Hitting the back button after a PDF paginated capture work exactly as it should, but since it cuts off the text at the bottom, it is still a problem and only useful if you are happy with the first page of the tweetdeck.
If you open tweetdeck, open a tweet and then use formatted note to capture it, that works,
but when you hit the back button, DT3 opens a new, blank window and if you do this more than a few times, DT3 crashes. It also adds a download entry to the downloads window
Fireshot now has their beta available on Mac.
I’ve been using it on PC for a while and kept bugging them to make it work on MacOS.
They’ve done it. It’s not yet as powerful, but it will let you quickly generate PDF screenshots that inclde text and hyperlinks.
Works for Tweetdeck from safari and then I can drag all the PDFs into DT3…
Please choose Help > Report Bug while pressing the Alt modifier key and send the result and a web archive causing the issue to cgrunenberg - at - devon-technologies.com - thanks!
Looks like Tweetdeck will soon change.
Twitter is revamping it and they are prompting to test it.
It doesn’t work as nice as the current version, so I’m going to stop using it.
I looked harder for a replacement this time and found something that works very nice: https://nitter.net/
This is a twitter client and allows you to be anonymous.
It’s very nice because it doesn’t use javascript and therefore the web archive mode in devonthink works perfectly for a capture. In addition, the title of the post is captured as the name of the web archive, making it a lot nicer than “tweetdeck” for every post as I was dealing with before.
To download videos, I am using ytdlp yt-dlp -U 'enter the nitter address here' -o '%(uploader)s (%(uploader_id)s)/%(upload_date>%Y-%m-%d %a %b)s Twitter %(title).150s - (%(duration>%Hh%Mm%Ss)s) [%(resolution)s] [%(id)s]' --write-auto-sub --convert-subs=srt --write-description --write-info-json -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/mp4'
That generates a nice title up to 150 characters from nitter and gives you a high quality video download.
Images can be downloaded inside DT3 by opening them and choosing capture image. But I couldn’t figure out the videos, so those I will just use terminal command.
If you have the crashing situation for tweetdeck still in the backlog, feel free to toss it aside or mark it as a lower priority if there is nobody else requesting.
Thanks for all your help, as always. Hopefully this idea will help some other people that need or want to save information from Twitter. I find it very nice short of DT3 offering some sort of integrated tools.
Nitter.net does allow an RSS feed and that can be put into DT3 as a feed folder so might be handy for some people as well.
