Twitter Archive

There’s a new iPad app out called Tweet Library. It imports up to 3000 tweets (+ new ones) and puts them into a searchable database.

DEVONthink should somehow be able to do this. Any ideas on how best to go about it? I can’t seem to get Twitter’s RSS to work (and even if, it probably wouldn’t go back hundreds of entries).

Twitter’s RSS feeds return only the last 20 tweets, therefore the only possibility right now is probably to access the Twitter API via AppleScript.

To backup my twitter archive I use a shell script. You can probably run the shell script within applescript and then have the results outputted to a DEVONthink file rather than (as I do) to a simple text file.

sites.huginn.net/scripts/shell-scripts

#Save the date:
mydate_folder=`date +%Y.%m.%d`
#Create a new folder for today's backup:
mkdir ~/shell/backups/twitter/$mydate_folder
#Download 5 xml files of 100 twitter entries each from Twitter
curl -o ~/shell/backups/twitter/$mydate_folder/#1.xml -u username:password "http://twitter.com/statuses/user_timeline.xml?count=100&page=[1-5]"
#After the download tar and gzip the folder, and delete the original folder.
cd ~/shell/backups/twitter/
tar -czf $mydate_folder.tar.gz $mydate_folder
rm -rf ~/shell/backups/twitter/$mydate_folder
#Determine the size of the update to include in our log file:
mysize=`du -ks $mydate_folder.tar.gz | awk '{print $1}'`
#Add an entry to my log file (which I have already added entries to for other backups made tday)
echo "\nTwitter backed up. $mysize KB in size.\n" >> ~/shell/backups/log.txt

Ahh…I just noticed my weekly twitter backup script stopped working after August 8 this summer…looks like there were some changes at Twitter.

The error returned by twitter.com:

Basic authentication is not supported

I found more info here:

danzambonini.com/migrating-from- … edentials/

Two other options in the meantime:

  1. Use a service like Tweet Scan to grab a big collection of your tweets (tweetscan.com/)

  2. If you just want an archive of your own tweets going forward, set up an RSS feed in DEVON but configure it “Manually” remove articles.

Tweet Scan was a great tip, thanks!

The RSS import looks really bad, though, I think I’ll schedule Tweet Scan backups instead.