DevonThink and Zotero - wish to script

Hi all!

As an academic, I keep all my articles in DevonThink, and only use a citation manager for citations. My current workflow has me initially downloading articles into Zotero, which automatically renames PDFs (I use ZotFile, but could use native functionality just as well) and moves the PDFs into a folder that DevonThink watches. DevonThink then pulls the article into its database - it means that Zotero gets confused and doesn’t know where the attachment is, but I just never use Zotero to actually manage attachments. So far, so good.

Download directly to Zotero -> rename PDF within Zotero -> moved to DevonThink watched folder -> imported into Devonthink = citation in Zotero, PDF in DevonThink

However, within DevonThink my articles now have limited information. They have whatever author, year, and title info I put into their name, but nothing beyond that. With the new scripting capabilities of DevonThink, I know there must be a way to launch or trigger a script that should look for the article info in Zotero, and add that info as various metadata types in DevonThink (i.e., tags/comments/etc.)

I don’t know that much about scripting, but I’m willing to learn. I’ve seen various scripts that pull data from Zotero, but no good explanation of how queries to Zotero work, or how to pull specific types of information from such an external source and set them as various DevonThink attributes.

Any and all help is welcome!

3 Likes

Zotero has a list of development efforts of various clients and plugins - here’s one starting place: https://www.zotero.org/support/plugins

I use zothero (which is an Alfred script) to pull data from Zotero for copying/pasting finished citations. Obviously, this is a very different scenario than you’re proposing, but it might offer a model for how to query the Zotero database. Several other plugins on that page might be simpler and thus easier to figure out how they’re working.

My workflow is rather the opposite of yours (Download to a cloud-synced folder that is indexed in DT, then manually add to Zotero for the auto-renaming and metadata; DT automatically updates the indexed file). There are drawbacks when involving DTTG, but I’m curious about what you see as the advantages of your system?

I do something similar. There’s a Zotfile plugin that helps rename and move files attachments added to Zotero references. I use that plugin to put files in an indexed folder and open those PDFs in place. The file gets updated everywhere as expected, and Zotfile can further extract annotations and do some other things. It works wonderfully!

1 Like

@lutefish, thanks! I’ll check out that page for ideas (as soon as the semester is over…)

I prefer keeping most of my documents in DevonThink, rather than indexed; which is why I use a workflow that renames everything before putting it into DevonThink. If there was a way to script it on import I would go that route, but Zotero doesn’t support scripting.

(I find that moving items around in DevonThink is easier with internal folders than with indexed folders, which default to a one-way rather than a two-way sync.)

@ryanjamurphy, your workflow is great assuming you want to use indexed folders. I really only use Zotero for managing citations, and do everything else within DevonThink, so I prefer this route.

1 Like

Zotero does support scripting (to an extreme level – there are no restrictions on what you can do with this scripting as there’s not really anything that separates your code from Zotero’s own code): https:// bit . ly / 32jRhhA for one-offs, https:// bit . ly / 2Sdg567 for things you want to run multiple times. If you have something specific in mind that you run over and over, a dedicated plugin is easy to create (I sometimes put together simple Zotero plugins in less than an hour, but I know Zotero fairly well). You can even change Zotero’s behavior using run-time patching of the Zotero code.

Check here Zotero sheet to DT

See Here:

Hello guys, I am new to this forum and actually just starting to consider DT and Zotero as a system for my PhD research. Your workflows seem great in different ways. I hope you can bear with my questions for a bit!

Can you demonstrate how one can replicate your workflows?

@ryanjamurphy When you manually add the file to Zotero, doesn’t Zotero copy the pdf into a new folder in its own database? How does the original file in your synced folder then get updated in DT?

@ryanjamurphy @lutefish Also, regarding DT incorporating metadata: I tried to test it manually (index a zotero folder in DT) and while the pdf does show up, not all its metadata does. What metadata does your own method generate in DT? So I don’t understand how, or if, your method solves OP’s problem (incomplete metadata).

And finally, to everyone, about indexing vs. importing. I have seen that this is a highly debated topic on this forum, but can you briefly explain why you chose one over the other?

Thank you all,

1 Like

I use Bookends now, although I still miss some aspects of Zotero. (It is much faster at capturing references.)

I used Zotfile, a fantastic tool to control how PDFs behave. You can easily use it to control where PDFs go. I indexed that destination folder in DEVONthink.

I tried to test it manually (index a zotero folder in DT) and while the pdf does show up, not all its metadata does. What metadata does your own method generate in DT? So I don’t understand how, or if, your method solves OP’s problem (incomplete metadata).

I never bother to store metadata in DT. That’s why I use the reference manager!

indexing vs. importing

As you intuited, you can’t easily keep a PDF associated with Zotero from within a DT database. So, indexing helps you have the PDF available both to your reference manager and to DT.

1 Like

Thanks for the swift reply!

It makes sense now. I’d be really interested in understanding your workflow. And why you chose to go with Bookends instead of Zotero.

For me the issue is the following: I currently have different types of files (photos, word documents, scrivener files, notes, handwritten notes, book reviews, books, pdf articles, citations, tweets, safari bookmarks, etc.) in different places (HDD, Google File Stream, iCloud, notebooks) and no universal way of indexing them—which now at the beginning of my PhD seems to me like a major deficiency. Ideally I am looking for a way in which I can tag any and all types of files, group and regroup them according to themes or projects, and retrieve them at will (hopefully finding connections between things I didn’t consider together before). All while not having duplicates, and being able to access them across devices. For me this is now not only about DT, but also figuring out how and where I want to store and backup my files in general.

Now the first year of my PhD is over and I have to figure this out and get working properly. I also have to say that I’m rather scatterbrained and so I need a low maintenance protocol.

So if you have any tips that would be greatly appreciated!

(Btw I tried browsing your reply history so as not to be redundant, and your reply on this other thread is very intriguing: Creating a commonplace book with DevonThink?)

1 Like

Forgot the link to Zotfile in the previous post: http://zotfile.com

Yeah, you’re asking the key questions. You’ll find many approaches to the problem you’re describing here, on the Bookends forums, and elsewhere on the web. There isn’t really a right answer—the key is to iteratively develop a system without spending too much time on the system, hah.

That thread details my approach. In my opinion, there’s no good hierarchy for complicated topics. At the same time tagging can be problematic: https://fortelabs.co/blog/tagging-is-broken/ (though Tiago has changed his tune a little on this over the years)

So, I’ve tried to reduce friction by saving everything in the same way and relying on search and a closed tagging system (e.g., don’t tag openly—predefine tags and use automations to only choose from those tags).

On reflection, though, I haven’t actually used those tags since I started maintaining them months ago. Like, at all. There’s always a possibility that’ll change, so I guess it provides some peace of mind. Still, I find I can usually find what I want because I remember authors’ names and paper titles. So I think I would advocate for using my automatic-organization-into-semesters-by-year approach and just use search. Don’t worry about other organizing paradigms.

I have also recently gotten into the PKM world, and while there’s a whole other chaotic class of apps, systems, and workflow questions there, I do think the philosophies behind Taking Smart Notes and related workflow philosophies add a valuable dimension to remembering and using the readings you’re using.

Finally, on Zotero vs. Bookends: honestly I’d stick with the former. Bookends is more powerful and fluid as a Mac app, but Zotero was definitely fine, and I think if I’d gotten better with Zotero and citekeys I wouldn’t have felt the pull for Bookends.

4 Likes

Thanks again, Ryan, much appreciated. This gives me a lot to think about. Back to the basics.

1 Like