I’m migrating from Papers 3, and after reading reviews and user comments decided to settle on Bookends as a replacement. I also want to use this switching opportunity to finally take advantage of the searching & AI features in Devonthink in my research workflow. I’ve been reading for 3 days, trying to figure how to do this, but I’m confused by the multiple possibilities and cannot conclude on how to best approach this…
Here’s what I want to achieve:
Use Bookends as my reference manager, mainly to import new papers through the Watch folder, auto-complete citing information & scan my manuscripts to automatically format bibliography when I write papers
Use DT for everything else: read, highlight, annotate, search, categorize (in groups & smart groups)
Hopefully keep DT & BE “in sync” for the basic info for each entry (PDF name, author, year). Keywords & labels sync would be a bonus.
The specific workflow would go something like this:
Import PDF in BE, then auto-complete the reference information
Have DT automatically index the PDF with the reference information (this is where I’m stuck… I don’t get how to use the Import from Bookends to link to the indexed PDF… is that even possible?)
In DT: annotate, add keywords (not tags, because they “pollute” Mac Finder’s tags), label as needed (to read, important, etc…), file in groups and/or smart groups
Write manuscripts in Word, insert needed reference (in whatever format, I don’t really care, as I usually just enter the reference quickly and format the bibliography in a separate step when I’m done writing)
Here is what I’ve done so far:
I exported all my Papers 3 entries (PDF, webpages, jpg) into Bookends using the app/script available here. Although it took a VERY long time (I have ~10000 items in my library), everything went smoothly: keywords and labels all got exported, annotations as well (in the form of note cards) with the corresponding highlight color.
I started indexing the Bookends/Attachments folder in Devonthink, which is still going and could take a while. But I’m realizing that the file name in DT is the PDF name (author-year-title-journal, as named initially by Papers), and that NO other information seems to be indexed (authors, paper title, keywords, labels, note cards…). Fortunately, PDF annotations done in Preview are correctly indexed (I guess the problem comes from the “note card” format in BE…)
Note that I don’t necessarily need to have all the metadata in BE (keywords, labels…), since I plan on doing all the work in DT. I just figured this was the easiest way to automate the transfer of my Papers 3 library into BE, and thought that the reference information would follow… I’m starting to wonder if I should have gone the other way around: import everything in DT, and export only the papers I’m going to cite into BE as needed… The fact is, I already have my whole Papers library indexed in DT, but just never really used it.
Any suggestion on the simplest way to achieve what I want?
I would not have indexed the entire collection of resources. The results of the direction you’re heading would have been more easily determined by working with a small batch of files. Currently, you will have to wait until the indexing process is done to see if it’s in the general area you hoped to get to.
The data in Bookends is internal to the application, so things like added Keywords aren’t actually applied to the attached file(s). This means indexing the PDFs will give you access to them in DEVONthink but the Bookends metadata will not be included.
There is a Data > New from Template > Education > Reference (from Bookends) that creates a rich text file from selected files in Bookends. Ths does include some metadata, but it’s a static file, i.e., changes made are local, not sent back and forth to Bookends.
I really wanted to be able to group items in DT according to keywords… Or tags I guess, if that’s the only way. I don’t see any problems moving forward, as I would add keywords to new items in DT since that’s where I would do all the work.
But I need to find a way to import the keywords associated with existing indexed items. I don’t mind having to batch process the files once if needed (in BE or DT). Being able to export references from Bookends is nice, but having a separate RTF for each PDF isn’t what I’m looking for. I want to have all annotations & keywords directly associated with the PDF. And as much as possible, I’d like to have the Bookends metadata populate the DT metadata (Title=Title, Author=Author, Publication date=Creation date, Keyword=Keyword, etc…). Without this metadata, smart grouping would be kind of unusable to me… I naively assumed that this would not be a problem to transfer… but I see that this is not currently (easily) doable.
I think a script combining some of the stuff discussed in this Bookends forum post might do the trick. But I’m running out of time to set it up right now. So I guess I’ll stick to Bookends only for now, and add DT later when I have time to figure things out.
At least the Bookends attachments folder is now fully indexed, so I can play with the AI thing to see if I find it useful.
Hi @guylaine, I’m the author of the “Papers to Bookends” export script that you mentioned using (glad to hear it worked for you!).
To also associate your original reference keywords with the indexed PDFs in DEVONthink, you could try my Papers to DEVONthink script. This, of course, assumes that you still have your original Papers library available.
You could also take the script as an inspiration to develop a script that directly does want you want. E.g., take the reference(s) selected in Bookends, index its PDF in DEVONthink, and set the index entry’s keywords and label accordingly. Similar to Papers & DEVONthink, Bookends is nicely scriptable, so this should be doable.
Hi Matthias! Thanks for that script, worked like a charm!
I do still have my Papers library available: I’ll probably never delete it, it’s currently backed up in like 3 different places haha! I had looked at your Papers to DEVONthink script, and although it would partially do what I need (import the metadata), I don’t want to have the annotations in separate RTF files, but as document annotations in DT. After waiting for the Bookends/Attachments folder indexing to finish, I can see that all annotations are correctly associated with the PDF in DT (whether done directly in the PDF, or done in Papers and therefore imported in Bookends). Not sure exactly how that happens? But that’s exactly what I want so I don’t really care I can also see that the metadata associated with the PDF originally (not the metadata from Bookends) is there as well, which makes sense I guess since it’s only the PDF that’s indexed in DT.
I’ll definitely try to tweak the available scripts to accomplish what I want (get the keywords, labels, ratings etc…) from BE to DT. But as I have very little experience with scripting (ie. I can modify existing ones to some extent, but can definitely not come up with my own…), I’ll have to spend a bit of time on this, which I don’t have right now.
Thanks for the suggestions though, it will definitely be useful when I get around doing this!
Hi,
I suppose I could open a new topic with my questions, but I think it’s better to reply here.
My first question is: while I’m happy to index Bookends Attachments folders in Devonthink, and importing references to the database, I’m not sure how to continue. For example: If I modify a paragraph in a reference file, the changes don’t seem to update in Bookends (and vice-versa, when I add a notecard in Bookends, the reference file in Devonthink is not updated).
There are some script or workaround available to do this?
My second question is If I can play with the references and/or PDF-attachment files in Devonthink (for example, to put some in a group) without breaking the sync between Devonthink and Bookends.
Note: There is no sync between Bookends and DEVONthink. If you are indexing the PDFs, changes are made to the file in the Finder, so changes should be shown in Bookends.
Hi, but there is any way the references files (which contains notes) are modified/updated in the same way the PDFs are? If not, it seems more convenient, to replace all the notes in Bookends with a link to DevonThink.
Hi, is there some news regarding such a script? What would be great is a simple script that re-imports all folders that have been imported previously (DTP shows a bookends symbol, so somehow those must be detectable). Then the mechanism of updating links to items in BE would be very similar to the process of updating indexed items that reside outside the database.
No there is no such script still. Also, a reimport of already imported items could easily become non-performant with many records and would need to ensure any user-added metadata, etc. was preserved.
Hello, I am trying to understand the File > References > Import from Bookends recurring functionality. I have done one bulk import, which is great.
But if I start working with the references (which I understand are RTF files) – replicating them, adding labels, etc – and then I want to do subsequent imports, as I add new references into Bookends-- what will that do to the references (RTF files) that have already been imported?
I see in the Manual that it says “Already Imported References are Updated,” on p. 136, but can someone provide more information about what exactly this means- will the new import replace the older RTF/ reference, or will the new metadata that I’ve added to it in DT remain?
I can also see that the metadata associated with the PDF originally (not the metadata from Bookends) is there as well, which makes sense I guess since it’s only the PDF that’s indexed in DT.