Suggestions for annotations between DT, Bookends and Obsidian

DEVONthink
1

I have recently installed DT, and I anticipate that I am a novice. Nevertheless, I am trying to make it the central hub of my research apparatus, which currently consists of Bookends as a reference manager and Obsidian for taking notes in markdown. Both Bookends’ folder of attachments and Obsidian’s notes are already indexed in DT, but I lack to devise a solid and smooth workflow.

The first thing is to switch back and forth between DT’s indexed pdfs and Bookends’ references, hardlinking them. I have read about some scripts that allow to create links in the “user20” field in Bookends, and on the other hand in the finder comments of the DT file. Just this would be very useful for me, especially if it is possible to create a smart rule that does this in an automated way every so often, independently searching for files. Unfortunately, I have no scripting experience and wouldn’t know where to start, not even where they should be put to make them appear in DT.

The other issue, perhaps more tricky, is to understand (and discuss with you) what the best practices are for taking annotations on files.

If anyone is familiar with Bookends they will know that in addition to highlighting pdfs you can make annotations in the “notes” section, handling them as plain text (not markdown) or as “notecards.” However, I would not know how to index these pieces within DT.

The other method is to take notes on a specific book by creating a markdown in Obsidian indexed in DT, but the problem arises at this point of hardlinking it to the specific file in Bookends in a consistent way. DT has a very interesting system for creating annotations: creating a subfolder named annotations, with files in mardown inside linked to the respective files. The point is that using Obsidian I have a better markdown editor, with various plugins, and most importantly it allows me to group files into the specific subfolders I want, not limiting me to the database structure. Also being able to keep them on Dropbox to synchronize them on the fly with my Android phone (whereas DT’s “Annotations” files – correct me if I’m wrong – are created exclusively within the database and not externally, and only then indexed).

I noticed that @ryanjamurphy created a script a while back to manage the workflow between the three applications, but other than not knowing how to install a script, I can’t figure out if it would work for me. Possibly there are more updated or alternative scripts on the interconnection between these three software?

I apologize for the possibly confusing post but I would be curious to know experiences from those who are more skilled than me and have been in a similar scenario.

2 Likes
2

This is a kind of running bee in the bonnet for me too. I will point out to you that DEVONthink 3 has a facility to import references from Bookends. Go File > Import> references from Bookends. I very rarely see it mentioned and think it time that it was.
It is possible, and I did used to do it, to add notes to those imported references and it made all my references searchable on DEVONthink 3 too. I have never settled on a consistent bibliography method to be honest and I don’t have to handle huge numbers of citations like some working in teams and so on now do.
I really think of Bookends as a means to format citations: there is always a fiddle though and I can do it as quick by hand using Markdown as it were for the few I use now.

3

There is now an option in Preferences > General to choose where the Annotation file is created:

  1. In a shared Annotations group, which is exclusively in the database, as you outline; or

  2. In the same group as the source file. If that group is indexed (like your Obsidian folders) then the new annotation will be also be indexed.

Screenshot 2022-08-08 at 03.13.08
Screenshot 2022-08-08 at 03.13.081536×1038 142 KB

Secondly, with either of the two options, you can subsequently move the annotation to any group (indexed or imported) and the links will remain. You can do this either manually, or automatically with a Smart Rule (though that will depend on exactly what you want, of course). E.g. If you have option 2 selected, you could have a smart rule which says ‘If any document is created in this (the Bookends source) group with the word “(Annotation)” in the title, move (or replicate, or duplicate) it to that/those group(s) (the Obsidian notes group)’ Again, an imported file moved to an indexed folder becomes itself indexed.

Manually, it’s simply a matter of using a couple of shortcuts: Ctl-opt-cmd-o while the Annotation pane is visible opens the annotation in its own window, then Ctl-cmd-m moves the file to the group of your choice. (You can also add your own shortcut to create the annotation in the first place…)

Does that help?

3 Likes
4

It helps A LOT. I solved it by simply replicating the Annotations group within an Obsidian/Dropbox folder (indexed in the same database). That way all the annotation files I create within that database end up there and I have them available and sync’d in Obsidian as well. However, one thing is still not clear to me: what does the check in the “move annotations automatically” option imply?

At this point the last hurdle would be create a (possibly clickable) link from DevonThink’s indexed attachments (x-devonthink-item-type) to the Bookends counterpart (bookends://sonnysoftware.com-type) and vice versa. I see that the user20 field is often used (while on the other hand I am not clear if finder comments, aliases or custom metadata are the best choice). Maybe it is not such a difficult process but I have no idea where to start :slight_smile: In this way I could create a custom template in the shared annotation file with two placeholders that reference both the internal DT file and the reference in Bookends.

5

AFAIUI, the Move checkbox means that if you were to move the source document to another DT3 database, the annotation would move with it. I’m not sure how that applies when the annotation is not in the top level Annotations group though.

I don’t really use the Bookends integration so can’t really help with the other questions, sorry!

1 Like
6

Check out how I use the programs you mention. It may help as you develop your own system.

7

@brookter is correct on this. It makes the annotation file follow the referenced file when moving it between databases.

2 Likes
8

I tried the user20 thing for a while… until I realized that there’s scriptable ways to just open the PDF by finding the attachment in Bookends, instead of depending on fragile metadata.

In other words, you can use scripting from DEVONthink to say “Hey Bookends, open the reference with this PDF.” Similarly, you can use scripting from Bookends to say “Hey DEVONthink, open this publication’s attached PDF, please”.

It works perfectly and doesn’t require any extra metadata configuration or management. (I have no idea why the user20 solution became popular.)

The script below checks whether Bookends or DEVONthink is the currently active app and does the appropriate actions to open in the vice-versa.

💻 Script: Open current file in Bookends/DEVONthink 
-- get frontmost app and trigger the right function depending on if Bookends or DEVONthink are active.
tell application "System Events" to set activeApp to name of application processes whose frontmost is true
if (activeApp as text) contains "DEVONthink" then
	my openBookendsReferenceFromDTItem()
else if activeApp contains "Bookends" then
	my openDTItemFromBookends()
end if

on openDTItemFromBookends()
	tell application "Bookends"
		set theWindow to front library window
		set theSelection to theWindow's selected publication items
		repeat with eachItem in theSelection
			set theAttachments to get eachItem's attachment item
			set theAttachment to item 1 of theAttachments
			log theAttachment's path as string
			tell application id "DNtp"
				set mainframeDatabase to get database with uuid "D0CA3444-A862-4C99-9A20-3B93E6F24CA8"
				set libraryDatabase to get database with uuid "956EB0B2-F8DE-4955-873A-A065F0D096B7"
				set archiveDatabase to get database with uuid "ADB9D9D0-0304-43F4-A187-945F15A4777C"
				set attachmentDTRecord to lookup records with path (theAttachment's path as string) in mainframeDatabase
				if (count of attachmentDTRecord) is 0 then
					set attachmentDTRecord to lookup records with path (theAttachment's path as string) in libraryDatabase
				end if
				if (count of attachmentDTRecord) is 0 then
					set attachmentDTRecord to lookup records with path (theAttachment's path as string) in archiveDatabase
				end if
				if (count of attachmentDTRecord) is 0 then
					display notification "No corresponding DEVONthink record found for this reference's first attachment."
				else
					set attachmentDTRecord to the first item in attachmentDTRecord
					set openRecordWindow to open window for record the first item in attachmentDTRecord
					activate
				end if
			end tell
		end repeat
	end tell
end openDTItemFromBookends

on openBookendsReferenceFromDTItem()
	tell application id "DNtp"
		set theSelection to get the selection
		if (count of theSelection) is greater than 1 then
			display notification "More than one record selected in DEVONthink. Opening the first in Bookends."
		end if
		repeat with eachRecord in theSelection
			set thePath to eachRecord's path
			set theName to eachRecord's name
			tell application "Bookends"
				set thePublications to sql search "attachments REGEX '" & theName & "'"
				try
					set thePublication to the first item in thePublications
					my openBookendsItem(thePublication's id as string)
					activate
				on error
					display notification "This record was not found as an attachment in Bookends."
				end try
				--if (count of theSelection) is 1 then
				--else
				--set targetGroupName to the text returned of (display dialog "Name the group for these items:" default answer "")
				--tell front library window
				--	set targetGroup to make new group with properties {name:targetGroupName}
				--						
				--	repeat with eachPublication in thePublications
				--		add eachItem to targetGroup
				--	end repeat
				--end tell
				-- end if
			end tell
		end repeat
	end tell
end openBookendsReferenceFromDTItem


on openBookendsItem(someBookendsID)
	open location "bookends://sonnysoftware.com/" & someBookendsID
end openBookendsItem

As for how to install and run scripts — there’s a variety of ways. The first step in DEVONthink is to search for script in the documentation (menu Help → DEVONthink 3 Help) and learn a bit about the possibilities. Then I suggest adding this script to the DEVONthink 3 Toolbar.

As for Bookends… I actually don’t know the “official” way of launching scripts in that app. Which means you’ll probably want to find a script-launching tool of some kind. macOS offers a Scripts tool in the menubar for this, though it needs to be enabled. Personally, I have scripts like this tied to Keyboard Maestro macros and launch them via a Stream Deck, Alfred, and all sorts of other procrastination automation tools.

As for linking Bookends, DEVONthink, and Obsidian, I still mostly use the same workflow you linked to, though with some augmentations, such as streaming annotations into markdown. Don’t forget about the plugin I made for using Obsidian and DEVONthink together (DEVONlink and the companion script.

7 Likes
9

Thank you Ryan. On the hard links between DT and Bookends exactly the words I wanted to hear! Something simple but solid. I have started reading some scripting basics and am trying now to set up your script. Can you point me to the parts that I need to customize with my data? I guess they are the ones I quoted above, but I can’t discern what the differences are between mainframe, library and archive databases. What precisely do I need to enter?

On DevonLink I am already using it, I have already investigated your work. Really very useful.

1 Like
10

D’oh, sorry, I shared the script without even reviewing how it worked.

Those lines are hardcoded database UUIDs (that’s unique universal identifiers). I only keep my readings in certain databases, so I wanted to hardcode those in the script rather than iterate through all databases searching for my references. (It’s just a little better, performance-wise).

The following version of the script does the latter:

💻 Script: Open current file in Bookends/DEVONthink v2 
on run {}
	my openThisInThatOtherApp()
end run

-- get frontmost app and trigger the right function depending on if Bookends or DEVONthink are active.
on openThisInThatOtherApp()
	tell application "System Events" to set activeApp to name of application processes whose frontmost is true
	if (activeApp as text) contains "DEVONthink" then
		my openBookendsReferenceFromDTItem()
	else if activeApp contains "Bookends" then
		my openDTItemFromBookends()
	end if
end openThisInThatOtherApp

on openDTItemFromBookends()
	tell application "Bookends"
		set theWindow to front library window
		set theSelection to theWindow's selected publication items
		repeat with eachItem in theSelection
			set theAttachments to get eachItem's attachment item
			set theAttachment to item 1 of theAttachments
			log theAttachment's path as string
			tell application id "DNtp"
				set theDatabases to every database
				repeat with eachDatabase in theDatabases
					set attachmentDTRecord to lookup records with path (theAttachment's path as string) in eachDatabase
					if (count of attachmentDTRecord) is not 0 then -- found a matching record
						set attachmentDTRecord to the first item in attachmentDTRecord
						set openRecordWindow to open window for record the first item in attachmentDTRecord
						activate
						return openRecordWindow
					end if
				end repeat
				display notification "No corresponding DEVONthink record found for this reference's first attachment."
			end tell
		end repeat
	end tell
end openDTItemFromBookends

on openBookendsReferenceFromDTItem()
	tell application id "DNtp"
		set theSelection to get the selection
		if (count of theSelection) is greater than 1 then
			display notification "More than one record selected in DEVONthink. Opening the first in Bookends."
		end if
		repeat with eachRecord in theSelection
			set thePath to eachRecord's path
			set theName to eachRecord's name
			tell application "Bookends"
				set thePublications to sql search "attachments REGEX '" & theName & "'"
				try
					set thePublication to the first item in thePublications
					my openBookendsItem(thePublication's id as string)
					activate
				on error
					display notification "This record was not found as an attachment in Bookends."
				end try
				--if (count of theSelection) is 1 then
				--else
				--set targetGroupName to the text returned of (display dialog "Name the group for these items:" default answer "")
				--tell front library window
				--	set targetGroup to make new group with properties {name:targetGroupName}
				--						
				--	repeat with eachPublication in thePublications
				--		add eachItem to targetGroup
				--	end repeat
				--end tell
				-- end if
			end tell
		end repeat
	end tell
end openBookendsReferenceFromDTItem


on openBookendsItem(someBookendsID)
	open location "bookends://sonnysoftware.com/" & someBookendsID
end openBookendsItem

It’s the kind of script that should work most of the time, but might fail occasionally depending on a variety of edge cases. In my experience it has been easier to just go get the appropriate record manually in those cases than to try to code around the edge cases, hah.

2 Likes
11

I thank you very much. I think I will go for the non-redundant version, in which lines should I put my personal link database and which ones should I remove instead? Thank you very much.

1 Like
12

Sure.

You’ll be editing the following block:

				set mainframeDatabase to get database with uuid "D0CA3444-A862-4C99-9A20-3B93E6F24CA8"
				set libraryDatabase to get database with uuid "956EB0B2-F8DE-4955-873A-A065F0D096B7"
				set archiveDatabase to get database with uuid "ADB9D9D0-0304-43F4-A187-945F15A4777C"
				set attachmentDTRecord to lookup records with path (theAttachment's path as string) in mainframeDatabase
				if (count of attachmentDTRecord) is 0 then
					set attachmentDTRecord to lookup records with path (theAttachment's path as string) in libraryDatabase
				end if
				if (count of attachmentDTRecord) is 0 then
					set attachmentDTRecord to lookup records with path (theAttachment's path as string) in archiveDatabase
				end if

You’ll need one line of set [someDatabase] to get database with uuid "[some-uuid]" (lines 1–3 above) for each database you want to hardcode in.

(To get a database’s UUID, you can right-click on the database and Copy Database Link. Then paste that link somewhere and delete the x-devonthink-item:// part.)

Then, you’ll want to have one line for the first database you want to check (choose the one most likely to contain the reference):

set attachmentDTRecord to lookup records with path (theAttachment's path as string) in [your-first-database]

and then you’ll need an if block for any remaining databases you want to check:

if (count of attachmentDTRecord) is 0 then
	set attachmentDTRecord to lookup records with path (theAttachment's path as string) in libraryDatabase
end if

That’s it. I think, anyway!

13

Thank you! It works wonderfully from DEVONthink. Now I just need to figure out how to trigger the script from Bookends, since it doesn’t have a dedicated key/toolbar script nor anything else. I’d like to avoid having to buy Keyboard Maestro (I’m looking if Raycast can make up for it).

14

Look into the Scripts menubar item I mentioned above, too!

15

Solved Bookends trigger with this. Maybe helpful for others:

  1. Open Automator.
  2. Make a new Quick Action.
  3. Make sure it receives ‘no input’ at all programs.
  4. Select Run Apple Script and type in your code.
  5. Save!

Now go to System Preferences > Keyboard > Shortcuts. Select Services from the sidebar and find your service. Add a shortcut by double clicking (none).

Finally go to System Preferences > Security > Privacy > Accessibility and add Automator and the preferred app to run the shortcut.

3 Likes
16

Oh man.
I should have known about this script before I spent my days running scripts that write links in User20 fields, and the url field in Dt.

This is magic.

2 Likes
17

Maybe it is too late to ask about this, given that the thread is so old, but I think this script may be just what I am looking for as I had problems with what others called the “user20” method. I have one possibly useful comment for anyone who finds this thread, and one question.

Comment
In DT the easiest way to get a keyboard command is to make the script’s filename end with the characters “____F3” (where F3 is whichever function key you want, and there are 4 underscores) and place it in the DT scripts folder. Then Command-F3 (or whatever function key you have chosen) runs it. This worked a treat for me, so thank you to those who wrote the script. [I tried the automator method but with no success at all, in either direction.]

However I had no success at all the other way round, going from BE to DT, so I edited the script so that it simply told Bookends what to do, and I got this to work using the OS scripts menu (I am told that because this is present, there is no need for a scripts menu within BE; the DT developers clearly think differently). Attempts to assign it to a shortcut key using automator failed, but it works as a menu item.

Question

The script opens the selected PDF using the DT viewer. Instead of this, is it possible to replace it with instructions to open the Group that contains the PDF, with the PDF itself selected? This would be useful because one purpose of the script is surely to show you where the item is in DT, so having the PDF alone in a separate window is not helpful.

I assume something needs to be changed in the following code, but I don’t know what:

set openRecordWindow to open window for record the first item in attachmentDTRecord
activate
return openRecordWindow

It would be great to know how to edit this (of course, the issue may not lie in these three lines) or where to look for the answer.

Incidentally, I do not see an easy way in DT to go from viewing a PDF in a separate window to the enclosing group/folder; the nearest menu option is to go to the root folder of the database, when, even if the PDF is visible, it is not selected. Maybe I am missing something that should be obvious.

18

I’m not clear on exactly what you’re expecting to see but you can see the enclosing group in the Instances dropdown of the Info popup for a document. Tapping it navigates to that group.

19

The place the script takes me to is here:

screenshot_227
screenshot_227884×912 128 KB

Sorry if that was not clear. Nothing here seems to give that popup menu.

20

:thinking:
You asked about DEVONthink To Go and I answered based on that.

1 Like