I have started to put together a database of notes in the spirit of the Zettelkasten (ZK). Backlinks (also referred to as incoming links) are a feature that lends itself particularly to digital versions of the ZK. For various reasons, I am not keen on yet another dedicated piece of software for the ZK, and online options such as Roam or Remnote did not convince me either (export of your information is a massive concern, but they also look insanely busy for a serene ZK, with all those hovering pop-ups). Hence, I started implementing the ZK in trusted DT using Markdown as the note format. At this point, DT does not do backlinks. Recently, Bernardo_V (forum thread) and JacobIO (forum thread) posted very nice solutions. However, for my needs, I want something slightly different. Here are my requirements:
- Backlinks must work in Devonthink To Go.
- Backlinks must be immune to future name changes of files.
- Web export of the DB must fully maintain the link structure ( ➝ 100% safe exit strategy, as generic html will always work).
This implies:
- Wikilinks are out (don’t work in DTTG and break when filenames are changed).
- Item links are the way to go, but finding them must be based on UUID, not filenames.
Bernardo_V’s scripts almost fit the bill, as they can handle item links. However, they use the name to match the link. In principle, it would be easy to match the UUID instead of the name. Unfortunately, DT these days searches the rendered markdown content, not the source (see e.g. this forum thread)
My solution: copy the markdown source of all markdown files in the ZK DB as plain text into a custom metadata field of type “multi-line text” (but exclude the potentially already existing “Backlinks” section, to avoid going in circles). Then, search this field for links based on their UUID.
When should one act with such a script? One could attach it to markdown files such that when a file is opened in DT, the backlinks are updated on the fly. However, this will not work for someone like me who reads the notes mostly in DTTG. Hence I decided to do bulk backlink updates periodically (say once a day and whenever necessary).
The attached script takes a list of selected markdown files (but does not check the filetype!), and first copies for all of them the source md into the custom metadata field. Then it again loops through the list and searches the custom field for backlinks, adopting one of Bernardo_V’s scripts, but searching for UUID instead.
That’s a lot of searching: for 96 files, this takes 29 seconds on my lowly 2016 1.2 GHz MacBook. Naively, I’d expect this to scale as N^2 for N markdown files in the DB: N searches and in each search N items to search. However, in a first, imperfect, test, I found a largely linear behaviour: 192 files took 54 secs, 384 took 123 secs (I only have a few, independent markdown files in this DB, so I duplicated them multiple times over, which, however, creates a large number of almost identical files, but on the other hand I get massive levels of backlinking, which is also unrealistic).
Seems that for a large ZK of several thousand notes, this can still easily finish over lunch or overnight.
Below is the code. Please note that I have only fleeting familiarity with Applescript. I also can’t figure out how to make the code appear with colour highlighting (update: a fairy swooped in and inserted the triple backquotes).
(*
Purpose: Add UUID-based backlinks to a markdown note by appending them
to the end of the document, making the backlinks available in DevonThink to Go,
and in a DT web export of the note collection.
GG, 2020.09.14
Credits:
Based on an early version of a script provided by Bernardo_V, who deservers all
the credit for the difficult stuff
(see https://discourse.devontechnologies.com/t/return-links-back-links/54390 ).
This version of Bernardo_V's base script was distributed with the ebook
"Taking Smart Notes with DevonThink" by Kourosh Dini
( https://www.kouroshdini.com/course-books/ ).
What's different: Unlike the other scripts, this one identifies item links by
their UUID, NOT THE FILENAME. This is the only way to protect links
from future file name changes. To me this is a must-have feature (ruling
wikilinks effectively out, which also won't work in DTTG).
The problem: UUID, while present in the .md file in the link, CANNOT
be searched in DevonThink, because it only indexes the RENDERED
markdown content. See
https://discourse.devontechnologies.com/t/search-for-x-devonthink-item-link-not-working/57652 .
The solution: Prior to searching for backlinks, this script copies for all
selected files the markdown SOURCE into a custom metadata field
"GGtext", which subsequently, the is searched for links based on UUID.
Slight complication: already existing backlinks must not be copied into
the custom metadata field, otherwise they would be interpreted as
forward links. Hence, the "backlinks" section is excluded in part 1.
Usage and practical considerations:
- This script is run periodically to update all backlinks in bulk
- technically, one could use a trigger that runs this script for a single
file when it is opened in DT, so you get JIT backlinks updates whenever
you look at a file. However, no such mechanism is avaible in DTTG,
which I mostly use for looking at my notes. In that case periodic bulk
backlink maintenance is the way to go.
- this script has to loop twice over all selected markdown notes and carry
out DT searches for each file. It might get slow with a large number of notes.
I'm testing with 96 markdown files and it seems to take around 28 seconds
on my lowly 2016 1.2 GHz MacBook. Naively, 1000 markdown files might
then take around 290 seconds. Part 1 is much faster. The repeated searches
in part 2 slow things down.
For large markdown note collections, run it over lunch break or start it
last thing in the evening.
1. Select all markdown files you want to include in the backlinking process
(in the simplest case, make a smart folder that lists all markdown files in the
DB, and then select them all).
- note: the script does not check explicitly whether all selected files are
indeed markdown (could be added, but I'm not an Applescript whiz, and have
to move on to other stuff now). I have not checked yet what it does to other file
types. Proceed at your own risk.
2. Run the script, wait, and done. Hopefully there are no major bugs that
eluded me.
3. Once DB is synced, backlinks will be available in DTTG.
*)
use AppleScript version "2.4" -- Yosemite (10.10) or later
use script "RegexAndStuffLib"
use scripting additions
property MainUUID : ""
property theDelimiter : "###### Backlinks" -- Delimiter of choice. e.g. # Backlinks
property PlainText : "GGtext" -- name of the custom metadata field for the .md source
-- ===================================================
-- PART 1: for the selected list of .md files, copy the markdown source into the
-- custom metadata field "GGtext", but omit any existing "backlinks" section
tell application id "DNtp"
set theSelection to the selection
repeat with theRecord in theSelection
-- uncomment one of the following
-- copy the document's UUID into the comment
-- set the comment of theRecord to uuid of theRecord
-- copy the plain text of a markdown document into the comment
set theText to plain text of theRecord
-- return theText
-- Remove the backlink part
try
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to theDelimiter
set delimitedList to every text item of theText
set AppleScript's text item delimiters to oldDelims
on error
set AppleScript's text item delimiters to oldDelims
end try
-- return delimitedList
try
set theTruncatedText to item 1 of delimitedList
end try
-- return theTruncatedText
-- choose whether to write the plain text to the comment or a custom field
-- set the comment of theRecord to theTruncatedText
add custom meta data theTruncatedText for PlainText to theRecord
end repeat
display notification "Success part 1!"
end tell
-- ===================================================
-- PART 2: Go through the list of selected .md files again, and search for links,
-- based on UUID information in the custom metadata field, and append an
-- updated backlinks section to the .md file.
tell application id "DNtp"
set theSelection to the selection
repeat with theRecord in theSelection
-- set theRecord to (content record of think window 1)
set theText to plain text of theRecord
set theName to name of theRecord
set theUUID to uuid of theRecord
-- return theName
-- return theText
-- return theUUID
-- Prepare the search string
-- set theSearchList to "\"" & theName & "\"" -- search for the name of the selected document
set theSearchList to "\"" & theUUID & "\"" -- search for the UUID of the selected document
-- return theSearchList
-- Perform the search
set theGroup to get record with uuid MainUUID
-- set theRecords to search "content:" & theSearchList & space & "kind:markdown" in theGroup -- search the contents
-- set theRecords to search "comment:" & theSearchList & space & "kind:markdown" in theGroup -- search the comments
-- Note: if custom metadata field has name "custom", then the search addresses it with "mdcustom"
set theRecords to search "mdggtext:" & theSearchList & space & "kind:markdown" in theGroup -- search custom metadata
-- return theRecords
set theList to {}
repeat with each in theRecords
-- Without AutoWikiLinks, we need the full address
set the end of theList to "[" & name of each & "]" & "(" & reference URL of each & ")" & " ⚫︎ "
--set the end of theList to return & "* [" & name of each & "]" & "(" & reference URL of each & ")"
end repeat
set theList to my sortlist(theList)
-- Remove old Returnlinks section
try
set oldDelims to AppleScript's text item delimiters -- salvar o delimitador padrão
set AppleScript's text item delimiters to theDelimiter
set delimitedList to every text item of theText
set AppleScript's text item delimiters to oldDelims -- restaurar delimitador padrão
on error
set AppleScript's text item delimiters to oldDelims -- restaurar delimitador padrão em caso de erro
end try
-- Add new ReturnLinks section
try
set theText to item 1 of delimitedList
set theText to my trimtext(theText, "", "end")
set the plain text of theRecord to theText & theDelimiter & return & theList as text
end try
end repeat
display notification "Success part 2!"
end tell
-- END OF MAIN PROGRAM
-- Handlers section
on replaceText(theString, old, new)
set {TID, text item delimiters} to {text item delimiters, old}
set theStringItems to text items of theString
set text item delimiters to new
set theString to theStringItems as text
set text item delimiters to TID
return theString
end replaceText
on trimtext(theText, theCharactersToTrim, theTrimDirection)
set theTrimLength to length of theCharactersToTrim
if theTrimDirection is in {"beginning", "both"} then
repeat while theText begins with theCharactersToTrim
try
set theText to characters (theTrimLength + 1) thru -1 of theText as string
on error
-- text contains nothing but trim characters
return ""
end try
end repeat
end if
if theTrimDirection is in {"end", "both"} then
repeat while theText ends with theCharactersToTrim
try
set theText to characters 1 thru -(theTrimLength + 1) of theText as string
on error
-- text contains nothing but trim characters
return ""
end try
end repeat
end if
return theText
end trimtext
on sortlist(theList)
set theIndexList to {}
set theSortedList to {}
repeat (length of theList) times
set theLowItem to ""
repeat with a from 1 to (length of theList)
if a is not in theIndexList then
set theCurrentItem to item a of theList as text
if theLowItem is "" then
set theLowItem to theCurrentItem
set theLowItemIndex to a
else if theCurrentItem comes before theLowItem then
set theLowItem to theCurrentItem
set theLowItemIndex to a
end if
end if
end repeat
set end of theSortedList to theLowItem
set end of theIndexList to theLowItemIndex
end repeat
return theSortedList
end sortlist
I suspect Bernardo_V needed those functions in his more general scripts. I also don’t know what 
