How to avoid creating duplicates in a script (& any general tidy up tips)

Hi, I’m trying to modify the script by zverhope to index items from Bookends with additional metadata and reflexive Bookends/DT links. I’ve got something which more or less works. What I need to do now is avoid creating duplicates - i.e., if the item already exists, update its metadata instead of creating a new item.

How can I test for that? Script below

    tell application "Bookends"
	tell front library window
		set theTemplateFile to "/Users/lyndon/Library/Application Support/DEVONthink 3/Templates.noindex/Education/Reference LD.md"
		set theIDs to get id of publication items of group item "Red"
		repeat with theID in theIDs
			try
				set myRefs to (publication items whose id is theID)
				set myItem to first item of myRefs
				set {theKey, thePath, theAuthor, theEditor, theTitle, theShortTitle, theAbstract, theDOI, theKeywords, theURL} to {citekey, path of attachment items, authors, editors, title, short title, abstract, doi, keyword names, url} of myItem
				if theAuthor = "" then set theAuthor to theEditor
				if theShortTitle = "" then set theShortTitle to theTitle
				set theRIS to format myItem using "RIS.fmt"
				set theDate to format myItem using "LD date only.fmt"
				set theFilename to format myItem using "LD file name.fmt"
				set theReference to format myItem using "Oxford Author-Date.fmt"
				set theBookendsURL to ("bookends://sonnysoftware.com/" & theID) as text
				
				set otid to AppleScript's text item delimiters
				set AppleScript's text item delimiters to linefeed
				set thePath to text items of thePath
				set pathCount to count thePath
				if pathCount = 0 then
					set AppleScript's text item delimiters to otid
					
					tell application id "DNtp"
						set theDatabase to open database "/Users/lyndon/DevonThink/Research.dtBase2"
						set theLocation to create location "/Library"
						set theName to (theFilename & ".md") as text
						set thePlaceholders to {|%reference%|:theReference, |%citation%|:theKey, |%doi%|:theDOI, |%url%|:theURL, |%bookends%|:theBookendsURL, |%abstract%|:theAbstract}
						set theRecord to import theTemplateFile placeholders thePlaceholders to theLocation
						set the name of theRecord to theFilename
						set URL of theRecord to theBookendsURL
						set aliases of theRecord to theKey
						tell theRecord
							add custom meta data theRIS for "ris" to it
							add custom meta data theDOI for "doi" to it
							add custom meta data theAbstract for "abstract" to it
						end tell
						set tags of theRecord to theKeywords
						set theUUID to uuid of theRecord
						set theLink to reference URL of theRecord
					end tell
				end if
				if pathCount > 0 then
					repeat with i in thePath
						set thisPath to i as string
						tell application "Finder" to set theName to name of (POSIX file thisPath as alias)
						set AppleScript's text item delimiters to otid
						
						tell application id "DNtp"
							set theDatabase to open database "/Users/lyndon/DevonThink/Research.dtBase2"
							set theLocation to create location "/Library"
							set theRecord to indicate thisPath to theLocation
							set URL of theRecord to ("bookends://sonnysoftware.com/" & theID) as text
							set aliases of theRecord to theKey
							tell theRecord
								add custom meta data theRIS for "ris" to it
								add custom meta data theDOI for "doi" to it
								add custom meta data theAbstract for "abstract" to it
							end tell
							set tags of theRecord to theKeywords
							set theUUID to uuid of theRecord
							set theLink to reference URL of theRecord
						end tell
						
					end repeat
				end if
				
				set user15 of myItem to theUUID
				set user20 of myItem to theLink
				
			on error errorMessage
				
			end try
		end repeat
	end tell
end tell

It’s not easy to dissect and troubleshoot other peoples’ code, especially lengthy ones and ones with active try blocks that usually hide errors. At first glance I would suggest…

    tell application "Bookends"
	tell front library window
		set theTemplateFile to "/Users/lyndon/Library/Application Support/DEVONthink 3/Templates.noindex/Education/Reference LD.md"
		set theIDs to get id of publication items of group item "Red"
repeat with theID in theIDs
set recordExists to my dtCheck(theID)
if recordExists = {} then -- no record was found
    try
         set myRefs to (publication items whose id is theID)
   ---  … All the rest of the code to the "end try"
end try
end if
end repeat
end tell
end tell -- Yes this is the last one.

on dtCheck(id)
tell application id "DNtp"
return (lookup records with URL ("bookends://sonnysoftware.com/" & id as string)
end dtCheck

Essentially, you are checking the id of the selected items in Bookends and looping through them. I merely added a simple handler at the end of the script and am calling that in the beginning of the repeat loop. Do note you will have to close the if statement as noted in my snippet.

I actually started work on something similar the other day. This script assumes the items have already been indexed and then does something very similar to what @BLUEFROG suggests in order to avoid duplicates. Rather than lookup records with url, however, I’m using lookup records with path because some of my Bookends records contain multiple attachments (e.g. multivolume works or records with both an epub and pdf copy and so on). If the lookup returns a record and the reference URL of that record is not already among the DEVONthink item links in the Bookends database, then it creates the link between the two databases. If the link is already in the DEVONthink database, then the script does precisely what you want: it checks the modification dates of both items. If the Bookends reference has been modified more recently than the DEVONthink reference, then it updates the DEVONthink reference.

As a caution, if you’re looking to update custom meta data, it looks as if you should do so via a command like this: set custom meta data of theRecord to {bibtex:theBib, abstract:theAbstract}. The add custom meta data command will add meta data if none already exists in the specified field, but doesn’t seem to update that meta data if there is already content in the field.

I’ll post the script here, but I haven’t finished or thoroughly tested it–and it will obviously need to be updated to reflect your own specifications. You’ll notice that there are a bunch of dialogues at the beginning as well. This is an attempt to make a single script work for any number of selected items, single libraries, and multiple libraries. You could delete most of this and just specify the front library window.

When I do have a finished version, I’ll be sure to post it.

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

set theNotification to "Here is a list of discrepancies (by citekey of record) between your Bookends and DEVONthink libraries:" & linefeed & linefeed
set theLibraries to {}

tell application id "DNtp"
	set theDatabase to open database "/Users/zhope/DTPO/Research.dtBase2"
end tell

tell application "Bookends"
	set theWindows to library windows
	set winCount to count of theWindows
	set winNames to name of library windows
	tell front library window
		set sel to selected publication items
		set selCount to count of sel
	end tell
	if winCount is 1 and selCount > 0 then
		set toProcess to display dialog "You have " & selCount & " items selected in 1 open library window. Would you like to process the selected items or the entire library?" buttons {"Cancel", "Selection", "Library"} default button 3
	else if winCount > 1 and selCount > 0 then
		set toProcess to display dialog "You have " & selCount & " items selected in the front library and " & winCount & " libraries open. Would you like to process the selected items, a selected library, or all open libraries?" buttons {"Selection", "Select Library", "All Open Libraries"} default button 3
		if button returned of toProcess is "Select Library" then set theLibraries to choose from list winNames with prompt "You have the following libraries open. Select which open libraries you'd like to process or choose to process all open libraries." with multiple selections allowed
	else if winCount is 1 and selCount is 0 then
		set toProcess to display dialog "You have one library window open. Would you like to process this entire library?" buttons {"Cancel", "Library"} default button 3
	else if winCount > 1 and selCount is 0 then
		set theLibraries to choose from list winNames with prompt "You have the following libraries open. Select which open libraries you'd like to process." with multiple selections allowed
	end if
	if theLibraries is not {} then
		set theWindows to {}
		repeat with i in theLibraries
			set selWin to (library windows whose name is i)
			copy item 1 of selWin to end of theWindows
		end repeat
	end if
	if button returned of toProcess is "Selection" then set theWindows to front library window as list
	repeat with theWindow in theWindows
		tell theWindow
			if the button returned of toProcess is "Selection" then
				set theRefs to selected publication items
			else
				set theRefs to publication items of group attachments
			end if
			repeat with theRef in theRefs
				set {theKey, thePaths, dtLinks, modDate} to {citekey, path of attachment items, user20, date modified} of theRef
				if thePaths is not {} then
					set dtLinks to paragraphs of dtLinks
					set theCount to 1
					repeat with thePath in thePaths
						tell application id "DNtp" to set theRecord to lookup records with path thePath in theDatabase
						if theRecord is not missing value and (count of theRecord) is 1 then
							set theRecord to item 1 of theRecord
							tell application id "DNtp" to set {theUID, dtMod} to {reference URL, modification date} of item 1 of theRecord
							if theUID is not in dtLinks then
								if theCount is 1 then
									set {thePath, theAuthor, theEditor, theTitle, dtpoLink, theID, theAbstract, theLabel} to {path of attachment items, authors, editors, title, user20, id, abstract, label color} of theRef
									if dtpoLink is not "" then set dtpoLink to dtpoLink & linefeed
									if theAuthor = "" then set theAuthor to theEditor
									set theBib to format theRef using "BibTeX.fmt"
								end if
								tell application id "DNtp"
									set URL of theRecord to ("bookends://sonnysoftware.com/" & theID) as text
									set custom meta data of theRecord to {citekey:theKey, bibtex:theBib, abstract:theAbstract}
									set aliases of theRecord to theKey
									if theLabel is 0 then set label of theRecord to 0
									if theLabel is 1 or theLabel is 4 or theLabel is 5 then set label of theRecord to 1
									if theLabel is 2 then set label of the theRecord to 2
									if theLabel is 6 or theLabel is 7 then set label of theRecord to 5
									if theLabel is 9 then set label of theRecord to 6
									set theLink to reference URL of theRecord
									set modification date of theRecord to ((current date) + (1 * minutes))
									if theCount is greater than 1 then
										set dtpoLink to dtpoLink & linefeed & theLink
									else
										set dtpoLink to dtpoLink & theLink
									end if
								end tell
								set user20 of theRef to dtpoLink
							else if modDate > dtMod then
								set theBib to format theRef using "BibTeX.fmt"
								set {theLabel, theAbstract} to {label color, abstract} of theRef
								tell application id "DNtp"
									if theKey is (aliases of theRecord) then
										set custom meta data of theRecord to {bibtex:theBib, abstract:theAbstract}
										if theLabel is 0 then set label of theRecord to 0
										if theLabel is 1 or theLabel is 4 or theLabel is 5 then set label of theRecord to 1
										if theLabel is 2 then set label of the theRecord to 2
										if theLabel is 6 or theLabel is 7 then set label of theRecord to 5
										if theLabel is 9 then set label of theRecord to 6
										set modification date of theRecord to current date
									else
										set theNotification to theNotification & theKey & " : Keys do not match"
									end if
								end tell
							end if
							set theCount to theCount + 1
						else if (count of theRecord) is greater than 1 then
							set theNotification to theNotification & theKey & " : More than one record with this path is in the database"
						else if theRecord is missing value then
							set theNotification to theNotification & theKey & " : No record in database"
						end if
					end repeat
				end if
			end repeat
		end tell
	end repeat
end tell

if (count of paragraphs of theNotification) is greater than 3 then
	display dialog theNotification
	set the clipboard to theNotification
end if