Problem with getting reference URL or UUID when adding email

I’ve been trying to create scripts to automate my email processing with OmniFocus and DEVONThink. The first thing the script does is creates a record in DT with the email selected in Mail. I then try to get the reference URL that I can add to OmniFocus to link the two. I’ve also tried getting the UUID. The problem I have is that if I run the script once, the reference URL and UUID I get back don’t work. They both appear to be a portion of the email’s URL. If I run the script a second time on the same email, the reference URL and UUID are correct.

Here’s the really fun part: I tried running the “Add message(s) to DEVONThink” script that comes with DT and I get the same results. So either there is a bug or there is something in my configuration that is confusing things.

I’m running DEVONThink Pro Office 2.9.16 on macOS High Sierra 10.13.2.

Any suggestions?

Please post your code. Thanks

It’s not specific to my code. It’s happening with the DEVONThink-provided script to add messages from Apple Mail.

Thanks.

This script doesn’t do anything with the UUID or reference URL. It just imports the message into the database.

Sorry, I should clarify. I realize the script doesn’t do anything with the UUID or reference URL. When I use the script to add the email to DT, the record created by the script has a UUID that appears to be the message ID from the email (e.g., %3CC2.4A.16090.FA1BE3A5@momentum1-mta1%3E) rather than the format of UUID I see for DT records created through other means (e.g., AB8AA57A-4215-4A4C-B19F-22F501AEF8C0). If I run the script a second time on the same email, the UUID does look like the second example, which is what I would expect.

This appears to be causing problems for me when I try to use the UUID to retrieve the record from DT.

Thanks.

(There’s a reason my avatar says newbie. :smiley: )

I think I understand this better now. It looks like DT is using the message ID as the UUID when a mail message is imported by any means. Since the message ID should be unique, I can see how this would be okay. (I’m not sure why it’s needed vs the other format of UUID, but since I’ve figured out what I was doing wrong, it doesn’t matter.)

It also appears that the second time I imported the email, DT assigned a UUID in the format I expected because there already was a record with the message ID as the UUID.

My problem was that I was trying to extract the UUID from the reference URL. In the case where the UUID is the message ID, the UUID begins with < and ends with >. In the reference url, these characters are encoded and should be decoded before trying to lookup the record by UUID. Without realizing this, I thought the problem was the UUID. Once I figured that out that I needed to decode, it worked.

Sorry for wasting your time on this. Now, I can go back to building my cool automation with OmniFocus. Once I get it working - assuming I can do what I hope - I’ll come back to share.

Happy holidays, everyone!

URL, Reference URL, and UUID are different elements in the DEVONthink dictionary. URL is the URL of the source of the record (e.g., the web page or the Mail message URL). UUID is the UUID of the record added to the database (either imported or indexed or created internally). Reference URL is the UUID plus the “x-devonthink-item://” prefix. If you use the standard process for creating or importing or indexing records (e.g., the creation case: “set myrecord to create record … in location”) then your script can grab URL, UUID and Reference URL from “myrecord”.

UUID is not stored in the database with “< >” brackets – if UUID in your script has those brackets then something else is creating them.

You do NOT need to extract UUID from Reference URL – UUID is already separately stored.

Thanks, korm. Through trial and error, I think I’ve got a handle on these concepts now.

FYI. I’m extracting the UUID from the reference URL because I use it with OmniFocus. When I process email, I run a script that imports the email to DT and creates an OF task with the reference URL in the notes. This lets me use the link to get to the DT item from OF regardless of platform (i.e., macOS, iOS).

I’m in the process of writing a script to ensure my OF and DT organization is in sync. This script looks at OF tasks and, if a DT reference URL is found in the OF task’s notes, it extracts the UUID so I can get the DT record, check if the group it is in matches the OF project that the task is in. If it’s not, it moves the DT record. I need to extract the UUID from the reference URL because I don’t believe I can get a DT record using the reference URL directly.

Regarding the “< >” brackets, I’m seeing different results. If I import an email (doesn’t matter how - drag from Mail to DT, use the DT-provide script, etc.), the UUID I retrieve from DT for the item contains the brackets. For example, I dragged an email to DT and it created a record in the global inbox. I selected that record and ran this code:

tell application id "DNtp"
	set thisSelection to the selection
	set thisItem to first item of thisSelection
	set theUUID to uuid of thisItem
	set theRefURL to reference URL of thisItem
	get record with uuid theUUID
end tell

It produced this output:

tell application “DEVONthink Pro”
get selection
–> {content id 330410 of database id 1}
get uuid of content id 330410 of database id 1
–> “16C7D57C-D670-46B9-AA4C-7112F3A23FAF@PMUSAPP09.parkmobileglobal.com
get reference URL of content id 330410 of database id 1
–> “x-devonthink-item://%3C16C7D57C-D670-46B9-AA4C-7112F3A23FAF@PMUSAPP09.parkmobileglobal.com%3E”
get record with uuid “16C7D57C-D670-46B9-AA4C-7112F3A23FAF@PMUSAPP09.parkmobileglobal.com
–> content id 330410 of database id 1
end tell
Result:
content id 330410 of database id 1 of application “DEVONthink Pro”

The UUID is returned with the brackets and the reference URL contains the UUID with the brackets encoded. When my script tries to get the record, it needs to include the brackets (decoded if extracted from the reference URL) otherwise DT won’t find the record.

(Note: this only happens with email. Other UUIDs do not have the brackets.)

OK, yes about DEVONthink data content of UUID, Reference URL in eml.

But you still do not need to extract the UUID from Reference URL. Reference URL is the same data / same structure as your script gets, that you get when you use Copy Item Link for the selected item in a database. Over here if I paste Reference URL or the link (same as Reference URL) put on the clipboard via Copy Item Link, and paste that into the note of an OF action, I get a working link that opens that eml in DEVONthink when clicked.

Your script is doing unneeded work, IMO. But, that’s not a criticism!! We do what we do the best we can do.

I realize the reference URL will let me open the email directly. I want that functionality for when I want to look at the DT record from the OF task. However, I also want to make changes to the DT item via AppleScript without opening the DT record. Maybe I’m still misunderstanding, but I thought if I want to manipulate a record programmatically and not display it, I have to use the UUID to get the record (or the ID or location, but in my case, the UUID is easiest.)

I’m not at my Mac right now, but when I get a chance, I’ll upload my code. It will make it easier to show what I’m trying to do.

BTW, I’m not the least concerned about having my approach critiqued. I’ve read enough of the boards here to see how tremendously helpful everyone is. This is a great way to learn.

Thanks!

I’m back! :slight_smile:

Before I jump in with my code, I want to be clear that DEVONthink is working wonderfully for me. I don’t know what took me so long to figure out what a great tool it is for my needs. My script is doing what I want, although there is always room for improvement. I am posting this to get feedback on whether I’m missing anything, overcomplicating things, etc.

Here’s my use case:

  • I use a productivity method that is sort of a mashup between Agile Results and GTD.
  • My tasks are grouped by focus areas such as family, personal, career, etc.
  • I want to process all inboxes (email, paper, mind sweep, etc.) into OmniFocus with supporting material stored in DEVONthink.
  • I want to be able to collect and process on my iPhone and iPad if necessary
  • I’ve created automation that gets everything into OmniFocus and DEVONthink, but sometimes, I can’t assign to focus areas (folder) or projects when I do the processing step (usually this is because of limitations on iOS, sometimes I just end up moving things around later)
  • The AppleScript I’ve created (code is below) is designed to be run after I process my OmniFocus inbox on my Mac. It finds all OmniFocus tasks with links to support materials in DEVONthink. It then checks if the task and support materials are in the appropriate folders and projects in both applications. If materials in DEVONthink aren’t in the right place, the script moves them.
  • With the ability to keep every bit of supporting material in DEVONthink (I just love this!) and the consistent organization, I never lose track of anything.
  • Eventually, I plan to add features to archive DEVONthink materials when the OmniFocus task is completed and probably other similar clean-up tasks as I think of them

With that, lengthy preamble. Here is my code. It works and DEVONthink works as expected. I also work as expected since I have a better idea of what I’m doing. :laughing:

I welcome any feedback on how to make it better.

(Mods: If this post is too off-topic from my original post, feel free to move it.)

property dtSupportDB : "~/DEVONthink/Support.dtBase2"
property white_space : {space, tab, return, (ASCII character 10), (ASCII character 13)}
property taskRecord : {ofFolderName:"", ofProjectName:"", ofTaskName:"", dtUUID:""}
property taskList : {}

on run
	-- Get all OmniFocus tasks that have links to DEVONthink in the notes
	set theOFTaskList to GetOFDTLinkedTasks()
	-- Get count of OmniFocus tasks to loop through them all
	set theOFTaskCount to count of theOFTaskList
	-- If no OmniFocus tasks are found, exit
	if theOFTaskCount is 0 then
		return
	end if
	-- Loop through the OmniFocus tasks to check if associate DEVONthink record is
	-- using the same folder and project hierarchy used in OmniFocus
	repeat with theOFTask in theOFTaskList
		-- Get the corresponding record from DEVONthink using the UUID
		tell application id "DNtp"
			-- Get the DEVONthink record referenced in the OmniFocus task
			set dtRecord to get record with uuid dtUUID of theOFTask in dtSupportDB
			-- If the DEVONthink record is missing, alert (will create logic to clean up OmniFocus later)
			if dtRecord is missing value then
				display dialog "DEVONthink item not found." & return & return & "OmniFocus Info" & return & return & "Project: " & ofProjectName of theOFTask & return & "Task: " & ofTaskName of theOFTask
			else
				-- Get the DEVONthink record name
				set dtRecordName to name of dtRecord
				-- Get the name of the group (aka project) the record is currently in
				set dtProject to parent 1 of dtRecord
				set dtProjectName to name of dtProject
				-- If the DEVONthink record is in the global inbox, set the dtFolderName manually, otherwise, grab 
				-- the folder name from the record
				-- (TBD - figure out why I can't handle this programmatically)
				if dtProjectName is "Inbox" then
					set dtFolderName to "Global Inbox"
				else
					-- Get the name of the top level group the record is in
					set dtFolder to parent 1 of dtProject
					set dtFolderName to name of dtFolder
				end if
				-- Get the name of the database the record is in
				set dtDB to database of dtRecord
				set dtDBName to name of dtDB
				-- Check if the DEVONthink record is in the same folder and project that the OmniFocus task is in
				-- and if it is in the Support database. If any of these conditions are not true, the DEVONthink
				-- record needs to be moved
				if (dtProjectName is not equal to ofProjectName of theOFTask) or (dtFolderName is not equal to ofFolderName of theOFTask) or (dtDBName is not equal to "Support") then
					-- Get the ID of the destination group
					set destProject to get record at dtSupportDB & "/" & ofFolderName of theOFTask & "/" & ofProjectName of theOFTask
					-- Create the group if it doesn't exist already
					if destProject is missing value then
						set theDB to open database dtSupportDB
						set destProject to create location ofFolderName of theOFTask & "/" & ofProjectName of theOFTask in theDB
					end if
					-- If record exists in same database, move with from clause
					if dtDBName is equal to "Support" then
						move record dtRecord to destProject from dtProject
					else
						move record dtRecord to destProject
					end if
				end if
			end if
		end tell
	end repeat
end run

----------------------------------------------------------------------
-- Handler to extract DEVONthink UUID from reference URL
----------------------------------------------------------------------
on GetDTUUID(ofTaskNote)
	set oldTIDs to AppleScript's text item delimiters
	set AppleScript's text item delimiters to "://"
	set ofNoteParsed to (every text item in ofTaskNote) as list
	set dtUUID to item 2 of ofNoteParsed
	set AppleScript's text item delimiters to oldTIDs
	-- If the DEVONthink record is an email, decode the brackets in the UUID
	set dtUUID to ReplaceChars(dtUUID, "%3C", "<")
	set dtUUID to ReplaceChars(dtUUID, "%3E", ">")
end GetDTUUID

----------------------------------------------------------------------
-- Handler to retrieve tasks from OmniFocus that have DEVONthink link
----------------------------------------------------------------------
on GetOFDTLinkedTasks()
	tell application "OmniFocus"
		tell default document
			set allTasks to flattened tasks
			repeat with theTask in allTasks
				set theFolderName to the name of the folder of the containing project of theTask
				set theProjectName to the name of the containing project of theTask
				set theTaskName to the name of theTask
				set theNote to the note of theTask
				set theNote to my trim_string(theNote, white_space, "right")
				if theNote starts with "x-devonthink-item" then
					-- Extract the DEVONthink UUID from the OmniFocus task's note
					set theUUID to my GetDTUUID(theNote)
					set taskRecord to {ofFolderName:theFolderName, ofProjectName:theProjectName, ofTaskName:theTaskName, dtUUID:theUUID}
					copy taskRecord to end of taskList
				end if
			end repeat
		end tell
	end tell
	return taskList
end GetOFDTLinkedTasks

----------------------------------------------------------------------
-- Handler to trim white space from a string
----------------------------------------------------------------------
on trim_string(the_string, trim_chars, trim_parameter)
	set start_char to 1
	set end_char to length of the_string
	set all_chars to (characters of the_string)
	
	if trim_parameter is in {"left", "both"} then
		repeat with each_char in all_chars
			if each_char is not in trim_chars then exit repeat
			set start_char to (start_char + 1)
		end repeat
	end if
	
	if trim_parameter is in {"right", "both"} then
		set all_chars to reverse of all_chars
		repeat with each_char in all_chars
			if each_char is not in trim_chars then exit repeat
			set end_char to (end_char - 1)
		end repeat
	end if
	
	try
		return text start_char thru end_char of the_string
	on error
		return ""
	end try
end trim_string

----------------------------------------------------------------------
-- Handler to replace text within a string
----------------------------------------------------------------------
on ReplaceChars(this_text, search_string, replacement_string)
	try
		set AppleScript's text item delimiters to the search_string
		set the item_list to every text item of this_text
		set AppleScript's text item delimiters to the replacement_string
		set this_text to the item_list as string
		set AppleScript's text item delimiters to ""
		return this_text
		
	on error errStr number errorNumber
		set AppleScript's text item delimiters to oldTIDs
		error "Error in ReplaceChars handler. " & errStr number errorNumber
		
	end try
end ReplaceChars
1 Like

Thanks for the full script - I’m also trying to integrate OF and DT and hadn’t considered that it is possible to automatically mirror the project hierarchy.