Video Playback and Annotations

No worries and yes, UI scripting us fragile and not advocated by us in almost every case.

  • Playback speed can’t be changed in DEVONthink.

  • If you look at the AppleScript dictionary for DEVONthink, you’ll see a record has an annotation file property.

I’m in bed with a bad head cold so I’ll try to look at this tomorrow. (Yeah and still working. Bad work/life balance. I know. I know. :roll_eyes: :wink: )

Thanks, that was the key attribute I missed earlier, that’s really helpful!

Get feeling better!

You’re welcome and thanks :face_with_head_bandage::sleeping:

If you’re willing, I’d love to hear about the solution you landed on while researching this. I, too, just discovered the back link and would love to take better advantage of it when annotating files.

1 Like

Absolutely, I’ll post whatever I end up with.

1 Like

So I haven’t yet worked on the annotation component or even playback in DEVONthink (I will though) but I did discover that VLC has AppleScript support and includes a current time properly (to the second). So I created a script that will take the current time from VLC, get the Reference URL for the current file in DEVONthink and copy a markdown link with it and the ?time parameter to the clipboard.

I’ve got a bunch more I want to do, adapt a rich text version. Do a pair of matching scripts for QuickTime (and see if any other players have the necessary AppleScript options). I’d also like to do a more complete annotations workflow and probably adapt them for Keyboard Maestro as well but I thought you might like to see or be able to adapt what I have now.

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

# # Copy Markdown Link for DEVONthink Video Open in VLC at Current Time
# Since DEVONthink's video player is a bit limited (it doesn't support multiple playback speeds for instance) I wanted to be able to use a more flexible player but link back to the video file in DEVONthink at the current timecode. 
# One minor issue, VLC only lets you get the current time to the second, not the exact frame or decimal.
# This script will copy that URL as a markdown link in the format: [13:10:72](x-devonthink-item://45151EC1-8A15-4109-AFE9-1B4EA8C6DE2B?time=352)
# Created by [Christin White](

# # Set Options
# - `DatabaseName`     *text*: The name for the database to use, if a valid database is not found the current database will be used.
# - `PausePlayback` *boolean*: If true, pause VLC when script is run.
set DatabaseName to "Collection"
set PausePlayback to true

	# Get Reference URL from DEVONthink and Assemble Link
	set TheVideo to GetVideoFromVLC(PausePlayback)
	# Get Reference URL
	set TheReferenceURL to GetReferenceURLFromPath(TheVideoPath of TheVideo, DatabaseName)
	# Assemble Link
	set TimeCodeURL to TheReferenceURL & "?time=" & (TheCurrentTime of TheVideo)
	set TimeCode to FormatTimeCode(TheCurrentTime of TheVideo)
	set MarkdownLink to "[" & TimeCode & "](" & TimeCodeURL & ")"
	# Copy to Clipboard
	set the clipboard to MarkdownLink
on error ErrorMessage number ErrorNumber
	if the ErrorNumber is not -128 then display alert "DEVONthink" message ErrorMessage as warning
end try

# # Functions
# These will be moved to a script library.

# # AddLeadingZeros
# Add leading zeros to a number to reach the specified number of digits. Based on [Apple's handler](
# Parameters:
# - `TheNumber`         *number* : The number to add leading zeros to.
# - `TheNumberOfDigits` *integer*: The total number of digits.
# Returns:
# - *text*: TheNumber with the specified number of leading zeros.
on AddLeadingZeros(TheNumber, TheNumberOfDigits)
	set IsNegative to TheNumber is less than 0
	# Determine the maximum number from the number of digits.
	set TheNumberOfDigits to (TheNumberOfDigits - 1)
	set TheThreshold to (10 ^ TheNumberOfDigits) as integer
	if TheNumber is less than TheThreshold then
		# If the number is negative, convert it to positive
		if IsNegative = true then set TheNumber to -TheNumber
		set TheLeadingZeros to ""
		set TheDigitCount to length of ((TheNumber div 1) as text)
		set TheCharacterCount to (TheNumberOfDigits + 1) - TheDigitCount
		repeat TheCharacterCount times
			set TheLeadingZeros to (TheLeadingZeros & "0") as text
		end repeat
		# Make the number negative, if it was previously negative
		if IsNegative = true then set TheLeadingZeros to "-" & TheLeadingZeros
		return (TheLeadingZeros & (TheNumber as text)) as text
		# If the number is greater than or equal to the maximum number of digits
		# Return the original number
		return TheNumber as text
	end if
end AddLeadingZeros

# # FormatTimeCode
# Converts seconds into a timecode format matching `00:00:00`
# Parameters:
# - InSeconds *number*: Number of seconds to set timecode to. Accepts float but is converted to Integer
# Returns:
# - *text*: Timecode
on FormatTimeCode(InSeconds)
	set InSeconds to InSeconds as integer
	# Calculate Hours
	set CalculatedHours to (InSeconds div hours)
	# Calculate Minutes
	set RemainderSeconds to (InSeconds mod hours)
	set CalculatedMinutes to (RemainderSeconds div minutes)
	# Calculate Seconds
	set CalculatedSeconds to (RemainderSeconds mod minutes)
	# Convert to Two Digit Strings
	set CalculatedHours to AddLeadingZeros(CalculatedHours, 2)
	set CalculatedMinutes to AddLeadingZeros(CalculatedMinutes, 2)
	set CalculatedSeconds to AddLeadingZeros(CalculatedSeconds, 2)
	set TimeCode to CalculatedHours & ":" & CalculatedMinutes & ":" & CalculatedSeconds as text
	return TimeCode
end FormatTimeCode

# # GetReferenceURLFromPath
# Get the reference URL for a file in a specified or active database.
# Parameters:
# - `TheFilepath`  *text*: The path to the file.
# - `DatabaseName` *text*: That database to look in.
# Returns:
# - *text*: The Reference URL.
on GetReferenceURLFromPath(TheFilePath, DatabaseName)
	tell application id "DNtp"
		# Check if the database is valid, if not use the active database.
		if exists database DatabaseName then
			set TheRecordList to lookup records with path TheFilePath in database DatabaseName
			display notification "The specified database wasn't found, trying the active database instead" subtitle "Invalid Database"
			set TheRecordList to lookup records with path TheFilePath
		end if
		# Make sure we found a matching record.
		if (count of TheRecordList) is greater than 0 then
			# A list could return more than one record such as if a file is indexed to more than one group. Use the first item but notify the user.
			if (count of TheRecordList) is greater than 1 then
				display notification "More than one record was found, using the first record" subtitle "Multiple Records"
			end if
			set TheReferenceURL to reference URL of item 1 of TheRecordList
			return TheReferenceURL
			error "File not found in database."
		end if
	end tell
end GetReferenceURLFromPath

# # GetVideoFromVLC
# Gets metadata about the current video open in VLC
# Parameters:
# - `PausePlayback` *boolean*: Pause video when called.
# Returns:
# - *list* (indexed)
#   - `TheVideoPath`     *text*: The path for current video.
#   - `TheCurrentTime` *number*: The current second of playback.
on GetVideoFromVLC(PausePlayback)
	tell application "VLC"
		if (exists path of current item) then
			set TheVideoPath to path of current item
			set TheCurrentTime to current time
			error "VLC does not have an open video."
		end if
		if PausePlayback and playing then
		end if
		return {TheVideoPath:TheVideoPath, TheCurrentTime:TheCurrentTime}
	end tell
end GetVideoFromVLC

(My website is in-progress and not actually there yet. I’m also a monster who Pascal cases, soft-wraps and prefers tabs.)

Edit: Converted VLC related functions to a handler. Fixed hard-coded test value.

1 Like

One other note I forgot to mention, DEVONthink will use the format [00\:00] for markdown annotations, that works but seems unnecessary, in testing [00:00] worked fine in DEVONthink, Obsidian, iA Writer and Drafts while [00\:00] correctly interpreted the backslash as an escape character in DEVONthink, Obsidian and Drafts but iA Writer doesn’t.

1 Like

Wow, this is amazing work! Unfortunately, I’m not much of a developer, but I love the direction of what you’re doing. I am curious though, why would you prefer this method over DEVONthink’s built-in “Insert Back Link” option in the given file’s Annotations section in the Inspector?
CleanShot 2021-03-03 at 15.43.22

1 Like

A couple of reasons:

  1. As I mentioned, I wanted to use an external player with more options such as different playback speeds.
  2. The annotation panel in the sidebar is really small amd it’s finicky making sure it’s going to be in the format I want (Markdown, not rich text). Right now I haven’t fully implemented a create annotations file (if necessary and insert back link yet but it’s on the list.
  3. This lets me use other apps to take notes such as Obsidian for Markdown notes or if it’s a really complex video or set of videos I may create my annotations in MindNode instead and then export the outline back to Markdown. This depends on getting a rich text link though for MindNode or other apps that accept RTF instead of Markdown, that’s what I’m working on now.
  4. Perhaps most importantly, selecting the options in that right click annotations menu and working with the annotations panel programmatically isn’t possible without using UI scripting which is inadvisable because all kinds of slightly different change to the app stare and updates can brake it, it’s very fragile.
1 Like

Not sure I understood what you want to do but maybe the handler in this post could be useful. It copies a Markdown and a RTF link. Depending on the receiving app’s preferred clipboard type one of both will be inserted.

Awesome, thanks for all that info! I’m excited to see what you end up with for all of these needs. Thanks for sharing.

That’s a really interesting solution, I’d been playing with textutil as the solution but wasn’t sure how to get it where I was going. I love the idea of having both formats on the clipboard though, that’s perfect!

The only thing I was really hoping to figure out is a way to copy the rich text with the link set correctly but with no font/styling so that it just matches the format of whatever I paste it into. Something along the lines of this.

I’m not sure whether it’s even possible to have rich text without styling though and if it is I’m not sure it’s something I could use textutil to generate.

I’m wondering if it might be possible using the other technique mentioned in the thread your handler linked to using ASObjC, do you have any thoughts on that?

No idea whether that’s possible but I don’t think that one can create an attributed string without style attributes. If we don’t set them there’s probably a default used.

This script copies both types, RTF and Markdown, however I just realized that it doesn’t work with e.g. DEVONthink’s “Paste and match style” menu. No idea why, might be that it’s necessary to also add another type to the clipboard.

Here’s what happens, first line using “Paste”, second “Paste and match style”

But I overread your hint with Shane’s HTML technique, I’ll try that later.

-- Copy RTF and Markdown link

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

tell application id "DNtp"
		set theRecords to selected records
		if theRecords = {} or (count theRecords) > 1 then error "Please select one record"
		set theRecord to item 1 of theRecords
		set theName to name of theRecord
		set theURL to reference URL of theRecord
	on error error_message number error_number
		if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
	end try
end tell

set theTypeArray to current application's NSMutableArray's arrayWithArray:{}
set theDataArray to current application's NSMutableArray's arrayWithArray:{}

set theAttributtedString to current application's NSMutableAttributedString's alloc()'s initWithString:((current application's NSString's stringWithString:theName))
theAttributtedString's addAttribute:(current application's NSLinkAttributeName) value:theURL range:(current application's NSMakeRange(0, theAttributtedString's |length|()))
set theType to current application's NSPasteboardTypeRTF
set theData to theAttributtedString's pasteboardPropertyListForType:theType
theTypeArray's addObject:theType
theDataArray's addObject:theData

set theMarkdownString to (current application's NSString's stringWithFormat_("[%@](%@)", theName, theURL))
set theType to current application's NSPasteboardTypeString
set theData to theMarkdownString's dataUsingEncoding:(current application's NSUTF8StringEncoding)
theTypeArray's addObject:theType
theDataArray's addObject:theData

set thePasteboard to current application's NSPasteboard's generalPasteboard()
thePasteboard's clearContents()

repeat with i from 0 to (theDataArray's |count|()) - 1
	(thePasteboard's setData:(theDataArray's objectAtIndex:i) forType:(theTypeArray's objectAtIndex:i))
end repeat

I’m not sure about that either, I actually just started a new thread over at Late Night Software’s forum to see if anyone else may have more insight into this, there seem to be a number of ways to get a rich text link but all of them include the styling, my guess is that I’m chasing something that’s not valid RTF but it sure would be nice to paste a link into a Rich Text field and not have to manually reset the styling or paste it as plaintext and add the link myself.

I’m not too concerned about getting it to paste correctly into DEVONthink itself, just about anything I put in DEVONthink is going to be Markdown, really, MindNode is my personal goal here, it’s smart enough to reformat something pasted in as a node but not in the node’s notes.

I’ll take a look at your script there in a bit.

I now read your thread, you could try to get data from your RTF string with another handler from Shane which I posted here some days ago.

Ahmm, the same happens with a original DEVONthink link: “Paste” gives a properly named RTF link, “Paste and match style” gives a strange hybrid. Same in Tinderbox, so what you want won’t work, I’m afraid.

Hmm, interesting, that does take my string and encode it as rdat data okay but I’m not sure how to modify it to encode it as RTF.

Switching it to RTF this way

**set** theCode **to** *current application's* NSHFSTypeCodeFromFileType("'RTF'")

Doesn’t do it (rtf also fails). I’m definitely not understanding what’s actually going on with this stuff, it seems like I need to spend some time learning Apple’s Mac APIs and AppleScriptObjC — just a slightly larger scope than I was hoping to need for this project :joy:.

Thank you for all your help and advise, I really appreciate it!

1 Like

“Some time” :rofl: AppleScriptObjC is the greatest rabbit hole ever and there’s almost no documentation. You’ll find answers for Objective-C but once you think you’ve figured how something works BOOOM!, sorry, you can’t do that in AppleScriptObjC … If you get Shane’s book start with chapter 29. “Out of Bounds” - and keep it close

1 Like

I like Swift, I don’t know jack about Cocoa and I have no desire to learn Objective-C so yeah, I’m sure this is a huge rabbit hole. You’re really selling me on it too. :joy:

It seems like what Shane was saying was most of these datatypes were actually rdat if you check them but checking AEPrint appears to show RTF as RTF so I don’t know. :woman_shrugging:t3:

Shane’s idea to just set the formatting to match the destination is a good idea, even if it’s a lot less flexible.

1 Like

Hey @ChristinWhite, have you made any further developments on your script? I just came across this topic and it seems super helpful for my note taking workflow. Would be great if it worked on audio records ie. podcasts and lectures as well.

I’ve added the AppleScript you shared to the Scripts folder in DEVONthink, but whenever I try to run it, it gives me the error message “File not found in database.” I’m not sure what to do.

EDIT: Aha! I figured out it only works with media that’s already been imported into DEVONthink. duh… i’m new to all this. TY!!

I’m using it with Typora for taking markdown notes and BetterTouchTool to launch the script without leaving my note taking app. (The script is saved in DEVONthink’s default script location with the file name “Copy Markdown Link for DEVONthink Video Open in VLC at Current Time___Cmd-Alt-Shift-T”, so it’s triggered by the keyboard shortcut Command+Option+Shift+T. Within BTT, I configured Typora to send the keyboard shortcut Command+Option+Shift+T to DEVONthink, and the shortcut Command+Option+Shift+Space sends VLC the Space key to play/pause playback.

Still wondering if you’ve made any further improvements. But for my purposes it’s working great right now!