Video Playback and Annotations

I like Swift, I don’t know jack about Cocoa and I have no desire to learn Objective-C so yeah, I’m sure this is a huge rabbit hole. You’re really selling me on it too. :joy:

It seems like what Shane was saying was most of these datatypes were actually rdat if you check them but checking AEPrint appears to show RTF as RTF so I don’t know. :woman_shrugging:t3:

Shane’s idea to just set the formatting to match the destination is a good idea, even if it’s a lot less flexible.

1 Like

Hey @ChristinWhite, have you made any further developments on your script? I just came across this topic and it seems super helpful for my note taking workflow. Would be great if it worked on audio records ie. podcasts and lectures as well.

I’ve added the AppleScript you shared to the Scripts folder in DEVONthink, but whenever I try to run it, it gives me the error message “File not found in database.” I’m not sure what to do.

EDIT: Aha! I figured out it only works with media that’s already been imported into DEVONthink. duh… i’m new to all this. TY!!

I’m using it with Typora for taking markdown notes and BetterTouchTool to launch the script without leaving my note taking app. (The script is saved in DEVONthink’s default script location with the file name “Copy Markdown Link for DEVONthink Video Open in VLC at Current Time___Cmd-Alt-Shift-T”, so it’s triggered by the keyboard shortcut Command+Option+Shift+T. Within BTT, I configured Typora to send the keyboard shortcut Command+Option+Shift+T to DEVONthink, and the shortcut Command+Option+Shift+Space sends VLC the Space key to play/pause playback.

Still wondering if you’ve made any further improvements. But for my purposes it’s working great right now!

Seems DEVONthink could provide a basic video player with a notepad next to it and let us open both files, linked together, and go through the video marking off notes at any time code.
DEVONthink lets us take bookmark links to lines and paragraphs in text, why not include links to time codes in video?

Something like f4transkript | audiotranskription
With a nice audio scrubbing visualization, keyboard shortcuts for speed / text expansion / etc.

Video content and collaboration is becoming such a large market, it seems it would make sense to offer ways to tag video objects, audio objects, and transcriptions of the audio/video content.

I know I would really appreciate a real API to access video position and play/pause/rewind without needing to open another tool that doesn’t integrate with my notes.

I like InqScribe also. It has a simple text editor with no formatting and so they use the tab key to stop/start the video while you type notes and ctrl+; to insert clickable time codes. You can setup your own shortcuts for text expansions.

In any case, that’s something I would like to see in DEVONthink.
If your videos are private, this will give you a way to organize them with notes linked directly to timecodes. If your videos are on youtube, you can download them with youtube-dl ‘https link’ and then do the same thing…

Are there any OSX tools that generate links like these to video time-code frames?
x-devonthink-item://D5F023FB-E1AD-49BB-A2D4-63279B540531?line=40

1 Like

That’s already possible via the Annotations & Reminder inspector, e.g. you can insert back links via the popup menu.

1 Like

Thank you for the hint.

Ahh, that tiny little dropdown wasn’t something I would have noticed. Got it.
These hot keys work, so I won’t need the dropdown now.
image

So this works:
image

And I can export the RTF and see the time codes like this.
x-devonthink-item://0C673BF7-6C24-4A94-81D0-993715CA1F0B?time=685.947122

Are other time formats such as 11m26s supported?

I think I just need to understand how I can control that video player and perhaps how I can modify the font size and make that tiny annotation window large enough to type a full document? I was thinking of something more along the lines of where I can type in a real DEVONthink RTF or Markdown text note and while the video is playing hit a control key to add the time code marker.

So, can you give me another hint about how to stay in the annotation window and control the speed and pause/play/rewind of the video?
image

Are other time formats such as 11m26s supported?

Not at this time, no.

So, can you give me another hint about how to stay in the annotation window and control the speed and pause/play/rewind of the video?

There are no speed controls in the media player in DEVONthink.
Also, if the Annotation pane has focus, you can’t control the playback.

1 Like

The handler from Script: Copy RTF and Markdown link does that. Your script plus the handler seems to work fine over here

-- Copy RTF and Markdown link for VLC.app's video's current time and the corresponding DEVONthink record's reference URL 

-- https://discourse.devontechnologies.com/t/video-playback-and-annotations/62271/8
-- https://discourse.devontechnologies.com/t/script-copy-rtf-and-markdown-link/68151

use AppleScript version "2.4"
use framework "Foundation"
use framework "AppKit"
use scripting additions

property revealRecord : true
property includeSuffix : true

# # Copy Markdown Link for DEVONthink Video Open in VLC at Current Time
#
# Since DEVONthink's video player is a bit limited (it doesn't support multiple playback speeds for instance) I wanted to be able to use a more flexible player but link back to the video file in DEVONthink at the current timecode. 
#
# One minor issue, VLC only lets you get the current time to the second, not the exact frame or decimal.
# 
# This script will copy that URL as a markdown link in the format: [13:10:72](x-devonthink-item://45151EC1-8A15-4109-AFE9-1B4EA8C6DE2B?time=352)
#
# Created by [Christin White](http://christindwhite.com)
#

# # Set Options
# 
# - `DatabaseName`     *text*: The name for the database to use, if a valid database is not found the current database will be used.
# - `PausePlayback` *boolean*: If true, pause VLC when script is run.
#
set DatabaseName to "Collection"
set PausePlayback to true

try
	# Get Reference URL from DEVONthink and Assemble Link
	set TheVideo to GetVideoFromVLC(PausePlayback)
	
	# Get Reference URL
	set TheReferenceURL to GetReferenceURLFromPath(TheVideoPath of TheVideo, DatabaseName)
	
	# Assemble Link
	set TimeCodeURL to TheReferenceURL & "?time=" & (TheCurrentTime of TheVideo)
	set TimeCode to FormatTimeCode(TheCurrentTime of TheVideo)
	#set MarkdownLink to "[" & TimeCode & "](" & TimeCodeURL & ")"
	
	# Copy to Clipboard
	#set the clipboard to MarkdownLink
	my copyRTFandMarkdownLink({TimeCodeURL}, {TimeCode})
	
on error ErrorMessage number ErrorNumber
	if the ErrorNumber is not -128 then display alert "DEVONthink" message ErrorMessage as warning
end try


# # Functions
#
# These will be moved to a script library.
#


# # AddLeadingZeros
#
# Add leading zeros to a number to reach the specified number of digits. Based on [Apple's handler](https://developer.apple.com/library/archive/documentation/LanguagesUtilities/Conceptual/MacAutomationScriptingGuide/ManipulateNumbers.html).
#
# Parameters:
# - `TheNumber`         *number* : The number to add leading zeros to.
# - `TheNumberOfDigits` *integer*: The total number of digits.
#
# Returns:
# - *text*: TheNumber with the specified number of leading zeros.
#
on AddLeadingZeros(TheNumber, TheNumberOfDigits)
	set IsNegative to TheNumber is less than 0
	
	# Determine the maximum number from the number of digits.
	set TheNumberOfDigits to (TheNumberOfDigits - 1)
	set TheThreshold to (10 ^ TheNumberOfDigits) as integer
	
	if TheNumber is less than TheThreshold then
		# If the number is negative, convert it to positive
		if IsNegative = true then set TheNumber to -TheNumber
		
		set TheLeadingZeros to ""
		set TheDigitCount to length of ((TheNumber div 1) as text)
		set TheCharacterCount to (TheNumberOfDigits + 1) - TheDigitCount
		repeat TheCharacterCount times
			set TheLeadingZeros to (TheLeadingZeros & "0") as text
		end repeat
		
		# Make the number negative, if it was previously negative
		if IsNegative = true then set TheLeadingZeros to "-" & TheLeadingZeros
		
		return (TheLeadingZeros & (TheNumber as text)) as text
		# If the number is greater than or equal to the maximum number of digits
	else
		# Return the original number
		return TheNumber as text
	end if
end AddLeadingZeros


# # FormatTimeCode
#
# Converts seconds into a timecode format matching `00:00:00`
#
# Parameters:
# - InSeconds *number*: Number of seconds to set timecode to. Accepts float but is converted to Integer
#
# Returns:
# - *text*: Timecode
#
on FormatTimeCode(InSeconds)
	set InSeconds to InSeconds as integer
	# Calculate Hours
	set CalculatedHours to (InSeconds div hours)
	
	# Calculate Minutes
	set RemainderSeconds to (InSeconds mod hours)
	set CalculatedMinutes to (RemainderSeconds div minutes)
	
	# Calculate Seconds
	set CalculatedSeconds to (RemainderSeconds mod minutes)
	
	# Convert to Two Digit Strings
	set CalculatedHours to AddLeadingZeros(CalculatedHours, 2)
	set CalculatedMinutes to AddLeadingZeros(CalculatedMinutes, 2)
	set CalculatedSeconds to AddLeadingZeros(CalculatedSeconds, 2)
	
	set TimeCode to CalculatedHours & ":" & CalculatedMinutes & ":" & CalculatedSeconds as text
	return TimeCode
end FormatTimeCode


# # GetReferenceURLFromPath
#
# Get the reference URL for a file in a specified or active database.
#
# Parameters:
# - `TheFilepath`  *text*: The path to the file.
# - `DatabaseName` *text*: That database to look in.
#
# Returns:
# - *text*: The Reference URL.
#
on GetReferenceURLFromPath(TheFilePath, DatabaseName)
	tell application id "DNtp"
		# Check if the database is valid, if not use the active database.
		if exists database DatabaseName then
			set TheRecordList to lookup records with path TheFilePath in database DatabaseName
		else
			display notification "The specified database wasn't found, trying the active database instead" subtitle "Invalid Database"
			set TheRecordList to lookup records with path TheFilePath
		end if
		
		# Make sure we found a matching record.
		if (count of TheRecordList) is greater than 0 then
			# A list could return more than one record such as if a file is indexed to more than one group. Use the first item but notify the user.
			if (count of TheRecordList) is greater than 1 then
				display notification "More than one record was found, using the first record" subtitle "Multiple Records"
			end if
			set TheReferenceURL to reference URL of item 1 of TheRecordList
			return TheReferenceURL
		else
			error "File not found in database."
		end if
	end tell
end GetReferenceURLFromPath


# # GetVideoFromVLC
#
# Gets metadata about the current video open in VLC
#
# Parameters:
# - `PausePlayback` *boolean*: Pause video when called.
#
# Returns:
# - *list* (indexed)
#   - `TheVideoPath`     *text*: The path for current video.
#   - `TheCurrentTime` *number*: The current second of playback.
on GetVideoFromVLC(PausePlayback)
	tell application "VLC"
		if (exists path of current item) then
			set TheVideoPath to path of current item
			set TheCurrentTime to current time
		else
			error "VLC does not have an open video."
		end if
		
		if PausePlayback and playing then
			play
		end if
		
		return {TheVideoPath:TheVideoPath, TheCurrentTime:TheCurrentTime}
	end tell
end GetVideoFromVLC

on copyRTFandMarkdownLink(theURLs, theNames)
	try
		set theURLsArray to current application's NSMutableArray's arrayWithArray:theURLs
		set theNamesArray to current application's NSArray's arrayWithArray:theNames
		
		if revealRecord = true then
			repeat with i from 0 to ((theURLsArray's |count|()) - 1)
				set thisURLString to (theURLsArray's objectAtIndex:i)
				if (thisURLString's hasPrefix:"x-devonthink") then (theURLsArray's replaceObjectAtIndex:i withObject:(thisURLString's stringByAppendingString:"&?reveal=1"))
			end repeat
		end if
		
		if theURLsArray's |count|() = 1 then
			set theTypesArray to current application's NSArray's arrayWithArray:{"public.utf8-plain-text", "dyn.ah62d4rv4gu8yc6durvwwaznwmuuha2pxsvw0e55bsmwca7d3sbwu", ¬
				"public.url-name", "dyn.ah62d4rv4gu8zs3pcnzme2641rf4guzdmsv0gn64uqm10c6xenv61a3k", "public.url"}
		else
			set theTypesArray to current application's NSArray's arrayWithArray:{"public.utf8-plain-text", "dyn.ah62d4rv4gu8zs3pcnzme2641rf4guzdmsv0gn64uqm10c6xenv61a3k"}
		end if
		
		set thePasteboard to current application's NSPasteboard's generalPasteboard()
		thePasteboard's declareTypes:theTypesArray owner:(missing value)
		set thePasteboardItem to (thePasteboard's pasteboardItems())'s objectAtIndex:0
		
		repeat with i from 0 to ((theTypesArray's |count|()) - 1)
			set thisType to (theTypesArray's objectAtIndex:i)
			
			if (thisType's isEqualTo:"public.utf8-plain-text") then
				set theMarkdownLinksArray to (current application's NSMutableArray's arrayWithArray:{})
				set theURLsArrayCount to (theURLsArray's |count|())
				repeat with i from 0 to (theURLsArrayCount - 1)
					set thisMarkdownLink to (current application's NSString's stringWithFormat_("[%@](%@)", theNamesArray's objectAtIndex:i, theURLsArray's objectAtIndex:i))
					if i < (theURLsArrayCount - 1) then set thisMarkdownLink to (thisMarkdownLink's stringByAppendingString:(space & space))
					(theMarkdownLinksArray's addObject:thisMarkdownLink)
				end repeat
				(thePasteboardItem's setString:(theMarkdownLinksArray's componentsJoinedByString:linefeed) forType:thisType)
				
			else if (thisType's isEqualTo:"dyn.ah62d4rv4gu8yc6durvwwaznwmuuha2pxsvw0e55bsmwca7d3sbwu") then
				(thePasteboardItem's setData:(current application's NSPropertyListSerialization's dataWithPropertyList:theURLsArray ¬
					format:(current application's NSPropertyListXMLFormat_v1_0) options:0 |error|:(missing value)) forType:thisType)
				
			else if (thisType's isEqualTo:"public.url-name") then
				(thePasteboardItem's setString:(theNamesArray's objectAtIndex:0) forType:thisType)
				
			else if (thisType's isEqualTo:"dyn.ah62d4rv4gu8zs3pcnzme2641rf4guzdmsv0gn64uqm10c6xenv61a3k") then
				set theNamesWithLinefeedArray to (current application's NSMutableArray's arrayWithArray:{})
				set theNamesArrayCount to theNamesArray's |count|()
				repeat with i from 0 to (theNamesArrayCount - 1)
					set thisName to (theNamesArray's objectAtIndex:i)
					if i < (theNamesArrayCount - 1) then set thisName to (thisName's stringByAppendingString:return) -- necessary for OmniFocus. must be a return (a linefeed adds a blank line in Nisus Writer)
					(theNamesWithLinefeedArray's addObject:thisName)
				end repeat
				(thePasteboardItem's setData:(current application's NSPropertyListSerialization's dataWithPropertyList:{theURLsArray, theNamesWithLinefeedArray} ¬
					format:(current application's NSPropertyListXMLFormat_v1_0) options:0 |error|:(missing value)) forType:thisType)
				
			else if (thisType's isEqualTo:"public.url") then
				(thePasteboardItem's setString:(theURLsArray's objectAtIndex:0) forType:thisType)
			end if
		end repeat
		display notification "Copied"
		return true
	on error error_message number error_number
		activate
		if the error_number is not -128 then display alert "Error: Handler \"copyRTFandMarkdownLink\"" message error_message as warning
		error number -128
	end try
end copyRTFandMarkdownLink

2 Likes