Timeout of long running smart rules and batch processes

No, no! Definitely not. It’s the production-ready version of the script that I used to handle those 1400 records. Before “going live,” I use a test database containing crafted documents that I use while developing a script. In the next iteration, I want to dive into using (dynamic) templates to set up a clean test environment each time I change the script.

1 Like

Oh great! I was not aware of that command. Thank you very much!

Updated version of the script with logging in logfmt - personal preference:

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

property globalSuccessTags : {"auto-processed-rename-ai"}
property globalAiRequestTimeout : 120

property globalPrompt : "Generate a concise, descriptive filename for the following text.¬
# Rules/Expectations for the filename:¬
- 3-6 words maximum, no more than 50 letters¬
- Use only lowercase letters, numbers, and underscores¬
- Capture the main topic or purpose¬
- Be specific enough to identify the content¬
  - for invoices: if a company name is given always use it¬
  - for invoices: use the main product¬
  - ignore city names in invoices¬
  - for bookings: use the name of the place or the destination¬
- Ignore prices¬
- Avoid generic terms at all costs like \"document\", \"file\", or \"text\"¬
- Avoid any names of persons at all costs¬
- Follow the format: topic_subtopic_descriptor (if applicable)¬
# Steps¬
1. Create a management summary for the text¬
2. Capture the main topic or purpose based on the text¬
3. Extract dates from the text - ignore ones before the date saved in the NOW-variable¬
4. Order dates from newest to oldest¬
5. Use the oldest date in YYYY-MM-DD notation¬
6. Generate the name in German for the file and prefix the name with the date¬
7. Remove first names and surnames of persons + prices from filename
8. Translate the captured text into German - if given in any other language
# Expected output¬
The result is in JSON notation. The filename is stored with key \"new_filename\"¬ 
Example:¬
```json¬
{¬
 \"new_filename\": \"1970-10-01_this-is-the-file-name\"¬
}¬
```¬
# Variables

"

-- code to run within Script Debugger
tell application id "DNtp" to my performSmartRule(selected records)

-- ==========

-- test performSmartRule
property globalIsDev : true

-- copy template for smart rules
on performSmartRule(theRecords)
	local libraryHandler
	set libraryHandler to script "library-handling"
	
	tell application id "DNtp"
		libraryHandler's canRun(current database, theRecords)
	end tell
	
	local AiBase
	set AiBase to libraryHandler's loadScript("ai-base", globalIsDev)
	AiBase's SetupLocalAiRuntime()
	
	local theScriptHandler
	set theScriptHandler to libraryHandler's loadScriptSilent("document-rename", globalIsDev)
	
	local thePrompt
	set thePrompt to get globalPrompt of theScriptHandler
	
	local aiRequestTimeout
	set aiRequestTimeout to globalAiRequestTimeout of theScriptHandler
	
	tell application id "DNtp"
		-- phi4
		-- display alert "Dokumente werden umbenannt"
		theScriptHandler's ForAll(theRecords, globalIsDev, Ollama, "phi4", thePrompt, aiRequestTimeout)
		-- display alert "Alle Dokumente wurden erfolgreich umbenannt"
	end tell
end performSmartRule

-- Run job for all given records
on ForAll(theRecords, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local titleProgressBar
	set titleProgressBar to "Rename documents"
	
	tell application id "DNtp"
		show progress indicator titleProgressBar steps (count of theRecords) with cancel button
		-- https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptLangGuide/reference/ASLR_error_xmpls.html
		
		set len to length of theRecords
		set i to 1
		
		my globalLog("INFO", quoted form of "Job started", "count_of_records=" & len)
		
		try
			local theRecord
			repeat with theRecord in theRecords
				if cancelled progress then
					my recordLog(theRecord, "DEBUG", quoted form of "Cancel current run of script", missing value)
					
					exit repeat
				end if
				
				set theFilename to the filename of theRecord
				
				step progress indicator theFilename
				my recordLog(theRecord, "INFO", quoted form of "Work on record", "index=" & i & "/" & len)
				
				set modifiedRecord to my ForOne(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
				set tags of modifiedRecord to (get tags of modifiedRecord) & globalSuccessTags
				
				set i to i + 1
			end repeat
		on error errStr number errorNumber
			-- An unknown error occurred. Resignal, so the caller
			-- can handle it, or AppleScript can display the number.
			display alert errStr
			error errStr number errorNumber
		end try
		
		hide progress indicator
	end tell
	
	my globalLog("INFO", quoted form of "Job completed", "count_of_records=" & len)
end ForAll

-- Run job for single record
on ForOne(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local newName
	set newName to my GetNewNameLoop(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	
	my recordLog(theRecord, "INFO", quoted form of "Change name of record", "oldname=" & quoted form of (name of theRecord as string) & " newname=" & quoted form of newName)
	set name of theRecord to newName
	
	return theRecord
end ForOne

on GetNewNameLoop(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local newName
	
	local repeatCounter
	set repeatCounter to 1
	repeat
		try
			my recordLog(theRecord, "DEBUG", quoted form of "Send request to AI model", missing value)
			set newName to my GetNewName(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
			exit repeat
		on error errStr number errorNumber
			-- nothing
		end try
		
		if repeatCounter ≥ 3 then
			error "Repeat counter ≥ 3 for GetNewName(): " & errStr & ". Exit"
		end if
		
		set repeatCounter to repeatCounter + 1
	end repeat
	
	return newName
end GetNewNameLoop

on GetNewName(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local AiBase
	set AiBase to script "ai-base"
	
	local now
	set now to current date
	
	tell application id "DNtp"
		-- Get date components
		set currentYear to year of now
		set currentMonth to rich texts -2 thru -1 of ("00" & ((month of now) as integer))
		set currentDay to rich texts -2 thru -1 of ("00" & (day of now))
	end tell
	
	set currentDate to currentYear & "-" & currentMonth & "-" & currentDay as string
	
	set thePrompt to thePrompt & "¬
		CURRENT_DATE=" & currentDate
	
	-- log "[DEBUG] Validate chosen AI engine"
	-- set theEngine to AiBase's ValidateAndReturnEngine(theEngine)
	
	local theResponse
	set theResponse to AiBase's AskModelTimeout(theRecord, theEngine, theModel, thePrompt, "JSON", localAiRequestTimeout)
	
	if theResponse is missing value then
		log "[WARN] No (valid) result from AI"
		return missing value
	end if
	
	return new_filename of theResponse
end GetNewName

on recordLog(theRecord, level, msg, msgInfo)
	if msgInfo is missing value then
		set logMsg to "msg=" & msg
	else
		set logMsg to "msg=" & msg & " " & msgInfo
	end if
	
	if level is not "DEBUG" then
		tell application id "DNtp"
			log message record theRecord info "level=INFO " & logMsg
		end tell
		
		return
	end if
	
	if globalIsDev is true then
		tell application id "DNtp"
			log message record theRecord info "level=DEBUG " & logMsg
		end tell
	end if
end recordLog

on globalLog(level, msg, msgInfo)
	if msgInfo is missing value then
		set logMsg to "msg=" & msg
	else
		set logMsg to "msg=" & msg & " " & msgInfo
	end if
	
	if level is not "DEBUG" then
		tell application id "DNtp"
			log message "level=INFO " & logMsg
		end tell
		
		return
	end if
	
	if globalIsDev is true then
		tell application id "DNtp"
			log message "level=DEBUG " & logMsg
		end tell
	end if
end globalLog

@cgrunenberg I would love to see a log entry, if DT kills a long running smart rule.

Some time ago I wondered, what happened to a script, when I realized it was killed by the timeout.

image

I think it’s the OSA framework that kills scripts when they’re running too long. That might be out of the reach of DT.

In case of background scripts like smart rules DEVONthink uses its own timeout too. Especially as smart rule scripts are supposed to run automatically and not to require user interaction or input. Especially to ensure that the execution of other rules isn’t endlessly blocked.

1 Like