Timeout of long running smart rules and batch processes

Question:

  • Is there a time threshold for smart rules and/or batch processes after which the running AppleScript is stopped?
  • Is there some more logging available using the log window? This would be very helpful debugging long running rules/batch processes

What I found out so far

  1. Running the script - it’s not complete, but uses some outsourced methods - I found that the smart rule and the batch process are stopped after some time without any error/alert- maybe 5-10 min
  2. The very same script run via Script Debugger ran for about 47 mins without any error or some other trouble - I stopped it manually, as I was about to go to bed.
  3. Each request to the

Environment

  • I run phi4 with MstyStudio.app - best reproducible results for renaming files based on content
  • I configured Ollama for DEVONthink
  • Each “renaming” request (to the model) per record takes about 7-20 s

Next steps

  • Try to test the existing renaming script - I was not happy about the results and needed a reusable solution I can embed into another script

Here’s the script - if required I post the other parts as well. I tried th

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

property globalSuccessTags : {"auto-processed-rename-ai"}
property globalAiRequestTimeout : 120

property globalPrompt : "Generate a concise, descriptive filename for the following text.¬
# Rules/Expectations for the filename:¬
- Be 3-6 words maximum, no more than 50 letters¬
- Use only lowercase letters, numbers, and underscores¬
- Capture the main topic or purpose¬
- Be specific enough to identify the content¬
- if a company name is given always use it
- ignore city names in invoices
- Follow the format: topic_subtopic_descriptor (if applicable)¬
- Avoid generic terms at all costs like \"subscription\", \"license\", \"invoice\", \"rechnung\", \"document\", \"file\", or \"text\"¬
# Steps¬
1. Capture the main topic or purpose based on the text¬
2. Translate the captured text into German
2. Extract dates from the text - ignore ones before the date saved in the NOW-variable¬
3. Order dates from newest to oldest¬
4. Use the oldest date in YYYY-MM-DD notation¬
6. Generate the name in German for the file and prefix the name with the date¬
¬
# Expected output - Example¬
¬
```json¬
{¬
 \"new_filename\": \"1970-10-01_this-is-the-file-name\"¬
}¬
```¬
# Variables

"

-- code to run within Script Debugger
tell application id "DNtp" to my performSmartRule(selected records)

-- ==========

-- test performSmartRule
property globalIsDev : true

-- copy template for smart rules
on performSmartRule(theRecords)
	local libraryHandler
	set libraryHandler to script "library-handling"
	
	tell application id "DNtp"
		libraryHandler's canRun(current database, theRecords)
	end tell
	
	local AiBase
	set AiBase to libraryHandler's loadScript("ai-base", globalIsDev)
	AiBase's SetupLocalAiRuntime()
	
	local theScriptHandler
	set theScriptHandler to libraryHandler's loadScriptSilent("document-rename", globalIsDev)
	
	local thePrompt
	set thePrompt to get globalPrompt of theScriptHandler
	
	local aiRequestTimeout
	set aiRequestTimeout to globalAiRequestTimeout of theScriptHandler
	
	tell application id "DNtp"
		-- phi4
		-- display alert "Dokumente werden umbenannt"
		theScriptHandler's ForAll(theRecords, globalIsDev, Ollama, "phi4", thePrompt, aiRequestTimeout)
		-- display alert "Alle Dokumente wurden erfolgreich umbenannt"
	end tell
end performSmartRule

-- Run job for all given records
on ForAll(theRecords, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local titleProgressBar
	set titleProgressBar to "Rename documents"
	
	log "[INFO] Job started"
	
	tell application id "DNtp"
		show progress indicator titleProgressBar steps (count of theRecords) with cancel button
		-- https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptLangGuide/reference/ASLR_error_xmpls.html
		try
			local theRecord
			repeat with theRecord in theRecords
				if cancelled progress then exit repeat
				
				set theFilename to the filename of theRecord
				
				step progress indicator theFilename
				log "[INFO] Work on record " & theFilename
				
				set modifiedRecord to my ForOne(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
				set tags of modifiedRecord to (get tags of modifiedRecord) & globalSuccessTags
			end repeat
		on error errStr number errorNumber
			-- An unknown error occurred. Resignal, so the caller
			-- can handle it, or AppleScript can display the number.
			display alert errStr
			error errStr number errorNumber
		end try
		
		hide progress indicator
	end tell
	
	log "[INFO] Job completed"
end ForAll

-- Run job for single record
on ForOne(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local newName
	set newName to my GetNewNameLoop(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	
	tell application id "DNtp"
		set name of theRecord to newName
	end tell
	
	return theRecord
end ForOne

on GetNewNameLoop(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local newName
	
	local repeatCounter
	set repeatCounter to 1
	repeat
		try
			set newName to my GetNewName(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
			exit repeat
		on error errStr number errorNumber
			-- nothing
		end try
		
		if repeatCounter ≥ 3 then
			error "Repeat counter ≥ 3 for GetNewName(): " & errStr & ". Exit"
		end if
		
		set repeatCounter to repeatCounter + 1
	end repeat
	
	return newName
end GetNewNameLoop

on GetNewName(theRecord, localIsDev, theEngine, theModel, thePrompt, localAiRequestTimeout)
	local AiBase
	set AiBase to script "ai-base"
	
	set now to current date
	
	tell application id "DNtp"
		-- Get date components
		set currentYear to year of now
		set currentMonth to rich texts -2 thru -1 of ("00" & ((month of now) as integer))
		set currentDay to rich texts -2 thru -1 of ("00" & (day of now))
	end tell
	
	set currentDate to currentYear & "-" & currentMonth & "-" & currentDay as string
	
	set thePrompt to thePrompt & "¬
		CURRENT_DATE=" & currentDate
	
	-- log "[DEBUG] Validate chosen AI engine"
	-- set theEngine to AiBase's ValidateAndReturnEngine(theEngine)
	
	local theResponse
	set theResponse to AiBase's AskModelTimeout(theRecord, theEngine, theModel, thePrompt, "JSON", localAiRequestTimeout)
	
	if theResponse is missing value then
		log "[WARN] No (valid) result from AI"
		return missing value
	end if
	
	return new_filename of theResponse
end GetNewName

After 5 minutes to avoid that buggy scripts block the execution of other smart rules & scripts. One alternative might be a script for the Scripts menu which processes the selected items, another one the Chat - Query action and the Query Response placeholder of smart rules & batch processing.

Thanks. That works! Except in search: I’m not able to select a script - greyed out. Is this a known issue?

And the the Batch Process entry is missing although I can access it via Tools > Batch Process

Batch Processing isn’t available for search results though development would have to assess the behavior.
Considering the disabled scripts, if you already have a script running, you can’t launch another until the first is finished.

Ah! Ok. Yeah.

I created and started the Renaming Script as mentioned by @cgrunenberg earlier

It’s of course available. But smart rules & batch processing are blocked during the execution of a script.

How did you settle on phi4 ?

As per @MarvinMarvelouis’ comment, it can be run via the Tools menu or the hotkey but in the context menu of search results, Batch Processing isn’t shown as a command.

I didn’t know if that is intentional or not.

Trial and error. I ran different prompts on different models and chose the one with the best results.

Best means in this case:

  • JSON result on every request - gpt-oss failed on that as I remember
  • Usable naming
  • Sticking to the prompt - YYYY-MM-DD-xxxx - some left out the year
  • Results in a reasonable amount of time

I might try some other models in the future. The biggest challenge was crafting the prompt to get usable results. Using AppleScript was far more consistent than using the Chat action - but that might be on me.

But besides that, I really like the integration of AI tools via DEVONthink’s AppleScript integration.

That was a bit irritating; showing it “greyed out” would be more helpful - at least for me and more consistent with other menu entries.

Did you use the same role & prompt?

If you use FastScripts as your script launcher then you can launch multiple simultaneous scripts. I have found it works fine with DT4 scripts.

I duplicated the existing one and pinned the model I think - I removed the entry an hour ago or so.

This is only recommended if the scripts do not process the same records. In addition, only one script can use progress indication.

1 Like

Thanks. :slight_smile:

Stability/consistency trumps speed in “this domain” for me. And I use progress indication to stop the script during a run.

Currently I run the script to change the name of 1400 records. The only thing I really miss is some kind of logging. :wink:

See log message AppleScript command

But this is not logged within DEVONthink, is it? I used this quite a lot, but it was only useful within the Script Debugger and the like.

Does it make sense to issue a warning in the log or show an alert to signal the threshold to the user?

log message is a DT command. log is the one for the Script Editor.

You’re not trying to run the script on all 1400 while you’re debugging, are you?