Smart rules using Javascript with regular expressions

Are there any examples of Javascript scripts (instead of AppleScript) for Smart Rules?

Have been digging through the forums and experimenting for hours to come up with a solution to create the following smart rule with no luck so far.

  1. Parse PDF text to extract the invoice date
  2. Set the document creation date to the parsed date
  3. Rename the document to “YYYY-MM-DD - Vendor”

Caveats/notes:

  • the specific date to extract is buried in the text amongst 1/2 dozen dates which makes it non-trivial to extract as it is surrounded by “false positive” matches - I also have numerous other complex matching use cases that will rely on a full featured regex implementation

  • the following regex seems to do a reasonable job at matching for this specific use case
    (?!.*period.*).*((\b\d{1,2}\D{0,3})?\b(January|February|March|April|May|June|July|August|September|October|November|December)\s(\d{1,2})\,\s+\d{1,4})

However, AppleScript seems to be lacking regular expression support. Within AppleScript I’ve also tried shelling to sed, grep and egrep which at first looked hopeful only to discover their BSD regex support is dated and lacking functionality.

Ideally if I could just write the smart rule script in Javascript which has regular expressions as batteries included instead of AppleScript this should have been relatively straight forward task.

Attempted the following:

function performSmartRule(records) {

  var app = Application("DEVONthink 3");
  app.includeStandardAdditions = true;
    
  records.forEach(f => {

		app.displayDialog(f.plainText());
  });
}

Saved this via the Script Editor, but when trying to use it:

5:18:59 PM: ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/testing extract 3.scpt on performSmartRule (Error -1708)

in the DevonThink log and in console.log

2020-12-30 16:07:57.092 DEVONthink 3[4289:192641] Warning - unable to find template matching predicate "Script" == "/Users/recipedude/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/testing extract 2.scpt"

Seems DEVONthink doesn‘t support JavaScript in Smart Rules.

Yes, vanilla AppleScript. However you could use AppleScriptObjC. In this Script: Convert WikiLinks to Markdown links you‘ll find a regex „find“ and a „replace“ handler.

As @pete31 said: DT doesn’t support JS in smart rules. It is works fine in normal scripts, though.

As to your three questions above: I use Hazel and JavaScript in this one to do what you’re trying to do in DT. AppleScript is PITA if it comes to string processing. Event though it’s possible to do RegExes in AS with the ObjC-Integration, it is very cumbersome compared to plain JS. In my opinion.

Thank you - have been reviewing that post you linked to and script; initial testing looks promising.

Do have Hazel, however it would be nice to go directly into DT and skip the middleman if possible. What got me here was I ended up with a bunch of PDFs that were not OCR’d on scan and I found DevonThink could OCR for me and figured I’d go all in with DT.

AppleScript is beyond a PITA IMO. Heck, even JS isn’t that friendly with RegEx and string processing compared to say perl, but it’s a truckload better and infinitely more flexible at least.

I did get the regexes working via ObjC and Foundation framework, only to find that what works in the Script Editor fails when run in DevonThink’s smart rule - which have narrowed down to use framework "Foundation" blowing things up somehow.

That’s weird. Maybe @cgrunenberg can shed some light on that. But apparently others have run into the same problem: Can't log name of record via Smart Rule Script
So one is basically out of luck with Reg Exes in smart rules: neither JavaScript nor ObjCAppleScript are usable.
OT: I agree that Perl is more powerful re Reg Exes, but for me nowadays, JS is good enough. And it is a bit more legible than Perl :wink:

Not to be picky, but that seems to be a simple script. No on performSmartRule, as far as I can see.

1 Like

You can use shell script from apple script in a smart rule; and that does support regex (whilst you probably don’t need an example, Jim posted one for me here.

Of course, I just read the thread title …


@chrillek I’m confused. I thought using JavaScript in Smart Rules isn’t possible at all, then found this post

So isn’t it possible then to use an external JavaScript (and its regex) in a Smart Rule?

JXA is not supported when writing a smart rule script.

1 Like

@pete31, have the same question as I continue to dig into the linked post.

Got an error when trying to use an external Javascript script when saved in the ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules folder so was assuming no JS.

But comments in that post:

External scripts however can be in JavaScript.

For DEVONthink it makes no difference whether a script is actually AppleScript or JavaScript. But the internal script editor doesn’t support JavaScript

DEVONthink doesn’t even know whether a script uses JavaScript or AppleScript,

The key is that the script DEVONthink compiles for use in the Smart Rule does not contain any AppleScriptObjC references itself, only the script in the Script Library.

was able to get my script working (so far) by moving all of the code that required the use of AppleScriptObjC into a Script Library (e.g., to ~/Library/Script Libraries/MyScript.scpt) and then using it in my Smart Rule script

do seem to suggest there’s a workaround.

(apologies for not crediting each of the authors of each of the snippets above from the original post, am just trying gather some of the key points being shared that seem relevant)

So if the SmartRule’s external script is:

  • plain Jane native AppleScript
  • does not reference the Foundation Framework or obj-c in any way
  • not Javascript
  • in alignment with a various stars and planets

then it may be able to reference an external library script that does refer to the Foundation Framework and/or is javascript?

Don’t know about JavaScript but it does work with AppleScript over here, see this post

Kinda. You can use sed, but that’s even weirder than AppleScript. My best bet for doing it in the shell would be Perl, which also requires some learning.

Or Jim, which required none at all :crazy_face:

sed is definitely on the weird side, but weirder than AppleScript?

Once in the shell there’s sed, grep, egrep, awk, etc. However when I went down the sed/egrep path I discovered that sed and grep don’t implement PCRE (Perl Compatible Regular Expressions) which in turn didn’t support my regex that managed to crowbar the needed date out of the text. :man_facepalming:

It was also brittle - a bit of mangled OCR and it would break.

Here’s one attempt if it might help someone else down the road.

try
  -- save the text to a temp file
  set _cmd to "cat > /tmp/devon.tmp <<ENDOFFILE
" & theText & "ENDOFFILE
"

  do shell script _cmd
  -- regex
  set _cmd to "egrep -o '(?!.*period.*).*(\\b\\d{1,2}\\D{0,3})?\\b(January|February|March|April|May|June|July|August|September|October|November|December)\\s(\\d{1,2})\\,\\s+\\d{1,4}' /tmp/devon.tmp | sed -n '1 p'"
  set billDate to do shell script _cmd
end try

Again, that regex doesn’t work without PCRE.

As a fallback I came up with:

  1. save text to a temp file
  2. parse with and external script via the shell (in any language desired, probably ruby or perl)
  3. spit back results

Perl - Practical Extraction and Reporting Language - it’s made to fit the task; but it’s seems like overkill for something that could be done in a dozen lines of JS (or less in this case).

It actually is. I had my little heureka moment tonight during the fireworks: AppleScript is not case sensitive, JavaScript is. So
function performSmartRule(records)
does NOT work. Whereas
function performsmartrule(records)
DOES work in an external script.
Short example for an external JavaScript script in a smart rule:

function performsmartrule(records) {
  let app=Application("DEVONthink 3");
  records.forEach(r => app.logMessage(r.plainText()));
}

This writes the plainText part of the selected records to DT’s log window. After that, DT says

11:48:08: ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/Tryit.scpt on performSmartRule (Error: Error: Kein Fehler.)

in English: (Error: Error: No Error). Which is probably DT’s way to say “ceci n’est pas une pipe”. What a nice way to start a new year.

3 Likes

:heart_eyes: