How to use JavaScript in a smart rule (finally)

Checked for updates and got You are running DEVONthink Pro Edition 3.6.1. No newer version was found.

Ok, well the plot is thickening.

Using this:

function performsmartrule(records) {
  let app=Application("DEVONthink 3");
  records.forEach(r => {
	str = JSON.stringify(records);
	app.logMessage('records->'+str);
	str = JSON.stringify(r);
	app.logMessage('r->'+str);
    app.logMessage('before');
  	app.logMessage(r.plainText() || "NO TEXT");
	app.logMessage('after');
  } )
}

Spits out (in the log) window:

|5:05:57 PM: records->[null,null,null]||
|5:05:57 PM: r->undefined||
|5:05:57 PM: before||
|5:05:57 PM: ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/JS Plain Text Orig2.scpt|on performSmartRule (Error: Error: No error.)|

Uh, so the record in variable r is null, but the records array isn’t null but actually has 3 items each with a null value!?

P.S. (if I select 5 items, the records array ends up with 5 nulls)

Could you try a simple

app.logMessage(r.name());

inside the forEach? That should always work because every record has a name.
I’m not sure that the JSON.stringify works with these rather special objects,

Sure, actually that’s what led me to JSON.stringify to try and figure out what the heck is going on.

  let app=Application("DEVONthink 3");
  records.forEach(r => {
	app.logMessage(r.name());
  } )
}

5:37:19 PM: ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/JS Plain Text Orig2.scpt on performSmartRule (Error: Error: No error.)

Looks like those nulls in the records list aren’t really nulls, but seems like they are a function…

function performsmartrule(records) {
  let app=Application("DEVONthink 3");
  app.logMessage(app.name());
  app.logMessage( typeof records );
  records.forEach(r => {
	app.logMessage( typeof r );  
	app.logMessage(r.name());
  } )
}

Produces log of:

  • DEVONthink 3
  • object
  • function
  • on performSmartRule (Error: Error: No error.)

I’m curious why you want to do it in JavaScript. Didn’t moving the AppleScriptObjC part into a script library (see here) work for you?

Yes I do now have a handful of simpler but more complex than out of the box regexes working via AppleScriptObjC in an home-spun external script library script thanks to all the help here on the forums and in particular your awesome post. Once the dust settles and I have my scripts tidied up am hoping to post a complete follow-up example with what I ended up with.

However, that is just the tip of the iceberg. The AppleScript + ObjC solution is painfully unwieldy, verbose, brittle, obtuse and discourages further development. I can see the unmaintainable monster lurking and countless hours burnt once I start digging into more complex use cases.

Here’s an example use-case that I have in mind longer term.

  • import of invoice (via email or scan w/OCR) into DT inbox
  • script parses/extracts for basic data, such as identifying vendor, date and filing actions
  • further parsing to extract net, taxes, totals, description
  • using the extracted data invoice is then input into accounting software using REST calls, CLI, cough ApplesScript, MWScript, or generated CSV/XML

For an even more pipe-dream of a use-case:

  • business credit card statement (PDF with text) imported into DT inbox
  • script parses/extracts for basic data, filing/classification within DT
  • line items of transactions are parsed, looking for existing entries in accounting software, if no line exists an entry is made (and or dots are connected together)
  • e.g. Amazon CC line item could use amount to find the Amazon invoice/order summary previously imported into DT where it might be able to connect the order # by matching amounts
  • generates REST, CLI, CSV or XML for import into bookeeping software (or perhaps a report designed to make the manual entry less painful)

Considering the scripting yoga poses that were needed using DT + AppleScript + ObjC simply to capture a date using a no-so-complex regular expression… I can only imagine (in waking nightmarish form) what an AppleScript + ObjC solution might look like for these more interesting use cases.

Looking through the posts on the forums over the last few days, it sure feels as if AppleScript, the accompanying brittle-ness and, outright needless coding suffering is a road-block for so many to really unleash the power of DT and bring it to the next level.

If Javascript can be made to work natively, integrating more automation and intelligence becomes something one wants to explore and get excited about and opens up new possibilities.

IMO, getting Javascript working natively in DT is a massive game-changer.

2 Likes

Sounds like JavaScript to me :wink:

By the way, AppleScript is the preferred language of many, many Mac users, DEVONtech clients included. JavaScript is terse and indecipherable for many people with little programming experience. We use and support AppleScript natively as it provides less of a barrier to entry for everyday people.

1 Like

Sounds like JavaScript to me

Facile and baseless comment.

AppleScript less of a barrier to entry for everyday people.

Completely untrue. You may be confused by the fact that you are familiar with it.

  • The pattern of AppleScript is actually quite tricky to master
  • Its records are harder to work with, and more liable to explode in your face than any language I have encountered in 40 years
  • AppleScript is a total and unpassable barrier to macOS ⇄ iOS cross-platform scripting.

It’s lazy (and unhelpful to beginners) to be unrealistic about it on the grounds that it’s a language in which you have personally invested a lot of time.

1 Like

Please, let’s not get into language wars here. Everyone has their preference (I’m actually more of a Perl person).
JavaScript is as much a first class citizen of MacOS automation as AppleScript and others. Or as much third class as others, given Apple’s attitude to automation.
So if someone is having trouble with it, they deserve the same help as someone struggling with another problem. I think.

Slightly off-topic: JS is available across platforms. AppleScript is Mac-only. That, at least, is not a matter of taste :wink:

1 Like

JS is available across platforms. AppleScript is Mac-only. That, at least, is not a matter of taste :wink:

That’s right, and here’s another objective difference:

If we need a group of (key, value) pairs we can (in AppleScript or JavaScript) give a variable name to a value like:

{Alpha:70, Beta:60, Gamma:50, Epsilon:30}

AppleScript calls that a record, JavaScript calls it an Object.

When we need a list of the keys in our record/object.

In JavaScript:

let keyValues = {Alpha:70, Beta:60, Gamma:50, Epsilon:30}

Object.keys(keyValues)

// --> ["Alpha", "Beta", "Gamma", "Epsilon"]

in AppleScript, it quickly gets difficult, we have to import a library, and learn a verbose and exotic syntax:

use framework "Foundation"

set keyValues to {Alpha:70, Beta:60, Gamma:50, Epsilon:30}

(current application's NSDictionary's dictionaryWithDictionary:keyValues)'s allKeys() as list

And if we want to check the value of a key like 'Delta', which the record/object may or may not have (we’re not sure, and need to check):

In JavaScript:

keyValues.Delta
// -> undefined

keyValues.Gamma
// -> 50

But in AppleScript, the script just blows up with an error if we even try a key which the record doesn’t have:

To avoid tripping errors in AppleScript, we would have to use a ‘Foreign Function Interface’ to Objective C functions.

use framework "Foundation"


set keyValues to {Alpha:70, Beta:60, Gamma:50, Epsilon:30}

(current application's NSDictionary's dictionaryWithDictionary:keyValues)'s objectForKey:("Delta")

-- > missing value

OK, missing value is better … but if we use that same interface, when the record does have the key we tried, what we get back is a raw C pointer (don’t ask) and the poor beginner is now expected to handle its conversion to an AppleScript value:

use framework "Foundation"


set keyValues to {Alpha:70, Beta:60, Gamma:50, Epsilon:30}

(current application's NSDictionary's dictionaryWithDictionary:keyValues)'s objectForKey:("Beta")

--> «class ocid» id «data optr00000000D9FEBB59DFB6CA9F»

Checking the keys in a record, testing a key we’re not sure about – these are simple basics …

For example:

  • What is the full list of student names ?
  • Do I have an entry for this student ?

Very simple in nearly all languages. AppleScript is very unusual in making such basic things hard.

I have had great fun with AppleScript for over 20 years, and invested a huge amount of time in it. But I’m not going to pretend, just because I’m heavily invested in it, that it’s “easy”.

To be honest, it just ain’t.

When TaskPaper became JavaScript only, I experimented with learning a little JavaScript.

I soon noticed that it makes the basics quite a lot easier.

  • Not just easier records (see above),
  • but also much easier work with Strings and Lists (far more built-in operations for free),
  • and much richer libraries of things like:
    • regular expressions for pattern matching
    • url encoding and decoding
    • basic Math functions

Try to do some simple trigonometry for diagrams in OmniGraffle for example. Reach for AppleScript and you will be expected to write or bring your own sin and cos :frowning:

Reach for JS, and you will find them built in.

AppleScript “easy” ?

Fine if you’ve used it for years. Not a good place start. Especially now, in the context of iOS, and with AppleScript in sunset mode, and no longer actively developed.

2 Likes

No need for language wars. When is comes to languages just pick the best available tool for the job. Perl was made for this job but isn’t (directly) available.

The only reason Javascript is coming up here as a preference over Applescript is that it may be available natively, has infinitely better native support for the task at hand (uh, hello regex, arrays, etc.) and has an extremely large base of well-documented support.

1 Like

:slight_smile: Yeah, have to agree with ya there. Javascript is terse and its use of { }'s is overly zealous and irritating; not my personal preference just a better available tool for the job and, whether we like it or not, has a massive base of support load of examples for non-programmers and developers alike to choose from.

Personally I’d pitch for native Ruby support to expose many more people with little programming experience for readability and ease of entry but that’s not a (directly) available option.

One might question the assertion that

many, many Mac users prefer AppleScript.

I have a feeling that is because it was the only option natively available. A quick search led me to this quote in The unlikely persistence of AppleScript MacWorld article which sums it up better than I ever could.

What makes it so surprising that AppleScript survived … is that it was never loved by anyone. It was a fine theory and noble experiment, but it turns out that an English-like programming language didn’t really enable a large number of users to become programmers. And conversely, AppleScript’s English-like syntax often made (and to this day continues to make) things more difficult and confusing for scripters, not less.

However, I’m sure it may have some fans.

In my case however, AppleScript has prevented me from fully embracing DevonThink for 5 years now. Once or twice a year, I revisit DevonThink only to get frustrated by the self flagellation of AppleScript when running into a limitation. I’m certain that I’m not alone on this.

AppleScript is a barrier to entry for everyday people and seasoned software engineers alike; remove the barrier and more people will get excited instead of stifled.

No need for language wars, more tools in the toolbox is never a bad thing!

2 Likes

Now that hopefully everybody has explained why they (don’t) like this or that language, maybe we could

  • either get back to the problem at hand (if there is one)
  • or just leave it at that

I try my luck with the first possibility now. Here’s what I found out about when/if an external JavaScript script does (not) work in a smart rule. Take a very simple example:

function performsmartrule(recs) {
  let app = Application("DEVONthink 3");
   recs.forEach(r => {
    app.logMessage(r.name());
  } )
}

This code should only write the names of the selected records to DT’s log window. Which in fact it does not, it writes nothing. At least not for me. However (fasten your seat belts, please), if I add this snippet to the end of it

(() => {
  let app = Application("DEVONthink 3");
  performsmartrule(app.selectedRecords());
})();

the code does what it’s supposed to do in DT (and of course in Script Editor). See explanation at the end of the post. I must apologize that I didn’t mention that piece of code before, but it seemed irrelevant to me and only necessary to have the script run in Script Editor.

Some more experimentation showed that

  • the name of the function must be performsmartrule: changing it to foo in the definition and the call results in the script not doing anything.
  • the name of the function in the call (i.e. at the end of the script) must also be performsmartrule.

This seems to indicate that on the one hand, performsmartrule is called by DT. On the other hand, it seems also to indicate that the self-executing function added in step 2 above is also run.
If there’s somebody out there who can explain that … please, shed some light on this.

Explanation: The code added in the 2nd step above is a “self-executing anonymous function”. Anonymous, because it has no name. It also has no parameters, as indicated by the first pair of empty parenthesis. It’s self-executing because of the last pair of empty parenthesis following the function definition.

3 Likes

No. I like AppleScript.

2 Likes

Pete, shhhhh now! :smiley:

(:+1: @chrillek thanks for your perseverance & explanations)

2 Likes

The approach taken by Hook Productivity would work well here, and would be more consistent with Apple policy, the functioning of all other applications on macOS, and proper respect for the time and equanimity of DEVONthink customers.

(protecting customers from the rabbit holes, tar-pits, and XKCD-joke vultures that now cluster and circle so tightly and so disablingly around the use of regular expressions and JavaScript generally in DEVONthink Smart Actions)

In Hook scripts:

  1. Any script that begins with the two characters // is passed to osascript -l JavaScript
  2. any other script is passed to osascript -l AppleScript

(where the -l switch selects between available OSA components)

In Terminal.app, see:

  • man osalang
  • man osascript
  • man osacompile

Yes, there is still a problem that am still banging my head against trying to solve.

Running the very simple example produces:

  • on performSmartRule (Error: Error: No error.)

Adding the code in the 2nd step:

  • document name
  • on performSmartRule (Error: Error: No error.)

Now that is definitely progress!

Also tried changing r.name() to r.plainText() and woot! We are cooking with gas!

You are the (insert your preferred gender identity noun here)! Once I ice my noggin’ can get back to work after this 3 day distraction. There’s no way I would have stumbled on that missing bit without your help.

1 Like

Weird. Internally DEVONthink uses indeed performSmartRule, not performsmartrule.

An automatic derivation, I think, from the name used in the .sdef file: