Differentiation in smartrules

Timetrinus · January 13, 2021, 9:09am

Hello,

I just setup some smartrules to automatically handle different types of documents. For example for my insurance contracts. What I want to do is setting up a folder structure like this:
/Insurance Contracts/{CompanyName}{Policy No}{Insurance Type}/
For each of these folders I will have subfolders for 1. contract documents, 2. invoices and general correspondence.

At the moment I setup 1 Rule for each document type and for each company but I think for 15 insurance contracts, 3 bank accounts, different providers and so on this will end in a big number of smart rules. This is why I want to know if someone can give me an example about how to analyze a document within a script. Or is it possible to handle some complex conditions within smart rules?

Thank you.

cgrunenberg · January 13, 2021, 9:11am

A screenshot of an existing smart rule might be useful.

Timetrinus · January 13, 2021, 10:21am

cgrunenberg · January 13, 2021, 10:39am

Thanks! How many items do these rules actually process? Because they appear to be highly specific. Does the police no. have a certain prefix/suffix so that it could be easily retrieved from the document?

Timetrinus · January 13, 2021, 10:53am

It depends on the contract. For some contracts I get one or two letters per month, for some others just two or three per year…

Unfortunately the documents I receive don’t have useful names.

My Idea is to reduce the ammount of rules by scanning the document within a script. As I understood applescript is not able to work with regular expressions.

Do you have another Idea?

cgrunenberg · January 13, 2021, 11:03am

AppleScript does not support this but the command line tool sed does. In addition, the Scan Text action of smart rules supports regular expressions too. You can use the result(s) afterwards via the placeholder \0, \1 etc.

chrillek · January 13, 2021, 11:08am

You can make it kind of work with ObjCAppleScript (@pete31 has posted examples here). Also, JavaScript supports Regular Expressions. For JS to work in smart rules, you have to define a function performsmartrule(records) (note: all lowercase) and apparently also a bogus self-executing function that calls it

(() => {
  let app=Application("DEVONthink 3");
  performsmartrule(app.selectedRecords());
})()

Timetrinus · January 13, 2021, 2:30pm

Wow.

My charisma is project management not programming… I was happy to find out (with the help of some good examples here) a little bit about applescript. I’m a little bit familiar with python, bash scripting and at least ANSI-COBOL But I’m completely new to JavaScript, ObjCAppleScript and whatever …

A small sample script would help me a lot…

Timetrinus · January 13, 2021, 3:33pm

Thank you for you suggestion. I know I can use regex in Scan Text action and sed from command line. What I miss is the option to do some if then else or select case or whatever.

I think it can’t be done without a script inside a smartrule. Here is a very simplified example for what I wanna do:

Is documents content “Pfefferminzia”? Then call a external script:

If the document contains the stirring “contribution invoice” then set the document type to “contribution invoice”
If not
Does the document contain the string “contract”?
if so then set the document type to contract documents

u. s. w.

cgrunenberg · January 13, 2021, 3:40pm

Smart rules don’t support this, therefore the only option is a script executed via the Execute Script action.

BLUEFROG · January 13, 2021, 4:18pm

Is documents content “Pfefferminzia”? Then call a external script:

If the document contains the stirring “contribution invoice” then set the document type to “contribution invoice”
If not
Does the document contain the string “contract”?
if so then set the document type to contract documents

It sounds like you just described three different smart rules.

Timetrinus · January 13, 2021, 6:13pm

Sure. that’s how I do it right now. But maybe I haven’t clearly described it in this course: That’s exactly what I would like to avoid, because multiple rules ultimately mean that I would have to create around 50 - 60 rules.

Timetrinus · January 13, 2021, 10:17pm

where do I find an overview or a programmers reference to interact with devonthink?

rmschne · January 14, 2021, 7:12am

There is a whole chapter in the Manual for “Automation”. Same content in Help.

FYI, I use Hazel for all this complicated scanning of the PDF’s to make rules about changing files names and all that. As good as DEVONthink rules are, to do anything fancier than what is in the Rules requires scripting in some language. I can’t be bothered. Hurts my brain. I use Hazel (a separate app available on he App Store).

I wrote how I integrate Hazel and DEVONthink think here

This approach might be useful to you. Or plough ahead with what you are doing…

I can’t read your rules (in German and I don’t understand that language), it seems like you are setting up some very specific groups based on content. I don’t do much of that. I have some high level Groups in a database (or specific databases), e.g. dump all the Insurance contracts into one Group, then use DEVONthink’s rules to find a specific document by searching for the contact number, client, or content. It used to be in the days of paper filing cabinets that we needed all that structure in filing systems, but those days are past. Just a few file drawers and then search. Searching was hard in the paper file days. No longer. Just my two bits.

Timetrinus · January 15, 2021, 6:06pm

So I spent some time to drive into JavaScript and that’s the result:

function performsmartrule(recs) {

	app = Application("DEVONthink 3");
	records = app.selection()
	records.forEach (r => {
		t = r.plainText()
	
		myRegexp = /(([L]).??(\d{6}) ?(\d{3}) ?(\d{3}))/
		match = myRegexp.exec(t)
		if (match != null) {
			contract = match[2] + " " + match[3] + " " + match[4] + " " + match[5]
		} else {
			contract = "unknown"
		}
		if (t.match(/\nVersicherungsschein /))  {
			doctype = "Versicherungsschein"
		} else if (t.match(/\nErhöhung\n/)) {
			doctype = "Dynamik Anpassung"
		} else if (t.match(/\nInformation zu Ihrer/)) {
			doctype = "Info zur Überschussbeteiligung"
		} else if (t.match(/\nNachtrag\n/)) {
			doctype = "Versicherungsschein Nachtrag"
		} else {
			doctype = "Korrespondenz"
		}
	
		r.customMetaData = Object.assign(
			{},
			t.customMetaData,
			{
				['mddokumententyp']: doctype
			}
		)	

		myRegexp = /Nürnberg, ((\d{2})\.(\d{2})\.(\d{2,4}))\n/g
		match = myRegexp.exec(t)
		if (match != null) {
  			ddate = match[1]
		} else {
			myRegexp = /((\d{2})\.(\d{2})\.(\d{2,4}))\n/g
  			match = myRegexp.exec(t)
			if (match != null ) {
				ddate = match[1]
			} else {
				ddate = "unknown"
			}
		}
})
}
(() => {
  let app = Application("DEVONthink 3");
  performsmartrule(app.selectedRecords());
})();

If I run it from scripteditor everything is fine but when I run it from the smartrule as external script I just get an error:
18:58:11: ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/DEVONthink Nuernberger Versicherung.scpt on performSmartRule (Fehler -1708)

What did I wrong?

chrillek · January 15, 2021, 8:03pm

Difficult to say. The error code doesn’t tell anything. Just three points

records = app.selection()/records.forEach is not necessary, recs.forEach is just fine, since recs is the current selection (or rather: the records matched by the smart rule’s conditions.
t.customerMetaData in Object.assign(...) should always throw an error, because of t = r.plainText() - t is a string, and a string does not have a property customerMetaData. Unless you do something magical behind the scenes
as to the metadata part, I do it like that in one of my JS scripts:

app.addCustomMetaData("theString", {for: "theDataField", to: record});
/* In your case: */
app.addCustomMetaData(doctype, {for: 'mddokumententyp', to: r});

That works, but it is a simple script, not an external one in a smart rule. You could pepper your code with

app.logMessage(...)

to find out where the error occurs. This writes messages to DT’s log window.

Timetrinus · January 15, 2021, 9:32pm

Oh thank you so much! It’s doing what I want for the beginning. Now I have an idea about how to start these things and next steps are some finetunings.

One more question regarding:
app.addCustomMetaData
Where do I find some information / documentation about what objects, properties and methods are available to do the automation?

chrillek · January 15, 2021, 10:40pm

Open the ScriptEditor, then Files/Libraries (I don’t know the exact name) and choose DEVONthink 3. In the documentation, change the language at the top to JavaScript.
However, some of the more exotic methods don’t work as expected. You’ll find something on that in a recent thread in the automation subforum.

Timetrinus · January 18, 2021, 10:17pm

So some more hours of deep diving into JS … The script is now doing what it shoud do but just if I run it stand alone from the scripteditor. If I us it within a smartrule as an external skript it causes DT to hang and crash. So I removed everything from the script, step by step to find out what causes this behaviour. But I didn’t found it. So I created a testscript:

function performsmartrule(records) {
  let app=Application("DEVONthink 3");
  records.forEach(r => {
  	app.logMessage(r.name());	
  });
}

(() => {
  var app = Application("DEVONthink 3");
  app.includeStandardAdditions = true 
  performsmartrule(app.selection()); 
})();

but this also causes DT to crash when I use it as external script in a smartrule. What’s wrong?

pete31 · January 18, 2021, 11:18pm

I don’t know JavaScript, however Smart Rules don’t process a selection so I guess this line is not correct:

performsmartrule(app.selection());

Or is this just for testing with selected records?