JXA Sometimes Importing to Wrong Location

[rkaplan]

I have a question that perhaps some of the JXA gurus can help me with.

@8isnothing is helping me to write an API to access DT3 via the web and via Zapier. (I will post details here to share when done.). We have the download API endpoint working and are very close to having the upload endpoint working. But the uploaded documents are going into the root of the desired database instead of into the Group identified by UUID.

This is Python code which calls JXA inside the API:

This is the JXA script which is called by Python:

We know that this script correctly receives data from Python since it can identify the correct database and can also correctly add tags and custom metadata. But the problem is that DT is recognizing the UUID but importing the files to the root folder of the desired Group instead of to the desired Group itself.

Interestingly however this is a standalone JXA script which uses the same logic. If we run this script from Terminal or Keyboard Maestro or Textexpander, then the file does go to the correct Group. It only fails to go to the correct Group when the process is initiated from Python to JXA.

Any thoughts on this behavior?

[cgrunenberg]

There’s a typo in the first script (lenght instead of length), maybe that’s causing the issue?

[chrillek]

It would have been advantageous to include the script code verbatim (i.e. included in code fences ```) here, so that others could run it easily.

Apart from the spelling mistake @cgrunenberg noted already, I’d suggest to write the data you pass into the script to console.log or to a file or something. The only reason I can see that this is not working is that myDestUUID != "" is in fact true.

@rkaplan

But the problem is that DT is recognizing the UUID but importing the files to the root folder of the desired Group instead of to the desired Group itself.

How do you know that DT is recognizing the UUID? And is that UUID really a group? On a related note, wouldn’t it be easier to use group names (or hierachies) instead of UUIDs (in terms of readability and maintainability)?

What is the Object.keys(myDestGroup).length === 0 code supposed to check for? Also, why are you sometimes using “===” to compare for equality and sometimes “==”? Hint: You’d usually want to use “===” and “!==”: Which equals operator (== vs ===) should be used in JavaScript comparisons? - Stack Overflow

[rkaplan]

Thank you both - those are good ideas - we will look into those and get back with results.

[rkaplan]

@chrillek

How do you know that DT is recognizing the UUID? And is that UUID really a group? On a related note, wouldn’t it be easier to use group names (or hierachies) instead of UUIDs (in terms of readability and maintainability)?

Yes the UUID is really for a group - not only is this something I can personally verify in the database, but also it works in the standalone script to add the document to the group.

We know DT is recognizing the UUID because when we run the script in the API (using Python), it adds the document to the Inbox of the database which contains the desired destination group. If I try it with a different UUID representing a group in a different database, then it adds the document to the inbox of the corresponding database.

The reason to use a UUID rather than a group name is that names are not unique. For example if this is a database of client cases then I might have a main case group of “Doe, John” with subgroups of “Doe, John Urgent to Review” and “Doe, John Completed.”

Although the API will be usable on its own if desired, I am working on a web app front end using www.retool.com which displays the group names so for regular use I will not need to reference UUIDs if I choose not to do so.

I will let @8isnothing respond to your other specific questions on coding technique.

[chrillek]

@rkaplan

The reason to use a UUID rather than a group name is that names are not unique. For example if this is a database of client cases then I might have a main case group of “Doe, John” with subgroups of “Doe, John Urgent to Review” and “Doe, John Completed.”

That’s why I said “group name (or hierarchies)”. “Doe, John/Doe, John Urgent to Review” should be unique.

@rkaplan

We know DT is recognizing the UUID because when we run the script in the API (using Python), it adds the document to the Inbox of the database which contains the desired destination group.

That might just be a coincidence (this database being the “currentDatabase”, for example). I still suggest to console.log(theUUID) and to check that getRecordWithUuid(theUUID).type() === "group"

[rkaplan]

@chrillek

That’s why I said “group name (or hierarchies)”. “Doe, John/Doe, John Urgent to Review” should be unique.

If I use the UUID, then it is easy to do “Copy Item Link” and paste that into the appropriate query in the app (which actually accepts either pure UUID or an x-devonthink link). If I have to enter the group hierarchy, then there is no simple way to grab that.

That might just be a coincidence (this database being the “currentDatabase”, for example). I still suggest to console.log(theUUID) and to check that getRecordWithUuid(theUUID).type() === "group"

Agreed - we will try this if the other suggestions do not work. Thanks.

[winter]

As an aside, the use of exceptions for control flow tends to be seen as an anti-pattern. Google, for example, the discussion on:

anti patterns - Are exceptions as control flow considered a serious antipattern? If so, Why? - Software Engineering Stack Exchange

Here, I think the .length typo was probably harder to notice inside the black hole of a try ... throw error exception wrapping, where it would create the impression that the UUID was never being found.

(usually best to reserve exceptions for interactions with IO)

[chrillek]

I have a minimal example (which is always! the way to go) here:

function doIt(args) {
  dt = Application("DEVONthink 3");
  const file = args[0];
  const uuid = args[1];
  var tags = args[2];
  var meta = args[3];
  var params = {};
  if (uuid != "") {
    var destGroup = dt.getRecordWithUuid(uuid);
	if (destGroup.type() === "group") {
	  params.to = destGroup;
    }
  }
  const record = dt.import(file, params);
  console.log(`record ${record.name()} created in group ${record.parents()[0].uuid()}`);
}

doIt(["/Users/ck/Fotos ohne Location.scpt"), "FFC907C5-EF78-4934-B063-A6DA61061E39", "",""]);

that works ok. I didn’t do anything about the tags and metadata stuff, since you mentioned that your problem is the import.

Now that that’s out of the way, I suppose that the UUID is not passed from your Python code to the JXA script as you expect it.

[8isnothing]

Thanks for the insights, guys.

1. Thanks for pointing the ‘length’ typo.

2. Indeed, DEVONthink could be importing to the active database. It may be a coincidence. Great insight!

3. We had console.log before. It was stripped out after my personal tests, but it may be a good idea to bring it back.

4. Object.keys(myDestGroup).length === 0 is used to check if the return of getRecordWithUuid is not empty. As far as I tested, without it the code won’t return an error.

The most important and curious detail about this problem is that it happens in @rkaplan 's computer, but it works perfectly on mine. We are using exactly the same code (using github as source) and somehow software behaves differently. Also, as pointed out by @rkaplan, the standalone version of the script (I’ve made it just to debug the import feature) works flawlessly on both my and @rkaplan 's ends.

I’ll post the code again (as code, not as images) if that helps.

We have some other API endpoints that passes UUID from Python to JXA using the same logic as the code posted, and it works fine.

Question:
I was wondering: if this problem could be related to some DEVONthink configuration. Since the standalone script to import files work, we can discard this possibility, right?

[winter]

Does the same group necessarily have the same UUID on both systems ?

(I can easily imagine the UUID being locally particular, even more so if the editing history of the two instances diverges at all).

Might be worth identifying groups by paths rather than UUIDs ?

[8isnothing]

We have different databases. So the group UUID is definitely different.

Groups by path won’t be practical for this use case. We should really go with UUID. As I said, that method per se is not problematic. We are only facing issues with this import being called from Python.

[rkaplan]

@chrillek

That might just be a coincidence (this database being the “currentDatabase”, for example)

Very interesting theory - just checked that - it goes to the inbox related to the UUID regardless of what the current database is

[chrillek]

Very, very stupid question: Is the group in question may be “locked” or in some other way protected for writing?

The bad thing is that import is apparently not raising any error. It was driving me mad, because I tried to import a file that never appeared in DT and everything seemed to be ok. In fact, I’d forgotten the file extension, so what I passed to import did not exist in the filesystem…
@cgrunenberg: Maybe import could throw a error if the file can’t be imported, possibly indicating the reason for it? I know that I can check the returned record (which is null if something went wrong), but that does not tell me anything about the possible problem.

[rkaplan]

@chrillek

Very, very stupid question: Is the group in question may be “locked” or in some other way protected for writing?

Interesting, not stupid - but no it is not locked. And I have the same problem when I try it with various groups in different databases. And it works when we do the standalone script; the problem is only when Python is calling the script…

[chrillek]

Yes, I tried the ‘lock’ idea here and it works regardless of the group’s lock status.

Next question: Is/are your python versions identical? Did you try to run the script without python from the command line, something like

osascript -l JavaScript scriptfile "yourfile" "yourUUID" "" ""

(the last two empty strings are just the tags and metadata, which are not needed, I think). If that works, I’d go and fiddle around with the Python stuff.

[winter]

@8isnothing

Object.keys(myDestGroup).length === 0 is used to check if the return of getRecordWithUuid is not empty.

When there is no match, and you want to return an explanatory message string, rather than trigger an event, an alternative here is to branch on:

myDestGroup === null

or

myDestGroup !== null

(when getRecordWithUuid finds no target, the value returned is null rather than a reference).

[cgrunenberg]

@chrillek

@cgrunenberg: Maybe import could throw a error if the file can’t be imported, possibly indicating the reason for it? I know that I can check the returned record (which is null if something went wrong), but that does not tell me anything about the possible problem.

Thanks for the suggestion! While this might theoretically be nice, it might also break many existing scripts which do not expect this.

[rkaplan]

@chrillek

Did you try to run the script without python from the command line, something like

When I run that script from the command line without Python, it goes to the database inbox, not to the desired group.

Same Python version for both of us

[8isnothing]

Here’s the JXA code, in case someone can help us test it (I’ve replaced the ‘length’ as suggested):

(Please notice that ‘myDirectory’ is the path to the file/folder to be imported.)

function run(args) {
	// app variables
	const dt = Application("DEVONthink 3"); dt.includeStandardAdditions = true;
	
	// Get args
	const myDirectory = args[0]
	const myDestUUID = args[1]
	var myTags =  args[2]
	var myCustomMeta = args[3]
	
	if (myTags == "") {
		myTags = []
	} else {
		myTags = JSON.parse(myTags)
	}
	
	if (myCustomMeta == "") {
		myCustomMeta = {}
	} else {
		myCustomMeta = JSON.parse(myCustomMeta)
	}
	

	
	if (myDestUUID != "") {  // Check if MyDest is set
		try {  // Check if myDestUUID exists
			myDestGroup = dt.getRecordWithUuid(myDestUUID)
		
			if (myDestGroup === null) {
				throw "Error"
			}
		} catch (e) {
			console.log("Couldn't find group with uuid " + myDestUUID)
			return Error("Couldn't find group with uuid " + myDestUUID)
		}
		var myImportedRecord = dt.import(myDirectory, {to:myDestGroup})
	} else { // Default import in case it isn't
		var myImportedRecord = dt.import(myDirectory)
	}
	
	myImportedRecord.tags = myTags
	myImportedRecord.customMetaData = myCustomMeta
	
	return myImportedRecord.uuid()
	
}