Trouble getting all tags with JXA whose query

Just when I thought that I was understanding, I ran into a problem that I have isolated into the following code snippet:

(() => {
	const dt = Application('DEVONthink');
	const db = dt.currentDatabase;
	
	// get all tags in db
	const myParents = db.parents.whose({_match: 
		[ObjectSpecifier().tagType, "ordinary tag"]});
	console.log(`tags: ${myParents.length}`); // 1568
	
	const pNames = myParents.name();  // 1 AppleEvent 
	const pUuids = myParents.uuid();  // 1 AppleEvent
	const pTagTypes = myParents.tagType(); // 1 AppleEvent
	const pRecordTypes = myParents.recordType(); // 1 AppleEvent

	
	console.log(`myParents (all tags): ${pNames.length}`);
	console.log(`myParents (${pNames.length}): ${pNames}`); 
	
	let badRecordList = [];
	pTagTypes.forEach((tagType, index) => {
		if (tagType !== 'ordinary tag') {
			badRecordList.push({
				name: pNames[index], 
				uuid: pUuids[index], 
				recordType: pRecordTypes[index],
				tagType: pTagTypes[index]
			});
		}
	});	
	console.log(`captured ${badRecordList.length} bad records - they aren't tags!`); // 446
	console.log(JSON.stringify(badRecordList, null, 4)); // all {group, 'no tag'}
})()

Most of it is scaffolding to capture the error. The key line is:

// get all tags in db
const myParents = db.parents.whose({_match: 
	[ObjectSpecifier().tagType, "ordinary tag"]});

I expected that this code would return all tag groups in 1 Apple Event.

I have three “production” databases, and this code works as expected in 2 of the 3. However in the third, this code returns the tag groups - but a slew of other groups as well.

The console output is:

tags: 1568
myParents (all tags): 1568

myParents (1568): veh-USMC Trailer,DMV,doc-Title,doc-Registration,summaryNote,.pmf,pmf,showNumbers,Ahmet Kazan - Handicraftlamps,Arctic Scandinavia map & Travel Tips from Discover the World,
...
August/September 2017,Flights: LAX-SEA-KTN August 7-12, 2018

captured 446 bad records - they aren't tags!
[
    {
        "name": "Ahmet Kazan - Handicraftlamps",
        "uuid": "9240BC99-3CDD-4EA0-AF54-EA6FDD878B56",
        "recordType": "group",
        "tagType": "no tag"
    },
    {
        "name": "Arctic Scandinavia map & Travel Tips from Discover the World",
        "uuid": "690D9780-7881-49BB-9345-5EDB9C8A102F",
        "recordType": "group",
        "tagType": "no tag"
    },
    {
        "name": "Abisko Boutique - Swedish Sports Clothing & Gloves",
        "uuid": "7F29ABDD-F8A4-4D96-9BEA-613F5C05ABA4",
        "recordType": "group",
        "tagType": "no tag"
    },
...
        "name": "Flights: LAX-SEA-KTN August 7-12, 2018",
        "uuid": "072BD863-9F38-4B23-A729-99995F86ECCD",
        "recordType": "group",
        "tagType": "no tag"
    }
]


So, there are 446 regular groups that get caught up in the net. They are all {group, no tag}, so I don’t understand. I don’t (intentionally) use groups for tagging and I would have expected these to show as {group, group tag} in any case. Clearly I have a data dependent problem, so it may be hard for anyone else to reproduce.

The good news is that it is blazingly fast for producing hierarchical tag listings in the databases where it does work.

Any ideas @chrillek or @cgrunenberg ?

What happens if you try this with a search query? If that’s even possible, that is.

So, my faith in logic has been restored since the problem has gone away after rebuilding the offending database.

I had no real reason to suspect this database other than this strange error shown above where non-tags would be returned from a whose clause targeting tags via (parents with an “ordinary tag”), but then report the correct properties when those properties were targeted individually to report the error, i.e. {group, “no tag”}.

My only inkling was from the following behavior with the built-in integrity checks:

  1. Verify & Repair Database - always finished successfully, reporting Successfully verified in the log window, but…
  2. Check File Integrity - always ran, seemingly to completion according to the progress indicator, but often finished silently with nothing reported to the log window, but sometimes it would show Successfully verified the integrity of 29147 files.

Hopefully there is a clue somewhere in there for a developer?


---------------

I have a follow up question regarding the Rebuild Database… process that I have just completed.

I did this by first closing DEVONthink, duplicating my •PARA.dtBase2 database in the Finder, then loading the duplicated •PARA copy.dtBase2 in DEVONthink and running Rebuild Database…

I believe that during the rebuild process, I only had •PARA in the Open Databases section, as I would expect.

However, after completing the rebuild I now have all of my databases open:

I wasn’t watching at the critical moment when the lengthy process completed, but presumably DEVONthink relaunched, opening my other databases.

The confusion is that I now have •PARA instead of •PARA copy shown in the Open Databases section. I presume that this is really the •PARA copy database.

I just want to be 100% sure before deleting the original database, and renaming the copy as the original in the Finder.

Only the database that is being rebuilt will be closed during the process and then reopened.

I actually closed DEVONthink myself, duplicated the database in Finder, then clicked on the duplicate to open DEVONthink with that duplicate. I then rebuilt the duplicate.

My question is, “can I safely delete the original database in Finder, and rename the duplicate with the original’s name?”.

Sure. Why not?

Thanks Jim, just making sure! :+1:

I have read that the inbox uuid might change and custom metadata might not be preserved during a rebuild. Ate there any other things to be aware of? This is the first time that I have rebuilt a database, so have no personal experience.

  • If you duplicated a database file in the Finder, it retains the UUID.
  • A rebuild does not change the UUIDs of a database or its contents.

Ate there any other things to be aware of?

Yes. No one should habitually duplicate a database in the Finder.