Archive group: Script to create new database with selected group and archive new database

DEVONthink Automation
,
2

Nearly everything is scriptable, so the answer to

would be “most probably”.

Well, if you create a “text file or markdown”, exporting that as “JSON” seems to be a bit weird. But it’s possible. Only: What is it that you want to do with it later? Why build a text or MD file first instead of doing JSON right away?

What bugs me is this:

  • what exactly do you have? It seems to be a group containing indexed files. So, it’s perhaps an indexed folder? Or is it just a collection of indexed files? Or is it a group containing records and other groups, all of them indexed? Then you want to “create the content” of this group – what is this “content”? The content of all the records? Perhaps not, but somehow “the metadata overview” (which metadata, exactly?), and “index functions” – which index functions?
  • The outcome of this operation is to be a text or MD file which seems not to be used later on.

It seems to me that you want to move a “group” of indexed objects into their own database, archive this database, and then delete the original. I do not really get the point of having a ZIPped database somewhere which you do not use. If you need a database, why delete it? If you do not need the database (since you deleted it), why build it in the first place instead of simply ZIPping the content of the group and saving that somewhere? If you need a database from that content later on, build it then.

Maybe an image or something of this ominous text/MD file you want to create in the first step would clarify things.

1 Like
3

So are you trying to archive a portion of your database but also create a record of the contents for you to refer to outside of DEVONthink?

4

Thanks for your answers…

… to clarify my request:

  1. I want to archive a certain group (“file”) (= in general one indexed folder with files/foleders inside) and all the (custom) metadata of this files/folders (hereinafter I call this “data”). For legal reasons i have to keep this data, but i don’t need to work on it anymore. I have to separate every “file” for the archival purpose. That’s why i wanted to create a new database with the the data within.

  2. If I should need this data again (for legal reasons ;-)), I would like to have the data in Devonthink with all the (custom) metadata. I could also zip the indexed files and folders in the Finder or on the server, but then I don’t have any (custom) metadata (which ist very very useful; thanks to this again! :pray:).

  3. The text/md-csv files (from menu > table of contents (in german: Inhaltsverzeichnis) and > metadata overview (in german: Metadaten-Übersicht)) i want to use for a better overview without unzipping the database archive, if i need it for some reason. The JSON file i need to import the metadata data to another database application. For the last purpose i could also use the metadata overview csv, but there a some metadata fields missing (e.g. path).

Yes, that is also one reason. In case I used links, I would like to be able to use them after a database restore.

Thanks for your time!

5

BTW: Custom metadata with type formatted text are exported to the JSON file in a cryptographic form (because it is formatted). Is there a hidden preference feature to export this custom metadata as plain text?

6

So, basically you want a tree-like listing of all your files/folders with all the metadata. I’m not convinced that this easily done with MD/text or CSV. Suppose you have a structure like this

Folder1
   |---------File 1
   |---------File 2
   |---------Folder 2
                 |-----------File 1
                 |-----------File 2

then File 1 and File 2 would appear in two different folders – same name, different entities. Therefore, JSON might be the better choice. A basic (very basic) script might look like this:

(() => {
  const app = Application("DEVONthink 3")
  app.includeStandardAdditions = true;
  const initialGroup = app.getRecordWithUuid('28258F1A-FC1D-49C6-97B4-F5C8DA21265B');
  const metadata = [];
  getMetaData(initialGroup, metadata);
  console.log(JSON.stringify(metadata));
})()

function getMetaData(record, metadata) {
//  console.log(group.name());
  if (record.type() === 'group') {
    const groupInfo = {name: record.name(), type: "folder", metadata: record.customMetaData(), content: []};
    metadata.push(groupInfo);
    record.children().forEach(r => {
      getMetaData(r,groupInfo.content)
    })
  } else {
    const fileInfo = {name: record.name(), type: "file", metadata: record.customMetaData()};
    metadata.push(fileInfo);
  }
}

In this incantation, the script works on a particular group defined in the call to getRecordWithUuid. It outputs a hierarchical structure of folders and files representing all the groups and records included in the initial group, including all the custom metadata. You can, of course, add tags and other metadata like title, creation date, whatever. That would have to happen in the lines setting up groupInfo and fileInfo;

As to the path missing from the metadata overview: given that your data can reside anywhere in your file system, and you’re planning to move it around after saving the metadata, I doubt that this information is particularly useful.

As to “formatted text” in metadata: According to the documentation, that’s “Rich text”. You could use the command line utility textutil to convert it. But after I saw what a rich text field gives in JXA and AppleScript, I’d give up all hope on that one. Perhaps @cgrunenberg can shed some light on that issue? What I see here (in script editor) is this:

tell application "DEVONthink 3"
	get record with uuid "A95457F9-C064-4793-9906-16CA534DA2A7"
	get custom meta data for "mdrt" from content id 84902 of database id 2
Ergebnis:
error "«data ****72746664000000000300000002000000070000005458542E727466010000002EBC0100002B00000001000000B40100007B5C727466315C616E73695C616E7369637067313235325C636F636F61727466323730380A5C6
…000000000000000» to text

(That’s for AppleScript, in JXA the result is similarly unusable).

1 Like
7

Here’s how it can be done in AppleScriptObjC

-- Get plain text from RTF(D) Custom Meta Data

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property theCMD_Identifier : "testrtf" -- set your Custom Meta Data identifier

tell application id "DNtp"
	try
		set theRecords to selected records
		if theRecords = {} then error "Please select some records."
		
		repeat with thisRecord in theRecords
			set thisRecord_CMD_RTF to get custom meta data for theCMD_Identifier from thisRecord
			
			if thisRecord_CMD_RTF ≠ missing value then
				set thisRecord_CMD_RTF_plainText to my getPlainTextFromRTFDdata(thisRecord_CMD_RTF)
			else
				set thisRecord_CMD_RTF_plainText to ""
			end if
			
			-- do something with variable "thisRecord_CMD_RTF_plainText"
			
		end repeat
		
	on error error_message number error_number
		if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
		return
	end try
end tell


on getPlainTextFromRTFDdata(theData)
	try
		set theNSData to (current application's NSArray's arrayWithObject:theData)'s firstObject()'s |data|()
		set {theAttributedString, theError} to (current application's NSAttributedString's alloc()'s initWithData:theNSData options:(missing value) documentAttributes:(missing value) |error|:(reference))
		if theError ≠ missing value then error (theError's localizedDescription() as string)
		set theString to theAttributedString's |string|()
		return theString as string
	on error error_message number error_number
		activate
		if the error_number is not -128 then display alert "Error: Handler \"getPlainTextFromRTFDdata\"" message error_message as warning
		error number -128
	end try
end getPlainTextFromRTFDdata
1 Like
8

Ok, thanks so much. Will try to enhance that…

I’ll never used JS in Devonthink. The documentation says it cannot be used like AppleScript. So can JS only be used in smart rules?

Documentation >

JavaScript: In addition to AppleScript, JavaScript for Automation (JXA) is supported. However, it is bridged by the operating system from AppleScript, not specifically coded for.

9

JXA is supported in scripts, not just smart rules. But we do not create a script definition file for it. The scripting definition is for AppleScript but it’s converted when switching the scripting language to JavaScript.

1 Like
10

Sure, of course! :man_facepalming: Sorry!

11

Thanks for that. I’d of course like to replicate that in JXA, but I’m stumbling upon this line:

(current application's NSArray's arrayWithObject:theData)'s firstObject()
should be a (variant of) NSString. Or at least it is when I rewrite that line in JavaScript. But I don’t understand what the |data|() at the end is doing. It looks like a function/method call for data, but I don’t find that function/method in NSString. Simply replicating the code as

const nsdata = $.NSArray.arrayWithObject(theData).firstObject.data;

gives me a nil object.

12

Never cared about what it’s needed for, but turns out someone in the thread asked exactly that. Here’s Shane’s explanation:

When you create an array, any non-bridgeable AS items in it get converted to NSAppleEventDescriptors. So it’s effectively calling the -data method on an event descriptor.

(https://forum.latenightsw.com/t/convert-nsdata-into-raw-data-string/1914/7)

The only difference I can see between my AppleScript and your JavaScript is that I got brackets around the array part. What do you get when you use

const nsdata = $.NSArray.arrayWithObject(theData)

? What’s in the array?

13

an NSCFSting. It seems that we’re seeing again a problem that has arisen before: JXA receives a simple string from OSA, while AppleScript gets something special that can then be converted into a NSData object.

14

No worries! An easy misunderstanding.

15

Just to get some kind of closure here:

  • JavaScript and AppleScript are both based on Apple’s Open Scripting Architecture (OSA)
  • DEVONtechnologies even published a blog entry on using JXA with DT: DEVONtechnologies | How to Use JavaScript for Automation
  • Therefore, I find the wording in the documentation a bit awkward (JavaScript is, afaik, not bridged from AS, btw).
  • AppleScript is the older language, and it’s considered by many to be easier to learn
  • AppleScript is lacking a strict syntax definition
  • JavaScript is newer, it’s in use everywhere now (Web, desktop, whatever), but considered by many more difficult to learn
  • JavaScript is constantly developed by a standardization body and has a strict syntax definition.
  • Both are more or less dead in the water in terms of OSA: Apple is not doing anything with/to them anymore.
  • JavaScript for Automation (i.e. the OSA incantation of JavaScript) has never been finished, much less polished. There are some things that should work but don’t
  • One is the case of “formatted text in custom metadata”: In AppleScript, the value is an AppleEvent(Descriptor?) that can be converted to a string, as @pete31 demonstrated.
  • This doesn’t work in JXA (at least I can’t make it work). The only workaround would be to call the AppleScript code from JXA, which in fact works just fine.

Since producing JSON is a lot easier in JavaScript, I’d still suggest to stick with that for your purpose. If need be, we can continue the discussion in private.

1 Like
16

Today i had a little time to start this “project” …

Starting with the Applescript:

tell application id "DNtp"

	set selectedGroup to item 1 of (get selection)
	
	set documentsPath to path to desktop
	
	set dtaFolder to documentsPath & "Aktenablage"
	if not (exists dtaFolder) then
		make new folder at documentsPath with properties {name:Aktenablage}
	end if
	
	set newDatabase to create database with properties {name:(name of selectedGroup), path:dtaFolder as alias}

end tell

Result is a syntax error in the last line. If i remove the properties result is “missing value”…

Could someone give me an advice? Thanks…

17

Did you check that the various variables are what you want them to be? Running the code in Script Editor with its protocol tab active helps in figuring that out.

And perhaps the name parameter of the male should be a string? But I guess this branch of the code has not been executed.

18

Thanks for your help…

Creating the database with the name of the selected group works for me with:

tell application id "DNtp"
	set selectedGroup to item 1 of (get selection)
	set groupName to name of selectedGroup
	set desktopPath to POSIX path of (path to desktop) & groupName & ".dtbase2"
	set newDatabase to create database desktopPath
end tell
19

I’m assuming encrypted databases can’t be created via Applescript. Correct?

But is it possible to disable the spotlight index via Applescript?

20

Note: Your desktop isn’t a good permanent location for databases and working files.

I’m assuming encrypted databases can’t be created via Applescript. Correct?

That is correct. There is no direct command for this. And the encrypted property is read only.

But is it possible to disable the spotlight index via Applescript?

No. There is not a property for this.

21

We’ll consider this for future releases.