Moving from Mariner Paperless to DevonThink

[soffties24]

Thank you very, very much for your explanation re accessing Paperless files and opening them in DEVONthink. I followed them and they worked perfectly.
Like you, I found my files in folders by year/month. I have about 2,000 files in about 100 different folders and it is not really practical to open each file to find out what it is. Have you found any solutions as to how to identify an individual file short of clicking on it to open it?

[jimmy1]

I know this is a reply to an old document, but when I try to run this, I get an error:
error "Can’t set «class DTco» of missing value to \"\"." number -10006 from «class DTco» of missing value

When I call it with:
my addDocument("25-07-2011.pdf", "", "2011-10-19 15:35:39", {}, "CSFB")

and the link highlighted is the first. i.e: set comment of dp to notes
Is there something I need to run first. Devonthink 3 is running in the background when I run the script.

Any help would be gratefully received. Thanks.
(updated to clean the text up)

[chrillek]

Please include code snippets in backticks (`) to prevent useless “smart” stuff like quote substitution.
The way it looks now, we can’t see what you really used to call the function.

[jimmy1]

Updated the text to clean it up…

[chrillek]

@jimmy1

my addDocument(“25-07-2011.pdf”, “”, “2011-10-19 15:35:39”, {}, “CSFB”)

I can’t reproduce the issue. Here’s my code (using another file, of course):

on addDocument(filePath, notes, importDate, docTags, title)
	tell application id "DNtp"
		set dp to import filePath to (root of database "Test")
		set comment of dp to notes
		set creation date of dp to importDate
		set tags of dp to docTags
		add custom meta data title for "Title" to dp
	end tell
end addDocument

my addDocument("/Users/ck/Desktop/Bildschirmfoto 2024-11-28 um 10.10.49.png", "", "2011-10-19 15:35:39", {}, "CSFB")

If your issue persist, I suggest you run the code in Script Editor and turn on its protocol (Cmd+3).
But I guess that you did in fact use curly quotes in your call to addDocument.

[meowky]

DEVONthink’s AppleScript dictionary explains that import should be followed by a POSIX path (see chrillek’s example). In your instance the filePath is not a valid POSIX path, hence the error.

Also, the error message clearly indicates that dp is missing value, which implies that the import command has failed for some reason. Always try to decipher the error message – it is provided to you for a reason!

[jimmy1]

I now feel so foolish. I didn’t; know about CMD-3 (very much a novice when coming to script/SQL) but it showed up the error. My python program had a typo on the file path. It was only when I tested it on a simple string to compare to your example that I saw the error. ‘paperles’ and not ‘paperless’.

Anyway, thank you for your patience and advices, and now I can try to migrate in earnest…

Jimmy

[BLUEFROG]

I would recommend you try this first…

Moving from Mariner Paperless to DevonThink

I wanted to weigh in here from a non-coding lay(woman) who only has a basic understanding of how DEVONthink works, knows no coding outside of basic html, is coming from Paperless (and prior to that Neat) and just wants all paperless documents in one program. For a variety of reasons I do not use the full power of these tools, and rely 100% on the search feature provided by Optical Character Recognition (OCR) capabilities to find what I’m looking for. The coding above and conversations being he…

[chrillek]

@meowky

In your instance the filePath is not a valid POSIX path

Why not?
“25-07-2011.pdf” is a PDF in the current directory, equivalent to “./25-07-2011.pdf”. But what the current directory is, is not clear – it depends very much on the environment the script is run in.

[cgrunenberg]

@chrillek

But what the current directory is, is not clear – it depends very much on the environment the script is run in.

Whatever the current environment of the script might be, it’s not the environment of DEVONthink. Therefore the path has to be absolute or relative to the user directory.

[laguna_bob]

I just came across this discussion of importing Paperless files into DT. I am also trying to import files that need an “amount” and “document date” and saw that you seemed to have adapted chrillek’s script to achieve the import. I have been able to use chrillek’s script succesfully, but wondered if you’d post your revision; my coding experience is limited to a Fortran course I took 50 years ago, though I do have a limited understanding of what is happening here, especially using ChatGPT to decipher.

[MickeyT9]

I love that you said, exactly how I felt too…

The coding above and conversations being held are WAY over my head so I had to find another way to simply get my files into my system as easy as possible and I wanted to share incase this may help someone else.

Your system sounds very promising to me… I shall give it a crack!.. Thank you.

[PaulC]

I am also trying to move from Paperless to DEVONthink. Finding and dragging in the appropriate PDFs is fine. I have set up Paperless to include metadata in the PDFs and they appear as keywords in DEVONthink. They appear as below. Can I write a script to convert Keywords into Data fields. I have created data fields to match the fields I used in Paperless and which might appear in the DEVONthink keywords. Can I write a script to read the keywords, taking everything to the left of the “=” as the field name and everything to the right as the data? I thought I saw a script for that in this or another thread but cannot find it now. It would be very useful to me to preserve the metadata that I entered into Paperless. Thousands of records, I’m not up to entering this metadata by hand.

[BLUEFROG]

Welcome @PaulC
This has already been discussed but I would start with the post higher up in this thread…

Moving from Mariner Paperless to DevonThink

I wanted to weigh in here from a non-coding lay(woman) who only has a basic understanding of how DEVONthink works, knows no coding outside of basic html, is coming from Paperless (and prior to that Neat) and just wants all paperless documents in one program. For a variety of reasons I do not use the full power of these tools, and rely 100% on the search feature provided by Optical Character Recognition (OCR) capabilities to find what I’m looking for. The coding above and conversations being he…

[PaulC]

OK, I decided to go with the Javascript running in the Script Editor. Changed the Application to “DEVONthink” and the name of the DEVONthink database to the name I’m using. The log shows it reading the SQLlite database data. Ended up with a Error -1708 Message not understood.

This was on trying to import the first PDF file. It was running the command app.import. The path looks fine. It sends it to {“to”:app.databases.byId(1).parents.byId(2147483645)}. So I am wondering if it isn’t reading the PDF file, the target is wrong, or if it isn’t understanding the data from the database and perhaps I need to rewrite the SELECT command where I assume is mapping the data to fields in the DEVONthink database?

I used to write in AppleScript but it has been years and I’ve never written in JavaScript. I miss ScriptDebugger.

[BLUEFROG]

Did you read the post I linked to you?

[chrillek]

Seeing the problem would be a lot easier with context. Just providing a minuscule part of a script is not telling anybody anything.

[PaulC]

Yes. Which doesn’t mean that I am missing something but that convinced me to go with a script that reads the database rather than try to extract the key words from the PDFs.

[PaulC]

I should have been more clear that I am using the Javascript script provided by dxr on Feb 2024.

My script is almost identical to that one.

(() => {
const app = Application("DEVONthink");
const targetGroup = app.databases['Finances'].incomingGroup();
function addDocument(path, notes, date, tags, title) {
  const newRecord = app.import(path, {to: targetGroup});
  newRecord.tags = newRecord.tags().concat(tags);
  newRecord.creationDate = date;
  newRecord.name = title; /* that's _different_ from the path in DT! */
}
  
  
const queryString = `WITH DOC_TAGS AS (
		SELECT tags.Z_14RECEIPTS1 as ID, ZTAG.ZNAME as NAME  
		FROM Z_14TAGS tags
			JOIN ZTAG ON Z_PK = tags.Z_18TAGS
	),
	GROUPED_DOC_TAGS AS (
		SELECT DOC_TAGS.ID, GROUP_CONCAT( DOC_TAGS.NAME, '<<[]>>'  ) as TAGS 
			FROM DOC_TAGS
			GROUP BY ID
	)
SELECT 	ZRECEIPT.Z_PK as ID, 
		DATETIME(ZRECEIPT.ZIMPORTDATE + 978307200, 'unixepoch') as IMPORTED,
		ZRECEIPT.ZMERCHANT as TITLE, 
		ZCATEGORY.ZNAME as CATEGORY, 
		ZSUBCATEGORY.ZNAME as SUBCATEGORY, 
		GROUPED_DOC_TAGS.TAGS, 
		ZRECEIPT.ZNOTES as NOTES,
		ZRECEIPT.ZPATH as PATH
FROM ZRECEIPT
	LEFT JOIN GROUPED_DOC_TAGS ON GROUPED_DOC_TAGS.ID = ZRECEIPT.Z_PK
	LEFT JOIN ZCATEGORY ON ZRECEIPT.ZCATEGORY = ZCATEGORY.Z_PK
	LEFT JOIN ZSUBCATEGORY ON ZRECEIPT.ZSUBCATEGORY = ZSUBCATEGORY.Z_PK
`;
const basePath = '/Users/paulchernoff/Documents/Paperless Backups/Receipt Library.paperless'
const DBPath = `${basePath}/DocumentWallet.documentwalletsql`;
const curApp = Application.currentApplication();
curApp.includeStandardAdditions = true;
const rawDBResult = curApp.doShellScript(`sqlite3 -tabs "${DBPath}" "${queryString}"`, {alteringLineEndings: false});
const resultLines = rawDBResult.split('\n');
resultLines.forEach(r => {
  const resultColumns = r.split('\t');
  if (resultColumns.length === 8) { /* basic sanity check */
    const plDate = new Date(resultColumns[1]);
    const plTitle = resultColumns[2];
    const plCategory = resultColumns[3];
    const plSubCategory = resultColumns[4];
    const plTags = resultColumns[5].split('<<[]>>');
    plTags.push(plCategory, plSubCategory);
    const plNotes = resultColumns[6];
    const docPath = resultColumns[7];
    addDocument(`${basePath}/${docPath}`, plNotes, plDate, plTags, plTitle);
  } 
  })
})()

[chrillek]

And the error occurs here:

@PaulC

const newRecord = app.import(path, {to: targetGroup});

with the message 1708 - Message not understood. One of the particularly unhelpful OSA error messages. In general, it means that you use a method (aka “command”) that does not exist. In this case, it’s import. That one went down the drain, it appears, with DT4. As a look in Script Editor’s scripting dict tells you:

I suggest changing that line to

const newRecord = app.importPath(path, {to: targetGroup});

and see where that gets you.

Aside: Please post code included in backticks (`): one for inline code, three for code snippets and scripts. That makes it easily recognizable and also copyable.