A friend suggested that I find a way to automate the export of all my Files from DT for backup purposes.
Background - it was pointed to me that since my DT archive is squirrelled away, I don’t have eyes on the state of backups or database health.
While I don’t have the level of concern that my friend does around DB Corruption, I thought that ensuring I have duplicated copies of my research archives is just an SSD space problem.
I’ve done a File → “Export Files and Folders” for both my Inbox and my main research archive. Then of course being a madman, I added the whole thing to a local GIT repo, so that subsequent versions only store the changed files.
On the real question - is there an elegant way to automate the export weekly/daily? I’m sure I could use Keyboard Maestro but to quote one of my profs from undergrad, that feels like BFMI. Brute Force and Massive Ignorance. Is there something better?
Given that DT stores your files exactly as they are, they’ll be part of your normal backup process anyway. Why go through the steps of exporting them on top of that backup you have anyway? Or don’t you?
Someone I know has once suffered a catastrophic loss of data, through a corruption problem and didn’t know for years. They’re files that I might not access for years and would be impossible to find again now. So while my risk is only slightly larger than 0, it isn’t 0.
By exporting to separate folder and then committing to GIT. That risk is reduced. Overkill, maybe.
Supplementing @chrillek’s comments, to mitigate your risk as you describe it, why not simply copy the DEVONthink database “macOS package” somewhere else. All your files, unchanged, are there.
Exporting adds yet more risk to you, perhaps, e.g. did all the files get exported?, are they all verified as ok?, where did they go?
Me: I’m ok with normal backups and I do it a number of ways (TimeMachine, Backblaze, rsynch of really important stuff). I also test restoring from back-up every so often. Testing backups is as important as backing up, in my experience.
Take a look at built-in DEVONthink “backup to archive” command. I have setup an automated process that does that every week on the very important DEVONthink databases, when then are also backed up by normal backup processes.
My backups include a weekly export via AppleScript
Content, Groups and Tags are preserved
This is included in my Arq backups; incremental for versioning
tell application "DEVONthink 3"
set theGroup to get record with uuid "AA28F696-D9F9-4FC3-9089-B1EDBC96011E" --General Filing
export record theGroup to "/Volumes/Untitled"
end tell
I have wanted to use git with Devonthink for years. I COMPLETELY understand where you are going with this. Not only do you end up with a cryptographic store of the files but you can ask the all crucial question “what changed.” Using Time Machine gives no simple way to verify the integrity of databases stored years ago.
I love Devonthink but it is very unforgiving. It has no audit trail nor journal, afaik there is no way to check for missing files caused by a bug (historically in sync) or fat fingers. If you discover that a file is missing you can’t ask “what happened”. For me it is the single biggest failing of DT.
It makes complete sense to use a different tool to back up and git is just about perfect for keeping a verifiable archive which can be interrogated. I have never really understood why one can’t just use git on the main database folder with all Da. But I am always told NO.
Has that changed with DT3? I would love to use git.
Technically, a DT database is just a folder. So, it would be possible to add it to Git. OTOH, doing that would also add files and folders to the DT folder (eg .gitignore) that are not part of the DT database.
And as far as I know, DT doesn’t like its folder being populated by files not under its control. You could, perhaps, try to add the folder with all your databases to Git. But I don’t know if Git would then treat the database packages as opaque files or as folders. In the first case, you’d not get any version control.
No, nothing has changed in this regard. And @chrillek is correct about the issues that would arise from trying to turn a database’s internals into a git repo.