I have modified existing script “Daily backup archive” which is located at ~/Library/Application Scripts/com.devon-technologies.think3/Menu/Export/ to go through all the open databases and create the backup.
It also create back with date and time so that it can be created multiple times in a day instead of just once in a day.
Feel free to review it and let me know if anyone find any issues.
– Daily backup archive.
– Created by Christian Grunenberg on Mon Jun 22 2009.
– Copyright (c) 2009-2019. All rights reserved.
tell application id “DNtp”
set allDatabases to every database
repeat with thisDatabase in allDatabases
try
if not (exists thisDatabase) then error “No database is open.”
set this_database to thisDatabase
set this_date to do shell script "date +%Y-%m-%d-%H:%M:%S"
set this_path to path of this_database
set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, "/"}
set this_name to the last text item of this_path
set AppleScript's text item delimiters to ""
if this_name ends with ".dtBase2" then set this_name to (characters 1 thru -9 of this_name) as string
set this_archive to "~/Backup/" & this_name & "_" & this_date & ".dtBase2.zip"
set AppleScript's text item delimiters to od
show progress indicator "Daily Backup Archive" steps 3
with timeout of 1200 seconds
step progress indicator "Verifying..."
if (verify database this_database) is not 0 then error "Database is damaged."
step progress indicator "Optimizing..."
if not (optimize database this_database) then error "Optimization of database failed."
step progress indicator "Zipping..."
if not (compress database this_database to this_archive) then error "Backup failed."
end timeout
hide progress indicator
on error error_message number error_number
hide progress indicator
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
end try
end repeat
end tell
It generates backup files as
Inbox_2019-10-11-09:35:16.dtBase2.zip
YetToBeOrganized_2019-10-11-09:36:26.dtBase2.zip
etc…
You can create a script file as
touch script_name.scpt
open script_name.scpt
paste above code and put it in
~/Library/Application Scripts/com.devon-technologies.think3/Menu/Export
DEVONthink will pick up the new script and will show in UI
Thanks for the script! Just a reminder that it’s not recommended to replace/modify default scripts, it’s recommended to modify copies to avoid conflicts after updates.
-- Daily backup archive - all open databases
-- Created by Christian Grunenberg on Mon Jun 22 2009.
-- Copyright (c) 2009-2022. All rights reserved.
-- addition for all open databases by coolgoose85
-- https://discourse.devontechnologies.com/t/backup-all-open-databases/50932
tell application id "DNtp"
set allDatabases to every database
repeat with thisDatabase in allDatabases
try
if not (exists current database) then error "No database is open."
set this_database to thisDatabase
set this_date to do shell script "date +%Y-%m-%d-%H:%M:%S"
set this_path to path of this_database
set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, "/"}
set this_name to the last text item of this_path
set AppleScript's text item delimiters to ""
if this_name ends with ".dtBase2" then set this_name to (characters 1 thru -9 of this_name) as string
set this_archive to "~/Backup/" & this_name & "_" & this_date & ".dtBase2.zip"
set AppleScript's text item delimiters to od
show progress indicator "Daily Backup Archive" steps 3
with timeout of 1200 seconds
step progress indicator "Verifying..."
if (verify database this_database) is not 0 then error "Database is damaged."
step progress indicator "Optimizing..."
if not (optimize database this_database) then error "Optimization of database failed."
step progress indicator "Zipping..."
if not (compress database this_database to this_archive) then error "Backup failed."
end timeout
hide progress indicator
on error error_message number error_number
hide progress indicator
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
end try
end repeat
end tell
I’m only pasting again as code because I had to do some edits to fix the characters in the original post.
Gonna give it a try since it will let me run an overnight backup of everything I leave open, which sounds useful…
I have a question. What is the difference between running this archive system and simply shutting down DT3 and copying the databases folder?
Is there an advantage/disadvantage to doing this?
The script seems to freeze all action and it takes some time to complete.
So I would prefer a backup that doesn’t block continuing to work in the database or that can be done faster outside of DT3 with the database in a shutdown state.
Because it uses the DT commands to “verify” and “optimize” the databases?..which said databases would have to be open to perform verification and optimization. These functions can’t be performed on shutdown/closed databases I believe.
Thanks for updated code…this looks really useful for archival backups.
In my testing, it seems the “zipping” function is the one that “freezes” the action so maybe deleting that step, if the greater file sizes can be tolerated, might return control more quickly.
Adding below info to make it complete for everyone.
You can create a script file as
touch script_name.scpt
open script_name.scpt
paste above code and put it in
~/Library/Application Scripts/com.devon-technologies.think3/Menu/Export
DEVONthink will pick up the new script and will show in UI
I ended up using Script Debugger instead of Script Editor to run the script. I added info to the status messages so that I could tell where I was at in the process.
show progress indicator "Daily Backup Archive of " & this_archive steps 3
with timeout of 3600 seconds
step progress indicator "Verifying database " & this_name & "..."
if (verify database this_database) is not 0 then error "Database " & this_name & " is damaged."
step progress indicator "Optimizing database " & this_name & "..."
if not (optimize database this_database) then error "Optimization of database " & this_name & "failed."
step progress indicator "Zipping database " & this_name & "..."
if not (compress database this_database to this_archive) then error "Backup of database " & this_name & "failed."
end timeout
My second database took too long to backup.
1200 seconds isn’t enough. So I’ll triple this timeout parameter and try again tonight.
Have you allowed all Automation requests for DEVONthink in System Preferences > Security & Privacy > Privacy?
And you’re getting the forward slashes because you’ve got colons in the time. Those are reserved characters and can’t be put in a filename.
Actually so are forward slashes, but macOS is allowing for it while caching the proper filename under-the-hood.
Take the colons out and strip out the time altogether.
If they take out the time altogether, the colons should be gone in the process
@jsn Some remarks about what I’d do differently below. Just ignore them, if you wish. They’re mostly general remarks about coding style and habits, things I find helpful in obtaining clearer code. And clearer code is easier to understand as well as easier to debug and modify.
Move the set this date to do shell script "date +%Y-%m-%d" line before the repeat loop. Given that the date will not change while the backup is running (and even if it were, there’s no point in being overly accurate here) there’s no point in recalculating the same value over and over again. Generally, invariants should be moved out of loops to not obfuscate the logic.
Get rid of the set this_database to thisDatabase: What is the point of creating a variable with a slightly different name if you already have one (with a weird name, in my opinion, but that’s AppleScript lingo).
Move the line set {od, ... before the repeat loop. There’s no point of setting (and later resetting) the text delimiters if you only ever use the ‘/’ as a text delimiter inside the loop anyway. Same point as above with the do shell script line.
Analogously, move the line set AppleScript's text ... after the repeat loop.
And get rid of the set AppleScript's text item delimiter to "" completely - it makes no sense at all since you’re not splitting any strings after that point (and certainly not at an empty string).
In set this_name to (character 1 thru -9 of this_name) as string you could use text instead of character and thus get rid of the as string at the end. Again, that makes the code clearer (in my opinion), because taking a substring of a string should not require an explicit conversion to string. The concept is explained here Class Reference in the “Discussion” section for the Text class.
Then (you’ve found that out already the hard way): Using ~/Backup/ is not the best choice, since ~ is a shell shortcut for /Users/yourname/. Simply use /Users/yourname/Backup/ instead (replacing your name with your name, of course). There are other possibilities, too, but that one seems to be the simplest one in the given context.
I have doubts about the usefulness of display alert. If an error occurs, it will pop up the alert and then wait for user interaction. In the context of an automatically run script, this might not be ideal: I would not want my backup script to wait for user input in the middle of the night. You could alternatively use DT’s log message method that writes the messages to DTs log window. Also, error messages are a lot more useful if they do include the offending/broken object. Instead of "Database is damaged", you might want to use "Database " & this_database & " is damaged" and so on. You could pop up an alert at the end of the script informing the user that something went wrong and that they should inspect DT’s log.
-- Daily backup archive - all open databases
-- Created by Christian Grunenberg on Mon Jun 22 2009.
-- Copyright (c) 2009-2022. All rights reserved.
-- addition for all open databases by coolgoose85
-- https://discourse.devontechnologies.com/t/backup-all-open-databases/50932
-- Modifications courtesy of chrillek's advice: https://discourse.devontechnologies.com/t/backup-all-open-databases/50932/12
tell application id "DNtp"
set this_date to do shell script "date +%Y-%m-%d-%H-%M-%S"
set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, "/"}
set all_databases to every database
repeat with this_database in all_databases
try
set this_path to path of this_database
set this_name to the last text item of this_path
if this_name ends with ".dtBase2" then set this_name to (characters 1 thru -9 of this_name) as string
set this_archive to "/Volumes/YourBackupDriveName/Backup/" & this_name & " " & this_date & ".dtBase2.zip" -- was ˜Backup before, which created a folder under documents.
show progress indicator "Daily Backup Archive of " & this_archive steps 3
with timeout of 3600 seconds
step progress indicator "Verifying database " & this_name & "..."
if (verify database this_database) is not 0 then error "Database " & this_name & " is damaged."
step progress indicator "Optimizing database " & this_name & "..."
if not (optimize database this_database) then error "Optimization of database " & this_name & "failed."
step progress indicator "Zipping database " & this_name & "..."
if not (compress database this_database to this_archive) then error "Backup of database " & this_name & "failed."
end timeout
hide progress indicator
on error error_message number error_number
hide progress indicator
if the error_number is not -128 then log message "DEVONthink:" & error_message
end try
end repeat
set AppleScript's text item delimiters to od
end tell
Thanks for all of the tips.
I think I got everything but please if anybody sees a comment I missed or something I did wrong, correct it or let me know and I’ll correct it. I put the web links in the comments so that if anybody copies the script, they know where they got it and where they can go back to reference it.
I also changed the thisDatabase to this_database to stick with snake case convention.
And I changed the : to - in the time generation.
I did go through privacy parameters, but I think I’ve granted all the access that DEVONthink 3 requests in System Parameters, but I still have the error in my script. It’s an error that acts like a warning because the name is generated just fine. So perhaps I will see if there is a list of Privacy Parameters and make a visual guide to the System Parameters window. Wish all developers would make these parameter windows expandable and searchable with filters. Seems more logical to me if I could just type DEVONthink and then be shown all the settings related and check the boxes…
One interesting question is: how often do you get the error. Only once – then it’s happening outside the repeat loop. Or once for every database – then it’s happening inside the repeat loop (unless you have only one database, that is). Than I’d have the script run in Script Debugger or Script Editor and see where exactly the error happens. I doubt that it’s the line with the do shell script. I tried that single line over here and it runs just fine. I guess that there’s a problem creating the file itself.
This is a good point.
I think new people coming to this product don’t have an easy way to get started with the scripting.
I mean, there’s not a nice control panel inside DEVONthink for scripting nor a little tutorial build into that scripting menu.
If I were to rework this for a beginner, I would
Give the script menu a name (so that it can be automated with Keyboard Maestro)
Create a getting started panel where you can go through and set up some basics that every new user will need. For example, they need to have some global variables defined such as selecting their backup drive. They need to see all the scripts and have a button to get help on each one, change the hotkey for each one, edit each one, and a button to create a new script.
Create a forum post with a video if necessary for each script and link them in a way that allows a user to open the script menu and get help for all the common things.
Perhaps the scripts should be inside the DT3 system as a database and have a syntax highlighting system that allows the user to understand them, get help on them, organize them, duplicate them, and do so without ever modifying the original ones.
Seems nicer to do this than sending the user to finder to look at their scripts… especially since DEVONthink is already better than Finder, by far…
That would break the de-facto standard on a Mac. I’m not even sure it’s possible given that the visibility of the script menu is controlled by script editor (which I find a horrible idea).
The rest of your points: scripting has nothing to do with DT in particular. It is available across macOS as a whole, in many places and apps. And three basics as well as the finer points are explained elsewhere (i posted a link to Apple’s documentation before). Repeating that in every scriptable app is pointless and tedious. And AppleScript is not the only scripting language (just the oldest one and the one with rhe worst string, date and array handing).
Personally, i do never use AppleScript if i can get the job done with JavaScript. Therefore, my suggestion about moving the set text delimiter stuff outside of the repeat loop might have been misleading.
I was not at all sure I should muddy the waters even further but was sufficiently interested to have a working version of the script to suit my own purposes so spent some time on it.
Please note:
This version suits my use. It may not suit the uses of others.
I have removed all the credits at the start of the script—for which I apologise. I acknoweldge, however, all the help provided by others.
-- First set the path of the folder to which you wish to backup
property pbackup_path : "/Users/[your path]/Backups/"
tell application id "DNtp"
set this_date to do shell script "date +%Y-%m-%d-%H-%M-%S"
set all_databases to every database
repeat with this_database in all_databases
try
set this_name to name of this_database
-- We use the backup path property here
set this_archive to pbackup_path & this_name & " " & this_date & ".dtBase2.zip"
-- Set a general progress indicator
show progress indicator "Weekly backup of all open databases…" steps count of all_databases
with timeout of 3600 seconds
-- Show which database we're processing
step progress indicator this_name as string
if (verify database this_database) is not 0 then error "Database --" & this_name & " is damaged."
if not (optimize database this_database) then error "Optimisation --of database " & this_name & "failed."
if not (compress database this_database to this_archive) then --error "Backup of database " & this_name & "failed."
end if
end timeout
on error error_message number error_number
hide progress indicator
if the error_number is not -128 then log message "DEVONthink:" & error_message
end try
end repeat
hide progress indicator
-- Show an alert when it's all done
display alert "Backups of all open databases complete."
end tell
Stephen (who now blocks his ears and closes his eyes for fear of retribution)
Well, that caused me to scratch an itch. Here’s my take on it in JavaScript:
(() => {
const app = Application("DEVONthink 3")
app.includeStandardAdditions = true;
const homeFolder = app.pathTo("home folder", {from: "user domain"});
const pathToBackupFolder = `${homeFolder}/Backups/`
const date = new Date();
const dateString = date.toLocaleDateString("en-US", { year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute:'2-digit', second: '2-digit', hour12: false }).replaceAll(/[:\/ ]/g,"-").replace(",","");
Progress.totalUnitCount = app.databases().length;
Progress.title = 'Weekly backup of all open databases…';
app.databases().forEach((db, index) => {
const dbName = db.name();
const archiveName = `${pathToBackupFolder}${dbName} ${dateString}.dtBase2.zip`;
Progress.completedUnitCount = index+1;
if (app.verify({database: db}) !== 0) {
error(`Database '${dbName}' is damaged`);
} else if (!app.optimize({database: db})) {
error(`Optimisation of database '${dbName}' failed.`)
} else if (!app.compress({database: db, to: archiveName})) {
error(`Compression of database '${dbName}' failed.`)
}
})
app.displayDialog("Database backup complete.")
})()
Contrary to my usual habit, this is actually tested (well, I let it run once).
It’s a tad more portable since it determines the user’s home folder dynamically (see the pathTo call at the top). The progress indicator is a very sorry excuse for one, Apple just didn’t care to provide anything useful for JXA. The date formatting is really weird, but I didn’t find a better way quickly to do that without recurring to the shell (which I usually do not want to do unless I have to). EditSee my post below for a better way.
Also, I decided to change the three if statements in the repeat loop to a single if … else if ... else if to make the structure clearer. The logic is the same, though. And I skipped the error handling, simply calling error and hoping that this will end up in DT’s log anyway.
And I didn’t add the timeout thingy. If that’s necessary, it should be something like app.compress({database: db, to: archiveName}, { timeout: 3600 }). I.e. in JXA, the timeout is added to the method call as a final parameter object, not around a block. Why a scripting language would have a timeout parameter at all, is beyond me.
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
-- https://macscripter.net/viewtopic.php?id=24737
set this_time to replace_chars(time string of (current date), ":", "-")
tell (current date) to get "" & its year & "-" & (text -2 thru -1 of ("0" & (its month as integer) as text)) & "-" & its day
set this_date to result
set backup_date to this_date & "_" & this_time as string
-- https://macosxautomation.com/applescript/sbrt/sbrt-06.html
on replace_chars(this_text, search_string, replacement_string)
set AppleScript's text item delimiters to the search_string
set the item_list to every text item of this_text
set AppleScript's text item delimiters to the replacement_string
set this_text to the item_list as string
set AppleScript's text item delimiters to ""
return this_text
end replace_chars
This will remove the need to drop to a shell to get the date…
Perhaps the javascript idea is better…
I don’t like programming without an IDE that includes debugging and profiling tools.
What’s the best one for javascript?
No sense in buying Script Debugger which I still have on trial if AppleScript is so limiting and I can use something with JavaScript that does more.
