One interesting question is: how often do you get the error. Only once – then it’s happening outside the repeat loop. Or once for every database – then it’s happening inside the repeat loop (unless you have only one database, that is). Than I’d have the script run in Script Debugger or Script Editor and see where exactly the error happens. I doubt that it’s the line with the do shell script. I tried that single line over here and it runs just fine. I guess that there’s a problem creating the file itself.
This is a good point.
I think new people coming to this product don’t have an easy way to get started with the scripting.
I mean, there’s not a nice control panel inside DEVONthink for scripting nor a little tutorial build into that scripting menu.
If I were to rework this for a beginner, I would
Give the script menu a name (so that it can be automated with Keyboard Maestro)
Create a getting started panel where you can go through and set up some basics that every new user will need. For example, they need to have some global variables defined such as selecting their backup drive. They need to see all the scripts and have a button to get help on each one, change the hotkey for each one, edit each one, and a button to create a new script.
Create a forum post with a video if necessary for each script and link them in a way that allows a user to open the script menu and get help for all the common things.
Perhaps the scripts should be inside the DT3 system as a database and have a syntax highlighting system that allows the user to understand them, get help on them, organize them, duplicate them, and do so without ever modifying the original ones.
Seems nicer to do this than sending the user to finder to look at their scripts… especially since DEVONthink is already better than Finder, by far…
That would break the de-facto standard on a Mac. I’m not even sure it’s possible given that the visibility of the script menu is controlled by script editor (which I find a horrible idea).
The rest of your points: scripting has nothing to do with DT in particular. It is available across macOS as a whole, in many places and apps. And three basics as well as the finer points are explained elsewhere (i posted a link to Apple’s documentation before). Repeating that in every scriptable app is pointless and tedious. And AppleScript is not the only scripting language (just the oldest one and the one with rhe worst string, date and array handing).
Personally, i do never use AppleScript if i can get the job done with JavaScript. Therefore, my suggestion about moving the set text delimiter stuff outside of the repeat loop might have been misleading.
I was not at all sure I should muddy the waters even further but was sufficiently interested to have a working version of the script to suit my own purposes so spent some time on it.
Please note:
This version suits my use. It may not suit the uses of others.
I have removed all the credits at the start of the script—for which I apologise. I acknoweldge, however, all the help provided by others.
-- First set the path of the folder to which you wish to backup
property pbackup_path : "/Users/[your path]/Backups/"
tell application id "DNtp"
set this_date to do shell script "date +%Y-%m-%d-%H-%M-%S"
set all_databases to every database
repeat with this_database in all_databases
try
set this_name to name of this_database
-- We use the backup path property here
set this_archive to pbackup_path & this_name & " " & this_date & ".dtBase2.zip"
-- Set a general progress indicator
show progress indicator "Weekly backup of all open databases…" steps count of all_databases
with timeout of 3600 seconds
-- Show which database we're processing
step progress indicator this_name as string
if (verify database this_database) is not 0 then error "Database --" & this_name & " is damaged."
if not (optimize database this_database) then error "Optimisation --of database " & this_name & "failed."
if not (compress database this_database to this_archive) then --error "Backup of database " & this_name & "failed."
end if
end timeout
on error error_message number error_number
hide progress indicator
if the error_number is not -128 then log message "DEVONthink:" & error_message
end try
end repeat
hide progress indicator
-- Show an alert when it's all done
display alert "Backups of all open databases complete."
end tell
Stephen (who now blocks his ears and closes his eyes for fear of retribution)
Well, that caused me to scratch an itch. Here’s my take on it in JavaScript:
(() => {
const app = Application("DEVONthink 3")
app.includeStandardAdditions = true;
const homeFolder = app.pathTo("home folder", {from: "user domain"});
const pathToBackupFolder = `${homeFolder}/Backups/`
const date = new Date();
const dateString = date.toLocaleDateString("en-US", { year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute:'2-digit', second: '2-digit', hour12: false }).replaceAll(/[:\/ ]/g,"-").replace(",","");
Progress.totalUnitCount = app.databases().length;
Progress.title = 'Weekly backup of all open databases…';
app.databases().forEach((db, index) => {
const dbName = db.name();
const archiveName = `${pathToBackupFolder}${dbName} ${dateString}.dtBase2.zip`;
Progress.completedUnitCount = index+1;
if (app.verify({database: db}) !== 0) {
error(`Database '${dbName}' is damaged`);
} else if (!app.optimize({database: db})) {
error(`Optimisation of database '${dbName}' failed.`)
} else if (!app.compress({database: db, to: archiveName})) {
error(`Compression of database '${dbName}' failed.`)
}
})
app.displayDialog("Database backup complete.")
})()
Contrary to my usual habit, this is actually tested (well, I let it run once).
It’s a tad more portable since it determines the user’s home folder dynamically (see the pathTo call at the top). The progress indicator is a very sorry excuse for one, Apple just didn’t care to provide anything useful for JXA. The date formatting is really weird, but I didn’t find a better way quickly to do that without recurring to the shell (which I usually do not want to do unless I have to). EditSee my post below for a better way.
Also, I decided to change the three if statements in the repeat loop to a single if … else if ... else if to make the structure clearer. The logic is the same, though. And I skipped the error handling, simply calling error and hoping that this will end up in DT’s log anyway.
And I didn’t add the timeout thingy. If that’s necessary, it should be something like app.compress({database: db, to: archiveName}, { timeout: 3600 }). I.e. in JXA, the timeout is added to the method call as a final parameter object, not around a block. Why a scripting language would have a timeout parameter at all, is beyond me.
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
-- https://macscripter.net/viewtopic.php?id=24737
set this_time to replace_chars(time string of (current date), ":", "-")
tell (current date) to get "" & its year & "-" & (text -2 thru -1 of ("0" & (its month as integer) as text)) & "-" & its day
set this_date to result
set backup_date to this_date & "_" & this_time as string
-- https://macosxautomation.com/applescript/sbrt/sbrt-06.html
on replace_chars(this_text, search_string, replacement_string)
set AppleScript's text item delimiters to the search_string
set the item_list to every text item of this_text
set AppleScript's text item delimiters to the replacement_string
set this_text to the item_list as string
set AppleScript's text item delimiters to ""
return this_text
end replace_chars
This will remove the need to drop to a shell to get the date…
Perhaps the javascript idea is better…
I don’t like programming without an IDE that includes debugging and profiling tools.
What’s the best one for javascript?
No sense in buying Script Debugger which I still have on trial if AppleScript is so limiting and I can use something with JavaScript that does more.
date.toISOString returns something like 2011-10-05T14:48:00.000Z. The first replace call changes the colons and the T to dashes, the second one removes the point and everything after it. So it is a tiny bit more elegant than the AppleScript way, after all.
I’m sorely tempted to respond with an improved version of my AppleScript script (including the ability to cancel) but shall not because you will clearly win this little “contest”.
-- Daily backup archive - all open databases
-- Created by Christian Grunenberg on Mon Jun 22 2009.
-- Copyright (c) 2009-2022. All rights reserved.
-- addition for all open databases by coolgoose85
-- https://discourse.devontechnologies.com/t/backup-all-open-databases/50932
-- https://macscripter.net/viewtopic.php?id=24737
set this_time to replace_chars(time string of (current date), ":", "-")
tell (current date) to get "" & its year & "-" & (text -2 thru -1 of ("0" & (its month as integer) as text)) & "-" & its day
set this_date to result
set backup_date to this_date & " " & this_time as string
-- https://macosxautomation.com/applescript/sbrt/sbrt-06.html
on replace_chars(this_text, search_string, replacement_string)
set AppleScript's text item delimiters to the search_string
set the item_list to every text item of this_text
set AppleScript's text item delimiters to the replacement_string
set this_text to the item_list as string
set AppleScript's text item delimiters to ""
return this_text
end replace_chars
tell application id "DNtp"
set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, "/"}
set all_databases to every database
set AppleScript's text item delimiters to od
repeat with this_database in all_databases
try
set this_path to path of this_database
set this_name to name of this_database
set this_archive to "/Volumes/YourVolumeName/Backup/" & this_name & " " & backup_date & ".dtBase2.zip" -- Change YourVolumeName to your external backup drive.
show progress indicator "Daily Backup Archive of " & this_archive steps 3
with timeout of 3600 seconds
step progress indicator "Verifying database " & this_name & "..."
if (verify database this_database) is not 0 then error "Database " & this_name & " is damaged."
step progress indicator "Optimizing database " & this_name & "..."
if not (optimize database this_database) then error "Optimization of database " & this_name & "failed."
step progress indicator "Zipping database " & this_name & "..."
if not (compress database this_database to this_archive) then error "Backup of database " & this_name & "failed."
end timeout
hide progress indicator
on error error_message number error_number
hide progress indicator
if the error_number is not -128 then log message "DEVONthink:" & error_message
end try
end repeat
Okay. I’ve run this a few times and it’s doing what I need it to do.
It no longer gives me permissions errors because it handles the dates with AppleScript instead of a shell.
Really helpful to simply open the databases I have changed and kick off the script after finishing work.
So thanks for all the tips and code ideas, guys. I appreciate it.
If you decide to use it, just be sure to edit the YourVolumeName in the script and set it to the name of your backup drive… I’ve been running the script from Script Debugger which gives me log I can check in the morning to see if any errors occurred.
I think that’s the only thing I might like to change is to change all the messaging commands to write to the DT3 log, but I haven’t looked at how to write to the DT3 log yet, so this is where I’m leaving it for now.
Time Machine is better than no backups at all. It’s easy to set up and generally performs well enough. However, redundant backups of important data is always a good idea.
imho Time Machine’s incremental backups are superior to full database copies
However, Time Machine data is stored locally; I also want to use cloud storage (offsite)
edit: for this backup, I use Arq Premium
In addition to Time Machine, I want a data backup disconnected from Devonthink.
i.e. ability to retrieve individual files without the complexity of retrieving the DT database
edit; for this backup, I use the export files&folder feature
Oh, I guess I have some homework to do. I didn’t realize you could back up the files within the DT databases as individual files without dealing with the DT database. I understand it’s easier with an index structure, but I need to explore how to do so when my data is stored within the DT database.
I’m not sure of @DTLow’s method here but we don’t suggest you mess about in the internals of the database package, in case you were headed that direction.
Does anyone have a script that will backup all open databases rather than doing it individually? I would like to set up a few databases as my primary grows, and would simplify the backup process.
The database archive function isn’t intended for incremental backups. They are best used as periodic full backups. Incrementals should be handled by your primary back strategy, e.g., Time Machine and external drives.