I think in DEVONthink Manual that advice and recommendation is given–I will leave to you to confirm–to not to put databases in folders that sync elsewhere).
Here is what I do (on all three of my MacBooks):
All Databases ~/Documents/DEVONthink/
All zip archive backups ~/Backups/DEVONthink/ created by occasional use of the DEVONthink feature to do that
TimeMachine: to a NAS and occasionally to a USB drive
Arq to their offsite server (their default file setting) and a second process to the NAS of more files
Synching (macOS and iOS devices) primarily with Bonjour supplemented with Dropbox for a few databases when I know I’ll be travelling
Thank you for the reminder not to store even on a local Dropbox folder. I did use Arq and its offsite server, but decided that I was safe enough with TimeMachine, external SSDs, and Dropbox, especially as my macs are in different parts of town. The question in my mind is how to get better use out of Dropbox. I pay for 3TB of storage through my personal company. (No receipts issued that my accountant approves of for the personal 2TB Dropbox version). I use only 0.2TB for DT, and don’t have much data outside DT apart from photos and GMail archives.
I don’t have space on my macs to mirror my databases to a local Dropbox folder. I can’t, I think, use CCC to directly backup to Dropbox. So I guess I must first clone to an external SSD using CCC, and then upload to Dropbox from there. It doesn’t seem a very elegant solution (except that I will clone to the SSDs anyway, so perhaps it is not so bad). Maybe I am missing something.
If you don’t have local storage space for all you databases, maybe some of that money is better spent on hard drives.
(But if you absolutely want to put databases in dropbox – it should be fine if you zip them first, from what I recall. Of course that means you can’t open them from dropbox, only store them as static archives.)
Like @troejgaard I wonder if you have a solution looking for a problem.
That being said, rather than using CCC to copy synced files to Dropbox–which I think is just asking for trouble as doing it that way you have your backups on a “sync” location… if a flaw happens locally on your computer the flaw is synced.
Perhaps consider scrapping the CCC idea and instead use Dropbox Backup. Not Dropbox Sync. Dropbox Backup. I tested it for a while on one machine and it worked fine. A real backup service, and remote. Can handle multiple machines. My Dropbox account not big enough, so I did not proceed. But with your 3TB, surely enough to get a good and secure backup.
I don’t know if Dropbox backup software does “snapshots” to safely backup DEVONthink databases. Check that out. No matter. Just exclude from backup if snapshots (and how open files handled) are not taken or if you can’t find out. Backup the Archive Zip files which we pointed out above are a feature of DEVONthink.
I haven’t really looked into Dropbox Backup, so I thought it was a totally separate service with its own subscription. But you’re right: looks like it’s just included with all their plans. If you have 3TB to blow that seems like a no-brainer.
Of course check out the technical details of how it works, as you say.
First do you have space on your local system drives to keep your active DT databases running i.e. in the Documents folder, no cloud storage considerations, but syncing via iCloud, Dropbox or Bonjour?
I do have this and I think it is a recommended set up for stable work, saving and backing up which is the next step.
At a frequency (that you decide for your own comfort) I close DEVONthink and
run CCC clone to external SSD (daily for me)
Archive each DB to a ZIP file, add the date to the name and move the ZIP to Dropbox. (Fridays)
After each cycle of the Dropbox backup, I make the previous set of ZIPs “Online Only” so they’re not taking space on the system drive. I use a shortcut to automate the ZIP/rename/move.
Every once in a while I winnow the dropbox folder of anything older than a year. With your 3TB you could keep everything for a long time. My Dropbox is full of tons of other stuff so I am constantly cleaning it out after projects get done.
I like to check the ZIPs every once in a while by downloading them and opening them to make sure no corruptions have crept in. But the frequent closing of the DT and the DBs and the verifying that happens when you start DT afresh are also good health indicators. I’ve never lost anything or truly had to use a DEVONthink backup for that purpose but I see drives die all the time at work (say a prayer for the WD 5TB spinners and their short short MTBF) so I retain the urge to keep it up.
Yes, it is indeed a solution looking for a problem! Still, I have that all that extra space. Dropbox is non-negotiable for me given that I must have rock-solid sync especially for my accounting, and the free tier is just too small. In fact, I have over 60 databases. Maybe there is a way to zip them all easily, but doing it manually is a big chore….
I see cloning to the external SSD with CCC, but am wondering why you then Zip the databases? Is that to save space? On my external SSDs I have folders with a given date, and I just copy to those folders in the “raw” state. Those folders represent snapshots in time, and, like you, I thin them out as time goes by. I guess I should just copy those time stamped folders to Dropbox, although the SSDs in different locations are unlikely all to fail at the same time. Maybe Dropbox is best kept for the sync stores, the SSDs for the backups, and never mind the wasted space. In which case, might it not at least make sense to have some way to copy the Devonthink Packet Sync within Dropbox in a time stamped way (to which the response will maybe rightly be that I haven’t understood anything…)
Sorry not to understand you. Perhaps I am confusing the zipping that occurs with the Daily Backup Archive facility for some other form of zipping? Bill DeVille (bless him) used to argue for using Daily Backup Archives. But for a large number of databases, it’s a lot of work, or is there an automated way to do all this?
Have you tried that built-in script yet? Looked into what happened? Read the documentation about “Daily Backup Archive” script or “A word about backups” in the DEVONthink Manual?
Doesn’t sound like it.
If you don’t like the script, you can copy it, put into your own Scripts folder, and modify it to do as much more automation that you may want.
“As a secondary backup option, you can create an optimized and compressed copy of the whole database that you can then save on a server or other media. Use File > Export > Database Archive or Scripts > Export > Daily Backup Archive to create a ZIP file of the database”. I did read that. Modern macs don’t have much storage space. If I have 100GB of DT databases on a macbook, represented by 60 databases, I am in some difficulty finding space for it all if I create daily backup archives. So I fall back instead on using CCC to copy those 60 databases to an SSD in their raw state. Once on the SSD, in a folder with a given date, the question arises as to whether there is any real benefit in then having them as Daily Backup Archives instead, which itself is difficult because, due to space constraints on the macbook, would require a batch process….
I have modified the Daily Backup Archive so that it creates a zip directly on a cloud drive (in my case OneNote). I also changed the naming of the resultant zip archive. It would work for Dropbox too. The destination of the zip backup does not have to be on your local drive (as far as I know).
Easier to manage and date stamp and doesn’t clog up the Dropbox recent folder with all the millions of files in a backed up database
More importantly to me though is that a ZIP is instantly and definitively recognizable as an archive/backup
I work in a field where a good guideline is to make your filing system understandable to the most tired and sleep deprived version of yourself. I will never accidentally open a zip file from within DEVONthink, somehow thinking it is the active database and DEVONthink will not see all those zip files as potential DBs to open.
There were technical reasons to prefer ZIPs over uncompressed files for uploading in days’ past but I think a lot of new file systems and file transfers to cloud operating systems have lessened the need for that. Some of my habits are rooted in past difficulties and what I do maybe a wee bit overkill but I can sleep comfortably knowing how my system works for me, even when I am not firing on all cylinders.
The standard script (and hence my modified one) loops through all the databases you have open in DT. As I always have my important Databases open, one click creates individual zip archives of them all.
