Backup is Backup.

Dearests. DevonThink is a nice application in many ways. However, in one quite central point, it shows an unbelievable and potentially very expensive shortcoming. In short: If you use a common word, you need to use it in it’s common meaning. E.g.: Save needs to mean Save all recent changes, an not: Save the User from his or her own stupidity and erase all files. And Backup, my friends, means BACKUP WHAT HAS BEEN DONE. And not: Backup the file structure. It is of NO HELP, if then the Help section states that Backup actually doesn’t mean what every speaker of English on this planet, except you, understands as backup, but only a backup of the file structure. Unless of course you warn the user on installation that you have your own dictionary of otherwise common terms and that the user needs to look up every menu item in the help section, because it means something totally and reliably unexpected. Which you don’t. Hence if Backup actually means Backup File Structure, you have to name it so.
Let me extend this request. Please, as a first step, make an immediate announcement on other items, if existing, that follow your company lingo instead of common understanding. And please, correct them in any future edition of DT.
Thanks.

Don’t you have a Time Machine backup of the whole .dtbase2 file?

Of course I should have. But I have been around with a mobile computer without a second hard drive, and what I did was using a USB Stick as database carrier and a manual backup routine in copying the DevonDatabase to the hard drive every day. Under the assumption that an actual backup is part of that database, this made perfect sense. However, with no actual backup integrated in the database, copying the—as I now know: damaged—DevonDatabase to the backup location erases what would have been a backup.
And really: If we are supposed to rely on TimeMachine, why proclaim an integrated backup functionality anyway?

Because the backup is of the text in the files and DEVON-specific info. It’s not a backup of the FILES in the database. That’s always been the case.

This is a bit of a windmill, y’know.

yes, I know. But obviously, that’s a mistake, like I said. Backup means Backup. And Database, btw., means Database. Like Backup doesn’t mean Storage of File Locations absent their content, Database means Structured Data Storage, and not Structure Storage absent the Data. As for the Caballero de la Mancha though: The DevonThink people I had the pleasure writing and reading with so far seemed friendly and intelligent. I don’t think they’ll leave my reasoning without an answer.

See the “A Word About Backups” section in the user documentation. The difference between the internal backup of indexes (and other metadata) and a full database backup is discussed.

If, for example, I’m capturing large batches of new content or making significant changes in the organization of a database, I’ll periodically invoke Tools > Verify & Repair to make certain there are no errors (e.g., as might result when importing a malformed file), and then run Tools > Backup & Optimize. Now, if something goes wrong with the next thing I do (perhaps I may make a mistake in an organizational structure change that would otherwise require a lot of work to back out of), I can simply run Tools > Restore Backup and in moments I’m back to a previous state of the database before I screwed up the organization.

Suppose I had added new content after the time at which that internal Backup folder had been created. If I run Tools > Verify & Repair it will detect those new files and move them into a special ‘Orphaned Files’ group, from which they can be moved into groups as desired.

Once in a while a user reports that a database has become so badly damaged that DEVONthink can no longer open it at all. If there’s a good internal Backup folder, I can provide instructions on how to create a new and working database from an internal Backup folder plus the ‘Files.noindex’ folder in the database. So yes, those internal backups can be invaluable.

Of course, the internal backups are of no use if there’s a hard drive crash and the data isn’t recoverable. For that, we need full database backups that are stored on an external medium.

I use Time Machine, which creates backups on an external drive. That’s a great free feature included with OS X. But Time Machine isn’t necessarily the only backup that should be maintained. For one thing, if a database has errors, Time Machine blithely backs up the damaged database. For another, the Time Machine storage drive is usually located near my computer, so a burglar might make off with all my equipment, or a house fire might consume it. (BTW, it’s preferable for databases to be closed at the time a Time Machine backup is made.)

For users of DT Pro and DT Pro Office, the File > Export > Database Archive or Scripts > Export > (several options) routines are highly recommended. Importantly, before making a backup these routines check for errors and stop and report any problem found. If no errors are found, the resulting database archive file will be the smallest possible, zipped full backup of the database and can be stored on an external drive or in the cloud. I keep database archives of my most important databases on a portable drive that’s stored in a bank safety deposit box, and update them every now and then.

I spend thousands of dollars every year for home and auto insurance. Some of my databases are as valuable to me as the home and auto. A good backup strategy costs little and protects them.

Does anyone know of any way to back up Devonthink databases offsite reliably?

Specifically, from what I have read in these forums, neither Dropbox nor Crashplan can do this correctly. All of my other files are backed up to Dropbox (for syncing) and Crashplan (for offsite backup), but Devonthink databases apparently cannot be done this way.

I’d be most grateful for an answer to this question, it does not seem to be addressed anywhere. While my office is in a fairly secure building, my DT databases exist in duplicate on two external drives (and nowhere else). Anyone in my office could walk off with them at any time.

The issue with the cloud services are the mechanisms aren’t made to sync a live database package like our databases (and this is not just isolated to our software).

You can use File > Export > Database Archive… or Script icon menu > Export > Daily Backup Archive to create a ZIP file of your database in its current state. These ZP files are perfectly fine to use with these online services.

Hi. I index the files in a folder within SpiderOak (a secure cloud service). This provides me with an encrypted, intact copy of all my files on the cloud (offsite) in addition to my encrypted Time Machine backups (onsite).

christopher-mayo.com/?p=2376

As I mention in my blog post, indexing is an amazing feature in DT. However, it admittedly requires a little effort to wrap your brain around it, so if you use it, make sure to practice and test it out first in order to familiarize yourself with how it works.

In addition, I have my own backup regime. It seems to be working well so far (a decade or so).
christopher-mayo.com/?p=962

I agree 100% with the original poster of this thread. Just because I like Mac’s for their reliability does not mean I like to be forced to use every feature of the ecosystem that Apple intended. Of course I have a time machine backup - it’s a great feature but falls short in a number of areas. Either way, that’s a moot point. Something called backup should be exactly that. If Devonthink’s philosophy is to use Time Machine, then that should work for metadata too. If it doesn’t and requires a separate backup for metadata then that backup should be updated to include a complete backup.

In my case, my devonthink databases are stored on an external drive because the databases are huge and apple SSD drives are not. I cannot backup my entire external 2TB drive plus my internal SSD to time machine and neither will anyone else be able to without a giant NAS. I have a Giant NAS and a still can’t do that without sacrificing a HUGE amount of disk space because of the way Time Machine works. Unfortunately for me, I only found out about this problem after I ran out of space and had to move the DB externally and realised I might not be moving all the data it needed. It would be MUCH, MUCH easier just to implement a proper backup in Devonthink.

I clearly should have done my research better - Devonthink looked like it had all the bases covered when it came to backup, in fact it might, but if it does, they’re not called ‘backup’ but something else like sync. If that’s the case, Devonthink needs a terminology overhaul. I think I’ll write a blog article on how to properly backup devonthink, with translation into common ITIL backup terminology so the rest of the globe can figure out what it is that DevonThink is actully doing and keep their data safe!

Backing up your data is the user’s responsibility – as it is with every application. I guess I don’t grasp how a user could get so deep into using the product (massive databases, etc.) without researching the capabilities and experimenting, and developing a data assurance strategy. RTFM.

1 Like

As a long time fan, new full time user and technical writer for over a decade, I find this thread on backups fascinating. It seems to be an issue about user understanding and expectations. As application users, our understanding of average backups tends to have been fashioned on standard MS office-type flat files rather than the more complex database-like dtBase files with huge meta-data. We’re almost talking about filesystem within a filesystem.

My query is not dissimilar but I just need to clarify my thinking; our DTPro office is installed on my MacBook Air with obvious space limitations for the ever growing database files. I’m considering relocating the files to our NAS drive but I’m concerned about the following:

i. Will the performance of the application be degraded with the application on the Mac and the files on a NAS drive - I read somewhere that the Mac filesystem doesn’t necessarily play too well with non-Mac?

ii. Since the NAS storage is RAID configuration with 1 disk tolerance to allow for drive failure without data loss, alternatively, I can keep the database files on the Mac and continue to use Time Machine for back ups to the NAS.

Any ideas which option would offer a more secure and easier to synchronise setup without compromising performance? Thanks.