Greetings,
I’m using DT3 to store my personal documents such as contracts, invoices and others.
I’d like to have the same documents on iCloud for backup purposes, and in case something go wrong and I’m not able to use DT (so I can still reach those documents on iCloud).
My requisite is I need to preserve the same group structure as in the database, not the real folder structure.
How can I do this?
How do you do?
I’d like to sync because exporting the entire database on a given period of time (say each week or similar) would mean to export and sync some Gb each time.
afaik for incremental sync, your only option is the DT database structure
I have a sync store at iCloud
and local time machine backups to alternating external drives
I need to preserve the same group structure as in the database
I use periodic (weekly) export using the Files&Folder option (13GB)
to both iCloud (1 version)
and an external drive (multiple versions)
This also preserves the tag assignment
in case something go wrong and I’m not able to use DT
I’m also thinking legacy if I’m no longer here
My family are not DT users
Is there a way to automate:
each day or week, the new files and groups are exported outside DT3 preserving the same structure of the database.
This way I may choose iCloud as destination.
As stated here so many times, please do not rely on sync services (iCloud) as backup. If your hard disk goes “wrong” as you say, that “wrongness” could sync to iCloud and your so-called-backup will be dead.
As a starter, read about backups in the DEVONthink Handbook.
Sync = keep the data identical at all times on all machines
Backup = save the data somewhere at a certain point in time. Preferably newer data not immediately overwriting older data.
Sync ensures that you see the same data on your Mac(s) as on your iPad(s) and iPhone(s) all the time. Nice.
But if you delete a record on the Mac, it will eventually disappear from the other synced Mac(s), iPad(s) and iPhone(s), too. Not so nice if you deleted it by mistake.
Therefore, you need a backup: Snapshots of your database(s) at certain points in time that you can restore. So a backup you make today should not over-write all your data but keep the old one and augment it with the new one. That’s what TimeMachine does, for example.
So your title is not describing what you want: You want a backup of your data in case it gets lost. There are tons of solutions for that: TimeMachine and ArqBackup are some of those, CCC another one.
No backup software worthy of that name will make a copy of all your data every time it performs a backup. Instead, it will only save the differences since the last time it performed a backup (delta).
Personally, I’m using TimeMachine/ARC to backup all my data to a local NAS and a cloud service. Also, Arq backups the most relevant data like my DT databases to another cloud location. Other people, notably @Blanc have described their more thorough strategies in a lot of detail in this forum. The search function is your friend.
IF the sync store hasn’t been compromised, it can be used to recover your database
or create the database on a new device
Be warned - proper data backups are recommended
Is it incremental? Do you have to sync 13GBs each time you export your database?
No, it’s not incremental (export). I’m working with the full 13GB
You could index iCloud folders into your database so the files are always available in iCloud, but I would strongly not recommend you to do that, as iCloud is really aggressive with “optimizing” your local copy of the data and that can confuse DT.
Thank you all for your replies.
This group is very insightful.
I realized I was confusing between backup and sync, now it’s clear.
So I bought a new SSD and connected to my Mac for Time Machine.
Furthermore, I manage to export my “Personal” database to iCloud each week: I don’t think it will grow so much (it’s just PDFs of invoices and similar), so I can manage few GBs of upload each week.
@chrillek could you clarify how you ensure that Time Machine only backs up changed files?
I know that Time Machine is particularly efficient with APFS cloning - see Why nothing else can back up to APFS like Time Machine does – The Eclectic Light Company. Which leads me to believe that backing up without carefully considering how to export the files could cause Time Machine to back up every file, not just the changes. For example, I could create my backups by simply selecting the entire contents of my database in DEVONthink and dragging it onto my desktop, and then perform a Time Machine backup. Then before my next backup, I could do the same thing, replacing all those files with new ones. Does Time Machine know that some of these new files could be identical to the old ones?
I do not ensure that. TimeMachine does. Since several years, BTW. That is the whole point of this software. TimeMachine runs in the background, you do not really parametrize it (except telling it which folders not to backup, if you want to).
I’m not totally sure you are not at cross purposes here. When backing up a database, the database is one file; changes to the contents of the database will lead to the whole database being backup up. Certainly that has to be the case for encrypted databases (the backup software could not possibly know which individual file had changed within the database, as it would be unable to open the database), but to my knowledge it is also the case for non-encrypted containers too. So: any backup software worth the name will be able to produce incremental backups, only backing up those files which have changed since the last backup. But a container is one file, regardless of its contents - change the contents and in the eyes of the backup software the whole container has changed.
Yes; and or no. If you actually overwrite all the files, I would guess they would all be backed up (the metadata will have changed); but if when copying you select to copy only newer files, then effectively the same would apply to the backup. But why would you want to go down that route anyway, rather than simply backing up the database as is?
For those wishing to bend Time Machine backup to their will, see the terminal command “tmutil”. Fully documented with it’s own man file, “man tmutil”.
Me: I just run TimeMachine, excluding a few folders @Blanc mentions. It works for me. I’ve recovered a few full DEVONthink databases (macOSX “packages”) over the last decade. I don’t mess with it as I have other things to do.
TimeMachine to NAS (RAID5) that is redundantly backed up to USB and Backblaze B2
Backblaze on the “main” iMac
Chronosync copying very important stuff to USB attached to iMac. Important stuff includes the Zip Archives (which are created by a cron/AppleScript setup) of DEVONthink databases.
Over the top. For sure. Don’t care. Have had too many disk crashes in my time, esp. laptops.
Based on my understanding of how Time Machine handles backups on APFS, I’m not so sure that’s fully true. I’m pasting an except from a blog post that explains it. The author is talking about the benefits of backing up to APFS compared to HFS+ in time machine.
“There’s a clear advantage to this new scheme in that it functions not just with whole files, but with changed blocks within files. Just as a snapshot references the data blocks which make up each file, so a snapshot-based backup can back up individual blocks which have changed, which can be significantly more efficient in the storage space required.”
So as it pertains to DEVONthink databases, barring any special behavior that I’m unaware of, I would think that Time Machine backs up only the data that changes between backups in a DEVONthink.
Thanks very much for that; that coincides with what I have observed with regard space used, but hadn’t had time to investigate further (and actually my own post in the thread on backup strategies suggests something similar, so my knowledge has regressed ). Thanks again
(I think “only the data which changes” is probably also not true; based on my observations of space required, I suspect the “blocks” in question are not the size of individual files, but noticeably larger. I admit, however, that suspicion is not equal to knowledge.)
). Thanks again 