Safe backup timing with Time Machine + Synology Drive + Hyper Backup?

[Martin59]

Hello,

I would like to double-check whether my current backup workflow is safe for DEVONthink databases, especially regarding possible inconsistencies.

My setup

A. Time Machine (local)

I have a Time Machine drive attached to my Mac for hourly backups.

B. Synology Drive → Synology → Hyper Backup → Synology C2

Synology Drive is configured as a one-way upload (Mac → NAS) and continuously uploads my .dtBase2 files to the Synology.

Once a day, Hyper Backup performs incremental/versioned backups of the Synology copy to Synology C2.

C. Quarterly manual export

Four times a year I also do a folder/document export for quick offline access.

My concern

If Time Machine or Synology Hyperbackup starts a backup at the exact moment DEVONthink writes something into the database:

• Can this create an inconsistent or incomplete copy of the .dtBase2 package on Time Machine or on the Synology?

• Would it be safer to run Synology Drive only once a day, e.g. after finishing work or after closing DEVONthink, and let Hyper Backup upload that version?

I searched the manual and this forum but could not find a clear answer.

If there is already a relevant discussion, a pointer would be appreciated.

Thanks!

[BLUEFROG]

Synology Drive is configured as a one-way upload (Mac → NAS) and continuously uploads my .dtBase2 files to the Synology.

No, I would not allow this. You shouldn’t use a process that purports to be continuous or real-time& backups.

I searched the manual and this forum but could not find a clear answer.

Getting Started > A Word About Backups…

• Would it be safer to run Synology Drive only once a day, e.g. after finishing work or after closing DEVONthink, and let Hyper Backup upload that version?

Yes, this would be preferable. I don’t use their app but if it allows for multiple configurations, you may be able to exclude your DEVONthink databases from the continuous backups but include them once a day in a second config.

[Martin59]

Does this mean that Time Machine isn’t safe for DEVONthink databases?

[BLUEFROG]

Does this mean that Time Machine isn’t safe for DEVONthink databases?

I’m not sure how you arrived at that idea. Time Machine isn’t doing continuous backups. It is doing snapshot backups. Always has.

[rmschne]

Apple’s TimeMachine is discussed in the section of the DEVONthink Manual and Help that @bluefrog pointed you to. Please read it.

Important question. Are your DEVONthink database residing on the Synology NAS or on the local hard disk? My hunch from what you say that you might have them on the NAS? That may bring its own risks as discussed in previous posts here. Need a very fast network connection and NAS device is the advice from DEVONthink Support.

I have the same kit as you. But without trying to figure out what you are doing–as it seems different and more complex than me, but maybe how you explain it). What I do.

• All DEVONthink databases located local on ~/Documents/DEVONthink folder
• I use DEVONthink Menu: File → Export - DEVONthink Archive and save these zip files into ~/Backups/DEVONthink. From previous posts here I have a scheduled script to do this for all open databases once a week, and I occasionally do it upon manual instruction. A Hazel app process clears out all but the last five backup zip files.
• The above files are backed up (once per day) along with the entire machine using TimeMachine to two locations:
  • A share on Synology NAS specifically configured per Synologies instructions to be a backup location for Time Machine, and
  • A USB drive attached to the Macbook
• The local computer also backed up remotely to an Arq server every morning.
• I also use Chronosync with a weekly scheduled job to copy all from ~/backups/ (which has backups for other apps) to a Synology share named “Backups” and each app goes into its own subfolder. There is no conflict with DEVONthink as these are not DEVONthink database files–only zip file archives.
• I use Synology Hyperbackup with daily scheduled jobs to backup to two locations all the important folders/shares on the Synology NAS (more than DEVONthink stuff) to:
  • a big USB drive attached to the Synology NAS unit
  • selected important things (photos, for example) to Dropbox’s BACKUP service (not their Sync service)

I don’t understand what you mean by your Quarterly process and “quick offline access”… If I wanted a quarterly snapshot of the DEVONthink database(s), I’d made an Archive zip copy, give it a good file name, and store away securely, say ~/DEVONthink/Quarterly/DEVONthink, or on the NAS in a shared, say, named ARCHIVE/Quarterly/DEVONthink or something.

[Martin59]

@BLUEFROG

I’m not sure how you arrived at that idea. Time Machine isn’t doing continuous backups. It is doing snapshot backups. Always has.

Thanks again for your helpful answers.

There is one technical aspect I’m still trying to understand. When I add a PDF to the database, DEVONthink updates several internal files (DEVONthink-x.dtMeta, DEVONthink-Internal.dtMetaStore, previews, SQLite/WAL files, etc.) in multiple very fast steps. Synology Drive shows each of these files being uploaded individually, and when I close DEVONthink, a few more internal files are written again. These writes are clearly not atomic.

Because of this, I am trying to understand the difference between Time Machine and my Synology Drive → Hyper Backup process. Time Machine uses APFS snapshots, which freeze the entire filesystem at one atomic moment, so the very short update window inside DEVONthink is unlikely to be captured. Synology Drive, on the other hand, uploads each changed file immediately and can take several seconds to transmit them, so the temporary “in-between” state of the database remains visible on the NAS for a significantly longer time.

From a purely technical perspective, both Time Machine and Hyper Backup share the same theoretical limitation: if a backup occurs exactly while DEVONthink is updating multiple internal files, the snapshot could capture a partially updated state. Locally this incomplete state exists only for a very brief moment, but on the NAS it can persist for several seconds simply because Synology Drive needs time to upload each changed file individually. And since Hyper Backup backs up whatever state exists on the NAS at its scheduled time, it can preserve this longer-lasting incomplete version as well.

One further thought: if the Time Machine drive is permanently connected, it automatically decides which snapshots to keep (hourly/daily/weekly). Since these snapshots are created without regard to whether DEVONthink is open or closed, it is theoretically possible that Time Machine never actually retains a snapshot of a fully closed .dtBase2 file. In other words, even if Time Machine occasionally captures a clean state, its rotation may discard that and keep only snapshots taken while DEVONthink was open. This does not mean Time Machine is “unsafe,” but it does mean that retaining a known-closed version of a DEVONthink database is not guaranteed.

So I am wondering whether the safest practical approach is simply to work in DEVONthink during the day, close DEVONthink when I’m done, then let Synology Drive upload the finished database package, and let Hyper Backup run later in the evening. This way only a fully consistent, closed database gets backed up.

[Martin59]

Thanks for your reply. Just to clarify: my databases are all local, so none of the NAS-based risks apply in my setup. The timing aspect I outlined in my previous post already explains what I’m trying to understand, so I won’t repeat that here.

Regarding the quarterly exports: this is simply an additional convenience layer. Since DEVONthink cannot “look into” Time Machine or Hyper Backup archives to check whether a particular document exists in a given backup state, I would otherwise need to restore or open the entire 40–50 GB database just to verify one file. The quarterly folder/subfolder export avoids that: I can browse and search individual documents instantly with macOS (including OCR) without touching the main database.

About Time Machine: your setup makes sense, but this also highlights an important detail. If a Time Machine drive stays connected all day, its snapshots may well be created while DEVONthink is still open. Attaching the TM drive only after finishing work and after DEVONthink has been closed would avoid that and ensure that preserved snapshots contain a fully closed, consistent database. That’s essentially the core of the timing point I am evaluating in my setup.

[rmschne]

@Martin59

About Time Machine:

I don’t know that what you are concerned is not already handled by the DEVONthink app. What I read in the DEVONthink Manual suggests that TimeMachine takes proper backups. @Bluefrog, lead of DEVONthink Support, says same above.