Confused on the correct way to backup all my devonthink data, advice appreciate :)

I know @eboehnisch is using Arq for his Mac backups. However, I’d schedule the backup to happen at a time when you’re not using DEVONthink or DEVONthink To Go and syncing.

Thank for that. So Arq should only to be used when DT is shut down.

An application that uses a scheduled snapshot-style backup, i.e., it gathers info from one point in time, “freezing it”, then doing its backup in the background should allow for simultaneous use. However, as we always advocate caution on behalf of peoples’ data, the safer option is to do your backups during a period of inactivity.

Technically, this can also free up resources for the backup to happen.

2 Likes

Thanks for that. Is there are better way to backup to Arq?

I’m not using it here, so @eboehnisch or some other Arq user would have to weigh in on more specifics.

1 Like

As Jim mentioned, that’s not really necessary.

I just commented on this in another thread. Arq and Carbon Copy Cloner documentation also states that it doesn’t matter what you do while the backup is running. I even deleted a test database once while a backup was running and was able to restore it.

1 Like

I have Back Blaze and use Time machine backups onto some hard drives. However I keep my data files on a thumb-drive, 64 GB one that is solid state. I think I have used it once, to date I have never had to get anything from Back Blaze. However once woulld be enough to justify the cost. I still really use the 3-2-1 strategy. Three backups of my vital stuff in whatever form, including the thumb drive in my case, Two different ‘types’ of proper backup, not synching systems and the like one must note. And one ‘off site’.
I have to admit the thumbdrive is what I think of as my ‘real’ back up and the simplest and easiest to use. I have nothing else on my computer that is of any value to me other than what is in DEVONthink 3 by the way. So I don’t have things scattered around: even so I back up the whole system in various ways.

1 Like

Any chance to have a kind of incremental backup of a database?

I mean: I’d like to have the exact mirroring of the database on a cloud, with the folders and the files as I see them inside the DT3 App, not as they are stored in the HD.

I may export the database on a weekly or monthly base, but each time I’d have to re-sync the ALL the files, and it becomes cumbersome because I have many Gbs inside.

(hope it makes sense)

So effectively what you are looking for would be an incremental export; that could presumably be implemented by offering an “overwrite/duplicate/do nothing” option on export (currently export will duplicate files already available in the export destination; to my knowledge the user cannot currently influence that behaviour).

Agreed: files that have not changed don’t need to sync again.
Only the new files and the files that have been changed.

Are you sure that the DEVONthink sync does not already work like that?

Agreed, but my request is different.
I mean to have an exact duplicate of what I see inside the App, but on a Cloud.

I know I can export all the files to the Cloud, but each time I have to upload ALL the files, and it’s not viable for me due to a slow connection.

I’d like to have synced only the file that have been created or modified.

1 Like

You are aware that DEVONthink does not send recognisable (to you at least) files to the cloud sync location. if you need to duplicate indexed files to cloud to be any use to you, you need to do it yourself. Chronsync, rsync, and many others can do that. i cannot think how to get the imported files out of DEVONthink but surely you (or get someone else) could write some code to do that or maybe a menu command i am not aware of. but again, getting the files to the cloud in that form up to you.

this i do think is about backup so best to start a new thread for this.

I think you are talking at cross-purposes here; as I understand it, @IvanPsy is performing their own backup by exporting files from DT - the advantage being that if you need to replace only one file, then a backup made of that export will allow you to do so easily. Obviously this is not a complete backup - the metadata is missing. It is just an addition to the traditional backup of the database as a whole.

ok. easy to confuse a befuddled old guy. :wink:

I use Arq Premium backup software, it does incremental backups

I may export the database on a weekly or monthly base, but each time I’d have to re-sync the ALL the files …

I do the same; export and backup
Arq continues to be incremental with the exported data
(“Only the new files and the files that have been changed”)

1 Like

Arc, Chronosync, rsync, CCC and surely many others all do incremental copies.

I resurrect the thread to expose a problem not too dissimilar to @IvanPsy: I too have a rather slow connection and backing up DT to the cloud is pretty bloody. I use Arq to copy the database folder to cloud but every time the scanning and uploading job is very time-consuming: very heavy files like .dtMeta are always updated, in addition to any internal backup folders (perhaps they can be excluded?). To make it smoother I was thinking of creating copies via the sync function on the same machine and uploading those to the cloud, which I have noticed are more fragmented - and therefore probably more manageable. Is this a viable idea? And if so, could it possibly create problems for restore operations?

As a starting point, I recommend you re-read the section “A Word about Backups” starting on page 19 (3.9.0 version) of the outstanding DEVONthink Handbook.

Me, I would never backup to a cloud service that is based on sync, e.g. Apple iCloud, Dropbox, etc. That is because if a flaw develops on the local copy or on the server, then, “poof”, backup gone because the flaw is “synced”. And as you have issues with the slow network connection, the risk of corruption for you is probably high. IMHO.

I have a backup system which is probably “over the top”, but I don’t care. I won’t bore you with the details as it’s basically a very redundant 3-2-1 system (see the “interweb” thing to read about that.

Basically, I’d suggest you first get TimeMachine going to a local external/USB drive. TimeMachine backs up everything unless you exclude things. If you want offsite backups, consider a BackBlaze (or equiv) subscription. If you need to use arc or similiar, copy to a local external/USB Drive. Backblaze will backup connected external/USB drives.

In addition to Time Machine, I’m using Arq Premium to do an off-site backup (through their service), and no other cloud provider. However, loading and updating DTs is always very time-data-consuming, and I am looking for ways (or wildcards, e.g. on the internal backup folder to the databases themselves) to make them more agile.