DEVONthink Database Storage Strategy

Hi. “Cold storage” is a cloud storage service. The only difference with a “normal” cloud service is that you don’t have an instant access to your data. I do not expect to do anything else with my cloud data but to keep an additional copy of my files. In another words, if something goes wrong with my DTPO database files on my SSD I can eventually download a reserve copy of my DTPO files from my cold storage account.

Humm. My best advice for this question is to try to do a “restore” from your cold storage location and see if it works for you. However …

Remember that DEVONthink “databases” are actually MacOS “packages” holding many thousands of files. Will the restore of all this always work as intended? Dunno.

I guess (without doing any testing) that I would use DEVONthink’s Menu: File/Database Archive … to create a zip file archive backup of the database package and put that single file into cold storage (along with previous archives?). There is also a “Daily Archive” script available in DEVONthink. I’ve set that up to run weekly (with a cron job).

Before doing that, though, if you haven’t already implemented a backup regime on your device, e.g. using TimeMachine or equivalent, I’d do that first.

Also see section “About Backups” in the outstanding DEVONthink manual (and in Help) for further elaboration.

No you should not put your DEVONthink databases in any cloud-synced location.

1 Like

Ok, thanks. I assume only exported archived zipped files of my DTPO databases are recommended to be stored in a cloud account. And, I guess, it’s not recommended to do the same with the .dtbase2 files.

Zip files are probably fine, but you could also use ARQ to backup DT databases in encrypted form to your cloud storage services (archive storage such as AWS Glacier Deep Archive or Google Coldline / Archive are supported). Restores can be the whole database or single files from within, which won’t work with zip files.

.ZIP files are fine to put in your cloud storage.

1 Like

Does anyone have experiences (Performance) with many (50-100k) PDF files in DT indexed on a NAS using wireless connection?

May there be conflicts if the configuration is like this:

There are 3 (or later more) clients with each running a DT database. The DT database on the first client (Mac mini) “uses” the indexed files located on internal SSD of the Mac mini. The Mac mini shares the folders (with the indexed files) on the LAN. The other clients (No. 2 and 3) are connected wireless. Could the DT databases on the the clients No. 2 and 3 use the shares to index these files too? AND can the three DT databases being synchronized via local sync store without conflicts and data loss?

I’m in bed, so here’s a short response…
50-100,000 PDFs…
Indexed on a network volume…
Accessed via wireless connection…
equals expected performance issues.

You should be set up on a hard wired Ethernet connection, preferably gigabit, for what you’re attempting here.

2 Likes

:expressionless: … thanks for your response

Nonetheless, I have been wondering for some time why DT does not offer manual download. This is a wonderful option with DTTG. That would result in a faster sync and would save disk space. :man_shrugging:t3:

Shallow sync was implemented in DEVONthink To Go due to the lower available resources and iOS’ aggressive system management. macOS is much more resource friendly and as more easily extendible with external drives, etc.


There are 3 (or later more) clients with each running a DT database. The DT database on the first client (Mac mini) “uses” the indexed files located on internal SSD of the Mac mini. The Mac mini shares the folders (with the indexed files) on the LAN. The other clients (No. 2 and 3) are connected wireless. ?

  • On the emboldened part, how it it sharing?
  • Where is the NAS in this ?

via SMB (System Preferences > Sharing > …

In this scenario there is no NAS… I thought that with this configuration there would be at least one less connection to the NAS and thus some resources would be saved.

I thought so. But it would be super cool if one could only keep the data that is needed locally, especially with a very large database. In our use case most of the data is actually needed locally only every few weeks or even only every few months. We are currently thinking about how we can reduce the amount of data on client Macs in the future. Therefore, we are also considering swapping it out as indexed data on the NAS. It would of course be much easier if “shallow sync” would be available for DT. The background to this is that we want to use a single database for all client files. So far I have been the only one and the secretary to use DT. In the future, everyone else in our office also wants to use DT. In the case of a single database for all people in our office, this would mean that each consultant would also keep all client files of the other consultants local. After a few months / years that would be several terabytes of data (on each Macbook!)

Would it be very difficult to implement that in DT for macOS? Maybe that would be a feature that many DT users are looking for!?!? :raised_hands:t3:

Now that’s what we want to hear! :wink:

  • Are you creating one massive database containing all clients’ data?
  • Have you considered creating per client databases (which is how my mind naturally would build a system)?

After doing advertising the last three years for DT, I’ve decided that everyone has to use DT by mid-2022! :wink:

Yes.

From my current point of view (maybe this will change later this day ;-)) it will be impractical. The reasons at the moment are:

  1. Different consultants/lawyers need access to the data. Per client database would mean, to share/sync these databases separately (everyone has to enable the sync for each database).

  2. Sync would be slow with hundreds of databases. (Or am I wrong about it?)

  3. It would be time consuming to create a database that has certain settings by default (user / password for sync; file folder structure with indexed folders; …). (While i’m writing this, my point of view is changing a little bit.) - I will check, if there is some Apple Script code to do that automatically…

  4. We need to preserve the added date of an item for some reasons. If one move an item to another database (accidentally or not) it would get a new added date.

  5. And some more not such important reasons…

We are DEVONtechnologes and we approve of this message! :smiley:

  1. Different consultants/lawyers need access to the data. Per client database would mean, to share/sync these databases separately (everyone has to enable the sync for each database).

If you were running the Server edition, it would be likely you could share the databases from one master machine and provide access to the individual databases to those who need access.

  1. Sync would be slow with hundreds of databases. (Or am I wrong about it?)

Syncing one massive database wouldn’t be any more performant than syncing many, thoough Development can weigh in on this as well.
Also, individual databases would isolate any potential issues with sync stores.

  1. It would be time consuming to create a database that has certain settings by default (user / password for sync; file folder structure with indexed folders; …). (While i’m writing this, my point of view is changing a little bit.) - I will check, if there is some Apple Script code to do that automatically…

Again, this could be handled in the per-database, per-user permissions in the Server preferences.

  1. We need to preserve the added date of an item for some reasons. If one move an item to another database (accidentally or not) it would get a new added date.

You can limit the permissions of a broadcasted database so such changes can’t be made except by certain individuals.

  1. And some more not such important reasons…

We’ll ignore those for now :wink:

Thanks a lot for your help. We will try per-client/case databases and also with DT server later this year.

You’re welcome.

PS: If you wanted to try out the Server edition on a machine, you can do this…

  1. Quit DEVONthink.
  2. In the Finder, select the Go menu while holding the Option key, then choose Library.
  3. In the Application Support/DEVONthink 3 folder, move the Registration.plist file to your desktop.
  4. Relaunch DEVONthink and it will be running in trial mode with the Server features enabled.

When you’re finished testing, you can quit DEVONthink, put the Registration file back in place and it should again be licensed on relaunch.

Note: If you previously ran the trial, the time will not reset. You can only run the trial for the duration of the remaining time.