Help with possible retrieval - HowTo?

DEVONthink
1

Hi,
I’ve run into a bit of a strange / awkward situation and would like some help to possibly retrieve some lost data.

Let me first explain:

  • I’m working on multiple macs, only intel machines. My most portable machine is a 2019 i9 macbook, apart from that, a couple of 2018 mac mini’s (there’s more but those are irrelevant)
  • What these machines have in common is the very small 256 GB on-board harddrive, ssd or flashdrive.
  • I also have two Promise Pegasus R6 harddrive enclosures that are used for file storage. (Thunderbolt connection, the older, slower type)

One of those 2018 mini’s was running Catalina up until recently, they now run Ventura and Sequoia (in an attempt to keep +/- up to date)
These machines are barely shutdown ever, mostly only to clean out memory etc from time to time when the need arises.

The Sequoia 2018 mini has the /User/home folder moved on to the Promise Pegasus R6.
On the Sequioa 2018 mini, one way or the other, the Promise Pegasus R6 thunderbolt cable lost connection. (It was not disconnected, but it somehow lost connection)

Of course the 2018 mini hung when I wanted to start working on it again, and after some searching I closed down as much as possible, also needed to do some force closing, did a restart, and then another restart after powering down the Promise Pegasus R6 completely.
Then upon restart everything was working again…

Then, because I found Devonthink to be quite slow (on the Promise Pegasus with spinning disks) I closed down Devonthink completely, and moved the Databases to the internal 256 GB flashdrive /Users/erwin/databases/

Now to Devonthink.
When Devonthink was up and running, I noticed that I had approx 8 ~ 10 days of folders / work missing. Checking on my other mac’s, I discovered that the sync of the faulty 2018 mini had somehow made those 8 ~ 10 days missing.

  • first checked Time Machine, they were not there (need to update backups)
  • then I though of my macbook, that was laying around closed and unused.
  • I then disconnected my wifi accesspoints to avoid the macbook from syncing
  • this way I could recover a couple of files, but there are still plenty missing (I used a USB drive, created a new DB onto it, and duplicated the files that I could find on the macbook (offline) to the new database on the usb-drive, and imported those on the other machines. (including adding the new usb-drive-database to Devonthink sync (wired network))

Now from the Devonthink handbook, I learn that Devonthink does internal backups:
INTERNAL BACKUPS
DEVONthink backs up its database index
every week, either immediately after opening
the database (if the backup is overdue) or
during the day when appropriate.
Note: The internal backups are mainly a
troubleshooting mechanism for use in the
application. Backing up your files, including
your DEVONthink databases, is up to you

Now my question:
Apart from realising that I need to change my backup scenario, I started searching what is possible with Devonthink?
Is there a way to check those internal backups? It might be possible to compare these on my different machines..
Also: If Devonthink sync decides that some files need removal, what does it do with them? (I couldn’t find anything in Trash)

Attached is a screenshot to give you an idea of my year/month/daily folder structure.

Image 12 - i9 screenshot - what was on 2025-06 with wifi off - no internet
Image 12 - i9 screenshot - what was on 2025-06 with wifi off - no internet482×452 30.2 KB

Thanks for all help and suggestions!

2

I’d guess “database index” does not mean “database”. And personally I wouldn’t even want DT to backup my databases regularly. That’s what backup programs are for.

2 Likes
3

Yes, indeed, index.
Now it would nevertheles be interesting to be able to check what it contains. Maybe I could derive what specific files are missing and do a reconstruct from there.

Another thing that I wonder:
Suppose you have 10 macs that are synced to the same webdav syncstore.
One of the macs drops our for whatever reason, and has not been sync’ed for a number of days.
Then it comes back online.

What could make sync decide to say:

  • Ah, those other 9 macs are out of sync and I need to update (delete) items on those 9 macs
  • instead of: Ah, that 1 mac is out of sync, (it’s missing items..) and I need to update this 1 mac
4

I am curious… why do you have so many Macs running without a diligent backup procedure in place? Remember, sync is not a backup.

No, you can’t compare the internal backups between machines. You can hold the Option key and choose File > Restore Backup to rollback the metadata on a database. I would do this on the faulty Mini first to see if it’s just a metadata issue.

The most recently edited database’s state is propagated via sync.

5

Long story: mainly because I don’t throw stuff away.
More to the point:

  • somewhere 2020’ish, bought a 2013 trashcan and a 5,1 cheesegrater second hand.
  • also the two 2018 mini’s

The two machines that feel the most responsive are still the trashcan and the 5,1 cheesegrater.

Then, with electricity getting more and more expensive, started looking into power consumption. (and also fan noise)
That’s about the point that the 2018 mini’s started to take over.

Backup procedure:
I have a couple of synology’s, most important is (was) an 8 bay model.
Unfortunately disaster struck and I lost access to all data. Mind you, I still have the harddrives, but somehow the data got borked. I’ve been working on a method to recover the contents. The only piece of software that worked that I found is DiskDrill (If not mistaken this one is at least Wind + Mac).
Disadvantages:

  • slow
  • you need the equal amount of free data available to write the recovered file to

In my case that means that I need 64TB worth of extra disk storage.
And I have not yet come around to break the bank on that one..
Looking into mergerFS and the likes to see what I can cobble together using a bunch of WD MyBook usb drives laying around…

6

Interesting, and thanks for helping out!
Would you recommend doing this with sync being disabled? Or doesn’t that matter?

Shot 2025-06-11 at 19.03.34
Shot 2025-06-11 at 19.03.34708×551 72.2 KB

7

Yes, I would disable sync first so you can see what the result is. In fact, I would try to isolate one Mac and get the database into a master state, including passing a Verify & Repair. Then I would do a manual backup taking note of the date of the backup. I would then remove the compromised database from another Mac and import the newly curated one and verify it. I would then follow that procedure on other Macs, including doing a manual backup of each as well.

PS:

Long story: mainly because I don’t throw stuff away.

Unless you’re running a museum, maybe it’s time rethink this :wink:

8

Great idea!
Might make some money to buy an ARM Mac! :grinning_face_with_smiling_eyes:

1 Like
9

Ok, from here I’m currently freeing as much space as possible on one of my Samsung Portable SSD’s.
I’ve chosen one that’s formatted MacOS Journaled, not sure if I have one that’s APFS, but anyway, it will take a while to move / copy the data to a network share. This will take a while as there are also virtual machine files on the disk, and from experience I know that these take extremely long to move/copy.

Question:
Do you have any helpfull pointers on how to add these folderlocations to a Time Machine Backup?

~/Library/Application Support/DEVONthink 3: This includes the Global Inbox, templates, favorites, smart rules, etc.
~/Library/Application Scripts/com.devon-technologies.think3: Include this directory if you’ve created or added your own custom scripts.
~/Databases: If you’re using the default save location when creating your databases, include this directory to backup your databases.

Or maybe some tool to configure “smallish” (not full) backups?

10

Time Machine is more of a “turn it on and let it work” application, made for easy full and incremental backups. For more focused backups, you’d be best served by using a secondary application (e.g., ChronoSync, Carbon Copy Cloner, etc.) or process (e.g., rsync).

11

Thanks a ton!
I have CCC, I’ll look into that.

1 Like