DEVONthink: All custom metadata lost (and re-covered)

mysterious things happened to my largest DEVONthink-DB on my iMac (Sonoma) today: when I started DEVONthink the activity window displayed for a huge number of documents a “creating hash” (or similar, can’t remember the exact phrase). Afterwards, when I started to work with the database again, I noticed that all custom metadata had been erased.
At once I stopped DT-sync on all devices (MacBook, iPhone, iPad and tried to recover the .dtbase2 file from TimeMachine, but could not import that file. (Now I know why TimeMachine is not the way to restore a complete database)
But I did successfully recover all metadata this way:

  • exported all metadata from the DB on my MacBook (Ventura, all metadata intact) with “DEVONthink → File → Export → Metadata (JSON)” to a JSON file
  • converted that json-file to an xls-file
  • imported the xls-data into a FileMaker-DB
  • wrote an AppleScript which successfully re-imported the metadata from FileMaker to the DEVONthink-DB

I am very glad it took only 8 hours to repair the database. But after all the shit I experienced before with Sonoma I wonder if this problem too results from the Sonoma update.
Maybe this post helps if any of you in this community runs into similar problems.

1 Like
  • Where is the database located?
  • Is this the first time you’re handling this DB in Sonoma?
  • db is located in the user/databases folder (default DEVONthink db-location)
  • no, handled it before (always in use)

And had anything happened on the Ventura Mac, like a DB rebuild, crash, etc.?

nothing at all. Ventura works fine, problems only arise on the Sonoma iMac.

Imported or indexed data?

no indexed data, only imported into DEVONthink

Please choose Help > Report Bug while pressing the Alt modifier key and send the result to cgrunenberg - at - devon-technologies.com - thanks! Maybe there’s some useful information in the logs.

Just sent the log files to you and development. Ticket id is #704757

Thank you for the logs! The storages for internal and custom metadata in several databases were (and maybe still are) broken. These files are actually completely independent, therefore the only explanation coming to my mind are external causes.

Did you check the filesystem? And was there enough disk space available before installing Sonoma? Please note that the amount shown e.g. in the Finder is not necessarily accurate on APFS filesystems and that there might be a lot less disk space available.

I checked the FS. Results: enough free space, but the filesystem on the data disk is corrupt. “First aid” in Disk Utility told me that I have to run fsck on that volume. I have an open case with Apple regarding TimeMachine and iCloud and just sent a message to the second level technician regarding the disk errors. I am not a tech expert and do not dare to run fsck without technical support.

fsck can be run so that it checks only without modifying anything. man fsck should tell you about that.

Ouch :flushed: Fingers crossed you can get this addressed soon.

Thanks! Will be on the phone with Apple tomorrow and keep my fingers crossed.

Back on Ventura and all data recovered. :grinning: Apple Support was extremely helpful. Still waiting for OneDrive to sync but that should be only a matter of time.

congrats! :partying_face::tada:

1 Like

Rather than the procedure you used, couldn’t you (once the filesystem was fixed) just delete the iMac version of the database and copy the MacBook version to the iMac? Just wondering, in case something like this ever happens to me (I also DT-sync between an iMac and a MacBook). I also don’t use indexing, everything is imported. Perhaps this would only work if you don’t use indexing.

He had deeper problems that needed to be corrected.

I could not because the MacBook had not been synched for about 2 weeks, so the main database on the iMac had a lot of documents which weren’t on the MacBook. In my workflow tags are more important than custom meta data. So the decision was easy: recover most of the custom meta data or lose several tagged documents.