I ran Very and Repair on a database and it reported 3 missing files. I looked in the log to see what they were and established that the files were present in the correct place ( don’t understand why they were missing, but I opened them in their native app and they were fine the missing ones seemed to be dupes).
I read a post here that said rebuilding the database would clear the missing files, so I did a rebuild. After which there were NO files or folders in the Database in DT. The size of the database in ~/Databases was still 6 GB, so I figured the data had not gone.
I then tried Repair and Verify and ended up with 4000 orphaned files.
Then I restored the database from Time Machine from an hour previous, and am back up and running…phew!
As a new user of DT3 this has really scared me. What might I have done wrong?
This should definitely not happen and so far it’s the first report of its kind. Where is the database located, was there enough disk space available? Or is it an encrypted database?
Finally, rebuilding can’t repair missing files (only if there are also orphaned files having the same filename). But missing files are logged to Windows > Log and the contextual menu can be used to either trash or reveal these items.
Databases are in ~/Databases.
I have 70 GB free on the internal drive. I have six databases totalling 15GB. The affected one is 6GB.
Databases are not encrypted.
I understand that Rebuilding doesn’t fix missing files, but having established that they (or copies of them) were actually present, I went ahead to rebuild so that they would stop appearing as missing. I assume that rebuilding is like reindexing so if they are missing they won’t appear after a re-index.
Searching here for “Rebuild” does find some similar events eg this one, although there are differences.
A copy of your logs would be great. After rebuilding the copied database please choose Help > Report Bug while pressing the Alt modifier key and send the result to cgrunenberg - at - devon-technologies.com - thanks!
Also, though I have never personally seen the issue you reported (but have had a handful of tickets over the years), my first reaction would be to hold the Option key and choose File > Restore Backup and roll back the metadata.
When you say that your first reaction would be to “hold the Option key and choose File > Restore Backup and roll back the metadata”, do you mean that is what I should have done when I first hit the problem? …or if it happens again?..or now that I have restored from Time Machine?
I have done the test rebuild which did not repeat the problem, although the log showed a large number of other “no index”, “no text” and “unknown format” errors. These only apply to PDFs, which all appear to be fine when examined. It is all in BR #112867 which I hope you can see.
Duplicating a Database and renaming it for test purposes may not be such a great idea. I renamed the copy of my MIKE database to TEST and opened it. The log reports that it has the same UUID as MIKE and the name in the list of open databases was MIKE. I renamed it there (via Database Properties) to TEST to try and keep it separate from the proper MIKE.
Then TEST started replacing MIKE on my other sync’d devices and a lot of uploading and downloading went on. I think have now deleted TEST and got everything back to MIKE on all devices. I should probably have disabled syncing before starting on the TEST experiment.
I am rather inclined to quit while I am ahead and stop investigating/experimenting.
Can I make a suggestion completely off-topic with regard to the database situation? It does have to do with backups though.
@DrJJWMac mentioned “old school” CD/DVD’s as a backup medium a while ago. That made me reconsider them as an extra safety net to versioned on/off-site backups like Time Machine.
The most important reason to restart using them as an additional medium is the risk of falling prey to some kind of ransomware that seem to become more and more prevalent. Although optical media have it’s obvious downside they are the only media that are read only AFAIK.
To be clear, a good backup schemes should be able to withstand ransomware, but some apparently perform very nasty actions that are hard to protect yourself against. Hence the optical media.
I would not rely on them solely though, store them in the dark and test their content at least once yearly. Of course, there will always be some data loss, as you will probably not backup 15GB to DVDs or Blueray every hour. Whether you should use them is up to you of course.
Yep. But apparently they do still exist. The nice thing is that most drives have become quite cheap. Apple is actually still selling the Superdrive.
With ransomware issues appearing more frequently these days, I wouldn’t be surprised if there will be some kind of resurrection of these type of media. Or hard disks with a physical read only switch, but that requires human intervention.
People might also see the investment in a new drive as an investment in their bedroom, as it has the potential to improve one’s sleep
Thanks. Ransomware is indeed another good reason for offsite backups (along with fire and theft etc), since Ransomware can attack any attached backup, like Time Machine. I also backup exported DEVONthink Database archives to CrashPlan, which I find easier to keep up to date than CD/DVD/BDs. I do have a lot of photos and video which wouldn’t need updating as often, but with 4.3TB on CrashPlan, I would need more than few DVDs! Even BDs (which went up to 50Gb last time I used them) wouldn’t really cut it.