I hadn’t been aware of file compression utilities such as Cluster, since the old Classic days on Macs – when the capacity of hard drives might be rated as 40 MB, or even 20 MB. Back in those days there were several utilities available to squeeze more data onto a hard drive.
Sometimes they worked without obvious problems, and sometimes their use resulted in hosed data.
I’m not criticizing Cluster specifically, but I’m inherently suspicious of any utility that deliberately modifies OS X, and this one does. Apparently, Cluster can cause problems with some files, as the developer notes that it is “configurable for your safety”. Personally, after years of experience in software support during which I’ve seen problems caused by even very popular utilities, I stick to a pretty stock OS X on my own computers and I don’t have stability problems or data loss.
Did Cluster cause your problems? I don’t know.
You mentioned that you have had DT crashes in the past. Although most crashes are fairly graceful in OS X, any crash (System or application crash, Force Quit, power outage or forced shutdown) results in improper closing of open databases and potentially could result in an incomplete or damaged database, so I take them seriously. They should be extremely rare.
One of my computers that holds “master” copies of my databases hasn’t had a crash in more than two years of intensive work. It’s gone through a series of OS X upgrades and updates and updates of DT Pro Office flawlessly. When running Verify & Repair, I’ve never seen an error report on these databases. I’ve never seen a “missing file” report or other problem. That’s why I can say that my databases are rock solid stable.
Very often I’m working with prerelease betas of DEVONthink applications, and I generally do that on a different Mac, with copies of my databases. Even on that Mac, I’ve had only 5 crashes in more than a year, two of them resulting from the same situation involving import of an old WebArchive file that Snow Leopard doesn’t like. This is the Mac that I use when I ask a user to send a copy of a file or database that has caused problems on the user’s computer. So I have to say that even my “beta test” Mac is very stable.
Perhaps your database still holds the files that it reports are missing, but has lost track of them because of damage.
Quit the DEVONthink application. In the Finder, select the database file, Control-click and choose the option to make a compressed (zipped) copy of the file. Reserve that copy for possible future use in data recovery.
Launch the DT application and choose Tools > Rebuild Database.
When the rebuild operation is complete, check the Log (Window > Log) to see if there’s a list of files that failed to be included in the rebuilt database. If so, Save the list as a reference for attempts to find those files.
Now run Tools Verify & Repair. No errors should be reported. Now look for a special group name “Orphans”. Fingers crossed, your files that DT had “lost” will be included there, and you can now file them properly in your groups.
If problems remain, send a Support message citing this thread and the results of your rebuild attempt and we can try other approaches to recover data.