Database corruption

DevonThink Pro, v 1.2
Including earlier versions, I’ve been using the app for a year + and have accumulated 300+ stored documents, mostly single pages. The doucments are a mix of PDFs, e-mails, & web receipts. So, I’m thinking what a great app with no hint of the impending disaster. Today I go back and look for a recently saved docuement: it’s listed ok (along with the 300+), but selecting the document (single click) reveals it to be blank, as are the overwhelming majority of the other documents. I’ve had no hard drive crashes or known viral issues. So I select to Rebuild the Database and am told that there are several hundred errors; after the quick rebuild, all of my documents vanish, no longer being listed. Well, I’ve got 10 backups, so I’m not sweating yet. I select the oldest backup (August 16th) to restore, figuring maybe the corruption had not occurred that far back. The restoration went well and the 300+ documents are again listed. After examining a large number of documents, I find that about 3/4 are only minitures, or thumbnails, and are illegible when enlarged. Additionally, I have duplicate external hardrive backups that are done daily & weekly, but of course, these are simply replicas of the corrupted database.
Fortunately, I do have a copy of version 1.0 on my laptop and those files are ok through 8/10/06 - I hadn’t gotten around to updating the software on the laptop. I looked back and verified that version 1.2 was installed on my primary Mac (the one with the corrupted database) on the release date of 8/14/06. So, I guessing the database corruption is related to version 1.2. But, I’m open to other suggestions or input. Any one have a clue?

Thanks

I seriously doubt that the problem arose from installing version 1.2. I’ve been running pre-release betas as well as 1.2 for quite some time.

One advantage a notebook computer has over a desktop machine is that it’s not nearly as subject to problems arising from voltage fluctuations in the power supply. I don’t know what part of the country you live in, but many areas last August were experiencing low voltage fluctuations because of heavier than normal electrical requirements (all those air conditioners running). The result can be file corruption, even though the computer didn’t turn off.

I’ve got my desktop Macs on uninterruptible power supplies (UPS) for that reason (along with the occasional hurricane or thunderstorm).

Another thing to watch out for is free space on your hard drive. The operating system (and often your applications) need available free space in order to write temporary and swap files to the drive. Apple engineers recommend at least 15- to 20% of your drive space should be free. When you run out of free space, the operating system can start overwriting files, which will corrupt them, and the disk directory can be severely damaged.

My main database contains about 20,000 documents and I started it more than 4 years ago. I make it a practice to manually run the Verify & Repair routine, followed by Backup & Optimize whenever I’ve made significant changes in the database, so that I’m assured of having a good, recent backup. I have only three Backup folders, which should be enough. And I periodically use DT Pro’s Scripts > Export > Backup Archive to make externally saved archives.

I’ve never seen the “blank document” problem, and it’s been almost two years since I saw an error report after Verify & Repair – and that error was repairable.

It’s been about 2 years since I had to run Restore Backup, and that (fortunately) was a very recent backup. I think I did run Rebuild Database once, when Christian recommended that after the update to DT Personal 1.6 or 1.7 (I forget which). But I don’t see Rebuild Database as a normal routine and haven’t used it as such.

When document views become icons (small versions of the document’s first page), that means the Path to the document has been broken. Normally, that occurs when the document is externally linked and the document has been deleted or moved in such a way that DT Pro cannot find it.

Select one of those “shrunken” documents and open the Info panel. Check the Path of the document. Is the file to be found at the location specified by the Path? If so, try opening the file. Has it become corrupted so that it can’t be opened?

I recently had a similar problem to that of jtarpley. On moving my DEVONthink Pro database from my documents directory on my 2 year-old iMac to my new 24-inch iMac, the majority of my database entries seemed to disappear. Some groups showed zero documents within them or disappeared entirely and others were missing a large proportion of their documents.

I had run Verify & Repair and Backup & Optimize once a day or more often, and after the problem arose I ran Rebuild Database, but to no avail. The older iMac has about 40% of its hard drive space free, so that cannot be the problem. I’m obviously doing something very wrong, as others don’t seem to have similar problems. I would welcome any suggestions.

William, is it possible that DT Pro was open when the database was copied? That can result in database problems.

If you still have the older iMac available try opening the database and using Scripts > Export > Backup Archive. Then copy the archive to your new machine, unzip it and double-click to open it. Is it in good shape?

Remember that if you have indexed files they must be available in the same relative path on the new machine in order to be available to your database.

I’m having the same issues as William and jtarpley, but I have not switched computers, not had any power issues, have plenty of room left on my hard drive and don’t have any issues with the size of my dBase (it’s not very old or large).

Any other thoughts on what might have caused the problem? I have the time and patience to rebuild and restore once, but beyond that will return to using another dBase program (which doesn’t have some of the same features, but appears to be significantly more reliable at this point).

Thanks.

I think I may have solved the problem I was having concerning losing data from my DEVONthink database while transferring to a new iMac. I also switched to a demo version of DEVONthink Pro at the same time, assuming that DT Pro would open the DT database without difficulty.
What I did to solve the problem was to open DEVONthink (not DT Pro) on the old computer, whereupon the correct data appeared. I then did a verify & repair and an optimize & backup. The 1-10 databases in ~/Library/Application Support/Devonthink now had today’s time and date. I closed Devonthink.

I then copied the files database 1-10 to a CD and trashed the DT Pro demo on the new iMac, then copied the databases from the CD into the ~/Library/Application Support/DEVONthink folder on the new computer. Now when I opened the DEVONthink application on the new computer; the proper data appeared. I don’t think I need the additional features of DT Pro, so for the moment I’ll stay with DEVONthink.