I never used to get these, but now I do get “inconsistencies” when I run verify/repair. It’s such a vague description that, unlike “orphaned” and “missing” files, which are self-explanatory, w/ inconsistencies I don’t have a notion what the problem is. Thus I don’t know how to reevaluate my workflow to avoid it. Generally speaking, what are inconsistencies? How do they tend to come about in DTP, and how can I avoid them?
A DEVONthink database holds many “pieces” of information about every document in the database. Some of that information comes from OS X, such as the filetype, size, Creation Date, file storage location, etc. Some of that information is generated within or by the database, such as metadata about the group location, index, tags, etc. associated with a document. When Verify & Repair reports inconsistencies, that means simply that some of the information doesn’t check out properly.
When you start your computer, it loads important parts of the operating system into memory. OS X is a sophisticated and very reliable operating system, but if there are hardware problems such as data loss or corruption because of disk errors, CPU errors or RAM chip errors, there could be errors in the loading of the operating system code. Hardware problems, fortunately are pretty rare nowadays. Much more common are errors in the operating system caused by the installation of utilities that hack the operating system code to change its features. Although Apple has restricted the ability of third party developers to rewrite Apple’s own OS code, this is still allowed in some utilities, and problems affecting database stability could result, especially when the next OS update is released. It’s a good idea to be suspicious of utilities that modify the operating system in some way, to offer features not provided by Apple.
Let’s assume that your computer has loaded OS components, applications and application data flawlessly into memory. Errors may creep in, and the odds are that the longer the computer is running, the more likely it has accumulated errors in memory. Although there are ways of checking for and preventing such errors, some of those techniques would be too costly to include in hardware that ordinary humans can afford. The universe can be hostile. A cosmic ray zipping through a memory chip could “twiddle bits” and cause errors. More importantly, your computer isn’t static. In operation you will be continually loading and moving data into memory, both physical free RAM and, when free RAM isn’t sufficient, swapping data back and forth between RAM and the disk using Virtual Memory swap files. The likelihood of read/write errors in RAM and to/from disk isn’t quite zero, so errors may accumulate. Indeed, if Virtual Memory swap files are heavily used on a disk that is running out of free storage space, the potential for overwriting data files on the disk becomes serious. That’s why Apple engineers recommend keeping at least 10% to 15% of the rated storage space free – and if one is performing disk-intensive actions such OCR on large PDFs, still more free storage space might be needed for safety.
Often, when Jim or I receive a really weird report of DEVONthink behavior in a query to Support, our first question will be “Does that issue persist after a Restart?” Quite often, transitory errors that have accumulated in memory will be cleared up by that simple measure. By the same token, continuing to work with a database when flaky behavior is happening is an invitation to database damage by multiplying errors.
Apple provides the Disk Utility app (Applications > Utilities) to verify and correct Permissions and the disk directory. It’s wise to do that periodically. I use an OS X maintenance utility (C0cktail) and also recommend the free utility, OnyX. Not only can such a utility check and repair Permissions, but also a number of other routines otherwise available via the Terminal app can be performed in a more user-friendly way, such as clearing caches and rotating logs.
Some operations, such as moving large blocks of documents from one database to another, can stress your computer’s resources, especially if limited free RAM and free disk space remains available. If I’m planning to make significant changes to a database, I’ll first run Tools > Verify & Repair. If no errors are reported, I’ll then run Tools > Backup & Optimize, so that I’ve saved a good, recent state of the database. After making the significant modifications, I’ll again run Verify & Repair. If no errors were found, follow up again with Backup & Optimize. But if I screwed up the database, I can revert to a sound state using Tools > Restore Backup and choosing the most recent internal Backup folder.
I haven’t had database problems for a very long time. But I follow a backup strategy that not only includes those internal Backups, but also Time Machine and database archives produced by DEVONthink Pro/Office File > Export > Database Archive. (Remember that Time Machine doesn’t check database integrity, so it’s your responsibility to do that once in a while using Tools > Verify & Repair.)
What if your database has accumulated errors, and those errors exist in all of the internal Backup folders?
The next step to achieve an error-free database is Tools > Rebuild Database. This will export and reimport the database content, “kicking out” any files with which errors are associated. After the Rebuild, choose Window > Log to review a list of any files that may not have been included in the rebuilt database. (If you kept a copy of the original database, you may be able to export those files, then import them into the rebuilt database. But some files, especially older WebArchive files, may be problematic and result in introducing errors; if so, delete them.)
Bill brings up an excellent point…
While I am an advocate of TimeMachine, it will backup a bad database as easily as it will a good one.
We don’t want to imply that people should be paranoid about their data integrity so that you are running Tools > Verify after every interaction with DEVONthink (and most people’s experiences show that data integrity is very stable) . But we do advocate having a validation process such as the one Bill has outlined here as part of your backup procedures.
Very interesting answers, Bill and Jim, but doesn’t answer the OP’s question about what V&R means by “inconsistency”. I’m also curious about the meaning. 