Losing data after a computer crash

I have experienced data loss in DTP several times after computer crashes. Let me emphasize that DTP did not crash, nor was the database corrupted in any of these instances. I am faithful to back up and optimize regularly, usually at the end of each day. It may be significant to this question that the computer is never shut down, although I usually restart it every morning.

I live in DTP all day long, using it heavily in research, importing many URL’s and documents and doing quite a bit of writing as well. However, when the computer crashes all of this is lost, and the database returns to the state of whenever the program was last quit.

A sample scenario: I restart my computer in the morning and DTP obviously quits. I work all day long and the computer crashes at the end of the day before I get to do a backup and optimize. When I restart DTP, the entire day’s work is gone.

The Question: Does DTP put new data, imported documents, newly written documents et al into some sort of a buffer that is updated when it quits or a backup and optimize is done? Is there any way to “flush” the buffer and have the work saved to disk other than doing a backup and optimize or quitting?

I would hate to have to quit and/or do a backup and optimize several times a day just to keep from losing data. My budget will not allow me to get a more reliable computer at this time.

Thanks for a great program,
Mike

Mike, as you continue to add and modify files through the day those changes are being saved to your database – or at least DEVONthink is attempting to save the changes to your hard drive.

A Mac that frequently crashes isn’t a good working environment. Your Mac is designed to be stable. It’s time to do some diagnostics.

First, open Activity Monitor and look at the amount of free hard drive space that’s available. Apple engineers recommend at least 10% to 15% of the hard drive space should be free so that there’s room for the operating system and applications to write temporary files to disk (20% is better). Why? Because if your computer runs out of free space the operating system itself could start overwriting files, resulting in data loss and errors. That’s bad. I assume yours is an older computer, perhaps with a relatively small hard drive. If you are low on drive space, housekeeping is in order. Delete or archive to CD or DVD files that you rarely need or use, so that there’s “breathing room” on the drive.

Let’s eliminate one possibility for an unchanging database. In the Finder, select your database file, with the extension “.dtBase” as you are running DT Pro. Press Command-I to open the Info panel. Look at Ownership & Permissions (if you are running Tiger) or Sharing & Permissions (if you are running Leopard). You should have read & write permissions for this file; if read only, changes cannot be saved to it. If the permissions are incorrect, you can edit them to give yourself read & write permission.

Next, run Apple’s Disk Utility application. Run the repair permissions and disk verify procedures. If the disk verify routine finds errors, follow Apple’s recommendation to insert your Installation disc and repair the disk directory.

Hardware problems are fairly rare, but this possibility should be checked. Your Installation disc contains a routine to check hardware components in your computer.

Have you been running OS X maintenance? A utility such as OnyX or C*cktail can run a number of them that are built-in to your operating system. Run the cron scripts, rotate the logs and – I would recommend – clean out the existing system and user cache files. Corrupt cache files can cause problems. You might also check out your fonts, as corrupt fonts can cause numerous problems.

Fingers crossed, these checks and maintenance operations may identify the problem and/or make your computer much more stable.

Whenever I’ve made significant changes to a database, such as adding lots of new content or spending time writing something important, I don’t want to wait for an automatic backup. At break time, I’ll force a backup. If there’s plenty of free hard drive space, I’ll run Scripts > Export > Backup Archive. When I return from break the database has been verified, optimized and I’ve got current internal and external backup. Otherwise, at that break I’ll run Tools > Verify & Repair to check database integrity, followed by Tools > Backup & Optimize, so that I’ve got current internal backup. If something goes wrong, I haven’t lost my work. I really recommend running Backup Archive to produce a compressed and dated archive file once in a while, then storing that on an external medium such as a CD or DVD. That’s insurance against a catastrophic failure of your computer’s hard drive.

Thank you, Bill, for such a long and detailed reply. I’ll answer in the order you posted:

  1. Disk space: I have a 500gb main drive, with 275gb left. No problem there.
  2. The .dtbase file had the correct permissions.
  3. Disk permissions, disk verify. I fix permissions and run DiskWarrior every week before backing up the drive, so it’s kept in pretty good shape.
  4. Hardware problems: I’ve used the TechTool program provided by AppleCare and everything checks out fine. I suspect I may have a problem with one of my SATA cards. That shouldn’t affect DTP any, but it could be the reason for the computer’s recent instability.
  5. OS X maintenance: Since the computer is always on, the cron scripts are run at the appropriate times. I use C*cktail (do you do that to avoid the “naughty-word” filter?) to flush the caches on an irregular basis as needed. However, I will use the program to run all the cron scripts and empty the caches as soon as I am finished writing this and continue to do so as part of my weekly backup routine.
  6. Backup Archive. I’ve just been using the “Optimize and Backup” which I keep in the toolbar. I’ll use the script “Backup Archive” from now on.

I suppose the best advice you offer is to use Backup Archive during the day when I’ve been doing a lot of work (most days). I’ve been quitting and restarting some, too. Fortunately, I haven’t had any computer crashes since posting this question.

As you can see, I’ve already taken care of the obvious things. I’ve been using DTP since 2003 and have come to depend on it heavily. Something must have changed recently, since I’ve never experienced these problems before. What that change could be, I’m mystified.

I suspect this experience must be covered by Flagle’s Law of the Innate Perversity of Inanimate Objects: Any inanimate object may be expected at any time to behave in a manner that is entirely unexpected and totally unpredictable for reasons which are completely unknown or thoroughly obscure.

I guess I was hoping for a silver bullet, but I had prepared myself for the worst. As you say, a Mac that crashes frequently (or any) is not a good working environment, for DTP or the unfortunate user. I suppose it’s time to grit my teeth, break out the plastic and head for the Apple Store!

Thank you again for your kindness!

Mac system crashes are normally a sign of some hardware-related problem, which can be as basic as flakey memory (which customer utilities can sometimes detect) or something more complicated that may require professional diagnostic tools (like the Apple Store uses) to detect/isolate (that customer utilities like TechTool won’t).

Awhile ago my iMac G5 had hardware problems in both those categories. Some bad RAM I was able to isolate and replace myself. Something intermittent with the logic board and/or power supply (even Apple wasn’t sure) required lengthy Apple diagnostic testing before a solid failure convinced them it was a hardware problem, covered by AppleCare. I’d never not have AppleCare coverage on a system I may end up using for at least three years it covers, although I’ve waited until the first year of hardware warranty coverage is near expiration before adding the remaining two for software/hardware.

If there’s a disk problem, you might try opening the Console app (/Application/Utilities/Console.app) and looking for any suspicious messages around the time of crashes. Something like “diskXXX: I/O error” indicates the disk may have bad blocks (sectors), which can be(come) a serious data-corrupting problem. Those can occur without causing full system crashes. TechTools might be able to scan for bad blocks; I use Media Scanner from the OEM version of Intech SpeedTools Utilities for that. Whenever I see the dreaded disk I/O error (fortunately infrequently) it’s a sign to keep a closer eye on a drive and be prepared to replace it. And if it’s a system volume/drive I might not risk waiting until it gets worse.

Sometimes problems with external USB or FireWire peripherals can cause system crashes. Might even be a bad cable.

Not sure if any of that’s applicable to your current problem (which hopefully will be resolved!) but maybe it’ll be helpful in some other situation.

I’d pop out the SATA card for a couple days. It sucks if you have a RAID array or something on there, but that sounds like trouble to me. A lot of these idiots just want to be able to say their product is Mac-compatible, and don’t really care whether or not it can be expected to work reasonably.

Especially if the card was released during Panther/Tiger times and you’re running Leopard now. Their testing process basically seems to be “okay, installed Leopard… booting it up… alright, it sees the card. Add a note on the website, boys! Our work here is done!”

Just a followup, since you people have been so helpful.

The problem was the video card! I took out all of the other cards one by one, pulled and switched around my RAM, cleared caches, used DiskWarrior on all my hard discs. Finally there was nothing left but the video card. Fortunately, I had an old one that would work.

Bingo! No crashes. I’ve been a Mac user since 1984 and this is the first time I’ve heard of a video card causing a computer to crash!

I’m still following Bill’s suggestion to use backup and archive a couple times a day, then verify, optimize and backup from the toolbar at the end of the day. My work’s too valuable to take a chance on losing anything.

Now I just have to endure this 4 year old Mac until I can save up enough for a new Mac Pro.

Thanks again for all the help!

Glad you figured out the problem. Intermittent system crashes are really frustrating, especially with an increased risk of data loss.