Data I have been typing into this database for months has suddenly vanished. Gone. Tell me I did some little thing and it will all come back.
I don’t know anything about the state of health of your operating system, whether you’ve installed incompatible software that creates errors in the operating system, whether you’ve had a power outage, etc.
The first thing to try is to select Tools > Verify & Repair. If a lot of errors are reported and they can’t be repaired, stop.
The next thing to try is Tools > Restore Backup. You will see the internal Backup folders, most recent at the top (you can see the date of backup). Select the most recent Backup and confirm that you wish to swap it with the current database. Afterwards, check the database to see if it appears OK, including running Tools > Verify & Repair. If the database still has problems, run Tools > Restore Backup again and choose the next most recent internal Backup folder.
Or, if you follow my frequent sermons about backing up and use DT Pro or DT Pro Office, you may have been making external backup archives using Scripts > Export > Backup Archive. You can double-click on the most recent one (the archive files are dated).
I haven’t had to resort to a backup in more than two years, but I make it a habit to invoke Backup Archive whenever I’ve been investing time and effort in adding to or modifying my databases. When I take a break, I run Backup Archive, which will verify and optimize the database, then make current internal and external backups. When I return from my break those good things have happened, and I go back to work.
If you have to resort to a backup, you will have lost any material added after the time of the backup. That’s why its a good idea to make backups whenever you’ve made significant changes; don’t depend on schedules.
If the database problem was caused by a damaged operating system or disk directory, it may happen again. So it’s a good idea to run OS X and disk maintenance operations periodically, even as preventive maintenance.
First, fortuantely, I had dumped the database onto a thumb drive at 12.45PM, so I only lost a couple hours of work. Pure luck. I used finder and not DTP to do the save. Straight drag’n drop. Luckier still.
Secondly, the disappearance of data occured some time after 3.00 PM. I NEVER saw it happening. I just went to a file where I had been working – entering data from a book I was studying – and I found that my data had vanished. I looked in other files, same thing. So, even if I had done the save and back-up and archive procedures you suggest, Bill, I would have been ruined. I would have saved a dis-membered, mutilated, currupted database. So, saving is not the solution.
Third, my computer is new. I got it in mid-December 06 and got DTP soon after. After I got your reply, I did as you said and I checked my drives and all were working fine. No corruption, no repair. The other thing: I don’t use funky software. The only programs I had running were DTP, Safari, Word, Entourage, maybe DevonAgent.
I can only conclude that something nasty happened in DTP.
J.
If you make a Finder copy of a database while it’s open, the copy may contain errors, and those may grow worse with use.
It’s perfectly OK to make a copy of a database in the Finder, but make certain the database isn’t open. If it’s open, there can be information in memory that would be lost to the copy.
The first thing Backup Archive does is run Verify & Repair to check for database errors; if there are errors you will be notified, and I think the procedure will stop. (I haven’t seen a database error report in a very long time, and in that case the error was repaired.) So it’s unlikely that the backup archive would surprise you with a corrupt database.
Never assume that a new computer is error-free just because it’s new. The universe is a hostile place, with power supply fluctuations, cosmic rays zapping through the memory chips, and gremlins living on the hard drive and in the CPU. You have already done some OS and Security updates (if not, you should have done that). I run preventive maintenance routines at least weekly, and about one a month I run DiskWarrior, which invariably finds and corrects minor errors that if not repaired would begin to cause problems.
Thank you, Bill. I will take your advice. Do you use other maintenance software?
J.
I use three trouble-shooting and maintenance utilities:
AppleJack - see a description and overview at http://www.macfixit.com/article.php?story=2005041817191411.
C*cktail (the asterisk stands for “o” - the forum software thinks this is a bad word, hence the asterisk). This one is also mentioned in the MacFixit article about AppleJack. Note: Several other similar utilities exist, such as the free OnyX.
DiskWarrior 4.0 - my ‘gold standard’ for finding and fixing disk directory errors, then rebuilding and optimizing the directory for optimum performance. Also has a routine for finding and fixing minor file errors.
I always use Software Update to install Apple OS, Security and software updates, and I’ve never had a problem. But I always run Disk Utility’s permissions repair and disk verify routines before the installation, and the permissions repair routine again after the installation.
Finally, I’m cautious about the software that I install on my computer, as I try to avoid anything that modifies the operating system itself, or installs weird plugins. I’ll repeat that haxies have caused a great many problems, and not only for users of DEVONtechnologies applications.
Ishear,
I can fully understand your panic. The problem seems to be solved (to the most part), and luckily you have saved the database most recently.
I can understand you so well because once I made the same experience like you. Suddenly everything was gone, a whole precious database. The problem was my hard disk, on a computer that is checked frequently. The only thing I would blame DT is that it stores files in big bunches so that, if one of the ten files is corrupted by a corrupted HD, everything is gone at once. With thousands of small files the loss would not not be so fatal.
So this is another reason why I cannot wait for the new file format!
Until then I backup and also use the zip-Backup AppleScript in addition.
All the best,
Maria
Maria,
My big fear – as a result of this “crash,” for want of a better word – is that I didn’t recognize the warning signs. Having recently switched from PC to Mac, I put too much trust in the Mac’s vaunted stablity.
My biggest concern in the aftermath of this failure is that I didn’t see it. In other words, I didn’t recognize that I had a problem. Thus, if I had saved, I would have saved a corrupted database and lost months and months of hard work, irreplaceable work. Irreplaceable because so much of what I enter into DTP in terms of raw data/information is then commented on and noted by me. How could I possibly remember thoughts that I worked into DTP months ago? I could not, of course. Which is the reason for the database!
It seems the only rational response to this disaster is a paranoia. Perhaps this irony – paranoia as a rational response – should be the subject of someone Masters Thesis, either on computer programming or public policy. 
Jeff
I don’t encourage paranoia, which is usually irrational. 
Let’s call it an attitude of informed risk management, which can lead to some routine practices such as backups and OS and disk maintenance.
Such routine practices needn’t be intrusive or time-consuming, but can have a high payoff.
Sort of like checking the tire pressures on your Porsche, periodically changing the oil and checking other items that could cause problems if not attended to. Your Porsche will probably run for a long time if you do none of those things, but the ultimate result will be very expensive.
Bill, I actually check the oil for my Porche, while awaiting the day I can actually afford to own one! Seriously, though, DTP works for me, and you’re right: If you don’t take care of high-powered machines, they can crash, or just break down. Hey, I love DTP!
Jeff 
Jeff,
I am with you on that point as well. Of course I do not know the real problem, it just looks like a problem of the hard disk.
(1) Christian, could it happen that DT backed up a damaged database? Or would DT recognize the problem and warn?
(2) Mac’s stability. Yes, they are much more reliable than the Windows PCs my husband uses and I have to repair from time to time. But over years I have not heard about HD problems on Windows PCs, but I had myself 2 totally broken FW-HDs when the Pismo came out (destroying days of video editing), and I had the small but fatal problem with the hard disk on this G5. HD have nothing to do with OSs in the first place, and I still tend to think it is an accidental correlation.
Just some comments…
All the best,
Maria
Yes. DT checks the basic file consistency while opening a database (and disables automatic backups if the basic structure is not correct) but that’s not as thorough as the Tools > Verify & Repair command.
Christian,
Thanks a lot for the immediate reply (today is Ostern, so how come?)
It is good to know that DT would not backup an incorrect structure. Verify & Repair has become my friend anyway…
All the best,
Maria
Having been careful with my “verify and repair” and my “backup and optimize” after my problem, I also do “export backup archive,” as Bill suggested.
HOWEVER, I noticed that I got a log of two failed backups. Then later that day, I got a log of 20 or 30 failed backups, so many I didn’t count.
First, I am not clear which of the three procedures I used created this log; I suspect it was “export backup archive.” But I am left to wonder, in any case, why am I getting this growing log of failed back ups.
One more thing, today, in an effort to replicate the problem, I performed this “maintenance” in an orderly sequence, looking for a problem: so, first I did “verify and repair” and all was well; next I did “backup and optimize” and all was well; then I did “export backup archive” from the script file and once again all was well. No log and no indication of failures to back up.
Hmmmmm 
Jeff
Jeff, I talk about maintenance at two levels:
[1] Maintenance ‘inside’ the database including Verify & Repair and Backup; & Optimize (which are also run by the Backup Archive script).
[2] Maintenance of the computer’s operating system and disk directory.
In my opinion the second group of maintenance operations is the more critical. Your database is really quite stable, but it can be damaged if the operating system or disk directory go wonky.
Basically, the database can be damaged if damaged files are dumped into it (although Christian has made it much more robust than in the earliest versions circa 2002), if there are errors in memory (most likely resulting from errors in the OS when DT makes a call to the OS), or when an error occurs when data is being written to or read from disk (a power outage, a voltage fluctuation, insufficient free space on the disk or a damaged disk directory).
So if you have a database problem, use those internal maintenance procedures. But you should assume that maintenance of your operating system and disk directory is indicated as well. And you will find that preventive maintenance of the disk directory and operating system will drastically reduce any problems in your database. As a bonus, you will experience fewer problems with your other applications and data, as well.
However, if you are willing to download and install all sorts of strange applications on your computer without thought about problems they may cause (ShapeShifter is a current example), all bets are off.
Jeff,
did I understand you correct that you got logs of failed backups and went on working with a “verify and repair” but not checking the whole computer? If I got a message of a failed backup (I never did yet) all alarm bells would ring. That means: (1) I would verify and repair inside DT if possible, (2) I backup all my crucial data – those from DT and other applications on another hard disk, (3) start my computer from the installer DVD and repair my hard disk.
It is a pity that you encounter this problem with a new computer, but these things happen. Try to get everything cleared before you go on working. It should be OK afterwards. I am working now without any problem on two Macs (almost never shut down) for more than a year.
All the best,
Maria
I wish I could send you a snapshot. DTP’s tools says its verified, backed up and optimized. Nothing wrong. However, when I go to scripts/export/ daily back up, I instantly get this long log of files that DTP has failed to back up. Even while the backup process is still going on. Does that make sense? How can it be that even though the system says it’s been verified, backed up and optimzied, it won’t do a “daily back up” without failing to back up forty files or more? Is DTP not recongizing its own system of error protection? I can repeat the process at will. And the program is running perfectly.
By the way, I already used the original Mac disk and started with that to make sure my disk was clean, and it was. No repairs. Mac’s diagnostics detected no problem with my disk.
Could it be that there’s something twitchy with the “daily back up” process, which, incidentally, appears nowhere else in DTP except in the scripts menu. Or am I mistaken?
J.
Ishear,
now, after this last explanation, it seems to be something absolutely different, no need to panic: There may be a problem with long file names or filenames containing certain characters which are not allowed in the finder.
If this is the case, a simple problem. (I agree, DT should explain this with something like "file— could not be saved to the Finder because of forbidden characters in the filename).
The best solution is to send a screen shot to Christian. He knows best, and he is unbelievably quick in solving problems. May be it is just a little bug that has to be fixed. No problem with the integrity of the data base.
Tell us, how things turned out, I am curious.
Maria
Hi, Jeff. I hadn’t run the Daily Backup script in a long time, so I decided to run it.
The log reported 8 failed backups.
I checked out each of them. In every case, they were linked files that I had set up as synched to a test folder when I was trying out the PDF and .doc export modes of Pages. Later, I deleted that test folder that I had Indexed into my database, and forgot to delete the corresponding group from my database.
So there was nothing wrong with my Daily Backup run. Those files should have failed to be backed up. I did the right thing and deleted the test group from my database.
Check out (search by name) those files reported as failed in your log. You will likely find a good reason for the export failures.
Never forget that Indexed files documents lose data if the external file is deleted.
I think we’ve got it. It is a naming/linking issue. I’m going to check each file, check whether it is a deadlink and, no matter the case, if I wish to keep the file, I’m going to rename it, one by one. A little tedious, but a likely solution. Thanks for sticking with me, I’ll report in the next day or two.
J.