General troubleshooting tips

Whenever anything flaky happens, I suggest running the Verify & Repair tool. If errors are reported, try running this tool over and over until (hopefully) no errors are reported. (I haven’t seen an error report on my big DT Pro 1.9 beta database for a LONG time.)

If no errors are reported, close DT and do the housekeeping routines recommended below.

If the Verify & Repair tool cannot fix errors and DT recommends that you Rebuild the database, you have three choices:

[1] Assuming that you had previously run Verify & Repair and Backup & Optimize recently, when everything was working properly (it’s highly recommended that these tools be run whenever you’ve been adding lots of content, and before closing DT after a day’s work), you might consider the Restore Backup option. This will restore your database to a presumably sound backup that was made earlier.

Note: If you have been adding new content since the last time you ran Backup & Optimize, there’s a way to recover the recently added items. Just open the groups to which you’ve added material, do a date sort, select the new items and Export them using the Files & Folders option for each group containing new items (this will retain metadata such as URLs and comments for the exported items). Note: Create a new folder and give it a descriptive name for each group of selected items, if multiple exports are made.

Now go to Tools > Restore Backup and select the tool. This will restore your database to the last backup. Run Tools > Verify & Repair. If all is well, you can import the files you just exported, to bring your database back up to date. First, I would create a new group to which the imports will be sent, and go to Preferences > Import and select that group for new imports. Now choose File > Import > Files and Folders and select the exported folders. The content will be moved into your database to your selected new group. All you have to do now is replace the items into your preferred organization of groups. After completing this you may delete the temporary holding group that you had created to hold the reimported items.

If all went well, you have a working, updated database. Now run Verify & Repair followed by Backup & Optimize. Close DT. Highly recommended: Make a backup of your database to an external drive, CD and/or to another location on your drive.

Perform the housekeeping routines recommended below.

[2] Choose Tools > Rebuild Database. DT will export all the groups and their contents to your drive, then import them back. If you have a large database, this will take some time. Note: A “bad” text file or other problem could result in a stall during the export phase, and the Rebuild Database option may not go to completion. If this happens, try a manual File > Export > Files and Folders export, choosing groups one at a time. Then manually do the Files > Import > Files and Folders routine. Note: If there’s a stall during export of a selected group, you may need to select portions of it to export as much as possible, and you may lose one or more items that cannot be exported.

If Rebuild Database ran to completion, run Tools > Verify & Repair followed by Backup & Optimize. Close DT. Make an external backup. Run the housekeeping operations recommended below.

But if Rebuild Database failed and you did a manual export of your database contents, do this: Close DT. Remove your existing database file, for example, by copying it to another location. Now open DT and create a new, empty database. Import the previously exported folders, using File > Import > Files and Folders. Run Verify & Repair, followed by Backup & Repair. Close DT. Run the housekeeping operations recommended below.

[3] If you have made a recent external backup of your database, that may be the solution to your problems. Check the data of your last external backup, using Finder > Show Info. Have you added any content since that date? If so, follow the procedures outlined in [1] above to export information added since that date from your current DT database.

Now CLOSE DEVONthink (important). After doing the housekeeping operations recommended below, copy the external backup file over your DT database. Open DT and import any new items (as in item [1], above). Run the Verify & Repair and Backup & Optimize tools. You should now have a sound, updated database. Make an external backup!

HOUSEKEEPING OPERATIONS

Sometimes, just quitting and relaunching an application can get rid of flaky behavior. Corruption of the application in memory might have happened, perhaps as the result of a conflict with another application or a haxie that modified the system (DEVONtechnologies is generally suspicious of haxies). Or maybe a cosmic ray came through and twiddled a bit in RAM. Who knows?

Your System and hard drive do require periodic maintenance to keep them up to speed and to eliminate problems that can creep in over time.

Because OS X is a UNIX operating system, cron file routines need to be run periodically. There are a number of applications that can do this, as well as clean out cache files (that may become corrupted over time), update/rotate log files, repair Permissions, etc. I use Ccktail (replace the '’ with ‘o’ to read the name) to perform these functions at least weekly.

Disk Utilities can be used to do Permissions repair. To use Disk Utilities to correct directory errors that may happen from time to time, boot from your OS X installation disk 1 and choose File > Disk Utilities > Disk First Aid to correct any errors. I use Disk Warrior once in a while to find and correct any errors that Disk First Aid doesn’t catch.

Rarely, your DEVONthink preferences file can become corrupted. If all else fails and DT still behaves strangely, find and remove this file at your userdirectory/Library/Preferences/com.devon-technologies.think.plist. This file will be recreated next time you launch DEVONthink, but you will have to reenter any preferences you had previously modified.

Some people pride themselves on how long they can keep OS X running without a restart. That’s all very well. But with a large DT database and my practice of keeping a number of applications open at the same time, performance slows over time. A restart every few days speeds things up again.

Sorry for the length of this post, but these are some of the practices that have paid off for me. :slight_smile:

I answered all the questions you asked in your reply to my post about the empty database. I’m still having no luck at all. My database is still showing up empty even after multiple verify/repair/backup/rebuild sequences.

I cannot try any of the other suggestions you’ve made in this new post because I cannot manually export groups/files/folders that are not showing up in the database to begin with.

What I still don’t understand is why all this stuff you’ve listed here has to be necessary anyway. It seems like a whole lot of trouble and time wasted to have to babysit this database so diligently just to ensure it doesn’t become inoperable. No other program I’ve ever used requires this, especially after only a few hours of use on a single day.

What should I do now short of shipping everything off to Devon Technologies to see if they can retrieve the data? For multiple reasons, I really don’t want to have to do that.

insidertravels:

I’m sympathetic about your troubles. I expect DEVONtechnologies staff will respond to your postings.

Let me respond, however, that DEVONthink is in fact one of the most stable databases I’ve ever encountered. My database is over two years old, has passed the 13 million word count in my collection of reference materials, and has never lost data. I did have to rebuild the database once, while using an alpha of DT Pro. As it turned out, what caused my database problem was that I had dumped in a text file that had a Cocoa text problem – a “bad” file. The problem wasn’t DEVONthink’s fault, but a glitch in Apple’s Cocoa text handling.

As I understand it, you had dumped a great many files into DEVONthink, deleted the originals, and saved your database to your iPod. The database may have already been corrupt before saving it to the iPod (almost certainly, if it was saved while DEVONthink was still running). One or more of the files you imported may have been corrupt.

Try one more thing. With DEVONthink NOT RUNNING, remove your database file from its location (yourusername/Library/Application Support/DEVONhink) – move the DEVONthink folder from Application Support to the Desktop, for example. Then copy the DEVONthink folder from your iPod to the Application Support folder and launch DEVONthink. Any luck?

If not, send a help request to support@devon-technologies.com. If at all possible, they will help you recover data.

Have I lost data in other applications? Resoundingly, yes. Right now, FileMaker 7 users are discovering that large backup files are irretrievably corrupted. What’s good about the DEVONthink database design is that it is not only stable, but that most or all data can be recovered even if the database is corrupted.

The reason I “baby” my operating system, disk directory and DEVONthink database is that I’ve got thousands of hours of work represented in the files on my computer. That’s a big investment; to me, it’s worth many times the price of the computer and the applications on it. A few minutes spent every now and then to keep everything running right (and backed up) is cheap insurance. It’s not just DEVONthink that requires this kind of care; any or all of your files could vanish if the disk directory becomes badly corrupted.

I remember myself writing a similar sentence, something like “I have a Mac, I did not need this for more than 10 years.” At that time I had several system crashes – now I know they came from an old CM-PlugIn from CP NoteBook. They are absolutely gone since I deleted it.

But I took Bill’s advice quite seriously, and since then I run the repair and optimize commands in DT frequently, as well as I run the little app Onyx, which does all the optimizations and little repairs in a script, about every second evening. I enjoy the most stable system and DT dbase. Still, I got used to keep three backups of different versions on an external HD.

It is not much work, it has nothing to do with a particular software or hardware. it is just precaution that is necessary in this imperfect world.

Maria

How is it people are running a Pro version? According to the website, the Pro version hasn’t even been released yet. That was something I had wanted to ask you before but I forgot.

Does it appear to you from what you’ve heard so far that my data is irrevocably lost? It’s weird because the database files themselves are many megabytes big according to Finder info, so I know the information is in there somewhere. When I bring up database statistics from within DEVONthink, it shows that the database has something like 200,000+ words - although when I perform the function that adds up all occurrences of each word, intermixed with some recognizable words it gives me a bunch of mumbo jumbo that aren’t really even words. I’m not sure what that means.

That’s what I’ve already done multiple times with the same result every time.

I guess I’ll have to do that then. :wink:

I haven’t used FileMaker, so I cannot comment on that. But I’ve never lost txt, rtf, html, etc. files; the worst that’s happened is I’ve had to delete a few straggling gremlin-like characters after opening a file that’s not in the same condition I originally left it. I’ve never tried to put them into a “database” before though.

I take extremely good care of my operating system. All I meant was that I don’t want to spend more time trying to get the application to work right than I did actually managing the files manually before I ever found the application. That would be counterproductive. I downloaded this application to make my life and workflow easier, not more complicated. I had planned to use it in place of virtually every other app on my computer because I prefer to write everything by hand rather than using WYSIWYG tools for any number of tasks, and I didn’t want to keep editing multiple, probably rarely in sync with each other, versions of the same data.

If DEVON support can fix my database, then I’ll give it another shot (but the first thing I’ll do is export everything back to where it was before I ever opened DEVONthink so I won’t lose the files again. Now I know that when the help files said I could delete stuff once it had been imported into the database, it really meant that I shouldn’t delete them. :wink:

I didn’t say I should NEVER have to verify/repair/backup/restore the database. What I said was I had only downloaded the demo version the same day and had been working with it for only a few hours before I closed it for the evening. Then when I tried to move the backup from my iPod back to my freshly-installed OS (because I’d just gotten a new laptop when one unexpectedly became available for a huge discount via a firesale.)

Anyway, assuming I ever get my data back, I’ll be keeping a copy both on an internal partition and on my iPod, plus I’ll probably keep one on CD or DVD as well. I have no problem backing things up - I just didn’t think it should lose my data so quickly, that’s all. :slight_smile:

insidertravels:

Some of us brave souls (including Maria) have been beta-testing prerelease versions of DT Pro. So far, we’ve gone through 33 alpha versions and 3 beta versions.

“Alpha” usually means that software is so buggy and unstable that it’s more likely than not to swallow up your data, burp loudly, and then eat your System files. But even alpha 1 turned out to be so reliable that I transferred my database to DT Pro and have run my work under it since. It’s great!

Actually, you are braver than I am. :smiley: When I first started using DT PE, I didn’t trust it enough to discard the files that I imported.

From your last posting, it seems that your DT database does have data in it, so the likelihood of recovery sounds reasonable. In response to your email to DEVONtechnologies Support, they will probably suggest that you archive (compress) your database and FTP it to them.

climb on board the Beta tester train. It’s really been a great app to work with.

The alphas have been much more stable than most Beta SWs I’ve worked with and the Beta seem to have excellent stability.

The guys at DevonThink have really worked hard on this one. I think it’s one of the great undiscovered secrets on the Mac.

I’ve only seen one or two windows applications similar to it.

You are unlikely to find better developers out there, IME.

Having said that, backup and save often…

good luck

:slight_smile:

I believe it would be a positive service to the DEVON community if useful and informative contributions like this (thanks, Bill!) were somehow retained with high enough visibility that others might easily find them in the future.

Two possibilities:* Create a distinct subforum for selected “FAQ” topics, for easy referral and less need for redundant replication elsewhere

  • Make certain topics sticky
    Other ideas are welcomed. Or objections, if you don’t see benefits for this.

It seems your data is still in there somewhere. Please, zip the complete DEVONthink database folder (~/Library/Application Support/DEVONthink), give the archive file a distinctive name and upload it to the “Incoming” directory of our FTP server, ftp.devon-technologies.com. You’ll need an FTP programme for doing this, e.g. CyberDuck, Transmit or, of course, the ftp command in ther Terminal. Then, drop us an e-mail to support@devon-technologies.com, so that we know it’s there. We’ll have a look at it then and try both to recover the data and to find out what happened.

Best,

Eric.

I would like to weigh in for this idea as well. I am struck by the quality of volunteered advice on this BB. Immensely helpful.

Given their BB moniker, the uber-level file could be called “The DT Gods Speak” or “The DT Gods Suggest…”

Bill’s housekeeping suggestion, whether for DT alone or the OS, is prudent practice. At the very least, consistent housekeeping maintenance eliminates one level of concern when problems surface. It is helpful to know the exact procedures and paths DT p.r.e.f.e.r.s rather than the generic, but true, “backup your data” or “perform routine database maintenance.”

Few of us are UNIX geniuses. It’s useful to know which client-side routines that work best with a particular UNIX database application’s design.

Thanks, wdfs.

The FAQ sections on Shirt Pocket Discussions are part of the inspiration for possibly doing something similar here.

Discussing and refining ideas from previous forum topics has more productive potential for all participants than repeating “too much” of the original content. Increasing visibility of common topics can benefit newcomers looking for help or wanting to get up-to-speed about something more efficiently, while veterans aren’t “bored” with too much redundancy and can add new insights to older topics. Not trying to generalize since everyone can have insights and enjoy efficiency. Something like that. :slight_smile: