Help - Content Disappearing

I have lost confidence in this application. The content in many of entries is starting to disappear. The document title remains in the system, but when viewed everything is blank. I have had the problem with rtf’s, .doc’s and pdf’s.

I have to admit that I use the application a lot, but have not dug deep into all its features and capabilities, so I might be missing something simple.

I have done a rebuild of the database and got some of the entries back, but still have a disturbing number that are completely blank.

I also move the database between my laptop and desktop system when I travel. I move the data by using an application called ChronoSync and copy the content of the DevonThink file in Application Support in the Library folder from computer to computer.

I have stopped using the application until I can resolve this problem. I have all the updates installed.

Does anyone have a suggestion?

Are the missing files contained in the database or are they linked files? If they are only linked files then you have to make sure that they exist on both computers.

As Rudi suggested, links to external files that exist on one computer will be broken when the database is moved to a different computer that doesn’t have those same files in ‘the same’ location. The remedy in such a case is to ensure that all files are contained in the database or in its Files folder. I’ve got many hundreds of external links to PDF files. I’ll be migrating to a new computer soon, and will have to give careful thought to the migration to avoid broken links, as I’ve got external files on two different volumes.

I sometimes worry about the use of synchronization programs to swap files back and forth between computers, as ambiguities or improper settings of the sych rules could cause strange things to happen. A Finder copy over FireWire takes only minutes even for a large db; the only variable is deciding which computer is the source and which is the copy recipient. :slight_smile: As you might gather, I tend to be a pessimist about all the things that could go wrong, and usually favor those approaches that have the fewest variables to control.

Hi,
this reflects exactly my feelings about these programms, although I own one. I prefer to backup in one direction. My desktop has all main directories and the main DT database. My iBook is for presentations and working in a library where I do not gather material via the internet. If I have to add something to my DT Database on my iBook, I later export these files (with help of the history tool - Verlauf in German) and import that to the Desktop Db.

In addition, I do not have links to documents but rather store all files in the DT file folder. The database is quite large, but I have no speed problems, no files get lost.

Maria

Thank you for the suggestions. I may have some of the problems mentioned with links. When I finish this road trip and can sit down with my desk system and laptop I may be able to figure something out.

99% of my database in DevonThink consists for either rtf’s with no links or PDF’s of either pdf’s loaded from the internet, or pdf’s generated through the print window. I drop them into DevonThink and then through the original into the trash. Is that the correct process?

My method of synching the two computers is to treat one as the original and completely copy any documennts and folders that are different from one machine to the other. I perform the same procedure with the DevonThink folder in Library>Application Support. My assumption is that all of the pdf’s and rtf’s are stored in that folder. Is that correct?

Both depends on the settings in the preferences. If you set import of PDFs, media, unknown formats etc. to beig copied into the database, it is not in the Application Support folder, but in the database itself. I do not recommend this, because if you choose the other option (copy into database folder) you still have the original file accessible via finder. But you need not collect all links when you want to copy the whole database, which you would have to do with the third option, linking. In that case, trashing the original is of course a bad idea.

I do not really get your point here. Do you say that you do not find the files in that folder?

Perhaps you could check the info panel of one lost file and one file you still have. Do they show a path?

So far for now,
Maria

I now too have a similar problem with some of my pdf files but not all of them and I’ve no idea what would be the best way to get them back into the database because it didn’t happen to all of them. So I don’t know how to search to find the ones that got screwed up. Here’s the scenario:

Using newest DTPro beta, haven’t moved dbs out of documents folder, nothing like that, always verifying and repairing, backup and optimizing, yada yada yada. What DID happen was this…My hard drive failed on my laptop. Fortunately it failed only a couple of days after I made a complete drive backup, essentially a mirror, using SilverKeeper. So I installed a new drive in my PowerBook (not for the faint of heart or weak of knee, btw). I used SilverKeeper to place everything back onto the new drive from my external FireWire drive and make it bootable. So, when I finished it was an exact duplicate of the drive from two days earlier (or so). Everything has worked perfectly. Except DEVONthink. It seems to be unable to find some of my folders with pdfs in them. But, like I said, it isn’t consistent. The ones it cannot find appear in the window as reduced versions of the first page of the .pdf. Does anybody have any ideas about how I can search DT in a way that will isolate those that need to be reimported?

This has been a very frustrating experience. I was working desperately to get some work done on an actual paying project when I encountered this. As much as I have loved some of the things DT does, this hurts my opinion somewhat.

ChemBob

ChemBob:

Do you use partitions? I had a similar experience when I moved the contents of my TiBook to my new iMac. Everything got moved into the boot partition of the iMac. When I started up in the iMac, DT Pro couldn’t find PDFs that had been located on a different partition in the TiBook. (But everything was there, nonetheless. I expected the problem, but didn’t want to wipe, partition and reinstall everything on the iMac.)

Next, I set up an external FireWire drive with three partitions – again, corresponding to those on the TiBook. I used SilverKeeper to clone my iMac boot disk onto the external drive. Next, I moved all the items from the boot partition of the FireWire drive into the corresponding partitions they had occupied on the TiBook. (That was easy, as the Apple assistant had created named folders on the HD that held the contents of the other two partitions on my TiBook. All I had to do was drag the contents of those folders into their corresponding partition.)

Now DT Pro can find everything. Problems over. Actually, I’m currently running off the external FireWire drive. One of these days I’ll clone it back to the internal drive in my iMac, after wiping it and setting up partitions. (During summertime, my iMac runs cooler off the external drive, anyway.)

Tip: When you come across one of the PDFs that only shows an icon of the original, open DT Pro Info and examine the Path of that PDF. Where did DT Pro expect it to be? Now do a Spotlight search for that PDF file. Where is it? It should be there somewhere, unless your PowerBook drive was already failing at the time of the external backup.

Another question: What happens if you boot off the FireWire drive? Can DT Pro find the PDFs?

As Maria mentioned, it is important to consider the preferences setting for import (into database folder, into database or link only).

The problem with linking only is that they can, apparently, be broken (I asked about this in another thread today).

I think there is a real disadvantage to working with files outside of the database.

I hope you recover them.

cheers

:slight_smile:

Bill,

I don’t have partitions on my PowerBook. My external FireWire drive I use for backup and maintenance (of the PowerBook) is a 160 Gig partitioned into two 80 Gig partitions. One of the partitions is solely a boot drive that has utilities, such as Disk Warrior, on it for repairing PowerBook problems (or my Cube or whatever). The other 80 Gig partition is kept to match the 80 Gig PowerBook drive (which is now a 100 Gig drive so I’ll have to repartition the external to 60 boot : 100 backup at some point. So, basically, that partition had the exact data of the PowerBook on it and those are the data that were transferred to the new PowerBook Drive.

Regarding the paths, DT can currently follow some of them, not others, even though they read the same as Spotlight paths. Within the same DT Group, corresponding to Finder folders, some of the pdfs show up fine in DT and others do not, even though the word counts are there. I’ve discovered a lot of blank Word to rtf imports also, but not all of them are blank so go figure. It appears to be seriously screwed up. I’m going to check the DT dbs on the external drive tomorrow (I don’t leave it hooked up all the time, it’s only for backups and repairs). If they are not OK and are like this DT dbs, then I think I’m done with DT and will be out of here; possibly asking for a refund. I’ve done everything right, supposedly. Nothing else on the drive has had a problem. All the files open fine in their respective apps in the finder. Only DT has screwed up and, along with it, my entire workflow taking with it a huge amount of time and effort when I am in a serious time crunch.

I need to focus on my work more than I need to focus on constantly babysitting a program that clearly has file system issues and whose storage mechanisms are so complicated that no one understands them, at least the users don’t. I had zero problems with personal edition and I’m sorry I switched to Pro. This isn’t the only glitch I’ve seen, I reported another serious one to the developers previously.

Sorry to be venting here but I’ve been a loyal and happy user for months now and am extremely frustrated that so much of my hard work is evidently lost. Days? Weeks? Who knows? Do I really want to go through it all again and risk the same thing happening? Not really.

Sigh,
ChemBob

I just spent, literally, the entire day since 8:30 AM with only minor breaks fixing my DEVONthink Pro database. Hopefully it is not only as good as it was but better; I removed a lot of duplicate files that I didn’t realize I’d duplicated rather than replicated, etc. I couldn’t begin to relate to you how I finally repaired the db because I concocted steps as I went along, and as the types of issues changed the approach that I took changed. I will tell you that I used the Path & URL -> Update Paths script to correct large sections of problems. If not for this script I believe I would have given up because it would have taken days to fix. In addition, I used the DT search tool (two windows at a time, one searching paths the other searching names) in conjunction with Spotlight a great deal. If not for Spotlight I would, once again, have been dealing with this for days. I believe that my db problems were partially due to me having reorganized some of my Finder folder structure (not knowing that DT would not be able to cope with that) a couple of weeks before the hard drive crash and then some vagaries from reimporting everything onto the new drive. One lesson I’ve learned here is to never ever use any PDF import (such as the folder action “Action Index”) that doesn’t place the PDF itself into either the DT “files” folder (the approach I’m now using) or directly into the database (I thought “Action Index” did place the files in the database, hence part of my problem when I moved them from the folder I was using to implement this action). There is just too much possibility for the links to be broken for them to reside anywhere else, imho. Keeping them with the database would also simplify post-hard drive crash trauma.

Hopefully everything is OK and I can now proceed to updating to v1.0, which I will now proceed to download.

ChemBob

ChemBob:

I’m glad to hear that your database is running soundly, but sorry it took you time to put your external PDFs back in order.

In the future, you might want to set your PDF preferences for import this way:

[1] PDF & PS documents: Check “Create Thumbnails”

[2] Files: Check “Copy files to database folder”

[3] Index & Convert: Check "Use PDFKit (Tiger) – DO NOT ALSO CHECK “Convert to Rich Text”’ (if that’s checked the file is still external)

Before PDFKit became available, File > Index was the only way to get PDF+text imports of PDFs; but then the files were linked to externally rather than saved in the database. Now PDFKit allows PDF+text display, but you can store the PDF files either in the database or the database Files folder.

If you store your PDFs in the database Files folder, they will be seen and indexed by Spotlight.

Bill,

Yes, that is how I set the prefs earlier today to avoid a re-occurence of this problem.

Thanks,
Bob