Inexplicable corruption - please help

One of the groups in my database (DTP) has become corrupted, and I am at a loss for how to explain or fix it. If you can spare a moment, please help.

The database has been corrupted in the following ways:

(1) The content of (most) text files has lost all formatting. What remains are only capitalized words (no punctuation, no carriage returns, etc.). In other words, all of the text has been stripped of formatting and converted to upper-case letters.
(2) Many of the RTF files that I had created are still “located” where I created them (that is, the title of the RTF file can be seen in the list pane), but their size is 0 bytes. They do not contain any data.
(3) Same with Word files.
(4) Same with WMA and MP3 files.
(3) Same with PDF files, but “No File” shows up in the viewer.

Here is some context: As part of my dissertation research I have been doing research on about 30 organizations, and each one has a folder in my DT database. Today I imported a bunch of files (transcripts, audio recordings of interviews, pdfs, etc.) into DT and rearranged the folders. This all went over without a hitch. Realizing that the database was getting quite large, I created a new database and dragged some of the folders over to it. For whatever reason, this didn’t copy all of the files, so I deleted them in the receiving database and retried with the “Move to…” command from the sending database. Everything seemed to work fine. After working with the two separate databases for awhile, I realized that I actually preferred having all of the groups in the same database, so I used the “Move to…” command and sent the groups back to the original database. Somewhere in this process, I believe, the data became corrupted. It may also be relevant that while intending to follow the instructions here (devon-technologies.com/scrip … -database/), I rebuilt the original database after making all of these changes (I had in my mind to verify & repair, but I wasn’t thinking…).

In any case, I’m a huge fan of DT and want very much to believe that if this mess isn’t repairable then it is at least explicable. I would be tremendously grateful if someone(s) could lend a hand. Thoughts on how to fix the problem or how it might have occurred would be great. I’ve tried repairing and rebuilding the database without any luck.

Thanks in advance.

Me again: I’m not sure if this makes any difference, but I just discovered that the files from the first transfer mentioned above–where I dragged the parent group to the new database and only some of the files made it–are the only ones corrupted. Curiously, they weren’t corrupted in the original transfer (I just pulled them out of the trashcan). I don’t know how to explain this, but someone smarter than I might find in it a clue…

You might want to take a look at this thread, especially the most recent post (that I forgot to reply to…). It seems that one should avoid moving lots of files from one database to another. It seems to be a memory issue but it can also be interpreted as a serious bug

viewtopic.php?f=3&t=12135&start=0

Another strange development: Now DT won’t allow me to import or drag-and-drop certain files to the database I’ve been using. I can’t determine a pattern–it does not vary systematically by size or file type. Either it accepts the file as normal or will not (with the log reporting, “failed”).

Where are the databases located - on internal/external drives, a disk image or a server? DEVONthink isn’t copying/moving the files on its own, this is handled by Mac OS X/Cocoa and shouldn’t cause any issues as long as the filesystem is fine and there’s enough free space available. Or did you rename/move databases currently in use? Do you use Clusters too or did you force quit DEVONthink? And finally, has been anything logged to the system console recently?

The databases are located on a disk image. The data are confidential, so I keep them encrypted, following http://www.devon-technologies.com/support/academy/onlinetutorials.html?backlink=dt_tutorials.html&show=devonthink/tutorials/encryption.

There is plenty of free space.

I haven’t moved or renamed the databases recently (other than creating the new one mentioned above). I don’t use Clusters, and I haven’t used force quit.

I’m not sure what you mean by “logged anything to the system console,” so I guess the answer would be no.

Here’s the strange thing: Even if I accept that some of my data are lost, if I decide to start fresh and create a new database, there are certain files that still can’t be added to DT. I don’t understand.

That’s indeed strange. What kind of files did you try to add? Could you zip them and send the archive to cgrunenberg - at - devon-technologies.com? Thanks in advance!

After messing around with things for awhile, I think I discovered that the issue is lack of space on my sparse bundle disk image. In my defense: I originally set the size of the disk image at 20 GB. If I add up the size of the folders contained within the disk image (including the DT databases), they total to less than 15 GB, giving the appearance that I have more than 5 GB to spare. However–and this is what I just realized this morning–the info panel for the disk image shows that I am at maximum capacity–20 GB. I’m relatively new to the Mac, so perhaps this way of accounting for space is not a surprise to some, but it is to me (where are those extra 5 GBs?). In any case, I think think explains why my file transfers went awry and why I couldn’t subsequently add anything to DT.

I’ll take responsibility for problem, although I’m still perplexed at how the OS calculates and sums the folder and disk image sizes. If anything, it would have been great if DT had thrown up a simple error stating that there was no more room to add files. Since that wasn’t obvious from Finder, it would have saved me a day’s worth of work.

Hi,

I wonder how you set up the sparse-bundle… it should have grown with new data…

next time (when/after recovery) in Disk Utility choose “growing sparse-bundle” image ( the lower most option available in the dialog) – sorry I am on a German system where it is “mitwachsendes Bundle-Image”.

Perhaps someone on an English system can add the correct english term…

Regards,

Rolf

It’s “sparse bundle disk image”, with a preset/custom maximum size that defaults to 100MB:

sparse bundle disk image sizes.png
There’s no “infinitely expanding” size choice.