Re: Inexplicable corruption - please help
Postby rosegeorge3 on Thu Oct 27, 2011 5:29 pm
I am writing a book so using one book-dedicated database with many topic-based groups. I created the database a while ago by importing contents of my Mac folder. A few days ago, the database told me there were 1500 missing files. I created a new database from the same folder that I’d used for the previous database, again importing everything into DT, manually imported the files that were still missing, then spent all day today allocating files into groups. By late afternoon, every single pdf came up as “no file” and all rtfs were corrupted - all in caps, no formatting, no punctuation. In short, the same as this thread from 2008: viewtopic.php?f=3&t=12234&hilit=pdf+no+file
However, I don’t understand disk images or sparse bundles (the solution offered in that thread). I can tell that DT could be wonderful, but having written for support and got none and having now spent 3 days trying to sort things out, I am losing my patience. NB I am not a sophisticated techie so layperson language please. All help most gratefully received.
I certainly wouldn’t be satisfied either, with those results from a document management database.
But something very strange appears to be going on here. The forum search link that you cited (Inexplicable corruption - please help) describes problems very similar to yours. In that case, the problem was lack of space to hold added content within a sparse disk image. But a similar problem would result if there’s insufficient free space on any storage device where the database is saved (whether it’s on the boot drive, an external drive or a too-small disk image).
I’ve searched for the earlier message you indicated that you had sent to Support, but can find only one message dated today (28 October 2011), and it has less information than in your forum post. I’ll respond via Support, with a request for information about the available free space on the storage device that holds the database(s). That may or may not be the cause of your problems, but should immediately be checked.
More generally, the availability of SSD (solid state disk) drives in Macs provides amazing speedups in read/write access speeds, compared to conventional hard drive. But SSDs are currently more expensive and come in smaller sizes than HDs. I love my MacBook Air with a 256 GB SSD drive, but I would have serious problems were I to try to cram too many files from my Macs with 500 GB or 1 TB drives onto it. Apple engineers recommend keeping at least 10% to 15% of the rated drive capacity free, in order to provide necessary space for temporary files required by applications and the operating system, as well as for Virtual Memory swap files. Running out of free storage space could not only result in the kind of problem noted above, but also opens the possibility that the operating system may overwrite and so erase or corrupt existing data files.
Bill, thanks for your response. I’ve emailed you too.
So, my MacBook HD tells me I have 8 GB available out of a capacity of 120 GB. I will try to free up space, but my question is then, how much space does DevonThink need to run, using a database created from a 1GB folder (importing not indexing)? And if I do succeed in freeing up space, then will the no file pdfs magically reappear or do I need to re-import them?
I’m also puzzled as to why only DevonThink has problems when all my other applications & programmes are working fine?
Also that may explain the missing pdfs, but does it explain the corrupt rtfs, and the 1500 missing “no index” files?
Thanks
I know that at some point DTP is going to work smoothly and beautifully and I will love it. I have not reached that point yet.
I have spent the best part of a week now finding all empty, no file, no index, o byte files and re-importing manually. Some of those I filed directly in my chosen database. Some I put into global inbox to sort later. Having just imported the last missing file, I went to global inbox to start sorting. There were 180 files listed in the inbox icon, but in the next pane, only a bunch of folders and then tags, with only half a dozen documents in them. I - stupidly, it seems - decided to ungroup to free up the documents. of course they disappeared. I now have 200 files or so in my trash that I also can’t access; when I click on the trash icon it shows me half a dozen files.
please, someone, tell me what I am doing wrong? Have I once again lost the 200 files that I spent this morning carefully importing?
I have wasted a week on this that I cannot afford to waste.
Hi rosegeorge3,
I sympathise with you. I have a long history of using DTP to store my research and I have over the years had regular problems with empty, no file, no index, o byte files, orphans and then having to re-import them manually.
Today I was filing another batch of 12,000 orphans that I had parked in an ‘unfilled’ group in DTP and when moving a subset to an open DTP database they all became empty documents. I ran the ‘verify & repair’ and DTP detected 45,000 orphans! Then DTP crashed while repairing (it got up to document 900 or so). After restarting I immediately ran ‘verify & repair’ again and DTP reported only 38,000 or so orphans.
So I’ve given up for the moment.
Luckily I keep a copy of every bit of original intell that I collect so I am ready to find an alternative way of managing and using my research (Spotlight indexing??). Perhaps in a more industrial-strength database technology (MySQL??).
I will miss some of the neat DTP features but not the bugs/frailties.
Has anyone else come to this same point and do they have any ideas about what alternatives there are for life after DTP? I’m happy to persist with the agent product and then export each week the results to another solution.
Just for the record, I run the latest DTP office, agent and To Go. I have (had?) about 15 DTP databases holding in total 130,000 objects. Mostly office, PDF, image files. Adding about 1200 a week. Individual DB sizes ranged up to 20GB. I use the latest MB Air, OS and all software. I have tried to keep free 10 - 20 GB on my Airs HD.
Anyway, hopefully someone has some tips for me - they will be appreciated as I need this research badly!
Analyst, I don’t have such problems. Please locate a recent crash report from your DEVONthink application and include it in a message to Support.
I would suspect memory errors related to database size and/or insufficient free space to hold temporary files.