Hello,
I cannot sync to my sync store because I have one missing file in my database. The missing entry can be safely deleted from DT and I already moved it into the trash, but it still moans. How can I remove this entry from the database?
I don’t want to rebuild the database, I just want to remove this entry.
Bye,
Oliver
Hello Oliver,
I have had this problem a few times, and what I did was;
First take a backup so you can revert if this does not work.
Create a “Smart Group” as shown on the attachment.
Find the files which have 0 bytes (you will see they still show up in the database), and delete them. Do this on ALL the databases which sync via your sync store.
Verify/Repair and Backup/Optimise all the databases.
Only then do you try to sync again.
This worked for me.
If you mean to the Trash in DEVONthink, that is not deleted. The Trash is still part of the database. Emptying the Trash will remove the entry from the database.
But emptying the the trash will result in restoring deleting documents from other DTs via the sync store 
That did not work for me. I have reportedly 12 missing files after verify&repair, however, this smart group shows zero files.
I do not find the Smart Group technique reliable. Go to the report that DEVONthink provides – the Log. Try:
If for some reason “Reveal” is not working, the Log is telling you the path of the missing document, and you can navigate to that spot.
I have been trying to sync for two days now. Verify and repair reports nothing unusual but the sync starts chugging along and then stops because a file is missing.
The database contains 13,902 documents. So far about 10 have been found missing. This makes me start to question the reliability of DT.
Fortunately for me I have two cloud backups, a backup to a Synology, three hard disk based backups and one to a thumb drive. I use Arq, Time Machine and Carbon Copy Cloner so as not to rely on one technology. I bet not a lot of people are as obsessive about backups as I am.
There are tons of articles about going paperless all over the internet with people extolling the virtues of DT and how once it’s in there you can now shred the original. Oh, wait!
If I was designing a sync system, I’d damn well make sure it ran a verify and repair before starting and maybe it does. Unfortunately, it appears that V&R is also borked.
At the very least we need a tool which simply searches for missing files. Even if it’s an Applescript. The only problem is that the DT dictionary doesn’t seem to have a method of detecting that! B O N E headed!
Cheers,
Jason.
“This makes me start to question the reliability of DT.” :: Why? Are you indexing files?
No, all files are in the database.
Cheers,
Jason.
Sorry Jim, I missed the ‘Why?’
Aren’t missing files enough to question the reliability of a document cataloging and management system?
Cheers,
Jason.
No, I don’t think so. Every database has the potential to experience small amounts of inconsistency (or more if the database is poorly designed). (Even the indices for your filesystem or Spotlight periodically run into issues. This is a similar scenario.) This is why there are tools built to be able to periodically check your database’s health. (I suggest once a week for a heavy User.)
Where are you databases stored?
Wow, that’s like saying “but I did not shoot the deputy!” Setting the bar at “but everyone else does it” does not inspire confidence.
This would be fine if the bloody tools actually worked!
They are stored on my Mac SSD.
Cheers,
Jason.
I’m not trying to sell you on anything - just telling you the facts.
If the Verify & Repair is telling you you are missing files, it is doing its job. Look at Window > Log to see which ones.
PS: Not on what drive… what location in your filesystem? ie. is it located in a Dropbox folder?
Jason, have you run Disk Utility > Verify Disk?
I’ve got a MacBook Pro Retina with 500 GB SSD. Over the past year Verify Disk has reported errors requiring repair four times. In each case the repair was successful. (I’m not pleased by the frequency of errors on my SSD, but so far I’ve caught small errors early enough not to have had problems.)
I’m in the habit of running Disk Utility every few weeks. Had I not been doing so, I suspect that the errors would have accumulated to the point of data loss.
This is only one issue that can reduce the reliability of a computer, but still another that shouts for users to maintain good backup systems with external backup storage.
Every couple of weeks I run an OS X maintenance utility that provides a user-friendly interface to maintenance procedures Apple provided in the Terminal.
Habits like that have a payoff. My DEVONthink Pro Office databases are stable. Verify & Repair doesn’t find errors. I don’t have missing files. I haven’t had to resort to a backup for years.
I’ve already stated twice that the V&R doesn’t tell me anything. It thinks everything is okay. The log window is empty. Only after churning through what I can imagine are approximately 6000 documents (the couple of times a glanced at the progress before noticing the sync failed again, it was about halfway).
The database is in my home folder. It is not and has never been synced with Dropbox or anything else. It also looks likely that it won’t be synced using the built in system.
Thanks for your valuable time Jim, but let’s not waste any more of it on this. I’m going to start using a purely file based system and tagging (I’m already playing with Houdahspot and Leap as they seem to have matured a bit - didn’t you work there before?). At least that way I can restore individual files from my many backups.
Cheers,
Jason.
EDIT: spelling
Hi Bill,
I test my bootable backup every month and run Disk Utility on my SSD when I boot off it. No problems so far since I bought the MBPr a year and a half ago.
Cheers,
Jason.
It’s not problem, Jason (and I hope I didn’t come off as rude - not my intent). Have you done a Tools > Rebuild Database…?
PS: Is this a Dropbox Sync?
PPS: If you want to turn on verbose logging and send us a Console log… but I totally get it if you just want to go for the alternate approach.
Yeah, I worked for Ironic Software for 4.5 years and I used to be mad about Tagging. I am not off the Tag train but I do know that often tagging is used to compensate for substandard search functions. (This is why I much MUCH less frequently Tag in DEVONthink. The search functions are just that good and that fast that a contents-based search is often far faster than trying to remember a Tag I may have applied.) I still love Leap and sometimes still use it as a jukebox for my hard drive full of music. 
(Yep kinda morphed into a Leap clone. I always thought it should have stayed as it was in Yep 1.)
Cheers!
Verify & Repair is a good little utility, but there are some errors it cannot detect or correct and that can mess up Sync. For example, the problem may not be with the database’s metadata files, but with a peculiarity of a document file.
If Sync fails repeatedly, I would run Tools > Rebuild Database to ensure the integrity of my databases. Afterwards, check the Log (Window > Log) to see if any files were rejected in the rebuilt database.
More generally, if Sync fails, start a Support ticket. You will be asked to send in log files from a Sync attempt, that can help us diagnose what’s going wrong.
Hi Bill,
I’ve already started the process of exporting the files to disk. I started thinking of managing them by using folder indexing in DT as I’m not really feeling it with HS or Yep/Leap.
Is this viable for so many documents? Is tagging two way? Can I use my Applescripts as before?
I’m so used to DT that I’m hoping to get the best of both worlds. Moving files around inside the database doesn’t move them on disk does it?
Many thanks,
Jason.
Yes, although my personal preference is for self-contained databases, Indexed databases can work well for large collections of documents (the memory requirements concerning database size are about the same whether the document files are Indexed or Imported, and the most important measure of database size is total word count).
There are some important things to remember for Indexed databases. If an Indexed external folder or file is renamed, moved or deleted externally, the result will be missing files in the database, because the Path(s) to the folder or file(s) broke.
You can reorganize items in an Indexed database, but some confusion can result if one hasn’t thought about how Indexing works. The fool-proof approach is to leave the structure and contents of Indexed groups in place, and to use replicants of those items if you wish to file them into additional locations.
If you add content to an Indexed database (by creating new text documents, saving documents from other applications directly to the Global Inbox, using templates to create new documents or capturing documents to the database from the Web) the result will be a “mixed” database with some documents stored internally and some externally.
If you wish, non-Indexed content that is located inside an Indexed group can be selected and Control-clicked, then choice of the Move to External Group option will move the item(s) to the corresponding external folder and re-Index them back to the database.
To facilitate keeping a “pure” Indexed database, open the full Search window, set it to search across all open databases, then click the Advanced button and enter these criteria:
All of the following are true
Instance is not indexed
Then click the “+” button to name and save the smart group that lists non-indexed items.