Looking for a way to identify which files are “missing”

Everyone knows one should do a “Verify & Repair” of every database every now and then. I do it, so my databases are generally healthy. I say “generally” because there remains a bunch of “missing files”. My problem is that I’d like to know which files are missing. I would like to go straight to the reference(s) to said missing files, so I could either look for the missing file somewhere — maybe in an archive — or delete the reference altogether so that it doesn’t keep reappearing on the next “Verify & Repair”. I think the most elegant way of accomplishing this would be a Smart Group that would collect all these “lost pointers”. Unfortunately, I haven’t found a way to set one up. Maybe I overlooked something?

Can someone help me? If there’s no solution to this problem, I’ll consider posting a feature request.

Thanks in advance.

Olivier Spinnler

In my experience, the major of “missing” file reports result from not updating indexed items before running Verify & Repair. Files get moved in the file system, indices are not updated, DEVONthink doesn’t track external file moves and reassociate the moved files to indices, and thus things go “missing”.

This means that before using Verify & Repair, it is best to go through all of your indexed files and groups and use File > Update Indexed Items. Note, this command does not automatically update all indexed groups in the database – just the one that is selected. So, this process takes a while. (There is a script in the Support Assistant to update all groups in a database, but it only updates parent groups and not children.)

So, when you’ve done that task and then run Verify & Repair, if you still have a “missing” report, then open Window > Log. The Log lists all of the missing items and their location. Select one, and either choose “Reveal” from the contextual menu or command-R (“Reveal”), or merely navigate to the indicated location in the database. Resolve the missing aspect of each item, and run Verify & Repair again. The Log is cumulative, and it can be helpful to press the “Clear” button in the Log display before running a Verify & Repair so you get a report for just the most recent Verify & Repair.

(A “Smart Group” approach would be helpful, though not feasible yet. OTOH, the Log is a sort of Smart Group.)

Thank you very much for your reply. It did help… up to a certain point. OK, now I know how to go to the “missing files”. Next problem is this one:

  • I double-clic in the log on the “reportedly missing” files. Systematically, when this “missing file” is an internet bookmark, not only is it actually present in the database, but it seems to be perfectly valid, since the relevant webpage displays OK in the preview pane.
  • I tested if maybe after this «retesting» of the file/link/bookmark, it would be reported OK in the database. Unfortunately, it’s not the case: a new “Verify & Repair” of the base keeps reporting these same files as “missing”, although they are actually present and seem perfectly OK to me.

Thanks again.

Olivier

How are you verifying this?

  • I double click the file in the log →
  • this takes me to the file in a three panes view →
  • the file is selected and the relevant webpage is fetched from the internet and displayed in the view pane
  • on the “Info” about the file, I don’t spot anything particular.

Interesting… have you done a Tools > Rebuild Database? Even though it can take a bit of time with a large database, I am a fan of this process.

Also, make sure you are keeping your Trash empty (noting that it shouldn’t be used as another filing cabinet). These files matter to a database since they’re just in a different location but still “present” to the database.

Thank you for helping me solve this problem. In conclusion, I’m offering to the DT community this guide and piece of advice. Maybe it could be included in the user’s guides? ()

Databases health

In order to keep a DEVONthink database healthy, it’s a good idea to verify it from time to time. Here’s a proposed walkthrough:

Tools > Verify & Repair

This can also be invoked with [Shift – Cmd – y].

If a dialog box appears offering to repair the database, by any means, proceed with the repair. Some “orphaned files” might have been found. These are files that were actually present in the database but not referenced anymore. Now, they are indexed and referenced and they can be found in an “Orphaned files” at the root of the database. Check that you still need them and either trash them or file them.

Now comes the problem of the “Missing files”.

Missing files

After a “Verify & Repair” operation, DEVONthink might tell you that there are still “x” files missing. You can see in the Log window which files they are. If the Log window did’nt spontaneously appear, you can find it under “Window > Log”.

In the Log window, you can double-click the mentioned files, in order to go to their reference in the database and take the appropriate action. I.e. either:

  1. delete the reference or
  2. find the file somewhere — in a backup, in an archive or on any disk you might have copy it to — and import it again in the database.

Sometimes, files are mentioned “missing”, but they are not. This might be corrected by rebuilding the database. So, the last maintenance step should be…

Rebuilding the database

Select “Tools > Rebuild Database…” and every data will be exported, the indexes will be cleared, and everything will be imported back into the database, effectively recreating all the indexes afresh.

This “rebuilding” of the database should be performed from time to time, as a preventive measure. However, it’s not 100% garanteed that any “missing file” is not irremediably lost in the process. So, as always, it’s a good practice to keep a duplicate of your datafiles somewhere, just in case you have to desperately look for a file in one or many years from now…

I hope this helps anyone having the same problems as me in the future.