Hi all! I have a number of indexed folders in DEVONthink. By far the biggest and most active is my Obsidian vault. I often rename files there or move them into folders, using Obsidian, Finder, or a number of automations. Afterwards, DEVONthink reports the file as missing.
For example, if I have a file named Something.md and rename it to Something More Descriptive.md or move it into a folder in my Obsidian vault, DEVONthink will report it as missing.
How can I stop this and tell DEVONthink either to ignore “missing” files in that folder or to update its index of that folder regularly in order to minimize these notifications?
How can I stop this and tell DEVONthink either to ignore “missing” files in that folder or to update its index of that folder regularly in order to minimize these notifications?
You can’t ignore missing files and that would be a very bad idea. You’d end up with an inconsistent database and have no clue there’s any issue.
Where is the Obsidian vault located? I’m guessing in a cloud-synced folder, e.g., iCloud.
PS: For good reasons, indexing is not the default vector for getting data into DEVONthink. Have you read the In & Out > Import & Indexing section of the built-in Help and manual ?
Just to be clear, I wouldn’t want to ignore missing files in general, just in that folder. I just want DEVONthink to make the files in my indexed folder searchable from within DEVONthink and to keep up with any changes due to renaming or moving files without telling me those are errors.
The Obsidian vault is in an iCloud folder.
I have read the In & Out > Import & Indexing section of the manual. I definitely want these files to be indexed, not imported, because the goal is to have results from my Obsidian vault show up in real-time in DEVONthink and not grow stale. The problem is that DEVONthink isn’t picking up on changes, even file renamings, and even when I explicitly use the File > Update Indexed Items command.
I have been using File > Update Indexed Items, but DEVONthink typically doesn’t pick up any of the changes in that folder.
To illustrate the problem, this just happened a moment ago:
DEVONthink reported 9 missing files from my Obsidian vault.
I looked at each and confirmed it was a file that had been moved or renamed outside of DEVONthink.
I deleted each of those files in DEVONthink.
I emptied the DEVONthink trash.
I did a manual File > Update Indexed Items.
I did a manual File > Verify & Repair Database.
DEVONthink reports the same files as before as missing. It also reports one newly-created file as being empty, even though it definitely is not.
ETA: I did the above steps a second time, and, at least for now, DEVONthink doesn’t see any files as being missing or empty. But that’s a lot of work for 9 changed files, a relatively small batch of updates.
I have an Obsidian vault indexed in a DEVONthink database, mainly to seed the AI so that See Also recommends notes from that vault when suggesting related documents. That database is synced to DEVONthink to Go, but I never access the Obsidian vault portion of the synced database outside of DEVONthink on the desktop. The Obsidian vault is also synced in the background to Obsidian on other devices, where I add and edit content in the vault using Obsidian for iOS.
So, here, you can see I’ve set up a lot of potential conflicts by indexing – two different syncs of the same data occurring probably simultaneously because when I add or change a file using Obsidian desktop then both DEVONthink sync and Obsidian sync are touching the new / changed files. This is why I never modify indexed Obsidian files in DEVONthink, desktop or mobile.
But, what does happen pretty regularly, is that DEVONthink is doing a backgound V&R and reports that the database where the indexed files are located is damaged, or, sometimes, “missing” files. Neither is true – if I wait a minute, ignoring the messages, then the situation corrects itself. What’s missing is found by DEVONthink and life goes on.
I know there is a potential for conflict (that’s what’s behind the errors DEVONthink sometimes reports), so that’s my I never change Obsidian vault files in DEVONthink. Perhaps I’m overly cautious, but OTOH I have no reason to edit those files there; I just want to have the data on board for See Also purposes.
My idea is similar; I just want the notes there as “see also” material. Like you, I never modify Obsidian files in DEVONthink. But DEVONthink also doesn’t seem to figure out the changes; it just keeps reporting the same issues until I manually tell it to stop trying to index a given file by deleting the indexed reference and emptying the trash.
This is what I had found as easiest way to take care of the problem, which I did every time I opened devonthink (I think I found this on obsidian forum).
Window → log → clear
File → Verify and Repair → ignore
Window → Log → Select All → Delete
Trash → Empty Trash → Only in Database
Now practically I like the idea of obsidian and think it’s WYSIWYG editor and speed on mobile for templates, etc is better than DTTG, but the pain of sync and the multiple files always showing duplicates or missed name, etc wasn’t worth the function I got, so I just use DTTG for everything.
I’m seeing related behavior, since the last update.
My workflow includes saving stories in markdown to the Global Inbox, then moving those stories to a relevant INDEXED folder in Devonthink. Previously, doing so inside of Devonthink would add the file to the indexed folder on my hard drive. Now it adds the file somewhere else.
Updating the indexed folder does not make the file appear in the folder on my hard drive. If I move the file from the Global Inbox directly into Obsidian (instead of moving the file within Devonthink), duplicate files appear in the same folder on the hard drive.
You should not be messing about in the internals of a DEVONthink database.
PS: There is a bug in the 3.9.1 release related to files in the Global Inbox not deconsolidating in an indexed group in another database. This should be addressed in the next maintenance release. Thanks for your patience and understanding.
LOL. I’m not messing around with the database. I was curious where the file went, so within Devonthink I used the context-sensitive menu to “Show in Finder.” That’s where my screen-shot came from.
I guess the bug is where my difficulties are rooted. I’m looking forward to the maintenance release.
While you may be behaving, we unfortunately do have people who see things like this and think it’s okay to access and modify the files in a database’s internal structure, hence the admonition. Just trying to keep people aware and their data safe.
Got an update: as I made additional changes, I had the same problem come back.
I removed a bunch (64, I think) of files from my Obsidian vault. DEVONthink detected that they were missing, but “Update Indexed Items” didn’t resolve the problem. I deleted all of the missing files from DEVONthink, emptied the trash, and did another Verify & Repair Database, which resolved the issue from DEVONthink’s perspective.
But this just brings me back to the original problem. If the file is just indexed in DEVONthink, I just want it to be that: indexed. If it goes missing on the filesystem, then DEVONthink should note that and remove it from its index. At the very least, I would think “Update Indexed Items” would resolve the issue without having to manually delete all of the missing items in DEVONthink.
I see above that your files are in Apple’s iCloud. My hunch is that if DEVONthink app thinks the files are “missing” it might be because iCloud deleted them on your local machine to “help” you with reducing local disk space. Do whatever you have to do to make sure iCloud keeps all files local on your machine. I realise from experience that might difficult to control with iCloud. All this a hunch.
Also, do you really need the Obsidian files indexed into DEVONthink? Doesn’t Obsidian do a sufficient job of letting you find and manage the Obsidian files? Just wondering.
