I’ve been setting up DEVONthink 4 with a local model for working across a large collection of scientific papers, and I’ve hit a consistent problem with database search that I can’t resolve. I’d be grateful for a pointer on whether this is expected behaviour, a configuration mistake on my side, or a bug.
Setup
- macOS, Apple Silicon, 128 GB RAM
- Settings → AI → Chat → Provider: LM Studio, API URL http://localhost:1234/v1/chat/completions
- LM Studio server confirmed running; model confirmed loaded and responding
- A custom Role is set (instructing precise terminology and explicit uncertainty), which the model demonstrably follows
Models tested — identical behaviour with both, and both list tool use among their capabilities:
- openai/gpt-oss-120b (MXFP4, 32768 context)
- google/gemma-4-31b-qat (Q4_0, 131072 context)
Case 1 — single document selected: works perfectly
With the relevant paper selected in the item list, the chat returns a “Retrieved:” line naming the document, and the answer is accurate and specific, with page-level x-DEVONthink-item deep links for each claim. This is genuinely impressive and exactly what I want.
Case 2 — group selected: no retrieval at all
With “Datenbank” enabled in the chat Options, a group selected in the Navigate sidebar, and no individual document selected, I get no “Retrieved:” line whatsoever. The model answers entirely from its training knowledge. In my case the difference is unambiguous: the grounded answer correctly identifies the kinase and mechanism from the paper, while the ungrounded answer confidently states a different kinase and an unrelated mechanism.
The key point is that it doesn’t retrieve the wrong documents — it appears to retrieve nothing.
What I have ruled out
- Indexing: full-text search within the same group finds the document in milliseconds, with hits inside the PDF text. The text layer is fine.
- Filing: the document is confirmed to be inside the group I select.
- The model: two different models from different vendors, both tool-capable, behave identically.
- The database search setting: enabled and verified before each test.
- Stale chat state: tested in fresh chats after restarting DEVONthink.
Questions
- Is database search expected to work with local models served through LM Studio, or does it rely on tool-calling behaviour that currently only works reliably with the commercial providers?
- Is there a configuration step for group-scoped search that I may have missed? The Suche tab has its own settings — should a model be assigned there separately for database search to function?
- If this should work, is there any logging I can enable to see whether the search tool call is being issued at all?
Happy to run any specific test that would help narrow it down. Thanks for the work on DT4 — the document-grounded chat with page-level citations is excellent when it has the right documents in scope.
Devonthink 4.3.2., Tahoe 26.5
Welcome @ulf_wagner
What is the prompt you’re using when trying to work with a group. It logically wouldn’t be the same as the one you’re using with a document as the target is very different.
Here is an example naïve query with LM Studio on a document and the current location…
I just found a solution: if I delete the old chat and start a new one, than it works! Sorry that I didn’t write that earlier!
1 Like
No worries. And we were just discussing that here recently. Clearing the context of old chats before starting a new one is a very good idea. AI companies even recommend this. Less tokens spent up front, no muddied context window, etc.
Cheers!
Hi I ran into this same issue but haven’t had success. I tried a couple LM Studio models and could get an accurate chat response for the single doc (“The file … …categorized under the location “Orange.” It uses an accrual basis accounting method. The document was last modified on July 21, 2026, at 12:40 PM GMT+02:00.”) but when I used the exact same prompt “how many documents are in the current location?” the model could not answer the question directly: "To determine how many documents are in the current location, you would need to navigate through DEVONthink and check the folder or collection where this document is stored. Since I don’t have access to your specific workspace within DEVONthink, you can perform a search or browse that area directly.
If needed, you could use the search function (Cmd + Shift + F) in DEVONthink to find all documents with certain tags or keywords and count them manually."
Any suggestions? One model was 12B, the other 7B, context cleared aside from the initial single doc question. Thanks in advance.
Welcome @charlietuna
You need to provide information about what the current location was and what you had selected. See the Getting Started > AI in Practice > Controlling AI’s Access section of the built-in Help and manual for information on why these are important.
You also need to provide what actual model you were using, not merely the parameter count.
Thank you @BLUEFROG for your reply and continual support I’ve seen in the forums. I did manage to get connected with Hermes and am testing the success vs. failures. But the resolution may have just been restarting both programs. In Hermes it helped to confirm MCP connection by looking in Settings>Capabilities to see that it was connected to DEVONthink. After that I started a new Project with an empty folder selected in Hermes and the new session provided the results.
I will check out the Controlling AI’s Access for better understanding and again appreciate your response!
CT
You’re welcome!
And be aware that access via MCP in a third-party app like Hermes has a different level of access than the Chat assistant inside DEVONthink. So the subsection I pointed out relates to the Chat inspector / popover. MCP access is similar to running scripts, so the access can certainly be broader.
