Problems trying to chat with the Open Document in DevonThink & AI

DEVONthink Artificial Intelligence
1

I must be doing something wrong. I open a document, I click on the AI chat button, and I ask the model questions about the document. The model typically responds with something like this:

Function Call to Perform Wikipedia Search

To find information about the definition of darkness, we can perform a Wikipedia search. The function call would be:

{
  "name": "perform_wikipedia_search",
  "parameters": {
    "query": "Darkness"
  }
}

Selecting the text of the document and trying the same produces the same results.

If I use the chat button on the main DEVONthink window (instead of the reader window), and select the text, it answers my questions based on the selected text.

Intuitively, it seems I should be able to use the GenAI model to “talk” to the document when it is open in the document reader.

Am I doing something wrong?

image
image1920Ă—1672 213 KB

image
image1920Ă—1000 280 KB

1 Like
2

Welcome @darthjedi
You are making assumptions about the behavior. Nowhere in your prompt did you mention the document being viewed. AI is not prescient nor does it have eyes to see what you see, so it’s best to provide better instructions to it.

That being said, there are modifications we are working on to allow some of these looser questions, but there is no downside to greater specificity in prompting.

Have you tried commercial models, e.g., Claude Sonnet ?

in this document, What is the relationship between learning organization, learning orientation, and organizational learning.html (8.5 KB)

3

I figured I was assuming something and doing something wrong…

I assumed that opening up the PDF and opening up the chat attached to the PDF window, after configuring one of the supported AI APIs and models, would have fed the plain text of the document into the context window of the model so I could just “chat” with the document; otherwise, why not just load the document into an external LLM versus using the built in chat?

I do recognize that the integrations are early WIP - is there some place I can go to read about how to use, and what DT AI does support with the integrations, so I can disabuse myself of my assumptions, and level set myself in my expectations?

Thank you for the response! I’m new to the forums, but have been a customer for a LONG time.

2 Likes
4

Okay, I found what you probably have to type dozens of times a day, by reading one of the other posts; I’ll check this out tomorrow!

Please read the Getting Started > AI Explained section of the built-in Help and manual to see how external AI is integrated into DEVONthink.

5

Actually that’s not the case, it’s intended that contents are only sent if requested and if the model supports tool calls to save tokens & time and to improve the privacy. Therefore the initial issue was caused by invalid tool call responses, despite Ollama claiming that the used model should support them.

I would highly recommend Mistral 3.2:27b, Gemma 3:24b, GPT-OSS:20b or Qwen 3 instead. LLama 3.3 is kind of dated by now.

1 Like
6

That was a simple enough fix. I also went in and tweaked the role and the prompt to provide a more scholarly type of response [I am currently using DT as part of my tool chain for my doctorate].

Thanks for the suggestion!

1 Like