In my experience with AI issues within Devonthink, the solution is a cloud model like gemini to act as a liaison, in that, it provides all steps and troubleshooting when you are testing a model’s performance in Devonthink, which is difficult as models tend to feign ignorance and sometimes outright lie about their ability to access your database if your prompts aren’t confident. You must test them with foolproof prompts and you must force them to prove to you they are integrated and can see all your notes and tags. Oftentimes if you’re prompts are confident, the AI will provide incredible results.
You can’t just integrate a model and ask it in DEVONthink global AI chat or inspector AI chat. It will be confused and sometimes not even know where it is. Sometimes they’ll say inspector chat doesn’t have full access as global does, sometimes it will have all the answers no matter which chat. It’s haywire sometimes.
The only issues I’m hitting is if I’ve got a doosie prompt that takes let’s say 10 minutes, Devonthink times out the communication while the AI is still processing.
I want to understand the philosophy aspect from your title. Philosophy? The body of the post seems to be practical observations, and I haven’t seen the timeout issue, but my long prompts and long responses are made to an LM Studio instance running on a local network, so maybe that’s why I don’t see the timeouts.
Philosophy of doing it seems to be like the rest of DT: it’s an unbiased aperture through which access and function for your data is made available. I think there’s work yet as to how that aperture works, but it doesn’t seem much different from the other tools in DT: open a text file, save it, etc. The AI chat receives data and can do some things with your data, etc. I’ve been surprised at how much the router in DEVONthink can do — it’s smarter than the chat assist in LM Studio.
I think perhaps your comments on models has more to do with whatever AI the user chooses to bring to the data — DT just facilitates the meeting. So far, though, I haven’t seen the issues I think you are describing. How are you testing this integration? Is this when running Gemini through the API? Or are you using a different model in the chat interface for DT?
While AI access in DEVONthink is still somewhat nascent, we have quite a few people effectively using access to external AI in DEVONthink.
You have provided no information about the model you’re using, the prompts you’re attempting, and what you’re actually trying to accomplish. You mention Gemini (which is not the specific model) as some kind of “solution” but that doesn’t concretely state it’s what you’re using.