I’m having trouble figuring out how to switch the model in the help popup. It’s set to the smallest of LLM models and, for example, Haiku is not able to process any of my questions. I think this is because it’s a smaller model. When it was Sonnet, I had no issues.
Is there a way to stop the AI model from processing your request? I don’t see a stop button. If this isn’t possible, could it be?
Also, in the settings for AI Role for setting default instructions, is it possible to make this a multi-line box? Then one could scroll through and see what’s there now (also easier for editing).
The models don’t necessarily have the same capabilities. And what you’re asking (which you haven’t provided examples of) may be beyond the reach of certain models, or the answers will be more limited. If cost is no concern, you could opt for a higher-tier model all the time. But even then, the nature of the response you want may be more succinct than higher models deliver. This means your prompts will need to be even more well-crafted.
There isn’t a stop button. Development would have to assess adding an explicit button.
I’m sure it’s possible but are you aware that’s not a generic role but one for automation?
Yes, though, to be honest, I don’t spend much time with the smaller models. However, I didn’t choose any of these small models for the help; they are pre-chosen, and I cannot switch to a higher model. Here are screenshots showing the model selected in DEVONthink Help, the inability to choose a model (only a provider), the prompt, the “retries” I immediately get, and the error output showing there aren’t enough tokens (which is expected, since Haiku doesn’t allow for as many). I never had this problem before. I don’t remember which model was selected, but I know it wasn’t Haiku.
Oh, I wasn’t aware this was only for automation. Would it be possible for us to have the flexibility to add our own default instructions? And the suggestion to change that to “Automation Role.”
Development would have to consider this and there is some discussion on role management. But you can create a reusable batch process using a Chat - Query and a defined role.
With a pre-defined role of: You are a highschool math teacher providing explanations and lessons, including step-by-step instructions and practical examples. These are presented in well formed Markdown with tables or lists as appropriate. Use MathJax for equations. At the end of the response, include a horizontal rule and `
User Input`.
I understand the need to keep costs down and force the use of a cheaper model, but if that model doesn’t work, it provides no benefit. Personally, I wouldn’t have an issue with switching to a higher model that uses my API. Because, frankly, if I’m getting errors from the help AI chat, I’ll load the manual into a Claude Project and ask questions there, so it’ll cost me anyway.
I did find an answer for the toolbar thing. Thanks.
At the risk of being the lowest denominator here… I understand using batch processing for tasks that are repeatable, such as scripts for specific outcomes, or perhaps a structured set of tasks performed before chatting with the AI about it. But I don’t understand using it for something as simple as chatting with a model, especially since there’s a sidebar chat that accomplishes the same thing more easily. I can see how and why it could be used like this, but there isn’t anything in particular I’d use it for - at least not at this time. I find the chat sidebar easier and faster.
With that said, the instructions embedded in DTP are doing well and must be similar to what mine would be, as I hadn’t noticed that it wasn’t adhering to mine. LOL
Personally, I wouldn’t have an issue with switching to a higher model that uses my API.
I like to use Ollama to access local models. It can’t be used in the Help Assistant. I created a Group specifically for Devonthink help. I put the manual in there and select it for context and then run whichever model I want in the AI info panel. It supports the other commercial models obviously as well. Then save those conversations into the same group if I want to refer to them again or even use them as context for continuing the conversation. That might be one alternative to consider if you want different models.
That’s a good idea. But I’ll have to test it and see how fast it runs. Even though I’m running a newer MBP, I find the local models I prefer to run seem to take the longest (not surprising, really).
It is a way you could accomplish using a role, as you have inquired about. I already mentioned role management is in discussion, so I provided a possible interim solution and also an example of a batch processing configuration.
