Training local models

Hi,

I just testing the new functionality with local AI Models via Ollama and LM Studio. So far the models can just summarize a PDF but weren’t able to answer me a single question about the particular document. Do I miss some settings beside the AI pane? Do I need to train the models?

My usecase is: I have buckets (folders) of distinct cases which are a collection of documents belonging to one person. I want to be able to ask questions about that person like: When is xyz born, which health insurance was he, etc.

I suspect you are expecting more from a local LLM model than is practical at present.

Which model did you use?

I used llama3.2 and mistral via Ollama and deepseek via LMStudio.

Apparrently :slight_smile: Do paid models will meet my expectations?

I would recommend Mistral Small 3.1 or Gemma 3, ideally with 24b parameters if that’s in the range of your hardware.

1 Like

The mistral model gave me these instructions after I asked for a summary:


To summarize the document in DEVONthink, follow these steps:

  1. Retrieve Text Content: Use the “get_contents” function to extract the text from your PDF document.
  2. Manual or Automated Summarization: Once you have the text content:
  • Manually read and summarize the content.
  • If possible, use another tool within DEVONthink (if available) to automate the summarization process.
  1. Considerations:
  • Ensure that the “get_contents” function is compatible with PDFs in your version of DEVONthink.
  • Be aware of any limitations regarding file types and built-in functions provided by DEVONthink.

By following these steps, you can effectively summarize your PDF document using the available tools within DEVONthink.


What does that mean?

Which Mistral model? There are tons of Mistral models available.

Basically that the model you’re using seems to be completely confused. Or doesn’t support tool calls.

Which mistral model

ollama list
NAME ID SIZE MODIFIED
mistral:latest f974a74358d6 4.1 GB About an hour ago

Sounds like this one which is a rather old and limited model, its support for tool calls (see above) is obviously very unreliable:

This one is a lot better but requires more RAM and a faster computer: