Training local models

DEVONthink
1

Hi,

I just testing the new functionality with local AI Models via Ollama and LM Studio. So far the models can just summarize a PDF but weren’t able to answer me a single question about the particular document. Do I miss some settings beside the AI pane? Do I need to train the models?

My usecase is: I have buckets (folders) of distinct cases which are a collection of documents belonging to one person. I want to be able to ask questions about that person like: When is xyz born, which health insurance was he, etc.

2

I suspect you are expecting more from a local LLM model than is practical at present.

3

Which model did you use?

4

I used llama3.2 and mistral via Ollama and deepseek via LMStudio.

5

Apparrently :slight_smile: Do paid models will meet my expectations?

6

I would recommend Mistral Small 3.1 or Gemma 3, ideally with 24b parameters if that’s in the range of your hardware.

1 Like
7

The mistral model gave me these instructions after I asked for a summary:

To summarize the document in DEVONthink, follow these steps:

  1. Retrieve Text Content: Use the “get_contents” function to extract the text from your PDF document.
  2. Manual or Automated Summarization: Once you have the text content:
  • Manually read and summarize the content.
  • If possible, use another tool within DEVONthink (if available) to automate the summarization process.
  1. Considerations:
  • Ensure that the “get_contents” function is compatible with PDFs in your version of DEVONthink.
  • Be aware of any limitations regarding file types and built-in functions provided by DEVONthink.

By following these steps, you can effectively summarize your PDF document using the available tools within DEVONthink.

What does that mean?

8

Which Mistral model? There are tons of Mistral models available.

Basically that the model you’re using seems to be completely confused. Or doesn’t support tool calls.

9

Which mistral model

ollama list
NAME ID SIZE MODIFIED
mistral:latest f974a74358d6 4.1 GB About an hour ago

10

Sounds like this one which is a rather old and limited model, its support for tool calls (see above) is obviously very unreliable:

This one is a lot better but requires more RAM and a faster computer: