I must be doing something wrong. I am trying to use my local LLM (Gemma3, in this case) via LM Studio to process a PDF with text of a receipt. As background, chatting with my documents using a local LLM works fine.
For my batch process, I have a prompt that is working when I manually process it in LM studio. When I try to write a batch process in DT though, nothing happens. The GPU does not spike at all as if it is not even engaging my LLM. Screencap below:
Best,
Jason
What is your prompt and what model is chosen in the Chat Query action?
My prompt is:
Please extract information from the attached PDF to rename it. It is a receipt. The resulting format of the name is “YY M-D ” where YY represents the year with only two digits (e.g., 25 for 2025), M is the number of the month with no leading zero, D is the date with no leading zero, DD is the dollars with no leading zero and CC is the number of cents. Payee should use typical title capitalization. Please reply with ONLY the suggested name based on the contents. Remove any leading zeros. Example output "25 2-5 Walmart 10d39
Regarding selecting models… Since posting, I have been able to get it to work with some models, but not others. Worked: microsoft/phi-4 (4bit quant), lmstudio-community/phi-4 (8 bit), deepseek-r1-distill-llama-8b
These did not:
deepseek-r1-distill-qwen-14b-mlx
gemma-3-12b-it
deepseek-r1-distill-qwen-14b-mlx
qwen3-8b
lmstudio-community/qwq-32b
Jason
I hope you know there’s no guarantee of any model working, especially when it comes to locally run AI. If you find one that seems to work consistently, use it. But still be aware it can be wrong from time to time.
Of course. I will just note that the models that do not work for batch processing (in other words, nothing happens) do work for chat within Devonthink. Thus my concern was that there was something breaking for these models when it came to the batch process function, specifically.
Thatnks for the clarification. Development would have to assess that.
Many models do not even work in LM Studio, e.g. crash the engine or cause error messages. That’s an advantage of Ollama which provides a carefully curated list of recommended models (Ollama Search)
In DEVONthink documents are not attached (like in AI apps) but selected usually. Therefore terms like the selected document or this document are usually recommended instead, both in batch processing/smart rules and the chat assistant.
