# DT4 - AI - Ollama for local LLM - Recommendations?

**URL:** <https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597>\
**Category:** DEVONthink\
**Tags:** dt4\
**Created:** [April 3, 2025, 8:49pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597 "2025-04-03T20:49:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![axelbuehler](https://discourse.devontechnologies.com/letter_avatar_proxy/v4/letter/a/d6d6ee/32.png) [@axelbuehler](https://discourse.devontechnologies.com/u/axelbuehler)\
**Post date:** [April 3, 2025, 8:49pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/1 "2025-04-03T20:49:11Z")

</div>

Hi,

just tested ollama with qwq on Mac Studio M2 with 32 GB - worked, but slow (a couple of minutes for a mediocre summary of four pages). Great!

As privacy is probably one main concern not to use any API / public model - anyone recommendation what llm to use, what settings, how important is context window (I arbitrarily increased it to 10k) - and maybe what minimum tech spec is recommended - 32GB memory seems too small.

Regards  
Axel

---

<div class="post-metadata">

**Author:** ![BLUEFROG](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/bluefrog/32/135_2.png) [@BLUEFROG](https://discourse.devontechnologies.com/u/BLUEFROG)\
**Post date:** [April 3, 2025, 8:53pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/2 "2025-04-03T20:53:23Z")

</div>

Running a local AI is not going to be feasible or performant for most people. 32GB is not a small amount of RAM… until you want to run local AI.

Again, you should read the **AI Explained** section in the Help in regards to managing expectations and a realistic view of AI in DEVONthink.

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [April 4, 2025, 7:22am UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/3 "2025-04-04T07:22:13Z")

</div>

> [@axelbuehler](#):
>
> As privacy is probably one main concern not to use any API / public model - anyone recommendation what llm to use, what settings, how important is context window (I arbitrarily increased it to 10k) - and maybe what minimum tech spec is recommended - 32GB memory seems too small.

E.g. the recently released Gemma 3:27b with a context window of 32k (theoretically up to 128k) works quite well on an M1 Ultra with 128 GB and supports tool calls (LM Studio) or vision (Ollama). But of course still no comparison to commercial models.

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [April 4, 2025, 12:40pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/4 "2025-04-04T12:40:03Z")

</div>

Is this " **AI Explained**" somewhere available without needing to download the beta?

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [April 4, 2025, 12:43pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/5 "2025-04-04T12:43:36Z")

</div>

See [https://www.devontechnologies.com/download/extras](https://www.devontechnologies.com/download/extras)

---

<div class="post-metadata">

**Author:** ![toao](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/toao/32/20634_2.png) [@toao](https://discourse.devontechnologies.com/u/toao)\
**Post date:** [April 4, 2025, 5:03pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/6 "2025-04-04T17:03:51Z")

</div>

am running deepseek-r1 with the 70b parameter model on ollama on an MBP with M1 max with 64GB ram…not exactly fast (1-4 minutes depending on complexity of the request), but workable for most stuff I have played around with so far considering this is truly private…

---

<div class="post-metadata">

**Author:** ![AW2307](https://discourse.devontechnologies.com/letter_avatar_proxy/v4/letter/a/8c91f0/32.png) [@AW2307](https://discourse.devontechnologies.com/u/AW2307)\
**Post date:** [April 4, 2025, 5:07pm UTC](https://discourse.devontechnologies.com/t/dt4-ai-ollama-for-local-llm-recommendations/82597/7 "2025-04-04T17:07:26Z")

</div>

I would recommend Mistral Small 3 for your case with 32GB RAM. It’s a good model with good token speed compared to similarly sized models.

Also, I’ve found local models to generally perform better via LM Studio than using Ollama, so that would be worth a try. “Flash Attention” in LM Studio, if enabled, does speed things up a bit.