Clarifying ai access to local files in DT

DEVONthink Artificial Intelligence
1

Context: I think I’m in that early phase of not knowing what I need to know and so I’m posting asking for some guidance.

Problem: At the end of a long chat thread I appear to be getting contradictory answers with regard to the capabilities of, in this case ChatGTP , working with D T.

Original Goal: Review a set of documents, markdown, in D T , analyze the content, synthesize the results into a description. To be more specific look at documents describing a business and create a summary that describes the business, revenue, profits, products, staffing, treatment of workers. Many of the answers are numeric but some require qualitative description.

I was having difficulty getting clear numeric answers and asked in the chat about what capabilities were available. The response indicated that for the chat to analyze the documents I had to paste them back into the chat or provide some other action to give access. The response is quoted below.

This seems to be at odds with my understanding of the interaction of the chat and D T, where I assume you can either point at a group or file to have the chat interact with it. Or, specify in the chat a local URL, for instance local x-DEVONthink-item://C2A26307-5C8C-4DD8-B2EF-…… as part of the prompt.

The response from the chat on accessing local files is

What I can access in this chat context

  • Local content you explicitly share here (paste excerpts, numbers, notes).

  • The DEVONthink item URL you provide as a reference, which I can use to label prompts and saving targets, but I do not automatically read or download every document inside that item unless you paste or authorize it.

  • Web results only if you or I trigger a web search tool in this session and you share or authorize the results.

What I cannot do automatically

  • I cannot silently load and read the entire contents of your DEVONthink folder or any local database without you performing an action (exporting content, pasting excerpts, or giving me explicit permission to process a batch).

  • I cannot autonomously fetch live web pages from your machine or automatically compile data without you initiating the web search or providing the results.

Setup Details
Using ChatGTP with API Key
GTP 5 Nano
Database Search is checked
All assistants checked
All searched checked

I looked through the help section about understanding ai, but did not find, or perhaps missed, an explanation of how this works.

From my naive point of view the ai seems to be saying that it can’t read my Devon files, it can only read what I paste back into the chat.

I’d appreciate any help that can help remove the mist and help me see clearly.

Thanks

2

Just another hallucinatory response from a cheap and mediocre model. I would suggest to try a more powerful model to get a (hopefully) more accurate reply.

1 Like
3

Getting Started > AI in Practice > Controlling AI’s Access discusses this topic. And though I too often opt for cheaper models, the mid-to-high tier models are a bit slower but more accurate.

4

Thanks

5

DT’s capabilities aside, LLMs generally are not good at numerical answers. They are text-based tools, and do not replace conventional mathematical models.

6

That used to be the case.

Current models which are good at reasoning in general tend to be pretty good at math.

Some models such as Claude can also write python or javascript code as an artifact to solve more complex math problems

7

That’s exactly the reason why DEVONthink 4 provides a calculator tool. I just checked this morning that it was used by Claude Opus 4.1 when doing some price calculations in a Markdown document for me - otherwise I wouldn’t trust the result :grin:

Artifacts are a feature of the Claude.app.

8

Christian, I kept meaning to praise the tools in DT4 chat. I noticed them a while ago and checking transcripts on my LM Studio was showing them get used, and it’s good stuff!

I wish, and I am probably not seeing the obvious, there was a way to turn on/off different tools at the chat window for different prompts — say, one moment I want web search on, but later I don’t. Same with models — maybe one is a qwen3-203B-a22b-2507 kind of task, but the next one (tagging some docs) is better with a little model, and later a script wants to have Kimi-k2 synthesizing a search result set in a database.

I know I can go to preferences and toggle, but I was wondering even for scripting if there is a way to specify. What I keep getting afraid of is allowing all of it in preferences, and having a script or autonomous trigger and next thing I know I’ve got runaway scripts searching the web! LOL. Worse, with my luck, it would be Kimi-k2 or deepseek caught in some thinking loop and randomly searching web results! Hahahah

EDIT: maybe a longer term thing, and likely you’re already thinking of things like this, but I would like to see a way to specify a few “model choices” (maybe like how the sets work in devonagent) — for instance, I probably don’t want to run a simple task on high cost APIs but on friendly local models, but other tasks are worth sending off to the heavy compute APIs($$$) or to a heavy local model for intermediates but high context (I do have some Rope using local models that can go way over 256k token context windows and they work well), but neither the context nor the parameter count would be needed for small tasks or similar. Plus, I’m starting to send different kinds of tasks to different models just because each model seems to have strengths fitting the task. While I can get away with it when just me using the chat window, I am seeing the possibility to automate/script more, and I’m wanting to stretch DT chat into more orchestration roles! LOL

9

You can already define the engine and model when scripting, e.g.

tell application id "DNtp"
	set rply to get chat response for message "Explain the mathematical principles in this document." role "You're a high school teacher." record (selected record 1) engine Perplexity model "sonar-pro"
end tell
1 Like
10

Nice! (My dictionary diving in script editor is obviously weak!) Thanks, Jim!
Are there switches or parameters for the web search or other options also? I’m thinking of using a script to orchestrate a cascade of actions.. the web search is too handy in DT, (beats fighting with bright data or searxng!)

11

These are the available parameters…

image
image1920×433 131 KB

1 Like
12

Thank you! I’m particularly embarrassed that I wasn’t finding that in the script dictionary!

1 Like
13

The model can be changed in the Chat assistant on demand, see pop-up menu in the lower left corner. In addition, the possibility to change e.g. search options likewise is already planned.

Usually I would recommend to choose a default model in the settings that is both fast and inexpensive and therefore suitable for all kinds of automatic and batch processing and to switch to better or more expensive models in the assistant only whenever necessary.

1 Like
14

I see it now! thank you! I never clicked that icon before – in the chat inspector, I just thought it was telling me that I had LM Studio (the icon I’m seeing) selected! hah!

Thanks for the insights also - as I mentioned, I’m coming to enjoy the DT chat interface even to the other options I have, especially when working with documents and random context!

1 Like
15

Christian, one more thing – with Tahoe, Apple opened up the hooks to access the Apple Private Cloud model (there is actually a decent, albeit no personality, LLM there). I have tested some questions and analysis to it using shortcuts. It’s not a spectacular model, but it really is better than many small and medium local models. Any chance that it can made an option in the AI settings?

1 Like
16

It’s on our to-do list but due to its severe limitations (4k context window, only 3b parameters, 2 bit quantization, no vision or multimodal support, no reasoning, limited support for international languages, knowledge cutoff 2023, extremely guardrailed) it’s not a top priority so far. Theoretically tool calls are supported but the context window is so tiny that it’s barely usable, e.g. the chat assistant’s complete tool call definitions would require more than 50% of it.

The only advantages for users are ease of use (no installation), no costs and maybe a little bit faster than comparable models (similar size, quantization and other limitations) but using MLX models in LM Studio is a much more flexible and powerful option, IMHO.

3 Likes
17

Thanks. I’m hoping they are going to slide that multimodal ~50B model they released as a research model might become another choice in private cloud, and they are allegedly in talks for gemini, so this might be a thing soon also – how cool would that be!

18

Most likely the Gemini integration would be reserved for Apple but not be available to third-party developers, especially due to the insane costs that cloud AI usage can cause.

1 Like