Smart Rule “Chat Query” executes but %chatQueryResponse% and %chatSuggestedResponse% remain blank

Hi everyone :waving_hand:

I’m experimenting with DEVONthink 4’s AI integration (ChatGPT via GPT-5 Nano model) and trying to build an automated Smart Rule for renaming receipts. The idea is to have ChatGPT read the text of each receipt and output a 3–6-word summary phrase (e.g., “snacks and stationery”) to use in the filename.

Here’s my setup:

  • DEVONthink (Free Trial) version: 4.1.1 (latest)
  • macOS version: 15.6.1 (24G90)
  • AI model: GPT-5 (Nano) via API key
  • Smart Rule name: Dollarama Receipts – Auto-Rename & Move
  • Conditions:
    • Kind is Any Document
    • Content matches “Dollarama”
    • Content matches “Transaction Record”
  • Actions (in order):
    • Chat – Query → “With Document (Auto)”
      • Prompt:
        Read the text of this receipt and summarize what was purchased in 3–6 lowercase words.
        Do not include store name, date, or price.
        Return only the short phrase, nothing else.
    • Change Name → Short Document Date – Suggested Company – Query Response – Suggested Price
    • Move → “Dollarama”

The “Chat Query” step runs (I can see DEVONthink’s activity indicator and API usage on my account), but the Query Response field stays blank: 2025-09-23 – DOLLARAMA – – 6.61

I’ve also tried:

  • Replacing with %chatSuggestedResponse% (no effect)
  • Using Change Comment → %chatSuggestedResponse% (still blank)
  • Confirmed text can be selected/highlighted (OCR verified)
  • Tried “On Demand,” “On Startup,” and “With Document (Text)” execution modes

Is this a known limitation or am I missing a required intermediate step to save Chat Query output into metadata?

Any guidance would be hugely appreciated — this automation would be incredible if we can get the AI response stored!

Thanks in advance :folded_hands:
— Brent

A screenshot of the rule would be useful. Valid placeholders can be inserted via the contextual menu or via the popup menu on the right side of input fields.

Thanks, Christian.

I inserted the placeholders via the popup/contextual menu (see screenshots: the blue “Query Response” token in Change Name comes from Chat Suggestions ▸ Query Response; nothing was typed by hand other than Dollarama).

The rule executes (Chat – Query runs), but the filename still shows a blank where “Query Response” should be. I also tried %chatSuggestedResponse% and Change Comment → %chatSuggestedResponse%, with the same result (blank). OCR text is selectable.

Question: Is Chat Suggestions → Query Response currently supported as a valid placeholder in the Change Name field for Smart Rules in 4.1.1? Or is the Chat–Query/Suggestions output not written to a placeholder that renaming can read?

If there’s a different placeholder (or required action order) that makes this work, I’d love to try it. Thanks!

Try this document instead of this receipt in the prompt as the chat does not know what the document might contain, its contents are only sent if necessary (to improve the privacy). You could also check the output of your prompt(s) first in the Chat inspector or popover.

Modified the prompt to:

Read the text of this document and summarize what was purchased in 3-6 lowercase words.
Do not include store name, date, or price.
Return only the short phrase, nothing else.

Re-performed rule with the same result.

Ran the prompt in the Chat inspector in the sidebar and got this result:

Given…

The AI provider and model, as well as the prompt, comes into play. For example, GPT 5 Nano and Mini return a reply with a Markdown page link…

BestBuy_2021-12-02 [Page 1](x-devonthink-item://091EDD6B-B4BF-4FD8-802B-C891BF40C2BF?page=0).pdf

4.1 Nano produced…

company name: BestBuy_2021-12-02

… while 4.1 Mini returned the correct name…

BestBuy_2021-12-02

Also, Claude Haiku immediately returned the expected response.

(Note: I accidentally deleted “date” but the AI gave me the expected date.)

And adding a slight improvement to the prompt and some filing via built-in actions…

The next release will also work with these models in case of the original prompt.

I’m having similar trouble. Given the document has this at the start of page 2

and given this prompt:

Read this document, which contains a recipe, and return just the title of the recipe. The title is usually the first item on the second page. Return the title as plain text, with no Markdown formatting or links. Do NOT cite the source. Do NOT return anything that is not the text of the title of the recipe.

it still insists on returning

Poached salmon & turmeric rice [Poached salmon & turmeric rice](x-devonthink-item://B3610699-56A7-4115-8076-7C4D204C931B?page=1)

rather than just Poached salmon & turmeric rice.

This is using gpt-oss:20b on a local Ollama instance. If I go into settings and ask in chat for it to rename the document to the title using the same criteria, it does not include all the extra guff, but renames it correctly. However, I don’t want to leave the AI able to alter metadata outside of smart rules.

How can I get it to return just the name of the recipe, without all the Markdown (especially considering that I explicitly said to 1: not return any Markdown, and 2: not to cite the source)?

Update: It looks like the default role was confusing it. Adding a custom role to this rule fixed the issue:

You are an excellent AI summariser that always obeys instructions, and never includes irrelevant links or citations when they have been specifically disallowed.

Second update: nope, that only works about 75% of the time. Is there a way to get this to be consistent? Or to process the placeholder in an intermediate step before the renaming (because this is, when you get down to it, easy to do with sed — well, as easy as anything to do with sed is :slightly_smiling_face:)

Please send the zipped document to cgrunenberg - at - devon-technologies.com and we’ll check this, thanks. But most likely this issue has been already fixed in the latest internal builds.

Thanks — done!

Switching from gpt-oss:20b to gemma3:12b seems to get better results (and faster, too). A shame that model doesn’t quite seem to understand the concept of using title case :cry:

Did you send it to the above email address?

I did, but got back from lunch to discover that the server had refused it, so it would have actually left my machine shortly after you posted.

One day, everything will just work. But clearly it’s not today :grin:

Our server or your email provider? How large is the document?

Not large (660Kb) and it was my own email server. Not 100% sure what happened — and nothing useful in the logs, of course, but it was successfully sent at around 13:15, so you should have it now.

Still nothing in my inbox :frowning: You could also send me the attachment via a PM.