How to Create AI Agents with DT (plus feature request)

[rkaplan]

Creating “AI agents” is an area of enormous discussion lately - albeit the definition of an AI agent varies quite a bit.

Notion 3 AI agents got my attention recently among other reasons because I view Notion as a web version of Devonthink. I suspect if Devonthink had a web-centric philosophy it would have been Notion. And if Notion had been created only for the desktojp it would have been Devonthink. They both are enormously flexible systems to manage documents and personal information. Notion excels in being able to publish/share this information since it is web-based; Devonthink excels in privacy, data security, and management of enormous local data stores since it is so strongly desktop-based. Thus they complement each other; DT4 is the unquestionable center of computing for me, but I do use Notion at times when I need the web-centric features that it offers.

Anyway - Notion 3 recently was released with AI agents which automate almost anything Notion can do, from researching the web to reorganizing a Notion database. Its agent implementation is particularly interesting because it requires no coding and does not even require learning a GUI front-end; its agent implementation is entirely done via prompting. It even allows creation of complex repetitive multi-agent scenarios by simply typing the @ symbol, which brings up a selection of existing documents similar to the [[ feature in the Devonthink editor. That makes it easy to choose among a custom library of prompts which you can string together into an agent request.

So a Notion multi-agent scenario can be enacted with a prompt such as “Use @PubMed-Search to search for causes of low back pain. Then use @Spansh-Translator to create a second copy.”

I thought this would be useful in DT4 - and then I realized the capability exists now.

I created a markdown file with a very detailed prompt for how to search PubMed and the format to use for the output - this is a snippet of that file:

Then I created a second much shorter prompt to translate to Spanish

Finally I executed the “multi agent workflow” via DT4 Chat:

As requested, the agent workflow created two nicely formatted reports per my specifications, one in English and one in Spanish:

Next Steps

So a multi-agent workflow based upon prompting can indeed be created in DT4. Arguably this is similar to what can be done with scripting but it is a lot easier since it only uses natural language prompts.

But that said there are two DT4 features which I suggest would make such workflows considerably more practical -

1 - For this test, in the Chat I referenced the Prompt 1 and Prompt 2 markdown files via x-DEVONthink Item Links. It would be a lot easier if the chat window could accept [[ syntax for selecting an existing file similar to the Wikilink feature of DT4 in markdown files.

2 - I suspect complex agent workflows in DT4 would fail because of the existing timeout feature in DT4, which I understand to be 10 minutes for reasoning models or 5 minutes otherwise. One of the major features other agent software promotes is that an agent workflow can run autonomously for quite some time – 20 minutes for Notion, unlimited for some workflow software such as n8n. If there were a way to modify the timeout limit to a user-chosen duration that would make DT4 notably more useful for this purpose.

[cgrunenberg]

1. A similar feature is already planned for upcoming releases
2. The timeout shouldn‘t matter as it‘s the timeout for each request, not for the complete workflow.

[rkaplan]

Excellent news - thank you

[rkaplan]

Have you considered allowing the ability to call a Script from a Chat session, ideally including passing and receiving a parameter(s).

This would allow accessing an external API as part of a chat session. That would allow very easy creation of scenarios that would otherwise require a much more complex script. One script defining the API access could then be used in many differente Chat scenarios without the need to create a new script each time.

(Notion does this as a “custom connector” - but in DT I think simply being able to call a script with input/output parameters would be equivalent.)

[cgrunenberg]

@rkaplan

Have you considered allowing the ability to call a Script from a Chat session, ideally including passing and receiving a parameter(s).

Considered but discarded. The reliability would be probably horrible (most AI generated scripts do not compile or do not work without minor fixes) and it would be a major security issue.

[rkaplan]

@cgrunenberg

Considered but discarded. The reliability would be probably horrible (most AI generated scripts do not compile or do not work without minor fixes) and it would be a major security issue.

I am not suggesting an AI generated script - I am suggesting a chat could call a script I write myself and explicitly call by request in the chat.

Why would that be any more of a security risk than calling a script normally?

[CAE]

Cool. When I get time I’ll have to try something like this based on my own workflow.

[cgrunenberg]

@rkaplan

Why would that be any more of a security risk than calling a script normally?

In that case it’s indeed no security issue but maybe that would be more a job for a script handling all of this (chat querying, response handling etc.)

[rkaplan]

@cgrunenberg

In that case it’s indeed no security issue but maybe that would be more a job for a script handling all of this (chat querying, response handling etc.)

I think it would be infinitely more useful to leave the scripting to the hard stuff (like writing an API interface) and be able to simply use natural language in a chat to then apply that API call in various ways.

Interestingly Anthropic just made something very similar available:

Claude Skills: Customize AI for your workflows

Claude skills can include external API access among other things. I see it is accessible when I use Claude.ai

Is there any way for that to carry over to Chat in DT4?

[Farath]

I also find being able to addressing different agents from DEVONthink 4 quite interesting. Have tried with some AI services but had no luck so far.

[scaba]

@rkaplan

Is there any way for that to carry over to Chat in DT4?

I think the DT devs would have to enable it, but it seems simple enough of an API surface:

Claude Developer Platform (API)

Agent Skills, which we often refer to simply as Skills, can now be added to Messages API requests and the new /v1/skills endpoint gives developers programmatic control over custom skill versioning and management. Skills require the Code Execution Tool beta, which provides the secure environment they need to run.

[rkaplan]

@scaba

I think the DT devs would have to enable it, but it seems simple enough of an API surface

Yes that was my question

I have some skills enabled already via the Anthropic API settings page, but they are not recognized at present by DT4.

[cgrunenberg]

@rkaplan

Yes that was my question

There are currently no plans to invest too much time in highly provider specific features as maintaining and testing all the providers & models supported by DEVONthink 4 already requires a lot of time and we prefer to focus on generic improvements so all providers and most models benefit. Anyway, thanks for the suggestion.