Understanding how AI API's work in DT4

I subscribed to DT4 as I was drawn to the new AI integration and I also want to keep supporting its development. I mainly just use ChatGPT in a browser and I think it would be great to be able to select a document in DT4 and have a LLM summarise it down to bullet points. However, I’m struggling to understand how API access works. I obtained an OpenAI API Key, but receive an ‘exceeded current quota’ message when I try a query (see attached), so I must need to pay for something.

Are there any AI products that provide free API calls to enable document summarisation in DT4? Or is everything pretty much pay per use? I didn’t really find the documentation at https://download.devontechnologies.com/download/devonthink/4.0beta1/DEVONthink%20Manual.pdf helpful for answering these questions, but perhaps I missed something.

Getting Started > AI Explained.

Depending on your account’s tier, the quota and tokens per minute are limited and not all models are available.

You can use LM Studio. I haven’t tried it but it’s listed there, so I assume it works. Download a small LLM and you can use it for free. Gemma, or Qwen are good at 14b

1 Like

@BLUEFROG So, having a Chat GPT Pro or Perplexity Pro account apparently isn’t enough to use the AI functions.
This is irritating. I didn’t find anythiing about that neither in your online description nor in the onboarding for DT 4.

Do you have more explanation on that?

As well as the Getting Started > AI Explained section of the help.

The entire AI industry has an entirely separate subscription/pricing structure for use of web-based chat features (such as ChatGPT or Perplexity) vs programmatic use of an API (which Devonthink does). There is no way around that.

There are some sofware vendors who get around that by charging a flat rate and hoping usage averages out among their customers; clearly that model is doomed to either fail or result in very limited AI features.

8 Likes

Yeah I felt the same way about this when I upgraded to the beta of V4 (I would have done it anyway as I use DTP so much and like it). There are some of us out there who aren’t up on API’s or that that way of using AI requires payment - now I know. But it would have been good to know that from the outset. I see the new link that Bluefrog points to does mention that.

As does the Help section I mentioned. The information is certainly available.

Ok. thankds I didn’t read the blog post.

Maybe I am just annoyed by OpenAI for making it so difficult to get an API key.
I tried to get on already a year ago, but none of my credit cards worked.

So for all users in Germany. You probably need a new credit card.

I just registered with Revolut, got a virtual credit card – and it was accepted by OpenAI.

2 Likes

my question is, why do we need to use an API key for ChatGPT? Apple AI accepts a username/password from the web version of ChatGPT and works without a hitch. Could it be Apple has a special deal with OpenAI to enable this? If not, this is what I was expecting from DT4.

Of course, they do.

And we have to work within the boundaries set by the AI provider. Third-party access for an app like ours requires using an API key

If not, this is what I was expecting from DT4.

You shouldn’t come into this with expectations. Hopes? Sure but not expectations. We have transparently discussed the integration in the Getting Started > AI Explained section in the built-in Help and manual.

It only “works without a hitch” to a point. There are usage limits beyond which your AI access is throttled or otherwise limited in some way.

If your AI usage is fairly low volume then your API bill will likely be minimal.

Conversely if your API bill is substantial then your usage would likely exceed the “free” allowance with a paid ChatGPT account.

1 Like

Thanks for the tip! it works :grinning_face: