A Proposal for the Integration of DEVONthink and ChatGPT API

With the release of ChatGPT API, plenty of work can be assisted with AI, including file management in DEVONthink, of course.

I have come up with a GPT-based workflow for academic research. Currently, ChatGPT has the excellent language capabilities, including natural language understanding and writing. However, what it lacks is the ability to observe and infer. So it would be great if I, as a researcher, can feed ChatGPT with a lot of specific materials which are saved in DEVONthink database, and then ChatGPT can learn them and return summaries and retrieve specific information I asked, of course, with corresponding sources for citation.

A month ago, I thought it was impossible in the current period, but now I think this workflow could come true. I really hope that ChatGPT API can be integrated into DEVONthink :beers:

There is a similar online websites that can do this kind of work, but I hope it can be achieved locally and across multiple PDFs; i.e. my DEVONthink database. Do you guys have any ideas?

3 Likes

You don’t need to wait…. Use applescript to feed the data to chatGPT already. Devonthink is perhaps one of the most ideal macOS apps out there in how well they let you get your data and use it - including making your own integration like this at any time you want it.

2 Likes

Wow! Could you share your use-case?

I just want to chime in here after trying out the website you linked @TomBen. After uploading a couple PDFs, I find the functionality of asking questions and getting answers based on the PDF’s contents super useful.

This kind of functionality in DevonThink would add a lot of value in my view. Especially if, as I believe the OP suggests, the questions could refer not just to one PDF but to a whole collection.

3 Likes

It’s a definite No if you don’t ask, but I would say inclusion or direct integration with ChatGPT is highly unlikely at this time.

1 Like

Can I ask why? Do you really think that tools like Chatgpt is just some fancy ephemeral trend which will be forgotten quite quickly?

Well it’s not nearly as good as actual integration but this is a new app in any event.

So far it works on my M2 laptop but will not install on my Intel Mac Pro desktop. Not sure why that is.

https://anygpt.app/?ref=producthunt

Thank you for sharing with me. However, this option is reliant on the utilization of text fields, which is not preferred by me.

Currently, LlamaIndex is probably the greatest option to index my own documents by using ChatGPT API.

For PDF documents, you could try this project:

2 Likes

Hi Guys,

No need to wait for the DT3 guys to build in ChatGPT.

You can do it yourself. It knows how to write applescript and you can prompt it to develop anything DT3 related as well.

Here’s my example code for summarizing a document, but feel free to modify it to do something interesting: Inspector Search in files with long lines - #10 by jsn

Which quite often is faulty.

1 Like

I suggest you (and anyone interested in the subject) take some time to watch an actual expert on AI, e.g.,…

He’s obviously not anti-AI, but he understands the short and long-term ramifications, good and bad, of AI.

@BLUEFROG - it is an interesting video but his criticisms seem a big vague and unsupported.

David Sharpiro has quite a growing set of videos about very practical in-depth current uses of AI, his vision of AI for the future, and realistic discussions about the risks of AI. I think it is fascinating, detailed, and balanced/nuanced discussion - very much recommended to learn about both sides of the discussion.

it is an interesting video but his criticisms seem a big vague and unsupported.

There isn’t one video. This guy isn’t some nobody with an opinion (which I consider myself in this field). I suggest you explore more of his channel.

1 Like

He appears to be one guy who is a PhD student with a focused interest in “AI Safety.”

Nothing wrong with that nor anything wrong with listening to his views.

But the channel does appear to be focused on just that - “AI Safety” - rather than a balance of both “Risk and Benefits” of AI, which might be a bit more helpful.

If we focused simply on the “safety” of water, penicillin, romantic relationships, eating, artisan Mac software - really any aspect of life - we might well choose not to do it unless we considered the balance of pros/cons in more detail.

https://www.linkedin.com/in/robertskmiles/?originalSubdomain=uk

1 Like

That is one channel of his.

I don’t see any videos where he discusses benefits or practical how-to on AI. Do you?

I’m starting to find tools that integrate chatgpt into my MacOS tooling. See mailbutler.io for a good example.

I think Devonthink will miss the boat if they don’t start investing in this capability.

1 Like

Hi, I’m working with LLM since their inception in 2018. For quite some I would back your sentiment 100%. There were no value in integrating them with DT mostly because they were super hard to use (required training data). This changed with GPT-3 zero-shot ability but it was still hard to use due to need for prompt engineering. But release of chatgpt fixed all bad sides.

But I agree that the use cases adopted by Notion AI or Craft are poor as they focus on content generation. In DT we have plenty of other tasks that can be automated using ChatGPT.

Consider this simple usecase. For every document that goes to Inbox we would fire chatgpt with the following prompt:

Update this json: {"file_name": "invoice.pdf", "tags": [] , "summary":null} with better file name, tags suggestion and summary. Output only json and nothing else.
Tags I have: Private, Business, Invoice / Bill, Small, Large, Law, Subscription, On time payment, …
Based on the following content of the document:

Here is what chat gpt returned for copied content of an invoice from apple:

{"file_name": "apple_subscription_invoice.pdf",
"tags": ["Private", "Business", "Invoice / Bill", "Subscription", "On time payment", "Small"],
"summary": "This is an invoice for an annual subscription of Craft - Docs and Notes Editor purchased through Apple App Store. The invoice was issued on 31 May 2022 with a total amount of 189.99 zł inclusive of VAT at 23%."}

We would then present some interface to accept chatgpt recommendations during review process.
It would be super useful if we would allow users to change such prompts to include examples, etc.

If you are worried about privacy etc, we are going to have commercial opensource models pretty soon that can run on consumer hardware. I was able to run aplaca 7B on my mac book air with super fast performance.

Let me know what you think and if there is any chance that you would reconsider the hard “no” for this technology. I would love to talk about other usecases like search, data extraction, summarisation of long documents etc. Think that we could easily create something like phind or bing chat but that uses our databases of documents instead of the internet. (if you haven’t yet tried working with how chat gpt works for search try using phind expert mode)

Here is the chatgpt conversation:

4 Likes

Thanks for the interesting information!

There isn’t a hard “no”.
There’s just not an "Of course! That’s the greatest idea ever!!! :wink:

Thank you for the interesting use case! We’re actually aware of the possibilities & risks and trying out various options but no promises.

2 Likes