I subscribe to ChatGPT’s Pro plan and have for a couple of years. Over that time, ChatGPT has built up a considerable store of knowledge about me and the output I need for my work, making ChatGPT more and more useful to me. It is, as folks in the software industry say, a flywheel.
When I try the same queries in DEVONthink’s built-in AI, the results are nowhere near as good, and I think it’s because API access to ChatGPT doesn’t have that built-in store of knowledge about me and my work.
I wonder if other DEVONthink users have found solutions to these problems, and if so how?
I am not a software developer or programmer. To describe the extent of my ignorance: I can’t even create or modify a simple AppleScript, and I’ve never been able to figure out Shortcuts.
I would say people should think about how much they trust their AI provider’s claims about privacy, data usage, etc. OpenAI and Google clearly have been stealing information for years, then try justifying it by saying “But AI wouldn’t be as good as it is if we had to ask for permission!” People have given far more of themselves to Google, Meta, Apple, and now OpenAI, etc. all for “results”. Whether that’s a tradeoff someone is comfortable with or not is a personal decision.
DEVONthink is doing what it can to keep your information private when dealing with AI. There is a lot more at stake for many of our customers, with masses of personal and professional data stored in their databases. Perhaps that leads to “inferior, but more secure” results. On the other hand, we have firsthand experiences of DEVONthink excelling beyond what some other AI applications deliver.
Due to your experiences your prompts are most likely fine-tuned for ChatGPT and do not necessarily work as well in other apps like Claude or DEVONthink. Even switching models in the same app can have a major impact.
This is very likely because ChatGPT includes “memory” now (it’s an option you can disable), which helps it build up a profile of you and who you are. The APIs are stateless, so they don’t remember anything between invocations.
The way this works is that ChatGPT remembers things about you as you use it, and then injects that context into prompts before passing them on to the stateless OpenAI APIs.
You could probably get ChatGPT to generate a good prompt based on its knowledge of you, then try it in DEVONthink to see how you like the results. Right now, I think you can’t tweak DEVONthink’s chat prompt, just its summarization prompt, but hoping that’ll be added.
I have tried to suppress ChatGPT’s personality. Here’s what I have in “Customize ChatGPT” > “What traits should ChatGPT have?”
Keep responses short. ChatGPT can have opinions. When providing answers to prompts, maintain a neutral and straightforward tone, ensuring the responses are professional, clear, and concise. Offer straightforward content summaries, and provide clear, concise feedback on writing tasks. Don’t use lists in responses unless specifically requested by the user, ensuring communication is direct and to the point. When responding to queries, include links.
I don’t want ChatGPT to tell me, “Great question” or any of that encouraging horsehockey. I don’t need approval from a robot.
