More Thoughts on Potential ChatGPT Integration with DT3: Plugins and DT3 API

As I have pondered roles for AI (including ChatGPT) with DT3, I believe there is soon-to-be potential for a much bigger role beyond summarizing documents.

ChatGPT Plugins are in limited Alpha now though hopefully will soon be advanced to allow more widespread adoption. But the documentation is available to all:

Interestingly - creating a ChatGPT plugin primarily involves creating an API to perform whatever function you wish. The additional manifest and other specifications are fairly simple. So that means if you create or already have an API for your planned integration then you are already a good way towards creating the Plug-in when you get access.

Well it turns out I posted code for an unofficial DT3 API previously:

I believe this API is likely to be helpful as part of a ChatGPT plugin. But with more API endpoints it could of course do much more.

I am thinking about potential natural language “scripting” or integration of DT3 with prompts similar to these:

  • “Find all my Devonthink posts tagged XYZ, use Zapier to create a PDF using Template 1, and save those to Dropbox”

  • “Query Google Scholar daily on subject XYZ and post a summary to my Devonthink group ABC”

  • “Find all my Devonthink posts from today tagged ABC, translate them to Spanish and Dutch, and add them to my Asana Active Project Group.”

  • " Check RSS feed XXX, find all articles containing the word “ABC,” add a corresponding tag, and post them to Devonthink Database AAA Group BBB "

Those are just a few for starters - any additional ideas?

I am hoping Devontech sees the potential and implements this on their own with our input.

If not, then we are fortunate that DT3 is immensely scriptable. I am open to suggestions, advice, Pull requests on my API code, and any other help to make this happen.

2 Likes

I am agnostic and not enthusiastic about the future of AI as we currently understand it. So much will change. Lots of reasons explains my thinking based on experience which I will not go into here.

But consider the business risk to DEVONthink if, say, on entered the commands you suggest above and it does not work correctly or not at all? An indeterminate result from software is probably impossible to validate. User questions and complaints would probably cause Support costs to go through the roof if DEVONthink continues support as they so demonstrate now which is very attentive and comprehensive. If they reduce that support what would be the impact to their reputation? And would future sales be hurt if AI did not work as expected or word got around that AI does not work whether true or not? Even now I read on other forums reports from disenfranchised people about erroneous problems in years past which cause them to continue giving negative myths about the product.

Even if above points turn out to be unfounded does anyone know enough now to commit €$ now to develop something? Surely not a trivial amount, much in licensing fees perhaps which of course affects costs to customers.

To ponder. Perhaps an AI engine could write a business plan and risk assessment for this now for DEVONthink?

Or is it too “wicked” of a problem for even AI to consider?

Maybe and probably someday. But not soon, IMHO.

3 Likes

Those are good points for discussion but I think there are two really important things to consider:

  • The usage I am proposing for DT3 here is simply the convenience of a natural language interface. With specific prompts the risk is minimal.

    Using the example of asking ChatGPT to do a Google Scholar search and upload the results to a specific Group in DT3, the only risk is that the wrong stuff gets uploaded or incomplete data gets uploaded or maybe that it uploads too much stuff. Hard to imagine anything tragic.

  • All the arguments you raise about using AI would apply equally to Scripting and Smart Rules. Devontech surely has not been overcautious there! Fortunately for many of us, they appear to have taken the approach that users writing a script or downloading a script or creating a Smart Rule do so at their own risk and should take appropriate precautions to safeguard their data in doing so.

I am thrilled that Devontech allows me the ability to judge risk vs reward with scripting and Smart rules; even though those could destroy databases, I don’t recall any situation where a user has blamed errors on Devontech.

I see AI integration as a natural extension of this scripting philosophy. There are known methods to keep your data safe with AI just as with scripting.

If nothing else why not set up a separate database for AI experiments and learn from that? Set up a new User Account on your Mac just for that purpose - just like you might do if you were a beta tester for DT3. Why is the risk of AI any different than running DT3 beta versions?

1 Like

Well, I’ve had severe “fights” with my friends related to IA. I’m at @rmschne side, and even colder.

Yes, it has a potential incredible future, but before that they must resolve three things:

  • Invented answers. When the conversational IA does not know something, it invents it, making any answer completely invalid because if you know the answer, you don’t need the IA, and if you don’t know it, you can never be sure the answer is right.
  • Really curated answers. For example, I’m the owner (with the same friends I fight with) of a Podcasts Network (“Sospechosos habituales”, habitual suspects), and I asked Chat GPT for the list of our podcasts. Well, it answered completely wrong, and listed only 3 podcasts (out of 21), and 2 of them were wrong. “La Órbita de Endor” is a famous podcast that is not in our network, but they should say “and the list of habitual suspects of today is…”. The second one is a more than 5 years dead podcast (La Puerta de Trantor, Trantor’s Door) that has nothing to do with us. The only it listed right was in part of the question, “WinTablet”.
  • Specifically, not blocking/restarting when I ask something more complex than stupid questions. :sweat_smile: :rofl:
4 Likes

Agreed - 100%

Do you understand the difference between ChatGPT vs. the OpenAI API and Playground? Do you understand how the Plugins work?

Have you tried the API or Playground with a Temperature of 0 and specific prompting instructions to give bounds on its sources? I have never seen hallucination in that situation - have you?

Do you understand that a ChatGPT plugin will let you curate its sources to an even greater extent than prompting?

Do you ever use Google Search? How do you deal with its inaccuracies?

The OpenAI API with a prompt restricting its sources to an open-source plugin you can inspect should be more accurate than Google Search. Obviously you need to verify all information - but dismissing AI of this nature seems to me like throwing out the baby with the bathwater.

Maybe the better analogy is DevonAgent - advanced software with custom plugins that can automatically place search results into DevonThink. It’s amazing software too, albeit maybe now superceded to some extent by AI. Why is DevonAgent a viable and stable product but OpenAI with a custom plugin is not?

1 Like

I am really fascinated by the suggestion of @rkaplan - and would be in favor of such an API that allows to perform prompts on DTP data. As with applescripts and smartrules, every DTP user would have the freedom to use or not to use it.
I would not use it with privacy-sensitive information at the moment. But I would use it with less critical information. If OpenAI is used with a temperature of 0 or at least <30 the risk of hallucinations is very very low.

Using ChatGPT is not the same as applescripts and smart rules.

From a technical and business perspective the analogy seems clear.

Both are advanced technologies where the upside is infinite customization and increase in capability but the downside is the potential to lose or corrupt your entire database if you do not clearly understand the underlying platform.

Given that DT3 supports scripting and smart rules more enthustiastically than any other software I know, I am really surprised that you are not equally supportive of AI. Puzzling. It’s such a natural extension of your company’s whole philosophy.

AppleScripts and smart rules are performing procedural tasks and I would never say DEVONthink is so smart because it renamed some files and put them in a group I told it to. :wink:

1 Like

Or they can bring in data from external sources

Or they can call an API to manipulate existing data

They can do whatever you invent for them to do

You’re still talking about procedural tasks and they’re still not smart.

On a personal POV philosophical note: If you read the Cliffs Notes or “watched the movie”, you didn’t read the book and you can’t logically and truly claim to know and understand the material. I see ChatGPT as the analogue here. And sadly, people will take ChatGPT results as if it actually read the book and understands the material when it does not. It understands nothing. It doesn’t even know what nothing is. It only knows it is a series of characters that occurs with other certain characters in instances with frequency n versus frequency m, etc. Actually, if it was simply being used in a fashion to report the number of instances of words in a document, it would trouble me far less.

The anthropomophism or adjectives referring to it as if it’s sentient – regardless if people deny they make such claims – is troubling. Again, it’s being touted as something it’s not. And people will treat it as if it’s something it’s not. It technology that’s far more ripe for abuse and misunderstanding than you’re admitting to IMHO, even if your intentions and uses are benign (as I’m sure they are).

And again, these are my personal views. Being a person, I am entitled to them. This has no bearing on what DEVONtech does as a company. I don’t drive our technology, though we of course internally discuss many things. However, knowing @eboehnisch and @cgrunenberg for over 11 years now, I trust their decisions are well-rationed and not merely jumping on a bandwagon because “everyone’s doing it”. They do what’s best for the company and also what’s undergirds the philosophy and values of the company as a whole. :slight_smile: How that all washes out in terms of any external AI integration remains to be seen.

But as the culture of our forums dictates, you (and others) are free to dream and postulate and make suggestions as you feel lead. No one will stop you. We may just not jump in along side you :wink:

6 Likes

Greetings @BLUEFROG

Might you be confusing ChatGPT with the OpenAI API? Might you not be distinguishing the API’s chat mode from a plugin?

I am not suggesting in any way that ChatGPT can or should be a source of information for my DT3 databases.

I am suggesting instead that it can be used to retrieve information from trusted sources I designate, via a process and rules that I designate.

Basically what I am suggesting is that the OpenAI API can be used to allow natural language as an alternative to Javascript or Applescript to specify what external APIs to access and how to do that.

It’s just a shift in language for accessing APIs. Natural language instead of a traditional programming language.

This would NOT utilize ChatGPT’s language model for creating any information. The source of any information received would be no less accurate than with any other script.

You are usually quite intellectually curious about new things. In this case however might you be lumping all “AI” together? Perhaps you have not yet explored the distinction between a plugin to the OpenAI API vs. mass consumer oriented ChatGPT?

1 Like

The GPT-like models will have always problems with hallucinating, the newer models might limit it extent, but they still might be useful in summarisation/exploring a data of user,
we should be cautious not to be overconfident with them and to believe everything what hype is saying

I would suggest to make the solution as flexible as possible to use other APIs as well, locking everything with OpenAI is bad idea in long term, there is more inititiatives like the open source Hugging chat

For people who want use ChatGPT now, the scripts using Python integration posted in this repo could be useful, I didn’t try them

EDIT: there is a separate thread about this Python integration

2 Likes

As a legal professional with no prior experience in regex or scripting, I’ve found GPT-4 to be an invaluable tool, enabling me to achieve what I consider to be remarkable results.

Integrating GPT-4 with DEVONthink 3 could prove particularly useful for beginners like myself who lack knowledge in coding, regex, or scripting. Despite my inexperience, I’ve managed to develop fully functional scripts simply by prompting GPT-4 with “Create a smart rule in DEVONthink 3 that executes an AppleScript that does…”.

While my achievements may not be extraordinary to some, GPT-4 has helped me create a script that scans the first and last pages of a document to extract bates labels (e.g., JOHNSON 000123; JOHNSON 000199). The script then adds only the numerical values to a custom metadata field I created, named ‘Bates’, while retaining just the two sets of numbers (e.g., “000123-000199”). All I need to do is drag the file onto the smart rule.

This software has empowered me, someone who would have never dared to code, to create scripts that greatly simplify my daily tasks.

In my opinion, every attorney should be utilizing DEVONthink, with the assistance of GPT-4.

4 Likes

It can rise to various levels depending on the user’s needs.

I worked as a programmer eons ago in high school through med school but since then have not had the time to keep up with all the changes in languages and frameworks even though fundamental principles of coding have not really changed.

ChatGPT has been a really useful electronic textbook of coding for me. If it can give me an example of what programming class/method/technique I need to use with sample syntax, that can save me lots of time finding the same answer in standard books and websites. dddd

It is also very good at answering questions when I get stumped - like “What is wrong with this syntax?” or “Why is this not valid JSON?”

1 Like

Just to put my 2 cents in (4.5 with inflation), IF there is a connection to an AI within DTP - ANY AI - I’d like the option to never have the plugin or code installed. Not just sitting unused, not an unselected option, but to not have the code in my app or on my system, in any way.

AI has a lot of potential, but we’re not there yet and we’re no where near close enough to having established safeguards. I don’t want in my apps or on my system.

4 Likes

Well, there are enough examples floating around of this program shelling out wrong code. And hallucinating facts, as well as inventing sources that „prove“ these hallucinations.
Which is not to say that you might not occasionally get lucky with code generated by it.

1 Like

Outright rejection of OpenAI (or other leading AI algorithms) as inherently bad and unworthy of use as tools would be shortsighted for DEVONtech as a company. DEVNONtech itself touts “Our commercial applications are based on our unique proprietary artificial intelligence (AI) technology.” As recently as December 2021 you approved of the following description of DEVONthink’s version of AI:

AI won’t make filing recommendations if it doesn’t “see” at least one group that holds other documents that it considers contextually related to the document you are viewing. For each group that holds documents, Classify tries to identify a pattern of contextual relationships in the text of its documents, that differentiates the group from other groups. AI Optimization - #3 by BLUEFROG

DT Pro often gets it “wrong” when making a Classification suggestion. That didn’t stop DEVONtech from incorporating it and doesn’t stop people from using the tool. Having used OpenAI plugins inside Obsidian, it is nothing short of amazing at returning information from notes, drafting, and providing suggestions. If DEVONtech chooses to refuse the benefits of these developing LLMs, they may find themselves losing out to other developers that embrace them.

3 Likes

Box is not a newcomer to the industry.

They are about to introduce Enterprise-Grade, HIPAA-compliant AI integration.

They get that there are ways to see through the media paranoia and do AI right - with existing technology and proper safeguards.

Fortunately since they also support APIs and other integrations, there will likely be ways to integrate this with DT3 - by bookmarks, scripting, API calls, and/or indexing.

Wow. Things are happening incredibly fast. If Box is doing it, Dropbox won’t be far behind. I’m 25 years and going as a legal professional here and used DTPro for a loooong time. Always referred to DTPro as my secret weapon due to its ability to surface related content/information (using its own version of AI algorithms). Organization was a distant second (tags and folders are baked in on Mac). If something else comes along that does it better, faster, more intuitively, this might not end well for DEVONthink.