I’m running DEVONthink Pro 3.9.8. Wondering if the built-in AI is has the ability to search documents on broader terms than the actual search criteria. For example, if I use Chat-GPT, I can essentially have a conversation with a PDF, saying something like: “Find all the sections in this document related to aviation accidents…” and it will return results that might be worded as “plane crash” or “airline safety protocols.”
Obviously I could add those secondary terms to my documents as Aliases, but that defeats the purpose of AI, plus a search would only return the document itself, not the actually content locations.
I may be asking way more than DT is currently capable of, and if so, that’s fine. But it would be great to handle needs like this from inside DT instead of having to export documents and use another AI.
No, there is no semantic search in the internal AI. You can’t search for a document filled with the term stones by searching for the word rocks.
Obviously I could add those secondary terms to my documents as Aliases, but that defeats the purpose of AI,
There is no singular definition of AI. What you’re referring to is not the same as the foundational AI in our products.
There are things under investigation in regards to LLMs (the thing you’re actually asking about), things that could allow for making queries about a document. But realize there’s not going to be a grand indexing of all your documents and free-for-all (especially not free) searching of your entire databases.
Also, be aware any integrations for this kind of behavior would almost certainly require a commercial AI model, which means sending your documents to someone else’s servers and will incur a cost (which you may already know from ChatGPT). Local hardware and LLMs aren’t up to the task
Thanks for the helpful info! Very interesting. I’ve just recently started tinkering with AI, so I didn’t know about the details you mentioned here, but it makes sense.
Great forums… always helpful.
By the way, do you ever sleep?! You always seem to be here answering questions.
You’re welcome!
AI is certainly interesting, but I believe there’s plenty of misunderstandings about what is really feasible in regards to document and information management - not only in terms of privacy but costs incurred. Using AI as if it’s Siri or Alexa isn’t a big deal but when you want to start asking questions about documents and more documents, the meter is running.
By the way, do you ever sleep?! You always seem to be here answering questions.
A commercial LLM model like ChatGPT has billions or in some cases trillions of parameters. These are used to define a “weight” matrix.
The prompt generated by the user includes whatever context the user provides. If you ask a two word question, it’s a two-word prompt. If you ask a two word question about an article with 10,000 words, it’s a 10,002-word prompt.
The model converts the prompt to mathematical “tokens,” from which it constructs a vector. “Solving” the query involves (very roughly) multiplying the weight matrix by the prompt vector, then feeding the result to another layer that does the same thing, repeatedly until the model converges on a solution.
My “working” database comes in just under two million words.
If you think about multiplying trillion parameter matrices by million-token vectors, you start to realize why these beasts use such enormous amounts of energy, and why the kinds of queries discussed in this thread are not going to be running on the desktop any time soon.
This might also be the reason why some people consider local LLMs already to be fast but just saying “Hello!” is a completely different beast than processing RAG (e.g. a large document or search results) or reasoning or handling tool calls.
And although clients like GPT4All or LM Studio now support larger context windows (depending on the model and its training), things get really slow very quickly - I’m not patient enough to use a larger context window than 16k on an M1 Ultra so far. Hopefully an M5 Ultra will offer a lot more gear some day
Finally, a local model with 8 billion parameters or less and 3 or 4 bits quantization is actually not fast, it’s just dumbed down compared to the full blown models having about 1000 times more information.
I have about 10k notes in one database in DTP…
It works well.
I did have to set up a way to add metadata to the individual notes in order to make RAG more workable, but it’ll get you by.
that’s interesting, but how can elephas offer this for this price? do they offer LLM services on the whole uploaded documents? do they offer unlimited uploads? how do you sync data between DTP and elephas?
You need your own AI API key, or you can use LMstudio or the like.
Unlimited uploads (although it’s local service. It takes your data on your computer).
Basically, it takes files and vectorizes (RAG) it to enable you to ask questions on the data.
How it works with DT data: Integrating DEVONThink with Elephas – Elephas | Knowledge Base
For telling the truth, I do not really understand what elephas offers. So I would use my OpenAI API key and pay OpenAI for all LLM anaylses. So if I upload 100 pdfs to elephas, they are send from elephas to OpenAi and I would have to pay OpenAI for that. So what exactly does elephas in addition to OpenAI? I have not understood the elephas service - but it sounds interesting.
Elephas is on my list to try but I have not done so yet as I have several other AI tools I am in the process of exploring.
That said - Elaphas seems very interesting becauser it describes itself as “workflow automation.” It lets you apply AI prompts to documents from many different sources and lets you choose from many different LLM models from different companies.
That means it saves you a whole lot of work writing code to access the API of each of these apps and LLM models - and you can easily change LLM models as new ones come out.
That said - the Elephas integration with DT is not “official” and may use a method which puts DT data at risk. If we had a green light from Devontech that what Elephas does is “safe” then I would be a lot more inclined to jump on this
non-destructive? huh? what do you think Elephas is gonna do? null your files and eat your apple? at most you’ll have to rebuild the database scheme. You can always do an intermediary folder where you mirror DT files in a folder outside DT and use that.
You shouldn’t be so cavalier about messing about in the internals of a database. It is something we warn against. And the instructions you followed from Elephas are not sanctioned by us so you assume the risks alone.
You don’t have to rebuild them by hand. It takes a few minutes at most.
anyway… i’m using elephas, and it’s worked well for me.
You have to decide if you want to use it on your own.
I can’t tell you what to do, but this thread was about using AI with DT.
Since I guess everything now has to have a preface, I guess I have to preface my opinion with “it’s just my opinion, don’t take it as gospel” then there it is. Feel free to use a helmet while delving into the internals of your DB
