Hi, I think I have two ideas. They are borrowed from obsidian.
occurrence as a main search result instead of a list of documents.
rights now search retrieves only documents. after clicking a document, you see results based on that document. I think just returning occurrences first is much better. I want the option to see occurrences as a main results window, not documents.
To set up the tabs as panes as I want. I would like to pull out the tabs from the panels.
instead of left, centre and right panels, I would like to see tabs as panels:databases, occurrences, annotations.
Actually, Obsidian has a unified search panel – it lists documents and under the document is the occurrences of the search target. DEVONthink lists documents in one pane and occurrences in another. Much more efficient, in my opinion. And, since DEVONthink isn’t Obsidian (thankfully) more reliable.
While this might make sense if there’s only one document type (e.g. Markdown), things get much more complicated in case of DEVONthink as its databases can hold any document type. Depending on the document type and the search options there might be no occurrences at all or retrieving them might require a lot of time (e.g. in case of huge PDF documents).
I understand the time, multiple documents and zero results. But that premise seems like it is aimed at documenting a single retrieval, not information retrieval. You have a customer base that wants information. As a former marketing database officer, I was used to running counts and then waiting a day for the computer to process. Now I am happy with making a cup of tea on my M1. And waiting while I get a comprehensive list of relevant paragraphs. Google is a lot quicker. But it is not my database. I want a Google SERPS for my documents, similar to what I get from DT3 Occurrences now. I am also happy with zero results in an information retrieval use case. As long as I do not have to click back and forth through documents, that do not contain relevant pragraphs. That is time saved. I have solved part of the problem with the wrong documents returned with “NEAR”. But I wish I could only get occurrences as an “option” for a main result.
To me, an Occurrence main result page is closest to a Google search engine results page. This makes sense because similar to Google I want only relevant paragraphs. This kind of result makes sense with multiple documents. Because I have multiple documents, and I only want to see the best sentence, paragraph or page for the search This use-case is not document retrieval. It is about finding relevant parts, not the whole. Then deciding later to read more.
Occurrence results make sense when I use the “NEAR” to get relevant paragraphs. But I find it fatiguing to click through multiple documents, to see the occurences.
Having an option to see see occurrence first is better for a NEAR search. I will only see the relevant text. I want a paragraph, not a document.
However, I understand that showing a document first is more relevant to someone looking for a document. They Just type and get the document, they do not want a zero result. Some may even use “AND” to get a more relevant document. I also understand with occurrences as an option, there may be a performance overhead. But this clicking is time-consuming for when I use NEAR/10 and want relevant paragraphs.
I would just like to see relevant paragraphs before even considering opening a document. Just like a Google search result.
?
Google (and search engines in general) don’t just return relevant paragraphs; they also return results that must be clicked (visited). And the tiny abstract it does show is very likely not the only occurrence on the page, but perhaps not even the most relevant.
I want these changes so that I can search PDFs, web pages, and texts. I might index Obsidian with DT3. If I have the choice of deleting DT3 or Obsidian I would delete Obsidian.
For me DT3 for answering questions on data:
Reading and highlighting
Keeping files in one place, document retrieval
Quick focused research on legal matters (NEAR/10)
What I find with these apps, especially in my case is a failing in self. My poor memory, poor planning, poor English literature and language.
I am a poor planner, so no matter how many journals, organisers, or calendars I buy I cannot replace due diligence. That is sitting down every day and looking at my calendar. It does not matter what kind it is. If you never use it then you will fail.
DT3 replaces my poor memory and poor reading. I forget where I put documents and read slowly. Obsidian cannot replace that. Also, I am slowly getting back hard drive space!
Due to poor English, memory and organisation, dumping information in Obsidian leads to poor results. Overwhelm again. Dumping it into DT3 is amazing.
I wish the Chrome extension was like Zoteros. But I am going now I am moving everything from Xzotero into DT3.
DT3 enhances my thinking, by allowing me to forget. The search is amazing, so I can concentrate on writing.
I was hoping Obsidian would get around to a block-level search. It never did.
The best I have done is to use data query to create my own search.
My favourites are:
last use files
files containing law on a topic with EWCA or EWHC
But I failed to maintain it. Because each time I needed something I needed a day or two to learn how to add it. What I needed was a database. But I got a project.
Logseq and Roam Research return all occurrences by default, as they’re fully blocked-based databases (called graphs in Logseq). In my experience, this occurrence-first approach is as irrational as they could be useful. Considerable efforts are required to pick out what you want, when there are more than a few occurrences. On the other hand, when mentions are few, it doesn’t really matter how those items are presented.
The occurrences are supposed to enable efficient discovery of connections; in reality Logseq’s occurrence-first approach is not frictionlessly efficient, and my mind generally dislikes going through a high-density list of random occurrences.
I’m aware that I can potentially tweak Logseq to achieve a smoother workflow by e.g. making a number of filter presets for search. But, as the OP has said …
When you go to a designer you do not need to tell them how to build something, you just show them a sketch and let them build it. I have no idea how DT works. I know what I want, and I have built “occurrence” retrieval, on my M1 mac, (Private GPT) with Javascript and a day looking at YouTube. I am not a programmer, and I am following YouTube guides, and have LLMs answering questions on my computer and information. I do not want a project otherwise I would be with Roam and Obsidian. I want to not just search my information, I want to ask questions on my information. This is a bit different.
I am not looking for connections. I moved from ROAM to Obsidian to DT3. I am looking for a quote, a phrase I can barely remember, or something I have never read. The difference with Law quotes is that you need to know what they said. Then look up somewhere else what they meant. The creativity is in summarising the law to retain the intent to make large quotes smaller, in rule statement. But 90% of the time I need that exact quote, in a skeleton.
As explained I have a goodish (postgraduate brain) but no system and a poor memory. If I was a proper researcher without dyslexia, I would keep a reading log. And also prepare a literature review. But I cannot physically improve my reading speed. I want to only read what is necessary.
So I think the next step for Devon Think (DT) is Large Language Model integration (LLM).
I have experimented with Retrieval-Augmented Generation RAG LLM using Javascript and VS Code. In an unknown way (seriously they do not know how) a a vector database takes your normal words phrases [dog] [rotwieler] [cat] database and translates it to vectors [000][93][18][43941]. And the closer a vector is, the more it is related. For example, you can ask a database for sausages, and it will return bratwurst, Cumberland, and duck orange. It will do this even if the document has no mention of sausage. Imagine DT4 with no boolean search needed, just optional.
Imaging an intent search in DT4. You will just ask, “how much did I make last year” and it will reply with not just documents but “you made £20, from selling sausages”. Or you could ask for dumpling recipes, and in completely different languages you get Hanji, French and Creole recipes from your database.
Imagine a summary in DT4. The other day I was using perplexity to create headnotes of judgments. I am not the first to think about integrating an LLM with DT in this forum.
The future is bright and all. In reality, no LLM service offers customer support that is actually helpful, even though the exact service they provide is supposed to “revolutionize” customer support.
AIGC is useful. Unrealistic expectations are not. Several decades ago we fancied a world with ubiquitous, helpful robots. Today’s bots on the web is, while ubiquitous indeed, anything but helpful.
Okay. I cant believe you have taken the trouble to respond without using ai. But are commenting on why it does not work.
This technology is around now. But not in DT. So I think you should at least have some direct knowledge of using ai.
I actually have several ai on my computer. I created a program to chat with my documents. I am not imagining their use. I am only imagining their use by DT.
I quite offended you responded without actually any direct experience of using some ai.
Try www.perplexity.ai upload a document and ask a question.
I’m aware that sometimes existing LLMs are capable of generating above-average insights. That’s quite impressive. REALLY impressive, I should say, in video compilations which highlight the best tries and omit the worst.
I just did a test with Perplexity with a book I’m currently reading. I asked 3 questions, each of which was extracted from the beginning paragraphs of a chapter. The AI failed to give an answer to any of the questions. It spat out, in each case, very sensible information that, unfortunately, has nothing to do with what I want to know.
I have did a number of similar tests with ChatGPT, Perplexity etc. in the past year. The results have never been encouraging in any way.
I’m not going to say «I’m offended by your making baseless assumptions about my personal experiences with AIGC.» I actually feel sorry for your being a victim of YouTube propaganda – the 2020s reiteration of old-school television ads. It seems that the majority of vocal AI enthusiasts learn about the technology through videos. Although the sample size of this observation is way too small to be representative.
Q1. What is the relationship between Judaism, Jewishness, and Zionism?
Perplexity tells me the relationship between Judaism and Jewishness. It also gives me a definition of Zionism. The most important relationship is not mentioned in the response.
Q2. How to achieve a radical redefinition of secularism?
Perplexity tells me how secularism is redefined in the book. Nothing is said about the political process (i.e. the how) that would enable the change of definition.
Q3. Why is the current state of “the political” criticized? (note: “the political” is a specific phrase used in the book)
Perplexity tells me how “the political” is criticized in the book. Nothing is said about why it deserves criticism in the first place. In other words, Perplexity tells me “Tom is a bad guy” without mentioning “Tom stole a purse”. A significant part of A3 is outright hallucinated by the AI.
… is the equivalent of “Why do people say Tom is a bad guy?”
It saved me 0 seconds, because I still have to dive into the book to find what I want to know, which I would do anyway if there’s no AI to “help” me. A negative amount of time, perhaps, because I, as a curious mammal, have a tendency to cross-check what I got from the book with what the AI said. That’s some additional workload.
I think the problem is we are both different people. And I am okay with that. Lots of different people. I do not want this to turn into a survey of one. Nice chatting with you
That’s not a problem at all! I’m aware that some tools powered by artificial intelligence, such as Apple’s Accessibility suite on the iPhone, are highly respected by people with specific needs.
However, success in a niche does not nicely translate into success at everything. In my personal view, it’s not rationally feasible to believe that AI, or any other human-made technology, could become such a cosmological success as to answer any and all of your questions. Such (not fully rational) beliefs in the potential of human technology have been termed the Californian Ideology. Advocates of this ideology typically argue that technology brings progress and happiness. Real-world surveys, however, have always turned up mixed results on both concerns.
That’s why it seemed as if we were against each other. You were talking a specific ideology that I don’t accept. Ideological battles like this one are fascinating and enjoyable, if not always so.
I agree that, “success in a niche does not nicely translate into success at everything”. However, LLM’s are successful at everything that DT does.
Better at search, relationships, summarising, and findings related documents.
You can’t convince anyone if you just say what you want to say, without considering what the other party is saying or refuting their objections. What you have been doing during this entire conversation is trying to instill certain thoughts/ideology into the other party. It goes without saying that indoctrination would not work if the receiving party is well aware of the ills of a doctrine.
Coincidentally, video, the format of expression from which your knowledge about AIGC comes from, is known to excel at indoctrination but fare poorly at convincing rational viewers (when compared with text).
