You can define what you want in a prompt, including a role, etc
Is it fair to say that the the AI Settings are convenience features that set defaults but can be over written in the prompt?
However, you aren’t talking about the simple examples I’ve used here.
Understood but I’m trying to understand how DT settings and prompts interact. I would like to use for more complex use cases but only if DT can support it.
And as far as targeting a specific search engine,
I meant how the Source and Web & Wikipedia settings work? My understanding is that only DEVONthink, Perplexity, and Exa.ai are the ‘search engines’ available. I’m not interested in other providers like Google, just what is the expected result.
NOTE: I’m unfamiliar with DT4 native search engine for Web and Wikipedia – I am user of DA but I didn’t know that DT4 had its own index. I assume this is on the DT cloud services somewhere, right?
@BLUEFROG It would be great if you can get an answer from development confirming the flow of data (prompts and responses between the Search Scope, Web & Wikipedia, and the selected model in the chat.
This seems like a useful feature but unclear how it works.
Also if there is no DEVONthink Search index how does it return results – I wouldn’t imagine it’s searching the entire internet from my device.
We have no “cloud services”. Never have.
I wouldn’t imagine it’s searching the entire internet from my device.
DEVONthink is searching as you would, assessing results, tossing irrelevant pages, searching more as needed. You can clearly see some of this behavior in the Chat assistant.
Where did you see the URLs that Exa returned?
Live hyperlinks in the output.
The Claude and ChatGTP models did not return any Exa.ai URLs for me to review. (posted earlier)This is using your example prompt “Tell me something about pickles”
On a simple prompt of Give me information and a life timeline about the popular painter, Bob Ross. with Exa.ai doing the web search.
Here is the same search using each option for Web searching:
web-search.zip (14.1 KB)
The flow is quite simple. After entering a prompt, the AI model might trigger a search or multiple ones to collect e.g. additional or more recent information. Afterwards the model analyzes & processes the results on its own to provide you the desired response, e.g. a summary of recent news. All of this happens internally, you just get the response which might include reference links, not the raw search results.
But you control which search options are possible (database, web, Wikipedia, PubMed, arXiv) and how to perform Web/Wikipedia searches (DEVONthink, Exa.ai or Perplexity) via the settings. In addition, the number of results is quite limited and depends on both the context window of the model and Usage mode.
Finally, a precise prompt like Search the web for... is the best way to ensure that the desired search and only this one is performed.
I recreated your Search settings and prompts. Source URLs were included with DT4 and Exa.ai but not Perplexity. I tried again with Perplexity this time with a prompt to include sources. See the response – it is still using Perplexity’s training data, not web search.
Am I doing anything different from your set-up?
Give me information and a life timeline about the popular painter, Bob Ross. with Perplexity doing the web search. Include your sources in the response.
I apologize-the web search function is currently unavailable. However, I have reliable information about Bob Ross from my training data. Here’s a comprehensive overview:
In addition, the number of results is quite limited and depends on both the context window of the model and Usage mode.
Do you where in there process context window and usage impact the results? Pre-Web search or Post-Web search?
Examples to illustrate where there small context window and lower quality usage mode:
- Pre-Web Search: LLM instructs web search to return top 5 urls.
- Post- Web Search: Web search returns 20 URLs but LLM only processes top 5
Since DEVONthink doesn’t have it’s own search index service how does it perform a web search?
I use DevonAgent Pro and there is a lot of configuration options re: what search engines to use. And with individual sites whether to crawl the site or return results from a Search Engines index of the site.
I’m trying to understand which search service(s) DEVONthink Web Search use to return results to the LLM
Quite similar to DEVONagent by querying multiple search engines and collecting and filtering the results on its own.
The Usage mode and context window affect both the number of requested and retrieved results.
