Here’s how I’ve been using DEVONagent recently. All of a sudden DEVONagent has become WAY more useful to me than it has in recent years:
I want to search the web for useful information, but there’s so much stuff out there! I figured I can use DEVONagent to help me out.
The way I use DEVONagent, web sites and pages are divided into three categories:
- pages that I know about and think might be useful
- sites that I know about and think might be useful
- pages and sites that I don’t yet know about that might be useful
I use three key search sets to find and use this information:
This search set doesn’t use any plugins. It contains a list of URLs in the sites tab, all set to “Crawl”. It represents the subset of known web pages that I find potentially useful.
This search set doesn’t use plugins either. Instead, it has top-level sites listed in the sites tab, all set to “Search”. This lets me search those individual sites for new pages. It represents the subset of known web sites that I find potentially useful.
This search set uses the DuckDuckGo and Google plugins, with no Sites specified. It represents the World Wide Web
Search set focus and information availability
Each search set has varying degrees of focus and what I call “information availability” – the scope of possible information that they can find.
- Pages – high focus, low information availability
- Sites – medium focus, medium information availability
- Web – low focus, high information availability
Pages can only find the pages that I’ve identified (although with follow links enabled, that’s not quite true… ). Sites can find web pages on any sites I’ve identified. Web can find pretty much any page on the internet.
How I use the search sets
Here’s the basic process I follow:
Do a Web search. Add any interesting pages to Pages search set, and sites to Sites search set. Use the “Search” setting for each site in the Sites set. There’s probably a lot of junk you’ll toss in these results, though the more specific your query the less junk you’ll have.
Do a Sites search. Add any interesting pages to Pages search set. You’re likely to find a higher percentage of useful results, because they’re limited to the domains you’ve previously identified as useful.
Do a Pages search. Work with those results. These are going to be hyper-relevant because it’s limited to web pages you’ve identified as useful.
Rinse and repeat. That’s why it’s called research
The Pages search represents a known subset of the internet that I find interesting. The Web and Sites search help me find new pages to care about.
The trick is to save any and all useful sites to your Sites search set, and pages to your Pages search set, even if they’re not related to your current topic. With your Pages search set, you can zoom in on anything you like. Because DEVONagent caches pages, subsequent searches through the Pages set will be really fast. I have a Pages search set with 4500 links in it. The first run through it takes just under 5 minutes, but subsequent runs take only 8 seconds.
A note about archived pages and new results
DEVONagent has some features that ensure you only see results one time – the “Filter archived pages” and “Only new pages” search set configurations. I admit I haven’t gotten into that yet, but I think they will be useful. I would enable those only on the Web and Sites search sets.