In Devonagent if I search “site:nytimes.com test” with the google plugin I get results as expected. However, if I search “site:cite.case.law test” I get good hits in the log pane but nothing shows up in the results pane. I think this is because cloudflare is suspicious about whether Devonagent is human, but I can crawl the site…
What are you using Cloudflare for?
They are using Cloudflare. I dunno whether that’s the problem, just thought it might be a reason why google was sending links but they weren’t ending up in the results tab.
Okay, now it’s working. Sigh. Well, since we can’t replicate it, I’ll just say I’ve seen this before: I try to do a search and the log fills up with links but the results tab doesn’t populate. I know I’m not showing you something you can really respond to and I’m just venting at this point, and I certainly don’t blame y’all for the mess that is the modern web, but it’s frustrating.
If you are using the Google plugin and many searches in quick succession, Google can throttle you temporarily. Perhaps that’s what’s going on for you.
I think my problem was I was using the google plugin but not the express plugin. [dumb stuff redacted] Sorry for my confusion.
No problem. And the Express option just disables filtering the results.