HELP, Google plugin not working at all

After 20 years, the DA Google plugin returns no results for any search. For example, log shows:

https://www.google.com/search?as_q=Test&num=100&ie=UTF-8&start=0&tbs=li:1

and when I click on it it does a google search with many results but the DA search returns no results. I have reinstalled twice and same result. No idea what to do. I have no archived pages and unchecking this and Similar Pages doesn’t help. A work around is to use the google search from the link to create a set and then search with that set using DA but that is a real pain.

Actually several things seems to be going wrong at the same time including plugins not returning sufficient numbers of results. I would like to do a complete erase and reinstall but not sure how to delete all traces of the program.

I have reinstalled twice and same result.

I would like to do a complete erase and reinstall but not sure how to delete all traces of the program.

Why? Reinstallation is not something that should be used as an early troubleshooting step. See…


It’s not a DEVONagent issue. It appears Google has decided to make fundamental under-the-hood changes, effectively breaking the search plugin. We are investigating what, if anything, can be done on our side but that’s unclear at this time.

2 Likes

" Reinstallation is not something that should be used as an early troubleshooting step."

It is when you pull out the rest of your hair trying everything else."

“effectively breaking the search plugin”

Thats really bad obviously for the future of DA

Well, it’s not an ideal situation but DEVONagent isn’t a front-end to Google. There are far, far more options in the application. And as mentioned, we are looking into the situation.

2 Likes

"DEVONagent isn’t a front-end to Google. "

Maybe, but for me my primary use case is getting very large numbers of Google search results into DT as well as a way to track the searches to download only new results. If there are other options to do this fully, I would love to know what they are because I am pretty handicapped at this point.

BTW I am having similar issues with the Bing plugin. It will do the search but returns a minimal number of results. Searches inside quotation marks return no results.

There are many other plugins and scanners available, including for other search engines.
The Web plugins are still partially functional, but obviously are affected by Google’s changes/

Regarding Bing, some people reported misbehavior but then said it was fine.
I just did this search…

Remember we don’t control these services nor the networks between them and any device.

I don’t know what plugin that is but when I did the same search with the Bing plugin I get six results, smae with Duck Duck Go. When I do the same search using the DA generated search string:

https://www.google.com/search?as_q=wireguard&num=100&ie=UTF-8&start=0&tbs=li:1

I get many many results. This is exactly what happens with the Google (and other) plugins)

And what do you have in the Settings tab?

I found a clear explanation in a GitHub issue thread:

It looks like Google has removed the ability to fetch search results from their javascript-free URL (which is what Whoogle uses for everything). This was a feature they supported for over a decade, so I’m kinda surprised they seemingly killed it with no warning. Usually they only do that after a year or so of supporting a product.

2 Likes

These are the settings for the Bing search on “Wirecard”. As I just reinstalled, all the settings are default. Using the search string from the log produces many many results.



Thanks for the link! To fix this issue create a duplicate of the Google plugin in the Plugins & Scanners panel and then add the following key to the new plugin (until Google will block this user-agent too):

	<key>UserAgent</key>
	<string>Mozilla/1.22 (compatible; MSIE 2.0; Windows 3.1)</string>
5 Likes

That fixed the Google issue for me. Thanks!

I just did the same search with the Google plugin using the new key which was suggested and it is working fine with the same settings so the Bing issue can’t be settings, I think

BTW:
You might want to add this too to make captchas less likely:

	<key>CrawlDelay</key>
	<real>2</real>
1 Like

Great.

dont know if you saw but having trouble with other web search plugins like Bing and DDG which only returning a few results.

The DuckDuckGo plugin supports only one page of results and therefore few results. But the Bing plugin works fine over here. Sometimes search engines return different results depending on the geolocation/IP.

Its not just different results, its NO results for the Bing plugin using various IP’s via VPN. All give basically the same result

Assuming your software is up to date (to eliminate the variable), it’s logically not an issue with DEVONagent itself given it’s working on multiple machines, in different countries, on different continents.

Here is a Bing searched VPN’d to Madrid…

And to confirm @cgrunenberg’s suggestion (from @meowky’s find) is a search in the modified Google plugin…

I can confirm that searching wirecard with the Bing plugin turns up nothing. It seems something isn’t quite right at the Bing server side. Tested using a Singaporean IP.

  • Searching carbon dioxide does return something, however all results are from the domain ccohs.ca. This is particularly weird, considering that I don’t have anything indicating I’m in Canada.
  • Searching carbon monoxide turns up nothing.
  • Searching backup* macOS turns up nothing.
  • Searching apple turns up 10 relevant results concerning the company (5 are filtered). None of the Apple sites are Canadian.
  • Searching oxygen returns 4 results from who.int, and 6 results from Reddit. Makes no sense at all; where is Wikipedia?
  • Searching Ryukyu turns up mobile Wikipedia sites, whereas Hawaii turns up desktop Wikipedia sites.

I have never used the Bing plugin before, so I can’t say if this is new behavior, or if Bing has a history of returning erratic results.

I dont know what to say because I have been using DA for almost 20 years with no issues and now all kinds of issues.