I’ve been using Kagi for a while and it helps with sorting out the garbage websites. I’m having a devil of a time setting it up in DevonAgent. I’m logged into Kagi via DevonAgent window before I attempt my searches. This is the query string that I am using:
httpX://kagi. com/ search? q=agentQuery
But DevonAgent doesn’t parse the results and just returns the search page, the map page and a link to you.DOTCOM.
edit: I had to add spaces and the X to the query string because discourse didn’t like me posting a URL. Also it didn’t like the youDOTcom url either.
I’m not getting any results now - they gave an error and said too many requests. I think I’m going to not use Kagi with DevonAgent - there’s a chance they could ban me and Kagi’s too valuable for that.
@BLUEFROG, I tried the Kagi plugin, but no useful results were returned. I added Kagi’s access link token to the EngineUrl parameter and got the same results.
I have attached screenshots of the results, the log, and the settings panes. I have also attached a slightly modified version of the plugin (with the CrawlDelay and the access token placeholder included).
I also logged into Kagi from the DEVONAgent browser and used the session link from Kagi’s user settings page.
DEVONagent can’t parse pages that load results dynamically via JavaScript.
Note that the string you use in EngineURL is different from BLUEFROG’s plugin shared above – which uses kagi.com/html/. I assume this gives you a static, paginated results page without JavaScript, which DEVONagent can parse. (I don’t have a login, so can’t test).
I’m not sure the Session Link is necessary, since no one mentioned it above. Isn’t it enough to login in DEVONagent’s browser first? What are your cookie settings?
But if you do want to use the Session Link, I think you ordered the query string wrong.
Thank you! This worked. The token is unnecessary once you log into Kagi from the DEVONagent browser.
A new problem is that the first set of results is still from Kagi’s search result page (the screenshot might explain this better). It’s very likely because of Kagi’s link aggregation.
Also, for anyone using Kagi on the metered plans (e.g., 300 searches for 5 USD/month), note that this approach will burn through the search credits—caveat emptor.
That doesn’t look right at all. DEVONagent is supposed to parse the result pages and use the acutal search results. That’s also what BLUEFROG’s earlier screenshot shows.
But maybe Kagi’s results page changed since June and the plugin needs to be fleshed out. I would try adding the LinksStart and LinksEnd keys in the plugin.
I also think you should choose a language in the settings tab instead of using International. The log from your first post looks like… the same search is performed for every language/region! (the &r= parameter) Your screenshot above could be the same, but I can’t tell. Are you in Uruguay or is that just a random results page?
the first set of results is still from Kagi’s search result page
What do you mean by “the first set”? Seeing the log and the results tab is honestly more useful.
Yes. I thought so as well. The search result rendering has probably changed, as what I see differs from BLUEFROG’s screenshots.
The language setting is a good idea and would have saved me some search credits (I am not in Uruguay, BTW). Unfortunately, I have run out of this month’s quota and will need to wait for the counter to reset or pay per search, which can get expensive.
What I meant by “the first set” is the initial set of results returned in the results tab. See the screenshot I attached (in the browser view after the search is finished. What I mean by this is that the first 20+ results were links to Kagi’s search result pages and not the target pages. I hope this clarifies the admittedly poorly worded description.
And built-in parsing is limited to few major search engines. For Kagi the keys LinksStart, LinksEnd, LinksMatching and/or LinksNotMatching are necessary.
The best way to test a plug-in is via the Plugins & Scanners panel and its Test toolbar item. This command retrieves the first two pages of results and shows which links are used (+) or not (-). It’s also easy to exclude complete domains or some URLs via the contextual menu. This ensures that unnecessary traffic is avoided.
Makes sense. I wasn’t sure about Kagi, since the plugin BLUEFROG shared didn’t use any extra keys for parsing. LinksMatching and/or LinksNotMatching would also have been my next suggestion after LinksStart and LinksEnd.
Good reminder about the Test toolbar item. That’s also where I started when looking to write some plugins. (Step 3 under Plugin Development > Creating Your Own Plugins in the manual )
As I don’t have a kagi account, I couldn’t do any testing.
)