Problems Creating Technorati Plugin

DEVONagent Automation
1

Following the instructions in the DEVONagent Pro manual, I’ve tried creating a Technorati plugin. When I carry out a search with this, however, I don’t get nearly the results that I do when searching from the Technorati site and at times those hits that I do get are often so far off topic that they are useless (e.g. definitions of the given term from thefreeddictionary.com, etc).

I just tried a search for “Ariel Sharon” using the plugin and got six hits using my plugin.

Using the Technorati site itself I get 8 pages of hits.

Here is the search string as culled from the Technorati site:

technorati.com/search?return … on&x=0&y=0

Here is the code from my plugin:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Description</key>
	<string>Searches the web using &lt;www.technorati.com&gt;.</string>
	<key>EngineUrl</key>
	<string>http://www.technorati.com/search?return=posts&amp;authority=high&amp;q=_agentQuery_&amp;x=13&amp;y=14</string>
	<key>Identifier</key>
	<string>www.technorati.com</string>
	<key>Info</key>
	<string>Technorati Plugin</string>
	<key>Name</key>
	<string>Technorati</string>
	<key>OffsetPerPage</key>
	<integer>1</integer>
	<key>Operators</key>
	<integer>59</integer>
	<key>ParseLinks</key>
	<true/>
	<key>ResultsPerPage</key>
	<integer>10</integer>
	<key>Start</key>
	<integer>0</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>
</plist>

Can anyone advise where things might be going awry? I’ll need to set up several specialist plugins similar to this (PubMed, Google Blog Search, Wink), so would really appreciate help ironing this out.

Many thanks,

Will

2

There are two issues:

  1. The EngineUrl string doesn’t include agentOffset and therefore the plugin can use only the first page of results.

  2. As there are no LinksStart, LinksEnd, LinksMatching or LinksNotMatching keys, all links of the results page are used by the plugin.

Here’s an improved version of the plugin:


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Description</key>
	<string>Searches the web using &lt;www.technorati.com&gt;.</string>
	<key>EngineUrl</key>
	<string>http://www.technorati.com/search?q=_agentQuery_&amp;return=posts&amp;submit=&amp;sort=relevance&amp;topic=overall&amp;source=blogs&amp;authority=high&amp;page=_agentOffset_</string>
	<key>Identifier</key>
	<string>www.technorati.com</string>
	<key>Info</key>
	<string>Technorati Plugin</string>
	<key>LinksEnd</key>
	<string>&lt;/ol&gt;</string>
	<key>LinksNotMatching</key>
	<string>*www.technorati.com*</string>
	<key>LinksStart</key>
	<string>&lt;ol class="search-results post-list"&gt;</string>
	<key>Name</key>
	<string>Technorati</string>
	<key>OffsetPerPage</key>
	<integer>1</integer>
	<key>Operators</key>
	<integer>59</integer>
	<key>ResultsPerPage</key>
	<integer>10</integer>
	<key>Start</key>
	<integer>1</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>
</plist>
3

BTW:
There’s already a plugin for Technorati, see category “Blogs”.

4

Hi Christian,

Thanks. I’ll go back to the manual and read up on the LinksStart, LinksEnd, etc. operators you mentioned above.

In the interim I’ve tried your improved script above and found it returned much better results. Thanks.

On a similar note, would my lack of familiarity in fine tuning plugin scripts also explain why when I created a plugin to search the Proz.com site (a site used by translators to discuss translations and terminology) I found that I was unable to get the same results I could get by just using Google? Here is an example:

When using Google to search Proz.com for the term “Vorstandsvorsitzende” I got a Proz hit directly addressing this term at the top of the page of returned hits:

Search pattern used:

insite: proz.com Vorstandsvorsitzende

Top hit returned:

proz.com/kudoz/german_to_eng … stand.html

However, when using the plugin I created, the above hit was not returned in the list of articles found. Here is the plugin I created:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>CrawlDelay</key>
	<real>0.125</real>
	<key>Description</key>
	<string>Searches on the website &lt;www.proz.com/kudoz/german_to_english/&gt;.</string>
	<key>EngineSite</key>
	<string>www.proz.com/kudoz/german_to_english/</string>
	<key>Identifier</key>
	<string>www.proz.com/kudoz/german_to_english/</string>
	<key>Info</key>
	<string>Proz DE&gt;EN Plugin</string>
	<key>LinksMatching</key>
	<string>*www.proz.com/kudoz/german_to_english/*</string>
	<key>Name</key>
	<string>Proz DE&gt;EN</string>
	<key>Operators</key>
	<integer>59</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>
</plist>

I’ll also see if I can find a tutorial on creating and tweaking plugins.

Thanks,

Will

5

Google’s ranking may or may not imply the ‘best’ result. Usually, for the purposes for which I’m running DEVONagent Pro searches, Google’s top ranked search isn’t the most useful one in the DEVONagent results list.

If, however, you see a good Google result that doesn’t appear in the DEVONagent Pro results list, take a look at the Log. That page may have been rejected because it was larger than the currently set size limit, which can be adjusted in DEVONagent’s Preferences.