Plugins driving me crazy

I can’t seem to get plugins to work, no matter what I try to do. The manner in which they fail is different for the two sites I’m trying, but in both cases I get nearly no results.

First, I want a plug-in to search the MSDN Online Library. This is what I have:


<dict>
	<key>Description</key>
	<string>Searches the MSDN Library</string>
	<key>EngineUrl</key>
	<string>http://search.microsoft.com/search/results.aspx?qu=_agentQuery_&amp;View=msdn&amp;st=b&amp;c=4&amp;s=1&amp;swc=4&amp;p=_agentOffset_</string>
	<key>Identifier</key>
	<string>com.johnwiegley.msdn_web.plugin</string>
	<key>Info</key>
	<string>MSDN (Web) Plugin 1.0 ©2005 John Wiegley</string>
	<key>Keyword</key>
	<string>msdn</string>
	<key>LinksEnd</key>
	<string>&lt;div class="resultsNavigation"&gt;</string>
	<key>LinksStart</key>
	<string>&lt;div class="results"&gt;</string>
	<key>Name</key>
	<string>MSDN (Web)</string>
	<key>ResultsPerPage</key>
	<integer>10</integer>
	<key>OffsetPerPage</key>
	<integer>1</integer>
	<key>Start</key>
	<integer>1</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>

When I search, it reports that there are no results. However, if I look in the log, I see that it finds the links to the result pages, it just marks them all as “Skipped”. I cannot figure out why. My search query is “datagrid”, which should return MANY hits.

Next, I have a plug-in for searching the World of Warcraft community mod site. This one actually returns a single correct result – but no others. The log doesn’t reveal anything beyond the search query, so I can’t figure out why it’s skipping the many results that are found. My search query is “buff”, which should return over 50 pages.


<dict>
	<key>DateEnd</key>
	<string>&lt;br&gt;</string>
	<key>DateStart</key>
	<string>Last Updated: </string>
	<key>Description</key>
	<string>Searches the WOW Community Mod site</string>
	<key>EngineUrl</key>
	<string>http://ui.worldofwar.net/listuis.php?searchtext=_agentQuery_&amp;x=0&amp;y=0</string>
	<key>Identifier</key>
	<string>com.johnwiegley.ui_wow.plugin</string>
	<key>Info</key>
	<string>WOW Mod Site Plugin 1.0 ©2005 John Wiegley</string>
	<key>Keyword</key>
	<string>wowui</string>
	<key>LinksEnd</key>
	<string>&lt;/table&gt;&lt;/center&gt;</string>
	<key>LinksMatching</key>
	<string>*ui.php?id=*</string>
	<key>LinksStart</key>
	<string>form action="/listuis.php" method=get</string>
	<key>Name</key>
	<string>WOW Mod Site</string>
	<key>ResultsPerPage</key>
	<integer>100</integer>
	<key>TitleEnd</key>
	<string>&lt;/font&gt;</string>
	<key>TitleStart</key>
	<string>&lt;td width=99% align=left&gt;&lt;font class=txc2&gt;</string>
	<key>Version</key>
	<string>1.0</string>
</dict>

I’ve read through the Plugins doc at least twenty times, and have tried every variation of settings I can think of. What am I doing so wrong? I even started my plugins based on the Google web search plugin.

Thanks,
John

Please take a look at this thread in the forums: Can Da be “neat” without XML?

In the upcoming release this post has been turned into a tutorial but it will use the updated features of DA and so may cause some confusion. The post above should give you a good idea how to debug your plugin, since I had to do this myself for a lot of new plugins. Please take a look at it, give it a whirl and tell us if it was useful.

Cheers!

Nope, nothing on that page helped. I tried every option I know of this time, and still I can’t get it to show more than one valid link from the WOW mod site. Is it considering all of the URLs on that page the same? It downloads the first entry great, but nothing more.

Well, the first thing I notice in the plugin for WoW is that the value for the “LinksEnd” keyword is too generic: it is found in many places in the HTML source. Try this value instead: “”. That happens only once at the end of the file and the tables with data end there too.
But, other than that your plugin is ok. What causes the problem is that the relative links seem to be malformed on the page. They should start with a ‘/’. I’m going to investigate a bit further but if we can have a workaround, it will be in the next version.

Your plugin with MSDN doesn’t give any results because probably in your preferences the microsoft domain is being ignored. Don’t ask me why that domain is in there but you can fix it easily. Go to the Preferences, click the “Search” tab and remove the “microsoft.” entry. Rerun your query.

Now I did run your query and you can optimize it, but I’ll leave that as an exercise for you. LinksMatching and LinksNotMatching are your friends…

Cheers!

Ah, thank you. You’ve restored my sanity for today at least. :slight_smile: I didn’t even think about the Preferences filtering.

The MSDN query works fine now, as well as six other plug-ins that I’ve written since. It’s only WOW that has me stumped. I’m looking forward to whatever you find.

Also, is Operators documented anywhere yet?

Thanks, John

The Operators keyword is described in the XML keywords document. But in order to find out which ones work, you have to dig through the target website’s documentation and do some experimenting because there are so many variations on a theme here…

Actually, the XML document just says “Description forthcoming”.

Oh, and a big feature request for DEVONagent (which I am really liking): type-ahead search, in the manner of Firefox.

In general I am a big fan of keyboard shortcuts. While DEVONagent is OK in this respect, it has some strangenesses, such as that SPACE pages forward in the browser and Digest view, but not in the Pages (Summary) view. I’ll collect a list of suggested improvements if you are interested.

In DEVONthink, keyboard navigation is far worse. The tool is almost not usable without a mouse. Who bound the view tabs to Cmd-Opt-1? Fortunately the System Preferences allowed me to rebind those to Cmd-1, etc. Or take Sheets: they are pretty much useless unless you are in love with your mouse. I can’t even make a new row without moving and clicking.

Perhaps you could take away your UI engineer’s mouse for a week. That would generate all the solutions I’m hoping for. :slight_smile: Another example: Allow Cmd-I and Shift-Cmd-I to do in DEVONthink’s browser what they do in DEVONagent’s browser. This is something System Preferences can’t fix for me. :frowning:

A great tool so far, John

Oops on my part. In the upcoming release this will be forthcoming. And since this part was also upgraded to fit more websites’ ways to deal with operators you’ll have to look forward to version 2 for the updated document.

But this should be valid for the current version (fingers crossed):


Operators integer Bitmap describing the possibilities of the search engine and the required syntax. Default value is 59 (1+2+8+16+32) meaning that AND, OR, NOT, phrases and parenthesis are supported. 
Bit Value Description 
0      1  Supports AND operator 
1      2  Supports OR operator 
2      4  Supports XOR operator 
3      8  Supports NOT operator 
4     16  Supports "" phrases 
5     32  Supports () parenthesis
6     64  Supports NEAR operator 
7    128  Supports AFTER operator 
8    256  Supports BEFORE operator 
11  2048  Use ANDNOT instead of NOT or - 
12  4096  Convert search term to lower case 
13  8192  Convert search term to upper case 
14 16384  Engine scans for any word by default (not for all words) without an explicit operator 
15 32768  Use AND/NOT instead of +/-