Help needed ebay classified plugin

DEVONagent Automation
1

Hello,

I’m trying to develop a plugin that searches the german website kleinanzeigen.ebay.de" for a given product name etc.

The “next” page link shows the following url:
http://kleinanzeigen.ebay.de/anzeigen/s-seite:2/tiffany/k0
(hint: seite = page)
and the result page in the browser talks about 2.800 adverts on ebay for that keyword.
My plugin loads 148 files and shows 68 results, this seems not complete.

The protocol tells me that the “interesting” next pages are “duplicates”, but I would like DA to scan these pages and show me the results.

protocol.jpg
protocol.jpg761×69 84.9 KB

Any idea what I need to change here?

regards

Joern

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Description</key>
	<string>Durchsucht das Web per &lt;kleinanzeigen.ebay.de&gt;.</string>
	<key>EngineUrl</key>
	<string>http://kleinanzeigen.ebay.de/anzeigen/_agentOffset_/_agentQuery_/k0</string>
	<key>Identifier</key>
	<string>kleinanzeigen.ebay.de</string>
	<key>Info</key>
	<string>1 Kleinanzeigen Plugin</string>
	<key>Name</key>
	<string>1 Kleinanzeigen</string>
	<key>OffsetPerPage</key>
	<integer>100</integer>
	<key>Operators</key>
	<integer>59</integer>
	<key>ParseLinks</key>
	<true/>
	<key>ResultsPerPage</key>
	<integer>100</integer>
	<key>Start</key>
	<integer>0</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>
</plist>
2

There are several minor issues but the most important one is that the plugin doesn’t define which links should be used and therefore uses all of them. Here’s a revised plugin adding a LinksMatching key:


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Description</key>
	<string>Durchsucht das Web per &lt;kleinanzeigen.ebay.de&gt;.</string>
	<key>EngineUrl</key>
	<string>http://kleinanzeigen.ebay.de/anzeigen/s-seite:_agentOffset_/_agentQuery_/k0</string>
	<key>Identifier</key>
	<string>kleinanzeigen.ebay.de</string>
	<key>Info</key>
	<string>1 Kleinanzeigen Plugin</string>
	<key>LinksMatching</key>
	<string>/anzeigen/s-anzeige/</string>
	<key>Name</key>
	<string>1 Kleinanzeigen</string>
	<key>OffsetPerPage</key>
	<integer>1</integer>
	<key>Operators</key>
	<integer>0</integer>
	<key>ResultsPerPage</key>
	<integer>25</integer>
	<key>Start</key>
	<integer>1</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>
</plist>
3

works great! Just tried it in combination with a search set and received my 500 pages.

Just two more questions:
How did you develop this “LinksMatching key”?
I don’t understand where the “anzeige” in “s-anzeige” comes from. I believe the “S” comes from “Seite” (=page) but how did you come up with the “anzeige” behind the s- ?

Second one:
When I check the protocol I see this kind of error message, is that of any concerns re to the results?

protocol.jpg
protocol.jpg761×69 84.9 KB

Thanks for your great help,

Joern

4

I believe I’m closer to the “linksmatching” solution, understood how you did that for the “Kleinanzeigen” script.
Anyway I tried it for eBay and it seems that I still overlook something:

ebay next page link:
http://www.ebay.de/sch/Uhren-Schmuck-/281/i.html?_from=R40&_nkw=tiffany&_pgn=5&_skc=200&rt=nc

ebay direct auction link
http://www.ebay.de/itm/Armband-Tiffany-Co-Sterling-Silber-Ruckkehr-Zu-Tiffany-Herz-Gebogen-/121354858054?pt=DE_Uhren_Schmuck_Echtschmuck&hash=item1c41502e46

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Description</key>
	<string>Durchsucht &lt;www.ebay.de&gt;.</string>
	<key>EngineUrl</key>
	<string>http://www.ebay.de/sch/Uhren-Schmuck-/281/i.html?_from=R40&amp;_nkw=_agentQuery_&amp;_pgn=_agentOffset_&amp;_skc=50&amp;rt=nc</string>
	<key>Identifier</key>
	<string>www.ebay.de</string>
	<key>Info</key>
	<string>eBay Test Plugin</string>
	<key>LinksMatching</key>
	<string>/itm/</string>
	<key>Name</key>
	<string>eBay Test</string>
	<key>OffsetPerPage</key>
	<integer>1</integer>
	<key>Operators</key>
	<integer>0</integer>
	<key>ResultsPerPage</key>
	<integer>25</integer>
	<key>Start</key>
	<integer>1</integer>
	<key>Version</key>
	<string>1.0</string>
</dict>
</plist>

I receive only 50 files and 2 results:

protocol3.jpg
protocol3.jpg1006×187 249 KB

5

I just had a look at the URL of each result. The URL of the search results pages looks like this…

kleinanzeigen.ebay.de/anzeigen/s … tiffany/k0

…and the URL of each result like this…

kleinanzeigen.ebay.de/anzeigen/s … ref=search

The updated plugin doesn’t cause any duplicates over here (and shouldn’t use the URLs shown on the screenshot).