DA- Make creating plugins easier for Researchers

As I review the DevonAgent plugin forum I see many requests for plugin codings. It seems the Devonagent plugins are just too general. Many researchers have specific websites and want to search within these for information. With the majority of users of Devonthink being researchers it should be easier for those who are not programmers to create their own plugins. Here are some easier ways that could perhaps be incorporated to make creating plugins within the grasp of researchers.

Many websites have their own search engines on their sites that can select the information researchers are looking for.

The browser OmniWeb provides “Shortcuts” that allows one to quickly create a shortcut that will use the selected site’s search engine to provide information from within the site. Firefox has similar “shortcuts” searching and Google has the ability to create custom site searches as well. The problem with each of these is they only seem to search one site at a time. This is where DA can be of great benefit because it allows for multiple selection of plugins within a searchset. If DA could link into or have a process to create plugins this easy then DA would be used instead of these other browsers.

I know the team at Devontechnologies is very busy but it seems that there should be a way to make creating plugins possible for researchers who are not programmers.

Below is a link to the manual that explains the process of creating “shortcuts” in two pages 55 and 56.

Thanks for all your hard work.

downloads.extremesims.com/omnigr … Manual.pdf

We’ll try to simplify this but it will be never as simple as a shortcut which is basically only loading a web page. DEVONagent has to parse the web page, follow the right links, extract contents & meta data, load additional pages etc.

I greatly appreciate your willingness to make the process simpler for non-programmers.
The “shortcut” examples I refer to seem to be templates or forms which a researcher can input their desired sites and topics where the result is a plugin for the browser.
If I may, the process of creating the DA plugin might even be done using Automator or Applescript if it asks the researcher where (website) and what topics are desired to be included in the plugin.

I am sure many researchers would extremely grateful for this and they would use DA as an integral part of their research finding.

Thank you for your response and considerations with this.

I’m afraid that’s not entirely what Christian meant. The problem is that most websites are a gigantic mess from a parser’s point of view. This is why it’s almost impossible to make effective “scrapers” (that’s what a plugin basically is) automatically. We can only make it easier by giving examples and a tutorial that shows the common pitfalls. And we have our consultants that can help you (probably for a fee) to develop a specialised plugin.

Thank you for your response.
I looked at the only tutorial I could find, which was in the DA Help: Creating your own plugins & XML Keys. The other tutorial videos were basic introductions of DA. The Help tutorial appears to be outdated and therefore not very easy to follow. This said I struggled through and found a temporary fix until Christian and the rest of the DA team can make creating plugins easier for non-programmers.

A PROCESS TO HELP RESEARCHERS: I have made a process that creates a basic plugin that allows the researcher a method of selecting their own sites that DA can search through. Some learning is needed to create the Google Custom Search Engine (GCSE), however the researcher will find that they can use their GCSE when away from DevonAgent and they can share it with others as well. (The XML text below searches my GCSE that includes sites that are faithful to Catholic beliefs.) To modify it for your GCSE follow the steps below. Let me know if this helps or needs improving.
Friar


This DevonAgent plugin after modifications will use your Google Custom Search Engine (or for short GCSE) that contains sites you selected.

STEPS:
1. Copy the XML text below into a new file using TextEdit application.
2a. Create your GCSE at http://www.google.com/coop/cse/ (you can modify and add more sites later.)
2b. Go to your GCSE homepage and with the box empty click on the Search button.
2c. Copy the url from “http://www.google.com/cse?cx=…” up to but not including “&ie=UTF-8&q=…”.
3. In your TextEdit XML file you copied from below, select and paste over “http://www.google.com/cse?cx=...5rwma” up to but not including “&ie=UTF-8&q=agentQuery&sa=Search”.
4a. Make your XML into a plain text file (in TextEdit, Format, Make plain text.)
4b. Save as mygooglecustom1.plist in your user folder username/Library/Application Support/DEVONAgent/Plugins.
5. Restart DA and it should show up at bottom of the plugins.

  (Note if you create more than one of these GCSE DA plugins, when in TextEdit you will have to change the names of com.my.GoogleCustom1.plugin and My GoogleCSE 1 as well as the filename: mygooglecustom1.plist. Replace 1 with 2 or something more specific to your custom search engines.)

---- XML Text below copy into new TextEdit plain text file—

<?xml version="1.0" encoding="UTF-8"?> Description This DevonAgent plugin, after modifications, will use your Google Custom Search Engine (or for short GCSE) that contains sites you selected. EncodingUrl UTF-8 EngineUrl [b]http://www.google.com/cse?cx=017643563084552897354%3Aloodmd5rwma[/b]&ie=UTF-8&q=_agentQuery_&sa=Search Identifier com.my.GoogleCustom1.plugin Info My Google Custom Search Engine Plugin 1.0 CC 2009 Friar.OP LinksEnd br clear="all"/ LinksNotMatching *q=cache:* *.google.* LinksStart Custom Search Name My GoogleCSE 1 OffsetPerPage 100 ParseLinks ResultsPerPage 100 Start 0 Version 1.0

:slight_smile: Hi Friars,
I tried your plugin and have some problems:
I made my personal google search and from safari it works perfectly but it doesn’t work as a DA plugin: for the same search DA returns me less results and no pdf (while the google search did)
Should I add some other preference in my custom set?

Anyhow: I didn’t know about custom search in google and it’s very useful :slight_smile:

Hello Nestor,

I am glad you found this helpful.
Here are a few things you may want to look at: After starting your DA plugin search click on “Log” and you will find your complete google custom search (GCSE) list as Safari would have shown you. You can search through this if “Digest” or “Pages” do not show you what you want.

(What is probably happening is that the DA plugin sees the links from this first GCSE page and then it can “Follow Links” through these first 10 hits. It could be broadened if there was a way to expand the GCSE to more than 10 on the first page. I would welcome recommendations on how to do this.)

If you continue scrolling down the “Log” list you will find in the “Error” column reasons why the DA plugin did not pass through the subsequent pages. You can adjust some of these allowing more through if you have limitations in the “Settings” that includes what your “Filter” and “Scanner” and “Secondary Query” may be limiting to pass through.

This is a work around and hopefully the ability and easier through site searches as found in the GCSE can be incorporated in creating future DA plugins. :slight_smile: