I’ve been using the wonderful Download Manager to grab archives of a couple of sites that I find really useful for my research (yet fear may disappear from the web at any time).
I have found that there are cases when I would really like to filter out parts of these sites as they really are not of interest to me. Therefore I would like to request the ability to add a list of URL paths or parts of a URL to exclude, such as certain directories or .php scripts that get called.
I understand that Will Ray was after some sort of plugin scheme in this post:
devon-technologies.com/phpBB … php?t=1488
What I’m after is a simple pattern match filter. At the moment I am having to do a pattern match at my firewall to block URLs containing certain strings, but this isn’t ideal. It would be good to have a global set of filters for all sites that the Download Manager archives and specific filter sets at the domain level also.
The ability to import sites for archiving is one of the hidden gems of DT Pro and easily overlooked. There are many tools that try to do just website mirroring or archival, but many are not a patch on the Download Manager. Almost all I have tried are unable to handle links in javascript, yet the Download Manager can. This alone has just saved me weeks of trawling websites and has been worth the purchase price many times over.
Edit:
I’ve taken to using Privoxy to do the required filtering with the Download Manager. Although not the cleanest of solutions, it does allow me to filter using regular expressions. Unfortunately it looks like DT Pro seems to ignore what I put in the Proxy preference setting and uses the one set up in the System Preferences Network pane instead, so I can’t have one setting for DT Pro and a different setting for the rest of the system. I imagine this has something to do with the use of WebKit.
Stuart