To a considerable extent, there will be problematic aspects of download manager and of other site suckers, until one can specifically ‘tell’ the process which links to follow, and which to exclude. If ‘follow link’ depth is too shallow, not all the desired pages will be captured. But, as many find out, if ‘follow link’ depth is too deep, it’s easy to run out of hard disk space.
As noted by others, download manager and other site suckers work with static Web sites, but not with dynamic ones. And of course the design and layout chosen by the site’s designer often impacts the scope and appearance of a site capture. Personally, I make very little use of Download Manager or any other page sucker. I can recall only one totally successful site download, i.e., one that brought in all of the wanted pages and didn’t bring in extraneous material. I sent a fervent letter of thanks to the site administrator. Some months later, I tried to download that site to check for changes. They were using a new site design and the download was useless. So it goes.
Most of the time, I do single-page downloads. Sometimes I do want to capture for reference purposes the exact state of a page, and now we can do a Web Archive download.
If I want to capture a page together with a lot of related links I use Acrobat’s Web page capture feature, as it gives me total control of the links that I want to capture (and the ability to discard pages that turn out to be unrelated, after download).
Example: I’ve got a PDF document that is over a thousand pages long. It contains technical references of the type that often change over time, e.g., laboratory procedures that may be revised. Each page of my downloaded PDF references the date and time at which the capture was made, which can be important for quality control purposes. Acrobat is smart enough to ‘internalize’ the links in the PDF file. That is, if I had captured a page that is referenced by another page I’ve captured, the link is followed in the PDF itself, not out to the Web. Note that there’s a lot of user decisions and some time invested in this approach. I went even further; because the actual page order was quite random rather than in linear topical order, I added Bookmarks to the document to make it very easy for topical navigation by users. That also took time.
In this case, I was providing reference materials intended for graduate student training in developing countries, and needed to assure that the included content was ‘on target’ and without irrelevant and potentially distracting or frustrating content. Unfortunately, at this time neither the Download Manager in DT/DT Pro nor any other site sucker software can produce results with the specificity and quality control of the example above. Maybe some time in the future?
Note: Johnson’s article in the New York Times has stimulated a lot of discussion among DT/DT Pro users about the ‘ideal’ size of documents in a DT/DT Pro database, especially for See Also purposes. I don’t ‘split’ large documents, but find that See Also still works pretty well for me. That 1,000+ page manual of lab procedures sits in my database. If I’m looking for analytical procedures for organic mercury compounds in fish tissue, it pops up – but it does not ‘dominate’ the list of suggestions, because relatively small portions of the big document are specific to that interest. And it’s a very useful reference, of course. Tip: Start See Also from a small document or selection, rather than from a big, diverse document.
