crawling a site

DEVONthink Tips
1

I searched the forums looking for this, but it just left me more confused… and I have no idea why this feature isn’t more obvious, I must be missing something right in front on my face…

I want to crawl a site (e.g. macintouch.com) in DevonThink; not search it, just crawl it, given a home page.

I tried creating a new Search Set, used a default search term - Macintosh - in both (?) fields, and it did manage to find a single result, but that’s all.

How can I get it to crawl the entire site?

thanks.

2

If you’re using DevonThink pro, you’ll want to File/Import Site… and put the home page as the starting URL.

3

I’m actually trying to use DevonThink, not DevonAgent; I don’t need to download the site, I need to analyze it the way that DevonThink does.

4

Assuming that DEVONthink should be DEVONagent (and vice versa) in the above statement, then…

  1. Activate the “Follow Links” checkbox in the Search Sets panel
  2. Use the highest “Follow Links” level (by moving the slider to the right)
  3. Enter “*” (without quotes) as the term to follow links (that way all links will be used)
1 Like
5

yes, you’re right, my mind was up mixed, I meant DEVONAgent.

your trick worked, thanks! (those 2 seach fields really ARE confusing.)

thanks again!