I want to use DevonAgent to download and monitor the different competitor websites.
Using DevonAgent I have set up two separate Search approaches: Crawl a domain, and Search (results from Google for that domain).
I’m surprised at how different the crawl vs search count is:
- Crawl: 45 results
- Search: 1,664
and when I download the entire site using DT3 I see
- Entire Site Pages: 2,344 pages for the domain (not counting assets)
Any reason for such large variance in results.
Aside from the technical issues I’m really just wanting to review and monitor several competitive domains. DA3 seems like the right tool but am trying to make sure I’m setting it up correctly for this purpose.