Automatically searching for new entries on a website and emailing results

Hi Everyone.

I’ve had Devonagent for about 10 years. What I’m trying to do is get it to report back any new additions to a webpage, searching maybe daily. Can’t seem to get it to work. There are short articles and links to the whole story behind.

2 Likes

This is exactly what I use DA for. I love it. I set up a Search Set. No Scanner. Under Advanced, I Search in the Title, Text, URL and look for HTML and PDFs. At first, I search for Results in All Pages but then once I’m up to date, I switch over to Only new pages. For me, it helped to examine the results I seek. The URL gives me clues about where to look for the documents I want. I Crawl those URL paths in Search Sets > Sites. I don’t use any Plugins. I checked in Actions that Results are emailed to me. I set up my email in Settings > Email. I set my Schedule to Weekly since my target website updates only a few times per year.

I hit a snag on setting up my SMTP, but played around with it and it works fine. It really helped me to spend a few evenings studying the manual. I never really understood DA for search, but now I think of DA more like an agent, like “agentic AI”, I feel like I’m better understanding how it is useful to me.