Backlinks Search Question

I want to set up DevonAgent to search backlinks to sites. I am not getting the results I see when using Google directly. Here’s an example comparing a native google search with an agent query:

Google Search - returns 2 recommended results; 42 in total

DevonAgent search set-up:

  • Query term = blink182.com
  • Follow Links = off
  • Express Search = on
  • Advanced: exclude domains: blink182.com (so I only see backlinks, no blink182.com links)
  • Sites: allmusic.com (I’m adding this to limit results to backlinks from allmusic.com In practice I would search for all backlinks across the web.

DevonAgent search is returning 5 links from other domains but not allmusic.com which I expected the sites setting to limit for.

Can someone take a look and advise how I should set this up – search set attached.
blink182.com Backlinks.agentSet.zip (1.0 KB)

DEVONagent isn’t a front-end to Google nor should you compare the two. DEVONagent is made for filtering results before delivering them.

If you want unfiltered searches, use the Express option. Alternatively, you can just use Google in a browser.

I’m only using Google Search to debug and illustrate the issue for this forum. I want to use DevonAgent for its ability to automate and capture results to DT3 in ongoing fashion (scheduled)

Express is enabled; again for debugging, ultimately I’ll want to filter out past results and similar pages.

How should I be setting up a query like this in DevonAgent Pro?

search for the term blink182.com excluding results from the domain blink182.com

to test my query I added ‘allmusic.com’ to the sites panel to limit the results to a single domain

  • In practice I would want results from across the internet. I don’t want to set up a custom search plug-in for allmusic.com
  • my project has nothing to do with music or blink182.com; I set them up only as examples for this forum.

@BLUEFROG,

Do you mind taking a look and answering my question?

I’ve read the DAP documentation back to front and haven’t found the answer yet.

Can you provide an actual example or are you truly intending to search for a domain name but not wanting to match the domain?

Yes.

The use case is common in SEO and content marketing – search for all backlinks to my site or my competitor’s sites. There are even specialized software with their own search index database for this – see the backlinks section on this page https://ahrefs.com/site-explorer

However, for my purposes the Google and other search engine indexes work just fine. The test I ran in with Google search operators (search for url “blink182.com” but exclude the results from the domain blink182.com produced the result set I expected.

If I’m understanding the DAP features from the docs I should be able to create a Search Set where I can exclude domains (advance tab) and where needed only Include links from select sites (the sites tab)

That’s how I’ve set up in the uploaded search set but it is not working as expected.

Note

  • If it was a one time need I would do as you suggested and just export the results. However I’m trying to use the power of DAP automation so that it monitors for this query and sends new results to DT3 as they are found.
  • the specific companies I’m monitoring have very common terms in their names. Using the name of the company will produce far too many meaningless results and so I’m choosing to use the URL of the site instead for it’s uniqueness.

Isn’t this what you’re looking for?

Why does blink182.com appear in the list of results if the domain is excluded?

Also, why does setting a domain (allmusic.com, for example) in Sites not restrict the results to just that domain?

I’m asking because I tried the example with DA as well, and it seems odd to me that these exclusions and whitelists seemingly have no effect on the results list. Just looking to understand the product better.

The screen capture clearly shows the domain is Excluded.

Also, why does setting a domain (allmusic.com, for example) in Sites not restrict the results to just that domain?

Did you add a plugin in the Plugins tab of the search set?

@BLUEFROG,

That’s exactly how set up and it appeared to work.

To doublecheck the results I have filtered DAP results looking for allmusic.com using the search bar on the upper right.

On the off chance that allmusic.com isn’t within the first 1000 results I’ve
also searched the results for the presence of blink182merch.com which is the number 2 ranked result on Google.

I understand not wanting to compare DAP with Google but I can’t explain why a top-ranking search result in Google isn’t showing up in the DAP result set. Can you take look at the screenshots below and advise.

What are the Settings in the search?

If you mean the application wide settings I haven’t modified those from default. Screenshot here.

I am also uploading the Search Set I created for Blink182.com example.

blink182.com Backlinks 2.agentSet.zip (1.1 KB)

No. The Settings tab of the Ssearch you posted screen captures of.

@BLUEFROG,

I’ve found the issue. It is not DAP, it is Google.

It is showing results for pages like allmusic.com which do not actually contain “blink182.com

The way I figured this out was to wrap the search with quotes “blink182.com” and only from site:allmusic.com. With the quotes wrapped Google finally displays a the alert text:
*No results found for blink182.com” -site:blink182.com +site:allmusic.com (see screenshot).

DAP on the other hand doesn’t show me any results that don’t contain blink182.com – exactly what I needed.

I think we can consider this issue closed. Thanks for all your help.

1 Like

Here’s a article on quotes in Google – I knew to use quotes with collection of terms – “blink 182” but didn’t think it was necessary for a single term like blink182.com

2 Likes

Thanks for the follow-up and link to the article. I wasn’t aware of these different quoting behaviors either. Learn something new, every day :slight_smile:

I think the Google search plug-in that ships with DAP is broken or needs an update.

Any term I search using just the google plug-in shows 0 links in the log and no results.

For comparison when I use Bing I see 28 of 30 links

Clicking on the URLs in the logs do confirm the presence of links in the search results.

https://www.google.com/search?as_q=Vanilla+Ice+Cream&num=100&ie=UTF-8&start=0

Maybe you’ve performed too many searches too fast and Google requires a captcha.

Your righ @cgrunenberg. I’m seeing results today that I wasn’t getting yesterday

Today:

Yesterday:

Thank you!