Scheduled searches work sporadically

Sometimes they run and other times nothing happens. I am guessing that it has to do with what is going on in Agent (a results window active?) when it is supposed to start, but I haven’t narrowed it down. What else can stop it from initiating?

Second question - is there any way to trigger a search? ie from search sets it would be nice to have a RUN NOW option on the context menu instead of having to initiate a new search and then searching out the set.

Final question - any way to BULK edit the search sets. I have about 50 and I want to change schedule for example. Do i have to go through one by one?

Thanks
Paul

Is the computer awake when the search is scheduled?

Yes. Computer is awake and DA is running. Still trying to narrow down variables. Wonder if VPN might have something to do with it? I’ll check that tonight.

No obvious variables that I can find. Is there a log file that might show an attempt and a failure? Maybe I could watch CONSOLE at scheduled time.

Computer is set to not sleep. Also set to wake for network access. Put hard drives to sleep when possible is unchecked. Mac Big Sur 11.

Some scheduled searches run just fine and others simply don’t happen. Frustrating since there is no way to batch run the failed searches and I have to go through and do them one by one.

I may have a clue here - after another failed search set not launching on schedule, I decided to quit DA. When I did, I got a warning that a search set was running, so I canceled the quit. But there is nothing running. So I did the force quit and restarted DA and the next test seemed to work fine. Maybe searches are getting “stuck” or not “releasing” or whatever the word is. I will try to remember to restart DA after running searches and see if that helps.

Next time please check whether there are any windows listed in the Windows menu which are not visible on the screen - thank you!

No windows showing in the menu.
Restarting DA seems to solve the issue. (sharing that info is the main reason for this post)
The scheduled sets fail to run when:

  1. DA “thinks” there is a search open even if there isn’t
  2. The schedule times are changed and DA not restarted.

Does this affect all searches or only certain search sets? What kind of actions do they perform?

OK, a lot more time spent on this and a few observations. The DA app ALWAYS says there is a search running when I try to quit and I get the dialog box warning. I wondered if having a scheduled search upcoming makes it report an active search but it doesn’t seem that that’s the case. The app must be restarted in order for scheduled searches to run again once it has “failed”.

So my setup has about 50 scheduled searches. For now, I have most of them set for hourly because it takes a bunch of attempts to get through it and I don’t want to wait a day or a week for the retry after restarting.

Nothing seems to “stall”. The search set starts, the search windows pop up and the process begins. Then it ends, and I realize that not all of them had run successfully. So I restart DA and 15 minutes later when the next run starts, it goes again and may nor may not finish.

Two questions that would help me narrow it down:

  1. can a scheduled search fool DA into thinking there are active searches going, or is there a stuck search that I can’t see (definitely none showing anywhere).
  2. If I set all of the search sets to QUIT WHEN DONE then what happens if there are 15 searches set to run at 1:15? Does the first one to finish quit the app and stop the process?
1 Like

This is more likely.

The app shouldn’t quit as long as it’s not idle.

So, after another few months, here is what I can conclude.
The search sets almost always fail at some point. So if twenty searches are set to run at noon, one of the searches will “fail” and no future scheduled searches will run until the app has been restarted (including dismissing the “there are searches running” (which there are not) message. It’s not any particular search set or search item. It seems random as near as I can tell. Is there any way to trace this back or do I need to just run all searches manually? If so, does that mean one at a time?
I love DA and what it has done for my workflow, but having to restart DA fifty times a day so my scheduled searches will run is obviously not ideal.

Why are you running 20 searches simultaneously and setting them to launch at the same time?

Fair question. With an hourly search there are only four time slots available. If there is a way to choose your own times, that would be great. But as it is, I have 00,15,30 and 45. So any number greater than 4 in your scheduled searches are going to require simultaneous searches. Maybe I am trying to do too much with DA.

Why do you need searches that frequently?

I work in movie marketing, so there are myriad of keywords to keep tabs on. i use FEEDLY as well, and GOOGLE ALERTS too, but mostly I rely on DevonAgent for the up to date stuff. It works perfectly for me, except for this issue of thinking there are active searches running where there aren’t and needing a restart to get it rolling again.
So, if i have a total of 40 searches I want to do each day, and I choose this schedule:
HOURLY at HH:00 searches 1-10
HOURLY at HH:15 searches 11-20
HOURLY at HH: 30 searches 21-30
HOURLY at HH:45 searches 31-40

I may get all of the results out of the first run. But the second run will likely not start until I have restarted DevonAgent and dismissed the “searches are still running” dialog.

So, in the perfect world, I would run 90% of the searches on a daily basis rather than hourly, so I could spread them out more. In fact, I was thinking of trying that just to see of volume is the problem. The problem is that when a search fails to run, you don’t get a chance to try it again until the next day, and you have to keep some kind of log as to which searches were actually being run. At least with a batch running every 15 minutes, you have a chance of completing all 40 searches at some point during the day.

I will try reducing the numbers first though. I can’t spread them out as there are only four times but I can simply reduce the numbers and see if that eliminates the need to restart DA.

Let me say again that DEVONagent and DEVONthink of course, are absolutely awesome and if I have to keep restarting that’s what I’ll do because the results and the workflow are simply too awesome to abandon.

Do they need to be 40 separate searches, or can you combine them with a big “OR” statement and then later in Devonthink use a Smart Rule to apply relevant tags or move them to relevant Groups?

Are you running search sets that follow links or shallower searches?

It’s a combination - but mostly zero links followed.

With the feedback from everyone on here, I did a bit more playing and here is what I have learned.

  1. turning on QUIT AFTER SEARCH for every single search seems to be the solution to searches getting “stuck” and leaving the app with the impression that tasks were still running. I have not narrowed down any specific searches that fail consistently.

Not sure if I am the only one with this issue, but I just wanted to put this here in case someone runs into the same challenge later.

Some apps are just so good that you will tweak and play forever to make it do what you want it to do. DA is one such app.

Hang on, hang on, back the truck up! I may have just found the answer. Is it possible, PERHAPS EVEN OBVIOUS, that the script being executed on the results of the script, is the search process that isn’t done yet?

What script are you using?