Search for EACH of two terms works; search for BOTH does not

Search in DTP(O) sometimes tears my hair… Here the latest:

I have a test document containing the terms “kallaro” and “pollari”. (kallaro in the body, pollari in the Spotlight comment - document for issue replication attached(*).)

Using the Search window with the term kallaro I find the document, using the Search window with the term pollari I find the document as well, but using both terms like this: kallaro pollari I don’t find the document. However it should find it, because from my understanding of the documentation space-separated terms are searched for with an and operator.

Why does this happen? It is pretty worrying: DTP(O) is such a great tool for me, but search becomes ever more important the more stuff I have in my databases. Not finding documents which I know are there weakens the trust in the tool considerably…

BTW, the problem is not limited to Markdown, happens at least with RTF docs as well.


(*) Unfortunately zipping the document does not preserve the Spotlight comment. After unzipping please add the term pollari to the Spotlight comment to replicate the issue.
Doc to find.md.zip (889 Bytes)

Using the search scope All, I’m finding this document with no issue.

That is strange! Just to make sure:

When you

  1. use the search string kallaro pollari (that is kallaropollari)
  2. in the Search window
  3. with the search scope All

the document is listed in among the results although

  1. the term pollari occurs only in the Spotlight comment?

Yes. It is the exact document you sent but with the word added to the Spotlight comments for the file.

Maybe it is related to the Search window? Here my story:

So why is this happening in the Search window? Why does DTP finds the doc for each search term, but not for both if one of the search terms is in the Spotlight comment?

And what can be done about this?

Thanks for any help!

Actually, I just saw my thesis was incorrect. You had added kallaro in the spotlight comment as well. It was hitting on both terms in the Spotlight comment.

No, kallaro is only in the body of the document.

But that is not the point. The point is: Given a document with one word in the body (say “kallaro”) and another in the Spotlight comment (say “pollari”) and then I search for “kallaro pollari” (with Search For option “All” and no other option) DTP does not find it.

What to do?

Incorrect. Look at my screencapture. The top line in the Spotlight Comments is from your. I KNOW I didn’t type it in and it certainly contains kallaro.

Criss could comment further, but when one scope is matched on a document, other scopes are not. ALL doesn’t mean, “Match some scope, then keep on matching on it”. It means, “I’m not sure what scope a match on a document will be in, so check Content / Name / etc.”

Thanks for coming back to this, Jim. :slight_smile: However I don’t get it: Let’s think of a document which has one term in the body and another in the Spotlight comment. When I choose “all” in the search for the one term and the other term at the same time - shouldn’t DTP find the document?

I don’t want to speak for Jim, but here is my understanding as to how this works. DEVONthink is not searching for one term and another term at the same time, it is searching for one term and only one term, which is (kallaro AND pollari). Setting the search to ALL, DEVONthink searches for (kallaro AND pollari) in Content, (kallaro AND pollari) in Name, (kallaro AND pollari) in Comment,…

That’s why initially Jim’s search found the document, as his document contained both terms, in the Comment. Now Criss would have to answer the question as to if what you are looking for is feasible, or even desirable, but the way it searches currently is consistent.

I expect the results you are wanting is possible with a Smart Group, should that be an option. This one should do it.
Screen Shot 2018-09-16 at 06.42.15.png

Thanks heaps, Greg, for taking the time to explain the search method in detail - I certainly needed it!

I think DTP’s interpretation of “All” is not really the most common way of thinking: Isn’t it more natural to assume that the “All” applies to each term? After all, no explicit boolean operator is used, just a space. Furthermore, DTTG finds the document when entering “kallaro pollari” without any problems…

Anyway, at least now I know why this happens - and I don’t need to worry that something is broken. :slight_smile:

In regards to the suggestion with the Smart Group:

Clever idea! However it doesn’t seem to work: When I create the Smart Group exactly as you showed DTP very, very, very unexpectedly puts the two “All” rules into one “All” rule!!! This is rather ironic, because the only reason to have two separate rules here is precisely to find documents with terms in different document areas - and DTP secretly prohibits this! Bug? Or maybe once more something I just don’t get the logic of… :confused:

PS: There is another way to do it though, via the search window; your suggestion with the Smart group brought me to it: In the Search window type one term (kallaro) into the Search box at the top and use for the other term (pollari) the Advanced dialog.

Try this-option-clicking on the ‘+’ sign on the first line of the smart group will let you create nested, or compound predicate statements. So in the example pictured, I option-clicked on the ‘+’ with the arrow twice to create two nested conditions under the top level ‘All of the following are true’ condition. One can create some pretty complex smart groups as the nested conditions can contain their own nested conditions. In other words, everywhere you see a ‘+’ sign you can option-click to create a compound predicate.
Screen Shot 2018-09-17 at 06.21.51.png

Actually, with DEVONthink macOS, a space is an implicit AND boolean operator. From the Pro Office manual:

As to how search works in DEVONthink to Go, as Jim would say ‘macOS is not iOS’!

Clever trick! Thanks Greg, works a charm. (Still, it is very strange that DTP puts two terms from rules on the same level into one rule although this changes the meaning of the expression! - Looks like a bug to me…)

Anyway, I know about workarounds for the issues we talked through in this thread, so thanks again for your help. :slight_smile:

A future version will definitely revise this.

Cool! Very promising! :slight_smile: