recommendations for organizing database for nested searches

Hello,

I want to use DT as a database to store notes on scientific journal articles for two main uses: 1) as a database with powerful search options, 2) AI features. Use #1 is much more important.

I envision having each journal article as a group so everything from that article is together. Within each paper group I would have sub-groups common to all papers such as: experimental design, analysis methods, experimental results, etc. Below is an example database with three papers. The letters below (e.g. “A”) represents a bunch of words (my notes) that might represent a concept (e.g. “A” is a bunch of words that might represent the concept A).

group: Paper 1
-group: experimental design
----note: “A”
----note: “B”
----note: “C”
-group: analysis methods
----note: “M”
----note: “N”
-group: experimental results
----note: “W”
----note: “X”

group: Paper 2
-group: experimental design
----note: “D”
-group: analysis methods
----note: “M”
----note: “Q”
-group: experimental results
----note: “Y”
----note: “Z”

group: Paper 3
-group: experimental design
----note: “A”
----note: “D”
-group: analysis methods
----note: “M”
----note: “R”
-group: experimental results
----note: “U”
----note: “V”

It seems this structure would be great for AI features in DT. For example, I can look at the experimental results from paper 3 and see what other groups DT believes is associated with this experimental result.

However, I am more interested in using DT as a database to search for information. For example I want to know for experimental design tag with text “A” and analysis method tag with text “M”, what were the different results and then being able to search through these different results. I am not sure of the proper term but maybe nested searches or subqueries - but using BOTH group/tag information in combination with text at EACH level of the search or query. For example:

(1)Search experimental design tag for text “A”, and within these search results be able to search through the different experimental results notes.

(2)Search analysis method tag for text “M”, and within these search results be able to search through the different experimental results notes.

(3)Search experimental design tag for text “D”, and within these search results search analysis method tag for text “R”, within these search results be able to search through the experimental results notes.

As far as I can tell DT and Ammonite cannot do this (Ammonite can do subqueries but, while all the queries can use tags, only the last query can use both tags and text). With the database as above, it seems to me the best way to do this is to run a search for experimental design tag, text “A”, record all the relevant papers (paper 1, paper 3), then do another search restricting the search to tags with paper 1 and paper 3. But with 1000’s of papers this seems very inefficient.

This seems like it should be a pretty standard thing with database software. Very easy to do in something like Excel. Maybe I missed something somewhere with DT? Anyone have suggestions for the best way to do this? Maybe either structuring the database differently or using the above database but using a different search strategy?

Thanks!

I believe you can do queries employing each of the three models you listed. For example, the “advanced” parameters of the search panel could be set up to do the following, which uses your third model (as far as I understand it):

Clearly, you’d want to test this on your own structure and data.

To set up this search, I opened Tools > Search, set “Search for:” to All. The search field at the top of the panel is blank. I opened the Advanced panel. On that panel, I start entering the queries. To enter a subquery, hold down Alt/Option and the “+” sign becomes an elipsis. Press that and the subquery starts.

There are two approaches to the last subquery. You could enter the text in a subquery in the Advanced panel, as shown, or in the text field on the main Search panel. If you put the text in the Advanced panel, then leave the text field on the main Search panel empty and press the magnifying glass button next to the “Similar Words” button in order to activate the search after you’ve set up all your Advanced parameters.

Note, I believe the group structure you outlined affects the queries only if you allow groups to be tags (Exclude Groups from Tagging is OFF for that database).