I recently added an applescript that creates a simple smart group associated with most new groups I create (it just looks for items in that group with a red label). As a result, most of my groups have a smart group attached to them and I use a lot of smart groups in general. However, now when I try to scroll past groups in the sidebar, the response is sometimes sluggish even when the smart groups are collapsed. It can take a second or two to view the contents of groups that contain a lot of smart groups, even if I am not viewing the smart group contents. I imagine that this might be due to the smart groups all actively updating their searches to remain up to date. I donât see the same sluggishness in databases without a lot of smart groups.
Is this sluggishness something I should expect if I am using a lot of smart groups? Is there any way to improve it, such as only updating the smart group search if I am trying to view the smart group contents?
For an explanation about why there is a strange clause at the very end (kind is any document and kind is not any document), see this post.
Note that I am not talking about opening these smart groups to view the contents, but just scrolling past them or expanding a group that contains the smart group.
I was confused by the self-contradicting condition. âkind is anyâ and âkind is not anyâ can never be true. So this condition doesnât select anything (but something like it is perhaps needed as a workaround).
Yes, it is a condition that will never select anything. I found that adding some extra condition was needed to act as a workaround to get the behavior I want, which is to see the child groups of the smart group appear in the sidebar. I didnât want to add a condition that would ever find anything. This one works well because it will never find anything. The need for this workaround is supposed to be fixed in the next maintenance release.
I want to revisit my original question here, because we got sidetracked on the âkind is any document and kind is not any document issueâ. That issue is now resolved and no longer within my smart groups.
The issue I still have is that opening a group that contains 9 - 10 smart groups takes several seconds in the sidebar, just to see the list of smart groups within the group. In addition, if 2 - 3 high level groups that each contain 9 - 10 smart groups are visible in the sidebar, it will hang when I just try to scroll up past the visible smart groups, even if the smart groups themselves are not visible. I am not opening the smart groups in these cases â they are just visible in the sidebar. Here is an example:
Each of these groups contains about 10 smart groups and scrolling up past this list takes several seconds and hangs, even if none of the smart groups are visible or open.
Below is an example of one of the smart groups rules. It is admittedly a little complex â is this the issue? Is there a limit on how many smart groups can be created or how complex they can be? I imagine that it is trying to perform all of the searches in the visible smart groups in some way and this is taking time (though that is just a guess) - Is there a way to only get the smart search to refresh when I try to open the smart group? Thank you
A screenshot of File > Database Properties would be useful. In addition, which version of macOS and of DEVONthink do you use? Does optimizing the database (see File > Optimize Database) make a difference?
It is a rather large database. Here are the properties. It would be very difficult to split. I will also try Optimize database, which I donât do very often.
Is the huge amount of smart groups really necessary? 10212 out of 58181 items in your database are local smart groups. A performance penalty therefore is not really surprising.
For me, yes, unfortunately. I didnât realize there was a penalty for having more. I thought they would search only when I opened the group, and not when I just scroll over the group or in the background when I scroll over a parent group. Why does a local smart group use resources when I am not opening the group? Is there any way to turn this off so that they only refresh the search when I open the group to see the contents?
If not, would upgrading my mac help at all, or is this a limitation regardless of my computing power? Thanks
Of course, I donât retype the text in the search field every time I need to search. I use a string (text expansion). In the case above, r space space. r for recent.
So there is no need for a smart group. The search is executed when I type the string.
This person has about 10000 smart groups. Hard to manage with keyboard shortcuts, I suppose
OTOH, I doubt that this number of smart groups makes sense at all. But thatâs another question.
Can you help me understand the issue with having a lot of smart groups? I have so many because I automatically generate smart groups for a lot of situations that I find enormously useful. About 1/3 of my groups automatically generate a smart group for items with red labels within that group (which pulls key documents) and items with certain text in the name that I use to designate meetings. I have groups for different people and this way I can easily find all of my meetings with that person throughout the database (not just in that folder). I find it convenient to always have a quick shortcut to these items. I didnât think it was using any resources just to have the group sitting there - I thought it would only use resources if I clicked on the group and the search was run. I also date a lot of my items and automatically generate smart groups for certain types of items by year. It adds up in terms of smart groups, but in terms of organization, it is working very well for me, or would be if the searches didnât consume resources unless they were activated.
I could potentially save a keyboard shortcut that would put text in the search bar to search the current group for red labels, but this wouldnât work for some of the more complicated automatic groups, like the years and others. Even the one that pulls meetings uses the personâs name (which is in the name of the group) in the smart group search, so I canât just paste the same request into the search bar each time.
If increasing the RAM would help, I would consider upgrading, but it would help to understand what is known about the limits of smart groups (how many is too many for example? or are there situations that keep the smart groups but use fewer resources when they arenât active)
In actuality, there are not only thousands, there are over 10,000 (in addition to over 10,000 groups). So is your setup full of thousands of items you have no current use for?
Yes, there are more than I thought. I use them as a way to dynamically create an organization structure. For example, if I have 100 proposal groups and the groups have years in their metadata, there is a smart group generated for each available year - so each smart group gives me all of the proposals from a particular year. I am using them in the same sense that I would be using a group of non-smart groups organized by year that I had manually filed. I am always using the organization structure even if I am not using the search. This may not have been the best way to set things up, given the resources that smart groups seem to take up, but it has worked great for me in the sense that I donât need to manually organize anything - the structure is created by a script. I do this for all kinds of metadata, years, organizations, countries, topics. It adds up quickly, but is extremely useful. What I didnât count on is that smart groups seem to use resources all the time (I am still trying to understand how and what the limits are).
Iâm just expressing an opinion here. But seeing 10000 smart groups along with 10000 groups and about 40000 documents looks like a case of over-engineering to me. Especially in connection with the screenshots showing the names of the groups and smart groups. You have on average four (4!) documents per smart group.
For example, with the screenshot here, you have a smart group search in a group named âdd.presON.drawerâ, and in its first âAllâ condition you are asking for non-documents whose name matches â[-drawer]â and âpresONâ. Which makes me think: Why does the group youâre searching in contain âdrawerâ and âpresON.â in its name, and then you put (perhaps?) other groups and smart groups and tags inside it also containing these name parts? Then thereâs the âmyType is not _SGâ, which I boldly take to mean âmyType does not indicate a smart groupâ â but youâre looking expressly for smart groups in the same condition.
Then, in that same smart group, you have the identical condition âYear is 2020â in the first and the second âAllâ condition. While it is not incorrect, it is too complicated.
And the document tags should be âI.pres.@final." or "I.pres@example.â. Does every document in the group youâre searching have either one or the other? And are these tags used anywhere else?
So, these smart groups and tags and names and what not come into being automagically?
What I can see here looks to me as if you are trying to recreate a relational database with SQLâs VIEWs and GROUP BY queries in DT. To stay with that image: A materialized VIEW, which a smart group in DT basically is, will consume resources. If I recall correctly, smaller smart groups are continuously updated so that the user doesnât have to wait for it when they open them. That might be what is biting you here.
