Thanks @cgrunenberg and @BLUEFROG. Got rid of all the tags which all came out of RSS feeds and deactivated tags conversion on import. Smooth flight so far.
You’re welcome and glad to hear it.
I honestly have never seen databases with more than 2000 Tags, so this was definitely eye-opening. 
It can happen very easily if you use DT3 regularly as an RSS news reader.
I’m very selective in the feeds I put into a database, so I guess I’ve never run into this.
I can understand that with regard to a permanent database on a given topic.
What I have found is that aside from all the “serious” uses of DT3 for professional documents and research, it is very useful as an RSS reader for more casual reading as well of mainstream news/personal interest feeds; this is particularly helpful given the new SmartRule features to add tags identifying the source of an item and identifying any other keywords of interest in the item.
So the result is one Smart Group continually updated during the day with all sorts of articles on various topics as well as tags to help further fine-tune which items may be of interest to read.
It can quikly evolve into a very large database, but it is also just as easy to delete older stuff from time to time. It runs all day in a separate window on my Mac and does not meaningfully impact performance of DT3 or of the computer itself otherwise.
for more casual reading as well of mainstream news/personal interest feeds
I avoid mainstream news like I avoid liver. (And yes, I hate liver 
).
I tend to rabbit-trail on my own, following my own curiosities and instincts.
I definitely have this same issue. Except in my case it is MUCH worse. I have 238,642 tags that have been generated from the hashtags in hundreds of RSS feeds; most of which I never read.
I have also seen extremely slowness in file save and open dialog boxes - I believe caused by this functionality interfering with Finder itself.
Now I need to first find a way in DEVONthink to clean up the RSS feeds that I have so that I can eliminate the ones I never use or read, then clean up the tags. I have already told it to stop creating tags from keywords or hashtags, but that only keeps things from continuing to go downhill. It does not correct the current problem.
If there is anything that could be added to the product that would prevent this problem, I would like to hear it. Perhaps storing tags in such a way that they’re in DEVONthink only and not part of the Finder or the OS? Or at the very least a way to have the conversion of hashtags and keywords into tags be prevented with RSS feeds?
Any crazy thoughts?
Well, that’s set in the preferences for RSS feeds. If you allowed over a quarter million tags to be created from RSS feeds and you didn’t notice that was happening, what would you suggest that DEVONthink could have done differently to prevent that?
Allowed? Not like I anticipated so many tags being created. Nor did I anticipate the interaction between large numbers of tags and the OS itself.
It’s not about anticipating anything. The tags were not created overnight. I’m just saying if you didn’t notice what was happening over the weeks, months, years that it took to create a quarter million tags, then what safeguards could DEVONthink do to have prevented this?
As to rectifying the current situation, do you still have all the RSS feed articles in the database that it took to generated 250k feeds? How much of your database is data that you have carefully curated and how much is junk from RSS feeds? It might be easier to move all your important data to a new database and delete the current database with hundreds of thousands of unwanted RSS feed articles and tags.
1 Like
Something that would have been potentially helpful would have been a popup alert when the first 1,000 tags were created from RSS feeds. Perhaps it could have said something like this: “Over 1,000 tags have now been created from the 300 RSS feeds to which you subscribe. Such a high number of tags can result in significant slowdowns in file open and save operations in macOS. Do you wish to update your RSS Feed Preferences to prevent tags from being created automatically?”
If the user clicks , the preference is turned off right then and there.
Even a notice in the help documentation would have been better. There are no best practice warnings in the docs that mentioned that automatically generating tags from hundreds of RSS Feeds can create hundreds of thousands of tags AND that having more than 1000 tags in macOS can slow Finder to a crawl. I like to save the content of feeds that I like, specifically because they vanish from the Internet after a time.
There’s no mention of architectural limitations, which I would really like to have called out. Best Practice guides, things to avoid, things to do to get the best experience.
That brings up a good question:
If I clear out the articles with the tags, will that also clear the tag entries from the macOS file system? If so, great! That should solve the problem, since it is caused by Finder trying to process hundreds of thousands of file system tags. If not, then that will help, but won’t solve the root of the problem, until I can delete the tag list from Finder itself.
If I clear out the articles with the tags, will that also clear the tag entries from the macOS file system?
No, this will not happen. Also, the Tags in the database will not be removed.
There are no best practice warnings in the docs that mentioned that automatically generating tags from hundreds of RSS Feeds can create hundreds of thousands of tags
Development could also comment on this but I would say having 300+ RSS feeds would be an unusually high number. In conjunction with the conversion of categories and hashtags enabled, this situation in not unexpected over the course of time.
The number of feeds isn’t unusual but the huge amount of different hashtags definitely is. What kind of feeds did you add? You could remove all tags by using e.g. the Remove All Tags action of Tools > Batch Processing…, this should remove the Finder tags too.
Hi all,
Thank you very much for reaching out to me to help! I just wanted to respond and let you all know the status of my situation, since you took the time to help out.
I went ahead and deleted thousands of extra articles and several hundred duplicate RSS feeds that had been added through imports from Fever RSS reader. That had a modest impact on system performance. However, things still were taking up to 15 seconds just to get a save dialog box, sometimes longer.
Then I took @cgrunenberg’s advice and used the new Batch Processing option to remove all tags from these Feeds.
That helped more, but I was still left with nearly 7000 tags. I then noticed, in a Finder search for a tag that I knew was not mine, that there was a DEVONthink metadata folder in my library that was holding metadata for the old feed articles that I had already deleted: Macintosh HD/Users/xxxxx/Library/Metadata/com.devon-technologies.think3
I decided to delete the old metadata folder and have DT regenerate the metadata folder. That did the trick!
I let my system run all night removing the extraneous tags from Finder and then rebooted in the morning.
Things are working AMAZINGLY now. That was a huge win!
I still have around 1,064 tags, but performance is now like a new Mac.
Evidently, when the tags in Finder get to be around 5,000 or so is when I started seeing a performance hit. I’m sure that varies depending on system speed, but it’s nice to know the limitations.
It still might be nice to have some sort of warning somewhere that subscribing to hundreds of RSS Feeds may result in thousands of hashtags being added, which could in turn heavily burden Finder performance.
Just a thought. Thank you all again!
The Spotlight index should be updated in the background, therefore changes might not immediately be visible in the Finder. But I’ll check that this does indeed work after removing all tags as suggested.
It’s indeed a bug, the next maintenance release will fix this.
Rock on! Thanks!
Do I get a DEVONthink mug for finding a bug?
Surely a face mask would be more en vogue - it could be imprinted with DEVONthink mug if you wanted 