Severe problem with Spotlight Indexing (Catalina)

I had a severe problem with Devonthink 3 and MacBook / Catalina regarding Spotlight Indexing.

It was turned “on” (Database properties and Database Folder not excluded from Spotlight Preferences / Privacy).

What happened was that EVERY saving process of a document on the computer (Word, Pages, Photoshop) froze for 20 seconds. I could not select a folder as saving location, in activities process - I believe it was com.apple.appkit.xpc.openAndSavePanelService - turned red for 20 seconds, and then I could continue to save. This was annoying and several calls to Apple Support and reinstalling everything from backup and then program by program by hand did not help.

I could not figure out, which program would be able to delay EVERY saving process for 20 seconds … until I turned Spotlight index for my Devonthink Database off and excluded the Folder of the database from the Spotlight index by turning privacy for that folder on in Preferences/Spotlight.

It took me a while so I wanted to share here. Is this a bug? Is this widely known?

1 Like

This is only the third report, and it’s only happened on Catalina.

until I turned Spotlight index for my Devonthink Database off and excluded the Folder of the database from the Spotlight index by turning privacy for that folder on in Preferences/Spotlight.

It would be better to do one or the other, not both. Adding two variables at once makes it unclear which has the effect.

Off course, but I was not acting as a developer, I wanted to end this abnormal behaviour by any means. I had an argument with Apple support because I doubted there can be a proram which has the power to screw up any other program running on the machine. Yes it can, I learned, and it happened via the Spotlight Indexing. It froze every saving for 20 seconds.

First I switched off the database spotlight indexing within devonthink, and i got the feeling this solved the problem. Then I put the database folder on privacy to exclude it from spotlight. And whatever which solved my problem, its solved. I can use both devon and my mac again.

Did you disable Spotlight for a particular database?

In File > Database Properties, what are the words / unique words counts for each database you disabled Spotlight for?

Thanks!
So this is the only database you disabled Spotlight for?

And do you really have almost 11,000 tags??
If so, that may come into play.

I should have known that when you ask for all - you mean that. For me it meant “the important” database. The one I sent contains my documents, so that one (“LDA”) I really need.

Probably I switched on too many auto tags while importing. I remember that during my imports the saving-problem phenomen started to develop, first with a little 5s delay, and at the end with full 20. If you asked me - I would not need all of the the automatic tags, I find them fascinating but never used them. I do use 30 manual tags or so which are important for my workflow.

So here are the other two databases which I don’t really need. The second (“LDB”) contains the files on my computer - indexing only, not stored in the database.

And the third (“LDI”) contains my RSS feeds. This one I do use regularly but a bit reluctantly because FEEDLY does a better optical and better user experience for reading RSS feeds.

I suspected you had RSS feeds somewhere and the third database confirms this.

In Preferences > RSS disable Convert categories and hashtags to Tags.

I would also suggest taking a step back and deleting extraneous tags - on LDA and LDI for sure - to see if the behavior improves.

OK, thanks for your thoughts and advice. I definitely appreciate it. I will keep you updated.

You’re welcome :slight_smile:

I switched on Database Spotlight Imaging for my Documents database, deleted all Tags except the manual ones I need, and switched off Privacy for the Database folder in Apple Preferences / Spotlight.

So far everything smooth - no problem with saving.

How many tags did your database contain?

Screenshots are in this thread a bit further up

Thanks for the hint! Actually a lot of threads, maybe the Open/Save panels can’t handle such an amount of tags (anymore)?

I finally realized this is what has caused quite a problem on all my Macs. This has been wrecking my life for months and I finally figured out it is DT that is the root cause of the issue.

The standard open and save dialogs take about 10 seconds to appear, as com.apple.appkit.xpc.openAndSavePanelService.xpc hangs. Now I see that in the past I did Spotlight integration without realizing that I was going to inject about 20,000 tags due to the RSS feed into the Finder tag space.

Found it here: https://discussions.apple.com/thread/250652926

It seems there is no efficient way to delete Finder tags through the UI, now looking to see is there a scriptable way to do it. I’ll share my findings here. If anyone has already figured out how to delete tags en masse in Finder, much appreciated.

Probably a good idea to have a very clear warning in DT3 before injecting so many tags into Finder’s world!

  • Eric
1 Like

Ok, it there is an easy way to delete all the tags: Finder:Preferences:Tags, select all the tags and click “-”. It takes awhile but seems to work.

Hi folks,

Today I find I’m very probably in the same boat…
Must say that it is very hard to really confirm, reason being that I had a OWC Aura Pro M2 ssd in my 2015 macbook pro, at a time I had extreme problems and issues, such that OWC has replaced the Aura Pro under warranty.
Since then things became better, but…
Very recently I see a lot of errors flying by in the (sync) log. At the same time I also realize that syncing some DB to either iPhone or iPad is probably not very wise since they are too big… and fail due to storage space limitations… (iPhone 256GB, iPad mini-4 128 GB)

I am wondering and not sure what has changed since the openMETA era, but I thought that macOS somehow sinced the Finder tags…

In the same way, I find my Finder tags overwhelmed with very long uuid like tags that I don’t know where they come from.

Looking around accros my systems to see if any have the settings wrong (RSS - convert tags)

Can someone confirm whether tags are synced accross machines?

Definitely for me.

Are you using Finder Tags outside of DEVONthink?

And yes, tags are syncing between my Macs here too.

Christian and Bluefrog,

Thanks for chiming in.
I do realize that my tags already got messed up a long time ago.
Probably the Preferences > RSS > convert categories and hashtags to tags has been the culprit, but it took me too long to stop and search for a solution.

I’m now looking into a possible solution using Brett Terpstra’s Vitag - link but I’d rather avoid going down that kind of rabbit hole solutions.