Performance Issues with Full-Text Indexing on Large Databases

Hello everyone,

I’m using Devonthink on macOS with full-text indexing enabled, and I’ve been experiencing some performance issues, especially with larger databases. I hope someone here might have encountered similar challenges or has tips on optimizing the setup.

Problem:
I frequently see the “spinning beach ball,” and the system response slows down significantly. I’ve noticed that only one CPU core is maxed out at 100%, while the others remain underutilized. My databases are quite large—around 30GB each.

Questions:

  1. Is there a recommended size limit for Devonthink databases where performance typically begins to degrade?
  2. Would splitting the database into smaller sections help improve performance? Or are there other optimization strategies that might reduce system load?
  3. Are there any settings or known workarounds to distribute the load across multiple CPU cores?

Thank you in advance for any advice or insights!

Best regards,

Andreas

Which action(s) frequently triggered the beachball? Or did the application start beachballing on its own, without your input?

Certain user actions, e.g. opening a very large epub file in Text Alternative view, are known to cause beachballs, and for understandable reasons. Knowing what’s causing the beachball helps with determining the cause.

It happens:

  • When I start DT
  • When I enter letters in the search field
  • When I scope the search by clicking on folders and subfolders

These are just some examples.
DT doesn’t seem to make use of multiple CPU kernels. It uses 7,3GB Memory, my MacBook Pro has 32GB.
Maybe my databases are just too large for DT.

Probably not, but in any case

  • what version of DEVONthink and macOS?
  • how much memory on the machine, and how much free disk space?
  • Where are the files being indexed–folder synched with a cloud sync service, local, or on a file server? How many? What type of files? What is total size of all the files you are indexing?
  • Do you by any chance have macOS “disk optimization” turned on?
  • Good to provide a screen shot of “Database Properties”
  • Have you done a “Verify & Repair”, “Check File Integrity”, et al. on the troublesome databases yet?
  • Tried any of the suggestions on “Troubleshooting” in the “DEVONThink” manual?

Where did you enable this? Probably somewhere I guess, but I can’t spot that option. Pls remind me.

Hard to tell without knowing the number of items & words as only these matter but not the database’s size. See File > Database Properties

Disabling partial matches while typing should at least fix the search issue.

Here is a screenshot of my biggest database.
What do you think? Too big for DT.
Will splitting help?

You comfortably should have a maximum of 250,000 items in a database. And yes, I would suggest splitting this.
Also, where are you indexing files from?

All files are index from the local SSD. An the database resides on the local SSD as well.

But where, e.g., iCloud Drive, your Documents folder, etc. ?