How to deal with data backup problems?

Can I directly delete the backup data files in the database folder? How can I disable backups or control them? My original database files are already large enough, and the backup data is becoming overwhelming for me.

Backup data should be stored separately from the database. That’s the whole point.

1 Like

Where do you see backup data? I can’t recognize anything from your screenshot.

1 Like

Screenshot 2024-11-23 at 10.04.54 AM
Screenshot 2024-11-23 at 10.05.06 AM

right :sob:

In my index database, the index file data is 34GB, and the two backup files together total 60GB.

Are you finding those backups by looking inside a Devonthink database? More power to you for learning how things work. I believe any manipulation of database internals is un-recommendable outside of dire circumstances.

1 Like

The backups are located within the database files, and I don’t intend to manipulate the objects inside the database files. However, it’s clear that the backups are causing a sharp increase in memory usage when opening the database. Even with my MacBook’s 96GB of RAM, it’s still not enough.

How is that clear? How do you know that DT loads a backup in memory to stay there?

How is that “not enough”? What does activity monitor say about memory usage of DT? Other apps? What does it say about files currently open in DT?

3 Likes

You should not be messing around in the internals if a database. You have also been pushing far beyond the recommended limits of a database. This logically has consequences.

Development can comment if they choose to weigh in but I don’t recommend you delete anything manually. Also, you should keep out of the package contents.

1 Like

Under the same conditions, I compared the database file size, backup file size, and memory usage at different time points. I feel that the rapid increase in memory usage is caused by the growth of backup data. (I created the index database, and when adding files to the source folder, the index data size increased only slightly, but the backup data size grew significantly.) Of course, I am not a technical expert, so this statement lacks rigor. It is merely a judgment I made as a user. I would appreciate any advice or suggestions from everyone.

THX for your reply.

The backups do not affect memory usage at all, it’s just the pure size of your database (amount of items, words, metadata etc.) that’s causing the memory usage. And memory usage might increase while using DEVONthink as not all necessary data is immediately loaded right after opening a database.

“Feeling” is not useful if you’re dealing with IT. Facts are what matters, like in real life.

FYI: I did check, and my running DT instance doesn’t have opened a single backup file, for any of my six or so databases. None. So, these backup files can’t have any influence on the memory usage of DT. None.

See lsof on the command line.

2 Likes

My suggestion is chill out and just enjoy your Mac and DT, both of which manage memory exceptionally well with no effort by the user.

If you look back through years of discussions here, running out of RAM or unified memory are exceptionally rare.

Unless you are having major software performance issues with DT there is no need to play memory detective.

Incidentally - if your computer does get into a RAM crunch you will start to see usage of memory swap noted or compressed memory in the “Memory Pressure” section of Activity monitor. If that happens I’ll place a wager that Google Chrome is way more likely your culprit than DT.

This is where you would look for such evidence. But even “Memory Used” is a bit misleading because MacOS is good at managing memory use depending on the hardware you are using. The same applications on a computer with substantially less physical memory will result in considerably less memory used. No need for you to adjust any settings - MacOS takes care of that for you.

Out of curiosity what does the Memory Pressure section of Activity Monitor show for you?

image

3 Likes

In addition to all the sage advice from others already, remember that the “backups” you are poking into are internal to DEVONthink and are not your backups. See page 19 of the 3.9.8 version of the “DEVONthink Manual”, section entitled “A Word About Backups, Internal Backups”:

INTERNAL BACKUPS

DEVONthink backs up its database index every week, either immediately after opening the database (if the backup is overdue) or during the day when appropriate.

Note: The internal backups are mainly a troubleshooting mechanism for use in the application. Backing up your files, including your DEVONthink databases, is up to you.

Suggestion: Read the other parts of this section to compare what is recommended vs. your backup regime for DEVONthink data.

3 Likes

got it ,THX