I have a small database of about 5 million words, 130 groups, 225 HTML pages, 256 images, 50 web archives, and 600 rich text documents.
Recently, the db has gotten really sluggish and slow (and I’m not talking about memory-intensive stuff like OCR). Sometimes, when I click on a group or on a document to display the contents, it takes 10-15 seconds for it to display in the preview window. Then again, sometimes the program responds instantly. Sometimes, creating a new group takes forever, sometimes it happens immediately.
I am a bit tired of the beach ball…
Does somebody know why this happens? I have a 2 GHz Intel Mac with 2 GB RAM, running OS 10.4.11. The other software I usually have running include Mail, ITunes, Word, Firefox, Endnote and some smaller programs. But I think 2GB should be more than enough RAM for all of them. Any ideas? Thanks.
I have tried to just ignore the slowness, which is why I haven’t responded to Christian’s questions until now. But the sluggishness is not going away. Just now, I had to wait for 20 seconds while DTP deleted two PDF files.
I checked Activity Monitor during and after deleting these PDFs:
Total memory is 2GB. Free memory is fairly low, hovering between 50-200 MB. Wired memory is about 300MB, Active Memory about 600MB and Inactive Memory about 1GB.
Pages in/out are between 90K-140K / 17.
DT uses between 2-10% of the CPU during this process.
Currently I am running Safari, DTP, Bookends, Skype, Skim, iCal, Mail, and Mellel. None of the other applications are slow, only DTP.
Some of the operations in DEVONthink are very memory-intensive.
Your data indicates that you are using a significant amount of Apple’s Virtual Memory, which involves swapping data back and forth between RAM and VM swap files on your hard drive. That can really cut down on responsiveness.
There are three or so general approaches to speed things up again:
Close other applications temporarily to free up memory before doing database work. (May not add all that much free memory, depending on the applications.)
Quit, then relaunch DEVONthink. That will usually result in much faster operation. Or restart to clear the VM swap files in use.
Add RAM. DEVONthink stays fast if there’s always lots of free RAM and there are no pageouts.
Your database is very small compared to my main database. My main database runs over 27 million words at the moment. On my MacBook Pro with 2 GB RAM it would sometimes slow down when I was working it hard, in which case I usually quit and relaunched DT Pro Office to get back to full speed. But on my new ModBook with 4 GB RAM it never slows down, and never triggers pageouts that would require using a VM swap file.
The next major release of the DEVONthink applications will reduce the memory footprint of a database, so the need for RAM will be reduced – until, of course, one opens multiple databases.
Given the relatively small size of the database, isn’t it odd that it would slow down during regular operations like deleting a file (or even sometimes clicking on a group? I understand that operations like indexing or OCR are memory-intensive, but it seems that opening a group, deleting an entry, or even starting a new record shouldn’t be memory-intensive.
As to the three strategies you outlined, I maxed out the RAM on my Macbook at 2GB, and I restart the machine every day. I don’t do any memory-intensive work like photo-, music-, or video-editing. It’s all pretty straightforward text-based research stuff.
Since my computer is an old beast, I can’t complain about performance issues. But I got to say that DTP 1.5 is way slower than previous versions.
4gb of RAM, is just A LOT. I think it’s not serious to give that kind of feedback to users, saying that “if you want it to run fast, use a supercomputer” (the only software maker that I know that argues like that is Microsoft). I think that changes in architecture are necessary. Other reason to wait for version 2.
Really? My understanding of Activity Monitor leads me to think that the problem has nothing to do with RAM. For one, his page ins exceed his page outs by an enormous amount, typically 5-9000x. His inactive memory is high, 50% of the total in his computer, so applications aren’t really fighting over the amount that’s there – a lot of information is already cached and being kept in memory simply because empty memory is useless. His active memory is 30% of his total, a pretty small amount.
By comparison, my page ins:out ratio is closer to 1.6:1, my inactive memory is just over 180MB and my active memory is over 340MB. I’m on a G3 iBook/600MHz with 640MB RAM. The database I’m currently using is tiny, true, but I’ve used databases larger than his on this laptop with no fuss and no muss.
The problem is something else, I suspect. You said your computer is a 2GHz Intel Mac; is it a Core Duo MacBook or Mac mini? Some of those, in case you haven’t heard, came with really shitty hard drives that send their read/write heads crashing into the platters with great zest and alacrity. It might be impending hard drive failure no matter the model you’re using, of course.
If this occurs only in DTP, then I’m somewhat puzzled. You might try trashing the app with, ah, AppZapper or something and reinstalling. A pain in the butt, but so is lousy performance.
You might try backing up and optimizing the database, too, although I don’t really know how much of an effect that has on performance. “It feels snappier.™”
Apple and Intel have been upping max RAM in laptops, so when I set the specs for my new ModBook (a tablet Mac based on the MacBook) I got it with 4 GB RAM, 2.2 GHz CPU and 200 GB 7200 rpm drive. Yeah, that would have seemed like a supercomputer not long ago. But now, for Pete’s sake, it’s only a customized MacBook, and not even the most recently introduced model.
So it’s faster than my MacBook Pro with 2 GB RAM and 2.0 GHz CPU, or than my Power Mac G5 with 5 GB RAM and 2.3 GHz PPC CPU. (I might not say that if I did lots of video work, though, as the graphics chip is more limited.)
My single 64 MB Virtual Memory swap file stays empty on the ModBook.My main database takes up 4.55 GB on disk, with a total word count of over 27 million words. DT Pro Office simply runs at full speed all the time.
The database architecture will change and there will be improvements in memory usage.
On my MacBook Pro I found that the “sweet spot” above which I would occasionally experience slowdowns was about 23 to 24 million total words in a database. Because my main database is also the temporary holder for new content that’s destined to be sent to other databases, it would sometimes grow well above that, to as much as 30 million words. When that happened it would experience delays, build 5 or 6 VM swap files and I would export content to other databases and prune the size back.
I’ll probably hold of on that pruning routine to see how large a database I can now run before hitting a lot of pageouts.
Earlier in the 21st century I bought a TiBook with a 500 MHz G4 CPU, 1 GB RAM and an 80 GB HD. I worked it hard for almost 5 years. It worked with DT Pro databases up to about 18 million words, and I got a lot of productive work done with it.
The guys with supercomputers these days are running Mac Pros with 2 quad-core 3.2 GHz CPUs, 32 GB RAM and an awesome amount of drive space. Yes, some of them are running DT Pro Office.
I’m happy with my little ModBook. I don’t even want to test how fast my databases would run on one of those Mac Pros. I might not be as happy, then. I can’t afford one of those beasts. Nor do I need one to get my work done.
At least until Apple’s low-end Macs reach that level of power. Which will likely happen in a few years.
Well, Bill, it sounds like you have a great machine. But that doesn’t solve my problem. And clearly, you can’t be suggesting that everyone who wants acceptable speed from DTP should upgrade to 4 GB of RAM…
I have a 2.0 GHz Macbook Core Duo with 2 GB RAM and a really tiny DB. My DB size is below 1 GB (!!) and counts only about 6 million words.
I do not understand why I am slowed down when simply browsing the content of the database by clicking on a group. And I am not talking about delays in milliseconds. I sometimes have to wait 5, 10, 20 seconds, while the beach ball comes out.
All other applications run fast. It’s just DT that’s slow. And it’s not always slow. When I start it up, it runs fast. But then, later, it slows down. It’s particularly annoying when I want to add material from Safari or Mail, and every time I add some text, I have to wait for half a minute until it processes it.
Unless there’s some sort of software conflict, something sucking up CPU cycles, using up RAM or whatever, I don’t understand why your database is that slow.
I got much faster performance on my old TiBook with a 500 MHz single-core G4 processor and 1 GB RAM, running DT Pro databases that were much larger than yours.
And a database of your size should easily run at full speed all the time on your machine, as it would on my first-generation MacBook Pro with 2.0 GHz dual core and 2 GB RAM.
Suggestion: Make a copy of your database and try a rebuild on it, just in case there could be some problematic files that are choking things. Take a look at the Log after the rebuild to see if any files failed to survive the rebuilding operation.
I work in a super-beautiful but slow Tibook DVI. I have a lot of speed problems, but I have the impression that deactivating Spotlight indexing turns the database faster.
I have a very similar database on a very similar machine, and performance is fine. What you are seeing is not normal DTP behavior.
Could be database corruption, could be a hard disk problem, could be some other program pounding the heck out of memory. But it’s not an inherent limitation of DTP.
Please open Apple’s activity monitor and select DEVONthink. Then do whatever is necessary to cause the 5-20 seconds delays, create a sample of DEVONthink and send it to our support address and we’ll have a look at this.
You might also have a look at the page in/out statistics and at other processes consuming CPU power (especially Spotlight’s mdimporter and mdworker).
