DA 1.6.1 CPU/disk usage control?

I’m getting started using DA 1.6.1 under 10.3.8 on a 1GHz PB 12" w/768 MB RAM booted from a fast external FW LaCie HD.

I have encountered a couple of situations where CPU usage and disk activity were so phenomenal that the entire system slowed to a crawl (overall CPU usage consistently at or close to 100 percent, massive disk activity including paging), app switching was difficult, window redrawing incredibly slow, and DA itself was too busy to respond to user input within less than a couple of minutes…

Can limits be set so DA does not take over the entire Mac and leave no choice but a force quit in the middle of a search?

Are there known software interaction issues otherwise?

Phil:

I’m pretty sure you have a FireWire 400 connection to the external hard drive. Although that’s pretty fast, the overall performance may not be as fast as your PowerBook’s internal hard drive.

If you’ve sent DEVONagent out to the Web to inspect a very large number of pages with a complex search strategy, it can take a while. Remember, DEVONagent is downloading relevant material to a cache on your computer, and digesting and organizing the information.

With the Internet (fast scan) search set and a simple query, DA should take only seconds to do the job.

But with a big search set, a complex query and instructions to drill deeply through pages, patience is in order. It will likely take more than two minutes. I try to time big searches to concide with break time.

If you change your mind and want to kill a search, just click on the Stop button. :slight_smile: If your CPU is pegged out, it may take a while to respond.

Do you notice especially high CPU usage with particular search sets?

Hi Bill:

It really got bogged down in a deep scan seeking a phrase and two words. It was at 281 docs collated out of 7K+ pages when I bailed out…

I wonder if DA could use a virtual disk in RAM rather than accessing the drive? Or be directed to use a second drive for the caches? You’re right on the FireWire spec.

I’m looking forward to migrating back to my desktop with two CPUs and a 2-drive RAID.

Phil

Phil:

I’m envious of your desktop setup.

I think DEVONagent tries to do a lot of caching in RAM. But with both DEVONthink and DEVONagent open, my 1 GB RAM can go south fast, and then we’re working in virtual memory. At the moment, I’ve got 309.5 MB inactive memory, and only 13.7 MB truly free memory. So if I click on a button, there’s going to be a lot of memory paging.

When Tiger comes out I’ll be looking at a 20 inch iMac. Current specs allow 2 GB RAM, and I’ll max it out. That will help, but I would expect to see virtual memory in use anyway.

In a few years we’ll probably have Macs with a hundred GB or more RAM and a terabyte or more HD space. Then Christian will add so much AI capability that we’ll be wishing for a terabyte of RAM. :slight_smile:

Bill,

I’m beginning to see… so the best appoach might be to try to set things up for the Mac to use space on a separate physical disk on a separate bus for its VM paging…

DA and DT are very deceptive. They pose as little apps like the competition’s, while there is actually a radical new technology at work in there.

Hmmm. What if you could allow the sharing of hardware resources between DA/T users on broadband. DTtech would then provide a centralized ‘boosting’ service that would manage the safe coordination of distributed resources, ‘leased’ on the fly on an ad hoc basis between licensed DA/T users.

Just a wild thought. I don’t really know what’s in there, except that I surmise Christian must have found a brilliant way to trade RAM and CPU cycles for time, from what you describe.

Correct me if I’m wrong, but DA does not store cache to disk, nor does it have a setting to let you do so.

Should have noticed this query earlier. Sorry.

DA has an option to clear cache on quit. If that option isn’t checked, the cache is in fact maintained on your hard drive.

That could be useful to know, e.g., if you’ve had a search interrupted or an Internet connection dropout. In such a case, you may be able to recover material from the cache by selecting the All Cached Pages set and entering your query to that set.

If all is well, you can always flush the cache files by choosing DEVONagent > Empty Cache.

Great point. The ‘search in cache’ feature is great for managing updated sets of results, as in tracking issues, too.

Regarding VM optimization, there is an interesting page here: http://www.resexcellence.com/hack_html_01/06-01-01.shtml.

Disk usage is my only real-life bottleneck. This might really help DA and DT fly, as well as other disk-intensive apps such as Mailsmith…