Slow response

I am quite new to DevonThinkPro. I have a large collection of teaching materials consisting of pdfs, docs, illustrator files, mp3s, ppts, and a few other odd file types which I use in my teaching. I have indexed everything into a database. The database is 3.2G. I find that every action takes an inordinate amout of beach ball spinning time, even just deleting or moving a file. It is very frustrating. Can anyone offer any advice? Is my database too big? Are there too many different file types. Am I doing something wrong. Any help will be greatly appreciated.

How much RAM does you computer have? If a database is that huge, there can’t be enough RAM :slight_smile:

Thanks for your reply. I have a G3 with 512MB RAM. Is this not enough?

No. If you’re using Mac OS X 10.4.x “Tiger”, then that’s just sufficient for basic tasks (like mailing, browsing etc.).

Yes I am using Tiger. It looks like a visit to the store for some extra RAM. Thank you for your time and advice. I’ll let you know the outcome of the RAM upgrade.

Hi,
I have a similar, albeit not so serious, problem with speed.
I have a MacBook Pro 17" with 2 Gb ram. My main database is about 600 Mb, but very often when I add a text file, or I perform some other operations I get a very slow feedback from DT Pro (1.1.1).
It seems it get worse with the time the db is open.

Any other text app I am using is so fast on the MacBook pro, but DTPro makes me wait sometimes quite a few seconds.

In the past I used to have the “Automatic Wikilinks” option on, but I stopped doing that because things were too slow. Now that option is off.

There is anything I can do to speed things up? I did a “rebuild Database”, with no effect.

I really like DTPro, but this slowing down effect is quite frustrating.

Some of the operations such as Classify are very memory intensive for a large database. That’s not surprising, as DT Pro is “looking” at the text patterns in the document being classified and comparing that to text patterns for your groups, so that it can suggest an appropriate location for the document.

I’m running a considerably larger database on a MacBook Pro with 2 GB RAM. After standing the database on its head repeatedly over time things can slow down. The quickest remedy is to simply quit DT Pro and relaunch the database.

The reason for such slowdowns is that when there’s not enough physical RAM to supply needed memory space, Virtual Memory comes into play, moving data back and forth from disk swap files to RAM. Unfortunately, disk access is very much slower than physical RAM access. :slight_smile:

I can run that same database for days without a single pageout on my PowerMac G5 dual core 2.3 HGz machine. But it’s got 5 GB RAM, which is more than enough to handle that database.

In the near future we will be seeing Intel Macs (perhaps even MacBook Pros and iMacs) that can hold more than 2 GB RAM.

Perhaps more importantly, Christian has noted several times that DT Pro 2.0 will result in smaller memory requirements for databases and so will be able to handle much larger databases.

I understand the points you said and I thank you for such an immensely useful product.

Can’t wait to start using version 2.0! Enroll me in the beta, if you have one…

Thank you.

Beta testing on DT Pro 2.0 hasn’t yet started.

Another comment on database size: over the years i’ve accumulated many, many gigabytes of files.

I’m somewhat spoiled. I like “instant gratification” when I ask my DT Pro databases to do something. I’m happy when a search is completed in a few milliseconds, but would get irritated if it took several seconds.

So I separate my DT Pro databases to cover topical interests that have little or no overlap.

Example: My main database focusses on professional interests in environmental science and technology, related legal and policy issues, and international environmental science exchanges – the latter interest often related to graduate student training in developing countries.

I keep that database size in the range of 20 to 25 million total words. That’s a lot of information. But on my computers a database of that size runs very quickly.

As my main database is also my default database, I use it to collect new information on a variety of topics, some of them not closely related to the focus of that database. So I periodically create new databases and dump material over to them, leaving more space for my constantly growing collection of primary interest reference material.

One of those other databases holds a very large collection of files about the Apple Newton. Another holds financial information. Still others deal with projects I’ve worked on involving collection of information about the effects of hurricanes Katrina and Rita on the health care infrastructure in Louisiana. And so on.

For my own purposes, I don’t Index my hard drives. My PowerMac has a terabyte of online storage. Indexing my PowerMac drives would result in a very large and probably slow DT Pro database. I wouldn’t be happy. Another aspect to consider is that the AI features such as See Also work best when there really are recognizable contextual relationships among some of the files in the database. By splitting my databases into topical collections, See Also works very well for me, and very quickly, as I’ve “pre-distilled” my databases by topical interests.

And of course if I had Indexed my PowerMac’s drives, I wouldn’t be able to work with that DT Pro database on my MacBook Pro. The referenced content couldn’t possibly fit on the 100 GB drive in the MacBook Pro, and the link paths for the PowerMac database would be broken and useless on the notebook computer. By making my databases self-contained, I can easily move a database from my PowerMac to the notebook and work on it on travel or show it off to a colleague at a distant location.

That’s not to put down Indexed databases; I’ve got a couple. An Indexed database requires less memory to load and is a good choice when one will need to edit files such as MS Word .doc documents.

Hello, I haven’t been here around but recognize you (and a few other names). A question:

What’s the best way to split a large database, as you described above, and “dump” a portion of it into another? I assume, select a group, and then Export files and folders, and then re-import? Does everything get moved over properly that way?

My PowerBook G4 has a lower memory slot failure, so I’m down to 500 MB RAM, and my database is ALWAYS slow now… I think until I get a new Macbook Pro with 2GB RAM, I’d like to split down my database size.

Yes, select the groups/documents that you would like to split out and choose File > Export > Files & Folders.

In a new database, import the exported material.

You will want to delete the exported material from the original database. But I suggest making a backup of that database before deleting the material, just in case you change your mind. Anyway, a backup is a Good Thing.

If you export a group containing replicants of items in a non-exported group then replicants won’t exist for those items when the former group is imported. Similar issue with duplicated items. And items with aliases and Wiki-style links could be affected, too.

One thing you may want to try is exporting a complete database, creating a new one, then importing the exported content (and moving it to the top level). Run Database Properties (option-command-P) on both databases and compare the results.

When I’ve checked my primary database like that the number of items differ and it’s not apparent why. Running the Export > Listing… script on both databases and comparing the results (not a straightforward task) might give a clue where the differences are. I’ve successfully done that with a smaller database but not my primary one since the script used to hang on it. That bug’s fixed in the current beta but now there’s a minor one: the output file for the listing has to preexist or the script won’t save the results.

I’ve also tried figuring out database differences by comparing exported hierarchies of an original and its imported copy.

Because of the possible export/import discrepancies I’d sometimes split a database by making a complete cloned copy the original, rename it, then selectively delete items from the original and its renamed copy.

This sounds safest. Better to have a few extra unwanted records, than to lose records that were thought to be kept.

With modifications like I suggested, Eric’s scripts for console jockeys could be useful for obtaining more detailed item summary info about database records. I welcome any tools and/or methods for discovering potential metadata-related issues, which is just as important for me in certain databases as the actual data content.