Interesting comments. But we have very different perspectives about the working environment of a database such as DT Pro.
I’ve used a Windows database application that, on its face, met your specifications for a “usable” database. I hated it, because although it unified the searching experience across everything on my drive, when the time came to work with my data everything was again fragmented into different working environments. And it was lacking in a host of features provided by the DEVOnthink applications that tie together the information contained in a DT database, from the glossary and related word analysis features to the AI support.
I rarely use Spotlight, because it seems draggingly slow to me, and the results are presented as what seems to me a fragmented and unfriendly environment. It’s a series of lists, that’s all. All I can assume is that each of the items meets the search query, but all I have to work with is a list of names and file types, which have to be individually opened in a parent application – at which point my working environment fragments all over the place.
I’ve never used nor advocated DEVONthink databases to gather together all of the files on my disks. Over the years I’ve lovingly accumulated collections of files reflecting my professional and personal interests.
My databases are topically organized, for several reasons. First, I’m spoiled; I want very fast searches, often just a few milliseconds – in other words, orders of magnitude faster response than Spotlight. But I’ve got many hundreds of gigabytes of files, and were these to be dumped into a single database performance would be unsatisfactory. Second, and very importantly, when I design a database with a focus of relationships among the component documents, the AI features become much more useful to me. Third, I like my databases to be self-contained so that I can migrate a database from my Power Mac G5 to my MacBook Pro, or the on-order ModBook. Were I to create a single DT Pro database on my Power Mac, which has a full terabyte of online disk space and hundreds of gigabytes of data files, it couldn’t be used on a laptop.
So I’ve got a ‘library’ of DT Pro databases, topically organized like the shelves of a library, with a collective content of more than 150,000 documents. They don’t contain all of the files on my computers. Instead, they represent those files that are especially interesting or useful to me for various purposes.
My main database reflects my professional interests in environmental science. technology and policy. It covers a diverse but related group of scientific and technical disciplines, from analytical chemistry to toxicology to conservation ecology to climate change. It covers a broad range of policy issues, legislation and regulations in the U.S., the EU and developing countries. It supports my interests in international environmental science exchanges, and related resources for graduate student training.
Because there are many practical relationships among this collection, the AI features such as See Also are extremely useful to me when I’m researching a topic.
But I’ve split out a related collection of highly specific collections of chemical analytical methodologies, environmental sampling and data evaluation methodologies and risk assessment literature from that main database for two reasons:[1] this collection will actually grown larger than my main database and if combined would severely degrade responsiveness and [2] it would degrade the utility of the AI assistance in suggesting related material by adding too much minutiae to the list.
I maintain other databases for banking, tax and other financial records. That lets me instantly find what I need, but has no significant overlap with other databases for other purposes.
I’ve got a very large database containing years of listserve files about the Apple Newton technology. And so on.
I try to maintain my main database size so that I don’t get pageouts (virtual memory use) on my MacBook Pro with 2 GB RAM. As a practical matter, the database remains extremely responsive up to about 24 million words total content. But as this is also my default database I use it to accumulate new content – especially from the Web – that will ultimately go into other databases. At the moment my main database is approaching 26 million words total and is showing pageouts on the MacBook Pro. Sometime soon I’ll export material to other databases and purge the exported items from my main database. But of course the current database size remains very responsive on my Power Mac with 5 GB RAM, and I expect it to be responsive on the ModBook with 3 GB RAM.
When I’m working on a research and writing project I ‘live in’ my database. My references are literally at my fingertips. I draft my notes inside the database and if I want to check out a term’s use in my references I can select it and press the Option key; there’s a list of other documents that used that term. Or I can select a paragraph or section of my notes and select the contextual menu option See Selected Text; DT Pro will provide a list of documents that are contextually similar, and I can explore ideas in that way. If I need additional reference material I can look for it with the built-in browser or devise a search query, select it and control-click on it to start a search in DEVONagent.
What I love about my DT Pro databases is the working environment and the ‘research assistant’ support for analyzing information content.
Are there weaknesses in my use of DT Pro in this way? Sure. Currently only one database at a time can be open, although it’s pretty quick to switch between databases. There’s no provision for searching across databases, so I’ve got to remember which database contains what I may be looking for – as a practical problem, not terribly limiting.
Because of the Tower of Babel problem of all those proprietary file types out there, and the fact that DT Pro can’t capture text content from many of them, I tend to restrict the file types I put into my database. I avoid use of Word, as .doc files aren’t captured into my database, so I tend to open those in Papyrus 12 and save them as hybrid PDFs for import to my database. I can’t directly read many file types such as PowerPoint, Keynote, Mellel, Excel and so on; if important, I’ll save them as PDF for capture to the database.
The way I work, Spotlight integration isn’t important.
The future DT Pro 2.0 will reduce memory requirements, so I’ll be able to run larger databases on the MacBook Pro without loss of speed. It will have Spotlight integration and that will likely address the issue of searches across databases. I expect that some additional important file types will be recognized for text capture and perhaps at least partial rendering. Multiple concurrently open databases are planned (limited by available memory). And search queries will have the full power of DEVONagent’s search queries (that will be one of the most important features of version 2.0).
Give me a Mac Pro with 32 GB RAM and I could run a database containing many hundreds of thousands of documents. But the capabilities of the AI support I currently experience with my topical databases wouldn’t be as good as my current experience, if only because the AI would have to be drastically revised to filter out trivia (the reason I separated my references containing primary and overview literature from procedure and methodology references). And I don’t expect to see laptops capable of that amount of RAM (and drive space) for the next five years or so.