Need usage advice from DT pro users

My Documents Folder is very well organized.

I bought DT recently to improve searchability and access to content.

I need advice from DT pro users before I will trust my well-laid out system to anything or anyone.

BACKGROUND TO MY QUESTION
I put a lot of time into organizing my life into nine main categories and then stepping down from that into what you will see is a grand total of 1000 folders in all. The system is numerical and categorical, almost like a thesaurus. Then I use the Jeter numerical system to start the name of every document within those folders. [I would only need the Jeter numbering, and not the cumbersome long title, if I could search quickly in DT, and thereby gain the added privacy Jeter numbering gives you.]

My Documents root directory has high-level 100, 200, 300, 400, 500 … 900 folders.

Within each high-level folder are #10, #20, #30, #40, #50#90 folders. Within each of these are the final ##1, ##2, ##3, ##4, ##5 … ##9 folders.

MY QUESTION
I question the viability of creating and maintaining my current structure within one DT database versus creating nine (9) different databases for each high-level group.

I view the latter choice – nine different databases – as negating the reason to use DT in the first place: I can’t search across them.

Further, my Documents folder is over 9 GBs.

I fear corruption and reduced speed if I go forward with one database.

Are my fears well-founded? Can DT handle this load?

I am testing DT Pro, and I am uncertain exactly how DT is set up at this time. you will need to examine DT more closely, including the Preferences, to see how this transfers.

with DT Pro you do not need to move your files at all; you can link to the files while leaving them in place in your Documents folder.

or, you can move the files inside the database folder which means that if you use “Show Package Contents” on the database you will find your files moved but not changed in any way.

or, you can move the files into the database itself.

it sounds like you might be most comfortable with the first option, keeping your carefully planned structure intact while adding the search and other features of DT to it.

It’s great that you’re testing DT Pro. I realized now I indicated in my initial post that I was calling for DT Pro users and there are none at the moment: only beta testers.

I, like everyone else, am using DT Personal.

Maybe I should just hold off until DT Pro is released. Maybe that will answer all my questions.

I’ve had too many years getting fancy with organizational systems not to go ordinary, generic, and easy to recover now. Huge chunks of my life are now locked in arcane, dead software. Unacceptable.

My entire life is PDF’d. What I was hoping is that DT would give me better access to what I was collecting about it.

Thanks

did you do this? Have you confirmed that the features you need are not in DT personal?

wdfs:

I’ve been using DEVONthink for more than 2 years, and have been running my database exclusively under prerelease versions of DEVONthink Pro for a good many months. So I’m very familiar with DT Pro.

My database contains 12,800,000 words of reference documents (many thousands). File formats include PDF, HTML, text, rich text and images. I’m currently using 772 groups (analogous to folders), including subgroups.

I’m using a 500 MHz TiBook with a 60 GB HD and 1 GB RAM (it will be 4 years old next March). Most searches are very quick, and the contextual recognition features such as “See Also” are acceptable in speed. But as DT will add more complex search strategies and increased artificial intelligence capabilities in future versions, I’ll be looking for more horsepower down the line.

I’ve imported all my PDF documents as external links. Because you’ve put a lot of time and care into the organization of your files, I would recommend that approach for your initial trials of DT Pro, setting your preferences to import PDFs as external links and to use the pdftotext option for capturing text into DT. Your files and their structure will remain untouched, but you will be able to search and access them with much greater facility than you can at present.

You didn’t specify the computer you are using. I can’t really tell whether your database is larger or smaller than mine, depending on how your PDFs were generated.

Although DT Pro allows multiple databases and in the future will be able to open multiple databases concurrently, I suggest you plan a single database initially, containing all of your files. From DT Pro, select File > Import > Files and Folders (either for your entire collection, or for subsets). Initial import will take some time, as DT will have to read the text of each PDF file.

You can continue your file naming and organizational structure under DT Pro. I say that with some irony, because I think that you will eventually discover that DT is really the solution for the problems that the Jeter filing system was designed to overcome!

DT Pro will soon have two-way synchronization (currently, one-way synch) between database groups and external file folders, so that as you add PDFs to your current folder structure, the DT database will be updated.

My guess is that you will love your DT Pro database. If you don’t, however, and run your trials as I suggested, you can walk away with your current files and organizational structure intact.

If you can’t wait or want to try preliminary tests, use DT PE 1.9a to import a subset of your PDFs, using the external link import preferences. Play with the results and see what you think. Be sure to try “See Also” and Keyword features. Note: I expect the upcoming Tiger OS release will enable DT to improve user views and searches of PDF imports, more closely resembling views in the Preview application.

I deeply appreciate the thought and time both of you expended to answer my question.

I am going to start with a subset of my Documents Folder and use the external link option and carefully go through the options and prefs available to me. Probably by the time I finish this exercise, the Pro version will be available to me.

For the record, I have a G4 dual 1.25 tower with 1 GB of RAM.

I do know that accessing multiple databases at once is not available in DT Personal 1.9a so I’ll have to wait for the Pro version for that. Nevertheless, exhausting what I’ve got will be a useful exercise.

Frankly, I’m suffering from the problem that this program sounds a little too good to be true. I’ve wasted thousands of dollars in difficult high-end database programs over the years, and I’m now looking for something that I wont have to overhaul, or dump, within the next five.

Again, I thank you profusely for your thoughtful remarks.

I know what you mean. Over the years, I’ve bought all of the major database applications available for the Mac. While I’ve successfully used them for other purposes, non of them was satisfactory for searching and managing collections of reference documents. I’ve played with a couple of databases designed for lawyers, intended to assist searching and management of text collections, but found them too limited (and much too expensive).

I’ve been using DEVONthink for more than two years. The fact that DT development has continued and the product keeps getting better all the time gives me a reasonable level of confidence that they will be around in 5 years. :slight_smile:

Keep us posted on your experience.

Hi,

I just now came across this interesting post. wdfs is perhaps the first person I have heard of who really managed to organize his vast amount of data and even keep it up organized. It is just amazing. :unamused:

My database has now about 3 GB, and I have to say, it is getting slow from time to time (the Pro beta on a G5 double 1.8GHz ). It seems that dividing databases from a size of 2 GB is a good solution, but as Bill said, I would try first of all. As soon as DT can handle multiple databases, your problem is solved, and lookng at the recent speed of development, I would trust that this will happen soon.

I understand your fear to loose data again to another proprietary format, but this is not necessary. Your data can be exported as original files (rtf, PDF, jpeg, ooutliner etc.). I would recommend such exports from time to time as another kind of backup.

As far as strict organization is concerned, I came from the word of relational databases, my archaeological research is all organized in a well designed and normalized db. I am looking forward to the enterprise version with its announced relational capabilities, but still I enjoy the freedom that DT gives me for my data storage and retrieval every day more and more. I hope that you will realize the chances that DT offers you for retrieving any useful data – whether filed in a certain group or not. Bill already mentioned the “See also” button.

Maria