Importing vs Indexing - a new user's solution

As the subject suggests, I’m a new DTPro user, and still finding my way around.

But through a lot of trial and error, I’ve come to some clear conclusions about my own usage choices that I hope might be useful to another neophyte. Bear in mind that all this applies to DTPro 1.02 - and will no doubt change as newer versions are released.

I had two huge collections of info to sort out, both in utter chaos:
-Category 1 was technical reference materials, quickly out of date, often added to but never modified.
-Category 2 was research and original material I needed to revise, copy from, rewrite, research further etc.

Here’s what has (so far) worked very well for me:

For Category 1 (technical reference material):

  • IMPORT all the info, with prefs set to COPY all non-readable files to a files folder in the database.
  • Use DTPro to grab any new info tidbits to this database
  • Reorganise and cull entirely within this database - i.e. don’t keep any more info of this kind outside of DTPro

For Category 2 (research and original material):
Using DTPro to store this kind of info was actually making it even more complex to deal with - confusion about versions and revisions, and the constant need to use other applications to revise the documents anyway.

But DTPro gave me the ‘meta-Finder’ approach that I needed, to easily move things around, sort, classify, find dupes, etc. So my solution here was entirely different than with the static collection:

  • INDEX all the contents of this info collection, with no importing or copying.
  • REORGANISE the index using DTPro’s tools
  • EXPORT the entire cleaned-up collection back out of DTPro
  • delete the DT database, which has now served its purpose.

At this point, work with the data as before, i.e. with the usual creator apps, and simply repeat the index/organise/export process as needed.

=============

So, nothing earth-shaking or ground-breaking here - but hopefully this may offer some ideas to someone else finding their way out of multiple levels of chaos, as I have been!

SN

SN,

indexing & synchronizing of the upcoming version 1.1 will be more powerful, e.g. indexing will be compatible to Phrase/Wildcards searching (great for static material like references & manuals) and synchronizing won’t create duplicates anymore (and fix several other minor issues). In addition, synchronizing will mirror indexed groups and the external folders they’re referring to.

This means that if you’re going to synchronize such an indexed group, DT will add new contents to its database, update modified contents and remove contents/groups not exististing anymore (neither on disk nor in the database). That way you can easily keep an indexed group up-to-date (by mirroring the folder’s contents).

Note: Synchronizing will of course NOT remove contents/groups imported to/created in the database.

SN:

For me, one of the reasons I rarely used Index imports has been the lack of compatibility of Phrase and Wildcard searching in items imported in that mode. Another reason I have rarely used Index imports concerns database portability when externally linked files are referenced (the ‘portability’ problem can be minimized with a bit of forethought, such as not linking to multiple volumes, etc.).

But Index-importing does have advantages for the size and speed of your DT Pro database.

I think you will love the improvements in DT Pro version 1.1.

The improvements in Index and Synchronization will allow you to not only continue to use your “Category 2” approach but to take advantage of DT Pro’s searching and contextual recognition features for your reorganized files.

Just import those exported files and folders back into a new DT Pro database and set up synchronization of your database groups to the external folders in the Finder. Now new additions and modifications will be reflected in your database.

Just remember to always edit the files that are located in the Finder! Many users have been confused when using the “Open with” Actions command, only to find that edit changes in documents showed up neither in the database nor in the externally linked file. “Open with” doesn’t mean “Edit with”. “Open with” has its uses, but the “Launch Path” commend should be used when one wants to edit the externally linked document under its parent application.

Tip: One can change the mode of import for ‘unknown’, image, PDF & PS and QuickTime files by exporting the current database, creating a new database, changing the import preferences, then importing the previously exported material. So you can switch from external linking of files to importing them to the database or the database Files folder, or switch to Index-importing (resulting in externally linked files located in the exported material folder structure) that were previously stored in the database or the database Files folder.

Note: In the current release of DT Pro, File > Index always results in an Index import that leaves the original files externally linked in the Finder. File > Import > Files & Folders allows the user to set preferences for file types that allow the option of leaving the files externally linked, imported into the database body, or imported into the database Files folder. In DT Pro 2.0 import choices will be simplified. Index import will result in external linking and Files & Folders will result in copying files into the database Files folder structure.

Many thanks to Christian and Bill for the clear info and good news about the changes in sync, indexing and importing in upcoming versions of DT Pro. The changes will really enhance my use of the programme.

And, Bill, thanks for the reminder about how easy it is to switch linking modes simply by changing prefs and exporting/reimporting.

Now, if you’d REALLY like to make me absolutely happy, all you have to do is tell me when we’ll see a real “undo” feature in database operations, to save me from my clumsy mousing! :wink:

Steven

Steven:

Sorry I can’t make you happy today. But frequent backups might help. :smiley:

So do I get it well if I conclude that with DevonThink Pro 1.1, I will be able to do a complete import (or index) of a folder. Then organise the imported information in DT and have that reflected in the folder itself. So two-side mirroring?
E.g. I drag a file in the finder to a different folder, it will move to that folder in DT and vice versa?

I really hope so. Hopefully someone can confirm.

Thankx a lot, Jan

Jan:

No. Don’t get carried away, now. :slight_smile:

If you import a hierarchy of folders into DT Pro, the hierarchy will be reflected in DT Pro.

You can now change the organizational structure of groups within your database. That is, you can add or delete groups, change the content of groups and change hierarchies.

The changes you make in your database organization and content will not be reflected in the organization of files and folders in the Finder. Nor will addition of a file into a database group result in its addition to the corresponding folder in the Finder. DT Pro is not a “Super-FInder” with the ability to override the structure and content of files and folders in the Finder. Nor should DT Pro be a “Super-Finder”.

Think about it. In my opinion, if DT Pro were to “override” the Finder and automatically make changes in the structure and content of Finder files and folders, that could raise issues about the quality and integrity of my files and folders in the Finder. From a quality assurance perspective, I don’t want anything to monkey with my files and folders in the Finder, no matter how much I may trust it.

That said, DT Pro provides me lots of flexibilities to control what’s going on, including the relationships between the contents of my database and the Finder contents, even up to and including my deliberate construction of a new Finder hierarchy that reflects the contents and organization of my database. Import and (one-way) synchronization commands will be simplified in version 1.1, and version 2.0 will maintain an internalized Finder hierarchy of structure and content that is still protective of the original Finder material. And DT Pro version 2.0 will introduce potentials of two-way synchronization, but with due attention to quality assurance and data integrity.

Version 1.1 will simplify some operations by reducing user choices that have sometimes caused confusion and unexpected results. For example, if I Index-import a folder of PDF files and then wish to add annotations to one of the captured documents, I can open that document under Preview or Acrobat Pro and add annotations, then save the document. The changes will be reflected in my DT Pro database content. And if I’ve imported the PDFs into my database Files folder, I can do the same, with the same result. Note that I can’t edit Index-imported files inside DT Pro itself, but only under their parent application (or other application that I define in the Finder Info panel). So DT Pro provides synchronization (one-way) from the original file to the displayed database content. I can open a PDF, Index-imported Word or image file, edit it under its native application, and see the results in my database. That’s good cooperation between the Finder and my database.

Certain file types including text, RTF, RTFD, HTML and a few others are currently stored in the monolithic DT Pro database. These files can be directly edited inside DT Pro and the changes are immediately evident in the displayed document. When DT Pro 2.0 is released, these file types will be stored in the Finder hierarchy inside the DT Pro database package and editing them will be handled like other file types.

What if I want to create a Finder hierarchy that mirrors my database hierarchy? I can do that now by exporting the contents of my database to the Finder (preferably to a target folder, so that files are not scattered all over the receiving directory). But this will be “static” in the sense that if I then modify my database structure, those changes won’t be reflected in the Finder hierarchy and content. Note: Under DT Pro 2.0, the internal Finder-like structure will be dynamic and will reflect changes to the database – but Index-imported files will be external to that structure.

Tip: Suppose I change my mind and want to copy into the database Files folder all files that I had previously Index-imported. I would export the database, create a new DT Pro database and import everything using the Files & Folders import mode. I make the choices (good or bad) and I’m in control (good or bad). If things go wrong, it’s my fault, not an artifact introduced by DT Pro. I like that. :slight_smile:

I’m following this thread because I’m trying to decide the best strategy for my work. You’re getting close to answering my questons but I need more…

I have a directory in the finder with subfolders for articles in progress that I write as MS Word .doc files. I imported it to DT and changed all the file types to .rtf. I can clip news items easily from the web using DT and store them in the appropriate folder. Great. Keeps all the info in one place. When I get to the writing, I may need several files open at once on the desktop and I use TextEdit for this. However, when I save my work, it doesn’t show up in DT. Rather, it saves the files to a hidden \tmp database. This doesn’t help much.

I tried another scenario: I indexed the original .doc files from the finder and edit using Word. I can synchronize them and that works fine. But when I export the files back to the finder directory, I get .rtf files instead of .doc files.

So I’m stuck. I want to work in one venue or the other, not both. I want the ease of organization that I get from DT but a simple and reliable way to save my work, either in TextEdit or in Word. Can you suggest a solution? Sorry if I’ve overlooked this in the documentation.

Thanx,
Mark

That’s a bug and will be fixed in the next version (the exported file is indeed a .doc file but with the wrong extension).

Christian:

Thanks for the info. I’ll try this again.

ciao,
Mark