DT Best practice

I am new to DT and am exploring the best way to use it. I already have a structured filing system using Finder and Folders but feel that DT could increase my productivity. I currently have the following file types: pdf’s, web archives, notes, circus ponies notebooks and excel files. All my current folders will contain at least one of these file formats.

Ideally I would like to just use DT in index and sync mode as I would like all my files to be available in Finder, however, as I use a laptop and sync all files between my desktop and laptop via phd’s this will not work as indexed file paths will be different on the laptop to the desktop. Therefore, it seems to me that to use DT properly on both machines I will have to import files into the DT database.

If this is the case, what would be the best way to manage the different file types. DT can handle pdf’s, notes and web archives but not excel and circus ponies notebooks files. Therefore, should these files be left in Finder or imported into DT. If imported should the original file in Finder be deleted?

If the best advice is to put all files and types into the DT database, I am slightly concerned with the overall size of the resulting databases. The current size of all my data is about 90Gb. I realise that I could split this into several distinct databases but if one has more than 6 or 7 does this become cumbersome to use?

I am interested in people’s advice/experience when using DT in practice with my setup and requirements.

You can have the same results with indexing and importing documents.

Not necessarily. I regularly transfer several indexed databases back and forth between the desktop and my MacBook. If you place your document folders at the same relative location from the user root (e.g., ~/Documents/…) and your databases are in a folder with the same base location, then indexing works transparently. So, I have my documents in ~/Documents/Work Files/Data (those are the files I index) and my databases in ~/Documents/Work Files/Databases on both machines. I synchronize ~/Documents/Work Files between the two machines (with databases closed!) and all the links are maintained.

DT can “handle” (i.e., display) any file for which the developer provides a QuickLook plugin, and it can work with PDFs, text, RTF, and webarchives natively. Circus Ponies - no QuickLook plugin (talk to the developer and request they create the plugin). Excel - yes. The Excel QuickLook isn’t terrific, but it is workable and more than adequate for browsing.

That’s probably beyond the range for good DT performance. What matters most is the number of words. Look in these forums for Christian’s postings on that. (Or my good friend sjk will find it for us and post the link below :wink: ).

No harm experimenting, though. Index the data and see how the performance is for you. The indexing won’t affect the originals.

I went from Indexing to putting everything inside my DPTO databases. I am indeed using multiple databases. My main database has over 10000 files and close to 11Gb, it contains only my ongoing work-related projects (this one is sync completely with my iPad). Works like a charm.

I have one for home and 4 for archiving completed projects and related information based on category. Overall, close to 60 Gb of stuff. The largest is database is 13 Gb so far and DTPO seems to handle it correctly.

I am excluding e-mail of all this for the time being (I have e-mail dating to early 1990s for many Gb).

Thank you for your replies - they are most helpful.

I have tried indexing and then syncing (via phd on laptop and network home on desktop) back and forth from my laptop to desktop and you are right that this works very well.

If indexing seems to work in this set up, it brings me to another question. With my limited use of DT so far, it seems that one of the big benefits is the ease of getting information into DT. i.e. clipping web sites, scanning directly into DT and then using the move to function to quickly file documents in the correct folder. However, if one indexes does this ease of use disappear to a certain extent as every file that is easily put into DT will have to be synced back to the underlying folder in finder and then indexed back into DT. This workflow does not seem very efficient. Is there a better way of managing the workflow to arrive at the desired outcome of having all files indexed into DT.

Once again, thank you for the help and insight.

I think you correctly perceive there is an efficiency trade-off between indexed and imported that is related to how you bring new data into the database, and how frequently. With recent changes to DT, the tools exist to readily move data into and out of the database with respect to indexed folders, and do this from inside DT itself – i.e., the Move Into Database, and Move to External Folder commands (which can also be scripted).

I think the best approach is to try using your database indexed, then see if your workflow or frequency of data imports and updates make that method unattractive or not. Then decide whether to stay indexed or go imported. It is relatively easy to convert indexed folders and documents into internal documents. Because of links, labels, etc., it can sometimes be a bit tricky to convert from internal folders to external. So that’s why I suggest first trying the indexed structure.

1 Like

Thank you for your helpful comments.

I will take your advice and index all my folders and see how that works for me in practice. Is there a script that will scan all folders and perform the ‘Move to External Folder’ command and a script that will synchronise all folders in the database.

I must say that I did found indexing, and going back and forth - moving information out into a folder structure, too bothersome in the end.

The AI is quite good and getting my Inbox to zero is very easy and very fast with DPTO. Like they say throw everything at it and let it chew :wink:

Luc

I found myself asking similar questions when I started using DT. One of the most important, to my mind, is “Do I really NEED to have things in the Finder?” In other words, is there any advantage to having things in the Finder and only indexed in DT? Eventually I opted to have my pdfs of articles only indexed because I was (and still am) using Bookends to manage my references, and this works better if the pdfs are in Bookends’ own attachments folder. Everything else (if it is to do with work) gets imported into DT. At least I know where it is if I do that! Occasionally I find a web page while using Safari that I ought to reference via Bookends, and then I always wonder whether to clip it with DT, or just print it to pdf and put it in the Bookends folder. I seem to have a mixture of methods, depending on the web page. I never use webarchives, as pdfs seem to “fix” a web page rather better. And they are cross-platform.

Martin

I’m reasonably new to DEVONthink as well - been using about a year now.

I for similar reasons to you went for the indexed mode for most of my databases, as I have a good file structure, and like it accessible outside of DT as well.

I have found the new commands ‘Move Into Database, and Move to External Folder commands’ very useful. I now use DT to create new files or folders, then just context click them straight away and choose ‘Move to external folder’ - job done. If you do this on a folder then all items in the folder move out of the database and become indexed. So you can quickly use the DT interface to build up data or capture data but make it indexed in one fell swoop.

Sometimes I forget to do this, but you can see which folder and files are not index references, by the icon indicator. I also have set up a smart folder that shows all folder or files that are ‘not indexed’ (criteria is: ‘instance’ ‘is not’ ‘indexed’) so I can quickly see those files, context click them and choose ‘Move to External Folder’.

Using indexed also means I can open a file in its native application if needed, e.g. for Omnigraffle diagrams - which are nicely viewable in DT, but not editable. I’m not sure if this would work if I was using in database ?

I do use a regular non-indexed Database for transitory data, that I know I will not need for long - e.g. for research and clippings.

1 Like

It will work.

DEVONthink can edit RTF, plain text, HTML and its cousins (just plain text, really), and CSV documents (the so-called “sheets”). Or, through the Open With comment it will use whatever native document editor(s) the OS X file system knows about. This includes OmniGraffle. It does not matter if the document is indexed or internal to the DEVONthink database – editing works the same way.

If the editor app provides a Quick Look plugin, then the file contents will be viewable in DEVONthink – again, the file viewer works the same way if the document is indexed or not. The quality of the view depends on the Quick Look plugin and is outside DEVONthink’s control (e.g., Microsoft has always provided very poor quality Quick Look plugins and there’s nothing to be had that’s better; while on the other hand Merlin provides superior plug ins). Finder uses the same method to display file contents, with the same shortcomings or advantages.

Viewing and editing are not affected by the index-vs-import decision.

Guys, I have DTP 2.1.1 and can’t find the “Move to External Folder” in my context menu when I right-click on a group within a database. Am I missing something? How would I get to it?

You can only use the move to external command if the selected files or groups are already within an indexed folder. So if you have an indexed a folder ‘FolderA’ and you created documents and/or groups within that folder in DEVONthink, you can select the items and the move to external command will be available.

Greg, I’m a bit confused now. If the group is an external folder already (since it’s not an imported but an indexed one), how come ‘Move to external folder’ applied to it make sense. In my understanding it’s already an external one, is it not? I think I might have lost the logic flow here.

Would you mind to explain me, please?

PS I stumbled upon the context menu later last night when I posted the previous question. I didn’t expect to see it in context of an indexed folder and still am having a hard time to understand the logic. Sorry! :confused: Confused.

Scenario:

You Index-capture folder A in the Finder, resulting in the creation of group A in the DEVONthink database.

Now you capture document X from the Web and move it into group A. But you would prefer that document X be moved to the external folder A and Indexed back to the database.

To do that, select and Control-click on document X. Choose the contextual menu option, Move to External Folder. Document X is moved to folder A and is now an Indexed file in group A.

Note that ‘Move to External Folder’ is available as a contextual menu option only if the selected item is not Index-captured, but resides in an Indexed group.

Bill posted his response while I was writing this one, so I’ll go ahead and add it anyway.

The potentially confusing part is because you can move files to an indexed folder in DEVONthink, but that doesn’t automatically move them to that folder in the filesystem. Let’s say you have indexed a folder, named “IndexedFolder” and its contents, which is say a total of 10 documents. In DEVONthink, you can move additional documents to the folder “IndexedFolder” but they are still in the database-they have not been moved externally to that folder in the filesystem (Finder).

You can also create new groups with DEVONthink inside the “IndexedFolder” but just like the above mentioned files, those groups do not appear in the folder “IndexedFolder” in the filesystem. It’s only when you select the “Move to External Folder” command on any file/group that is contained inside an indexed folder that the file/group is moved to the filesystem.

In other words, “Move to External Folder” is exporting those files and groups and indexing them back to the database in one step. One cannot “Move to External Folder” any files and groups that are not a sub-set of an indexed group as there is no designated place (an indexed folder that exists in the filesystem) to move them to. If you need to move files and/or groups that are not contained in an indexed folder, you’ll need to first a) move them to a folder that is indexed or b) export the files and/or groups to the filesystem and then index them.

Does this help? If not, let me know.

Bill, Greg, thanks a million! :slight_smile: Now I’m with you. It didn’t occur to me that I can move documents into indexed folders in DT. I must have missed this in the manual and haven’t had a chance to try it yet.
Also if I would I guess, I’d assume that document is copied into the external folder automatically.

Thanks again for making it clear!

Personally, I am glad that it does not copy the file to the external folder automatically. As one example of how I use this extensively, I index to DEVONthink some folders that are located in Dropbox and also some podcast folders within my iTunes library. I often make notes, have related documents, etc. that I want to keep in the indexed groups in DEVONthink, but I don’t want those documents saved out to the filesystem. By allowing me to pick which documents I want to move out to the indexed folders, DEVONthink gives me the flexibility to work with indexed folders and documents in a way that works best for me.

This is a good approach for i-device synchronization too. I keep a series of folders in Dropbox with work that I want to read and annotate in GoodReader, which has excellent synchronization and annotation features (thus avoiding the weird DTTG filenames, lack of annotation functionality, and unproductive round-tripping of documents on the iPad with DTTG). Those folders are indexed in various DEVONthink databases. And, as Greg described, I can keep back some documents in the folders from synchronizing to GoodReader by not moving those documents externally to the Dropbox folders.

When I’m finished with my work on the iPad, I just move the document internally and it disappears from the Dropbox/GoodReader process.

Another benefit is preserving the folder structure I want on the desktop without doing workarounds to the all-or-nothing folder synchronizing that DTTG imposes.

I have a folder with several hundred PDFs in the file system. Up until now, I’ve been importing them in DT, but I now realize that this is just taking up twice the space, and all I really need to do is index them (I still want to have external access to them). So what I’d like to do is get the files out of the database without losing my metadata, since I tagged them all. If I read this thread correctly, I can index a folder in Finder and then DT will index its entire contents? But if those files are already in DT will it skip them or duplicate them? I don’t want to have to re-enter all the metadata and would prefer to avoid deleting duplicates (not duplicates in the DT sense). If I index a folder in Finder, does that mean all the files I drop into it will automatically be indexed in the future? I’ve been keeping two separate folders, a PDF and a PDF Temp folder. I download PDFs into the Temp folder, then import them into DT when I have time, then move them from Temp to PDF. When I just index them, this breaks links. Obviously, there is some wasted effort here, but I did this so I don’t lose track of indexed items. It seems if I just index the folder in Finder directly in DT, I won’t have to worry about going through this process and I’ll save disk space… I tried finding the answer in documentation, but I’m nervous about doing anything to my database without being sure it’s going to work out. I spent so much time tagging everything!

I also don’t quite understand this:

“Indexed groups are excluded from tagging by default and changes to an indexed document are immediately reflexted in its OpenMeta metadata (and are so noticable by external tagging applications).”