I admittedly only took a quick glance at the beta of DEVONthink Pro 2.0 but I vaguely seem to remember that one of the big announcements for 2.0 had been a new way of storing documents directly in the filesystem (similar to the way Together, Eaglefiler et. al. are doing it). But there still seems to be a (more or less) proprietary db carrying such annoying necessities as “internal” db backups—that effectively grow the db’s size and in turn make online-backups etc. less convenient. Did I miss something? Was that feature scraped?
Yes, in version 2, files are stored outside the database in their original format. You can try the following to illustrate the difference between version 1.x and version 2 (at least for the professional version):
DT databases are stored in “packages.” In version 1.x if you right click on the *.dtBase package, and select “show package details” you’ll see the DT database. You’ll be able to see that files are not listed separately – that they are stored inside version 1.x databases.
Now try the same thing with version 2 packages–the dtBase2 package. Again, right click and select “show package contents.” You’ll find a folder labeled “files”–this appears to store the files in their original format, outside of the database (but inside the package).
So yes, files are stored “externally” from the database files, and you can edit them independently. (Although: I haven’t tried editing the files outside DT, e.g., by selecting one from the finder and editing.)
Interestingly, the original files are stored by type: txt, numbers, pages, ppt, mellel, etc. So I’m much more comfortable with version 2, because the original files are stored separately, and you can retrieve them even if you were to stop using DT (though why that would happen is a mystery to me, given the ability to keep multiple databases open, the AI features etc)
I’m not that psyched about this “open folders” concept. How is it helpful to only have the items separated by type? This really isn’t that different from version 1. The only difference, to my eyes, is an extra folders set. In DT 1, every single file was in the same spot. Now they are separated by type.
I thought the filesystem would ‘mirror’ the heirarchy in DT, folder-wise. This is what I was hoping for, anyways. The reason this would have been helpful, for me at least, is that I would have permanently moved my Bookends Attachments folder into the DT database of folders. Can’t do that I guess.
I’m sure there was a logical reason to it, I was just hoping the Finder folder hierarchy would match the hierarchy in the database. That is probably pretty difficult though.
When you get into the nether regions of benchmarking and so on, you can find out surprising things – like, for instance, that certain filesystems work good for x number of files in a directory, but any more and the performance stops scaling well. Some filesystems work very well for extremely large numbers of very small files; others work very well for small numbers of large files. Some (most consumer filesystems) are built to work decently with either situation, but of course it won’t do either as well as a filesystem tuned for either extreme.
So if you had a large number of files of all different sizes in one directory in one filesystem, it might be dog slow. I suspect that DTP organizes files the way it does in part because of the performance penalties that might result from a single all-encompassing folder.
A couple of users (including me) have voted for a folder hierarchy stored within the .dtBase that directly reflects that same hierarchy in the filesystem (even if it were just stored within the database, you could add aliases to other folders you wanted to see within DEVONthink). This would make it much simpler to import websites and manage directory tress that you need to remain hierarchical, etc. I’m guessing it would solve your Bookends problem too.
v2 is actually very different, let me explain because the crux of this is being lost in this thread. In DT v1 most files were stored directly in 1 of 10 DT database files, EXCEPT for things like PDF’s and image files which were stored in the Files folder. The big problem with this was that things like RTF’s and Word docs were not stored natively, so if you opened an RTF file in TextEdit from within DT using the “Open Externally” command, made edits, saved it, and then went back to DT those edits were not there! (The file save just went to an invisible temp file which you had no access to.) I am actually surprised that more people did not cry fowl about this, but it was a severe problem. Whereas if you did the same with a PDF file in Preview, the changes were reflected in DT v1. SO, you had to really know which file types were stored in which manner in DT v1 to know what kind of edits you could safely do. 4 years ago when I started using DT I immediately detected this issue and brought it to the developers attention. It was back then they promised the new format for 2.0, which we finally now have in our hands… I have actually held off for 4 years recommending this app to some people until this got fixed because I was fearful of them losing data.
Now, why is v2 so great? Because now EVERYTHING is stored as a separate file inside the database package, as described previously in this thread. The most notable thing here is for things like RTF’s which before were embedded in one of the 10 database files but now have their own file. You can see this by selecting an RTF in DT v2 and selecting “Reveal in Finder” from the contextual menu – in v1 you could not do this because there was no file in the Finder to correspond to. Now, in 2.0, you can open an RTF externally, edit and save it and the changes get reflected immediately in DT. So finally, you can truly forget how DT stores the data, because no matter how and where you edit it the file gets saved and reflected in DT. The 10 database files are still there, but they just store all the proprietary info unique to DT. In addition to the benefits I have mentioned, this makes everything faster too.
For a casual user this all may not seem like a big deal – but trust me, it is, and they have setup an architecture that is vastly superior than the old one. You should be excited about this
I do get the difference, but so much of what I had stored were things that in dt 1 are in the “files” folder (1 of my databases has 700 in there, the other has 1100) that I was hoping more for a hierarchy change.
I also think that the new format is superior to an approach where the folder hierarchy is reflected in the Finder hierarchy, mainly for two reasons:
You would run into difficulties with replicants: in which folder DT should store them?
I use Endnote to refer to pdfs stored in my DT-database. These links are not dynamically updated, so when an item is moved, the link does not work anymore. Imagine I introduce a new folder in my database where I move some pdfs in. All links in Endnote would cease to work.
So, I very much prefer a static structure. The important thing is that you can access your files without actually using DT (what is possible now).
I am able to access my files stored in DT 1 now external to DT. I just used quicksilver to index the “files” folder. I do it all the time. Another reason I was preferring a hierarchy
Can’t you point QS to index the Files folder in DT 2.0 as well? Just tell QS to index the subfolders too.
I too was hoping/expecting that the folder structure would mimic the structure in the Database to some extent as well, but upon reflection, it seems that this is the better way to go for 90% of the usages needed, for the reasons Mueller-Scheesel describes above. Having static links is important for linking to documents in DT from outside of DT, and also what would they do with Replicants – the Finder has no correlative file type.
What would be neat is if there was a QS and LaunchBar (which I use) plug-in that would allow DT users to navigate the actual DT folder structure within those apps.
@danco DEVOnthink Personal does not store it’s data in a package, at least in v1 it didn’t. It appears it may be the same in v2. In v1 everything is stored in the Application Support folder. Basically, the only difference in the Pro versions is that all those files in App Support get wrapped up into a package and can be stored anywhere. Since DT Personal only supports a single database this portability is not necessary.
Not so much for my usage. I use files I warehouse (media files, pdfs, research material in different formats) for multiple projects. My major writing program, Scrivener, allows me to create references to external files and view them within Scr. itself for my research. I can create split window views and transcribe video and audio, make notes, etc., all within Scrivener and all the while keeping my files nicely sorted in the Finder in one central place. If I had to export these files and duplicate them in order to use in different Scr.-based projects, I’d have a lot of memory get eaten up (some of them are quite large), and many instances of the same files.
I personally was hoping that there would be a way to use DT and also have this kind of centralized access to my files. Doesn’t look possible. Right now I can index these files, but I still have to organize them via the Finder. It also creates problems if I want to access something that I’ve stored directly in DT Pro.
The unique URL that DT 2.0 assigns each file in DT v2 should actually help you do what you want. I use Scrivener too, albeit rather lightly, so I decided to try this as it would be useful to me as well. I was able to add files from DT 2.0 as Project or Document References in Scrivener by just dragging and dropping them into the References pane in Scrivener. PDF’s and images (and probably audio I would think as well) can then be opened right into a Scrivener split pane. With RTF docs I noticed that Scrivener will only allow you to launch them in TextEdit (or your default text editor). I have not actually worked with this setup, but it appears it should be OK.
Well, I’ll be! You are quite right. I have to experiment more with this, but it appears I can do just what I need to do. I didn’t think to try dragging the file directly from DT to Scr. This is exactly how I need things to work. Rtfs can get copied directly into Scr., it’s really only the large media and pdf files I need to reference externally.
So it appears at first blush that perhaps 2.0 will do exactly what I need it to! This was the major reason I had to stop using DT Pro 1.0, much to my dismay at the time. Thanks!!!
Glad it works. If I were you I would put it through some tests before I trusted it with mission critical data. I’d do the following:
add references from DT to Scrivener
close Scrivener
edit/rearrange things in DT v2
reopen the Scrivener project and see if the links still work
Also, perhaps you can help me with a Scrivener question: In doing this I found that I could only open a reference within Scrivener if the Inspector pane was focused on a document. If the Inspector was only showing Project References within the whole Inspector pane the contextual menu to open in Editor would not appear. If I switched focus to a document and then changed the References section at the bottom of the Inspector from Document to Project again, then the references could be opened in Scrivener. Mmm? Seems like a bug in Scrivener to me, or am I missing something?
I do like the local URL feature, so I don’t want to give that up. As I said, I was hoping to store my Bookends attachments file right in my database, but that won’t be possible. Its okay, I’ll make it work and am still quite happy with 2.0
You read my mind. I thought to try this to see if the link works if I move things around. Probably not, but it doesn’t work in Scr. with Finder references either.
You are quite right that you lose the CM if the focus is on a folder or pdf file, where you will only get project-level access in the Inspector. But you can still drag the link and drop it in the header portion of the editor window to open it. Grab the reference icon in the Projects Reference pane, drag it to the header of the editor pane (it will switch from grey to dark grey and you’ll get a green plus sign), drop it and your reference will open.
Not sure why you lose the CM. I assume because it is in corkboard or outliner view and the editor is not available as it is in the document view. You might try posting this on the Scr. forum to see what Keith says about it.
@alexwein: I did the little test with moving things around and the links stay connected in Scrivener. This has nothing to do with Scriv but the way DT 2.0 stores the files, since DT doesn’t try to mimic the Group structure, even if you move a file within DT the link stays the same (or so it appears). This, really, is a HUGE advantage to the way DT is storing things in v2 – really, much better than if they tried to replicate the Group structure. I think they made a brilliant choice in how they implemented this.
Re. the Scriv behaviour, I think you are right in that there is no editor view available in that mode. Dragging and dropping works for me in this situation, I am content. Looks like Scriv and DT 2.0 are going to be really good companions!
