I cannot use “Classify” using Spotlight. (although smart folders are nice as well)
I cannot use Spotlight to build up a database of my interests trimmed to my needs and tightly focussed on the stuff I actually want to search for. (although the finder might be improved in this direction)
That’s probably the most important point for me: I need a space on my ibook dedicated to the organization of research. I don’t want to view my whole machine as a database.
Desktop search is a great thing. But for us serious information collectors, it does not replace a dedicated database solution. I hope enough people feel the same way to keep Devonthink alive.
As you see, I already made my decision. But I would be happy to hear further pros and cons. Might also help the developers to know what we expect from DT that Spotlight cannot/does not give us.
I will also continue to use DT for pretty similar reasons to you. In addition I use DT for GTD and I cannot see using finder files as a convenient way of doing the same.
I think the key thing for me is that Spotlight is mostly a Search technology whereas DT is an information organisation technology (of course it’s a search technology as well!) and that is one of it’s primary benefits for me.
I’m not sure. I love the concepts in DT, and, of course, there’s no equivalent of ‘See Also’ in Tiger. But the proportion of my machine that DT uses up is hard to justify, especially memory, on a laptop where adding RAM is quite an expensive process. The slowness, even after all my optimization attempts, and the startup time (if I decide not to keep it running all the time) mean that it’s not normally a quick note-taking tool for me. And it’s not entirely clear to me, as a former programmer, quite why its resource usage needs to be so high. With that amount of RAM, the search ought to be much faster, or, alternatively, searching at that speed, the memory use should be lower. It makes me suspect some memory leaks. But I may have missed something.
I’m also always worried about keeping important data in proprietary databases, but I think DT does an admirable job there both with the ease of export and with the automatic backup facilities. Still, it’s not quite as convenient as keeping files in the filesystem, and using DT to index that has never qorked quite so well for me.
I love DT, and expect to keep using it, but these are the big issues for me.
i have to say that spotlight promises to offer a lot of help with broader search. i am somewhat concerned about it picking up all the backup files (for my notebook applications, there are always a couple of recent backups lurking around in file/folder/rtf format (wouldn’t keep backup in a proprietary format). in spotlight, all of these backups would also get indexed (although no doubt there will be an option for omitting certain folders - if not in the current version, the next iteration).
this would lead to a lot of redundant files in any search, seriously compromising usability. in DT one has much tighter control of the base databse.
for me DEVONthink will shortly become the repository for third party documentation/documents in PDF. i expect i’ll just do a search on PDF on my hard drive and dump the lot in to DT where i’ll organise the PDF’s into folders and use DT to help me find the needle in the haystack more easily than searching through individual PDF’s. (btw, if anyone with more experience with PDF’s in DT sees a problem with this idea, please pass on any tips.)
on the other hand if spotlight can accomplish something with less system overhead, i will use it instead. i expect to do at least one round with DT as i don’t think i will be trusting my production mac to tiger for a few months anyway. major system upgrades in .0 versions are not for production machines.
Seems reasonable to suspect Spotlight will have similar issues as any search-based technology in regards to returning “relevant” information. Personally, I don’t particularly like the idea of it being a maintenance ritual deciding which files/folders are indexed (beyond some basic tuning). I’d rather consider it like web search engines that we don’t have much (if any) control over and hope attention will focus on making client-side tools more adept at finding/filtering/organizing items of (dis)interest. Seems that’s the approach DEVONagent takes, as I understand it.
I’d rather not “censor” Spotlight on the server side if there’s risk of excluding data I’m potentially interested in for reasons I may not know in advance. Unless performance becomes an issue I won’t mind it indexing as liberally as possible.
And I don’t see Tiger/Spotlight as a threat to my continuing to use DEVONthink.
I for one will be using DEVONthink in a Tiger world… for the simple fact that my 7600 would choke on 10.4 DT runs wonderfully using Panther and that’s the cat for me - for the time being.
To me, the most important feature of DT is its ability to find relationships among items in the database. Often, these relationships are surprising to me – something I would never have thought about without DT. And just as often the initial results lead to other surprises and insights as I dig deeper into what DT has helped me find.
And my research spans several seemingly unrelated areas, such as music, religion, psychology, linguistics, human anatomy, and several other oddities. Can Spotlight help me with relationships among these? I don’t think so.
Being presented by Spotlight with a list of documents will not help me very much at all, even if they all have some relevance to the search term.
Another issue with Spotlight is this: I have already put all the pre-DT documents I need for my work into DT. I deliberately left out irrelevant things. If, for instance, I search for “viola,” I don’t want to be presented with 23 receipts for viola strings and supplies, 14 student reference letters, 82 viola recordings, 53 Finale viola files, and skoodles of various student records, all intermingled with documents that might be relevant if I were to search each one individually! I want the kind of results DT gives me!
Sure, I’ll use Spotlight for finding stuff. But when I want to do real research, I’ll always turn to DT, every time!
Thanks for making this freeware. Hope you keep it, even in the face of Spotlight. I never use Finder Find. EasyFind is excellent. As good as RetrieveIt for OS 9.
It’s clear from reading these articles why DT makes the switch to using the file system to store it’s database in Tiger. It is also clear that Spotlight itself (at least in it’s initial form) will be incapable of performing the kinds of searches that DT can perform. For me at least it conclusively answers the question raised in this discussion. In my view those of you who feel that Spotlight might provide a replacement for DT are going to be disappointed.
The odds are it never will be. Meta data is an interesting way to catalog, but if it isn’t flexible enough out of the box to do what you want without coding, then there will always be a need for something flexible enough for those who don’t have the time to code.
I. Am. So. Buying. DTPro. When. It. Goes. Gold.
…I’ll have enough pennies saved up by then.
There are three databases I could grow immediately. And all for radically different reasons. No way, even with smart folders, can the job be done the way I need it to be done. It can and will be done with DTPro.
They aren’t very comparable at all. I’ve quietly read a lot of the DT vs Spotlight posts and wonder if people really understand Spotlight. Sure it handles an index of metadata, but it also does content searching. It does not do fuzzy searches and auto-classification, but it does do frequency counting and classification through tags. One might say that tagging files with Spotlight comments is cumbersome – others might say that importing stuff into DEVONthink and self-managing content until the sweet spot of words in snippets can be met to effectively use DEVONthink and then exporting stuff out for external use is also cumbersome. (i.e. anywhere the rubber doesn’t meet the road you are left to writing your own applescript)
I use DT Pro and DA for mining data off the net… but I make extensive use of Spotlight in my GTD process… I use an automator workflow to tag files in batch with spotlight comments and the tagbag dashboard widget to manage collections of disparate files by project and dynamic tagging for ToDo related file-based tasks.
Don’t forget that Spotlight provides one slight advantage – the user is in control of filetypes supported via importer plugins. If I am looking for something among my business documents I am also going to want my extensive collection of keynote presentations referenced – something which cannot be done in DT but is only a keystroke away in spotlight.
I’m not saying one is better then the other – I registered DT Pro a few nights ago after the extensive private beta testing and have been a DT PE and DA registered customer for a couple years. I will still be using both… but I think a lot of you are missing how powerful Spotlight is – for the fuzzy logic and auto-classification features it lacks, it makes up for it in other areas DT doesn’t cover.
I will love to see this integrated better, because personally I cannot give up either technology… although I have to admit, with Tiger on my system now, I am using DT less then I was in Panther. DT has been pushed to niche tasking whereas it used to be the bread and butter of my daily workflow. Not that it is a bad thing – I am actually welcome to the idea that if I have an application with a file format not currently supported in Spotlight and the file format is of an open specification, I can roll my own plugin – whereas in DT if I have an unsupported file format, I can only beg for support from the developers.
I don’t care how much code I have to roll (how far I need to crawl under the hood) to get my job done – but it has to be accessible and not blackbox technology or those areas that are blackbox are replaced with an open framework (and sometimes that represents a different technology). I realize this doesn’t apply to everyone, but sure as hell applies to the geeks / DIYers.
Irrespective of the arguments about which searching algorithms are superior (I too am using both in different ways), I focussed in on your above quote. I also created an automator to tag files and downloaded the tagbag widget to manage the tags but I’ve not had any time to sit and play with all of it enough to decide on tags, develop a workflow, etc. Haven’t really gotten into the GTD approach yet but I’ve been very interested in learning more about it. Is there any chance that you could provide information about your setup and how you implemented it?
…which is why they’re two seperate products, no doubt. Keystroke away in Spotlight? You mean coding your own plug in, don’t you?
I used to be that kind of geek. A Perl monger, no less. But the gray hair started to appear, and the grandkids…, and the time just wasn’t there anymore.
I’d like to see either DT integrate Spotlight so much that DT becomes a front-end requester to Spotlight for “unsupported” data types, a kind of on the fly plug-in writer, or Spotlight just index everything and DT become the real, workable front end/data wrangler.
For file based GTD I have to use Spotlight – DT doesn’t grok enough file formats for me to drag my business into it. The workflow is the same typical batch spotlight comment tagging you use made up of two finder automator actions that can be assembled in less then 10 seconds. (Isn’t automator wonderful? ) For tagbag, I follow the examples – I tend to use tags like Project.SomeProjectName (this code be a Xcode project name or task – e.g. Project.Resume, Project.Taxes), Cocoa.SomeCocoaTech, @Work.ToDo, @Home.ToDo.
In Finder I have two smart folders, one defining all my Xcode projects for work and play and the other listing all content I’ve worked on for work in the last two weeks.
Typically when I have a new task, and stuff has flowed from my Inbox → Next Actions (see GTD) – I “touch” the files appropriate for the task. If it involves using Word, Pages, Keynote, OmniGraffle, etc, I will launch the app and great a stub file. I will then tag that stub file with a spotlight comment so it is indexed… This all happens before the item is put in queue to get done. When I return to actually doing this action item, I recall Dashboard (TagBag widget), select the tag, Finder automagically pulls up the dynamic smart folder tagbag.SavedSearch, click the file, opens the app, finish what i need to get done, and close the app.
The part of my process that seems to not be defined is how best to handle the management of Next Actions, 43folders, Recurring Tasks (Dailies and Weeklies) – Currently I’m struggling on deciding between usage of VoodooPad (using VoodooPad->Remind script), Hog Bay Notebook (using GTD 0.1 from forum implementation) or DT (of which there is no way to handle this but manually and I’ll have to roll all my own AppleScripts when I get a spare moment).
I haven’t settling on ToDo – I’ve tried BurnOutMenu, iCal (naturally) and remind. I tend to lean toward remind because it seems the most powerful. I use a custom TextMate trigger when editting remind syntax through that application, and I use the all powerful quicksilver for doing quick todo appends on the remind text file includes – Of course I still use iCal for appointment alarms since it syncs with my cellphone.
I don’t think there is any magic to what I use, and I am certainly not settled on my choice of technologies – BUT the only place I have settled is using quicksilver and spotlight for filesystem based GTD. The flow of Next Actions I am still left to work out – I would love to use DT, but there is still much to work out here. The goal is not to have to manually manage the entire GTD system (e.g. each day a cron job should move my Daily recurring task into my inbox, and similar with weekly recurring tasks. The GTD 0.1 Hog Bay plugin works well here… However I am really against having to use several applications to get the job done… If I could do it all with the minimal amount of technologies I will. In order for GTD to work successfully, you have to 1) have everything as automated as possible – you should not have to self-manage your project management material TOO much and 2) you must be sure to access all the applications/techs used as a part of your GTD process – if you use too many, they will probably have different steps to handle each part of your process and decidedly becomes over-bearing and complicated. Only simple systems seem to be effective.
So yes, this is a hobby and a long quest… I’ve heard most people are happy with their GTD and finding it working ONCE they settle on a system – I just haven’t found a solution I am 100% happy with yet. I may have to write some gluecode before I can achieve that nirvana. Quicksilver and Spotlight have been the only two technologies I know I will be using for the files/folders part of the GTD system.
Oh no magic – keystroke away meaning spotlight searches are a function key away (versus having to application switch, or worse yet launching an app to provide a search of opaquely housed content) and Spotlight already supports a vast volume of file formats (including Keynote presentations) which DT can’t handle. That in of itself is why I am dependent on Spotlight – I must have a solution for rapidly finding keyword associations with my business files.
I know stuff is easy to pull in and out of DT – but it is still opaque, I cannot look into a dtBase file outside of DT – so long as that remains, it is opaque and requires an application memory footprint to get at these search queries. This is clearly acceptible for mined content, but for system wide content searching which happens far more often then trying to fuzzy search a needle from my virtual haystack, it is not very handy.
Believe me I understand the lack of time to be a geek… I am a father of 2 (5 year old stepson with functional autism and 5 month old son with a clean bill of health (so far)) I get most of my stuff done after the family goes to bed and often have to sacrifice a bit of sleep on occasion… I can’t give up my geektime – using Xcode for my own projects instead of just work-related stuff has to be a part of my diet or I will run screaming down the street
Happy Father’s Day BTW – to celebrate the geek-hood my wife bought me a ThinkGeek v1.0 shirt and my 5 month old a ThinkGeek v2.0 6 month baby shirt – woot!
…a part of me wants really, really, really badly to play with XCode. I was almost a convert to Smalltalk when I did a stint at IBM. I love the idea of truly object oriented programming. And at the time OS/2 was making their play for not only OOP, but an OOUI. We’re so close to going all the way with OOUI on OS X.
Thanks for the response mgrimes, I’ll ruminate about what you are doing in the context of the things I’m trying to do to improve my workflow. Time and efficiency is definitely an issue. All these systems take a significant level of effort to set up and it seems that the only way to determine whether it is a workflow that one will continue to use is to set it up and use it. Then, if you’re not happy with it, you’ve got to either modify it or start all over. So I’m trying to weigh the benefits of the process relative to the results and it is a difficult metric to determine.
With so many shareware solutions out there for Mac OS X it is indeed time consuming to check out which is best – one may spend everyday checking, not working.
After all, few apps have turned out to be essential, DT Pro definitely is. I am going to use it, get more used to it and wait for ver 2.0, where all documents are planned to be open and Spotlight fully integrated. I am sure, Christian will not let us wait so long again as he did with 1.9 – which was like a major release anyway.
In short: I will go on working like I do now and will enjoy DT becoming more open in the near future.
DT runs wonderfully using Panther and that’s the cat for me - for the time being. 
