Difference between Tiger Spotlight and Devonthink

Dear Devonthink, I have purchased your software and find it to be a useful and stable addition to my suite (along with StickyBrain 2 and NovaMind screenwriter’s edition) for both creative and business purposes. But I’ve watched the pre-cancerous Steve Jobs in the video about Spotlight and wonder what differentiators future versions of Devonthink will include to distinguish it from the built-in search capabilities of the forthcoming new Mac OSX please? Will you concentrate on enhancing retrieval and manipulation of data within Devonthink, or enhance the Devonthink “indexing” function to complement and extend the capabilities of Spotlight? Kind regards, Jason Romney

DEVONthink is of course much more than a pure search index like Spotlight. However, for those users using DEVONthink as a pure search index version 1.9 will still offer some important benefits:

Content indexing is currently (latest builds of v1.9) 4.5 times faster than the one of Panther, not limited to 100k per document, not case insensitive, does include all words and all numbers and it’s possible to preview found documents (including highlighting of occurrences). And version 2.0 (featuring automatic synchronization & indexing, the possibility to edit linked/indexed documents and likely even more speed) will be available before Tiger. But of course Spotlight looks prettier :wink:

And to answer your question - we’re going to extend all existing possibilities of course and to add more speed and new possibilities. Version 1.9 will be just the first step (and the biggest revision ever).

Ever consider a brushed metal interface for DEVONthink, like Near-Time Flow? :wink:

Actually we don’t like the metal look at all anymore and therefore this look will be probably removed from DEVONagent in a future release :wink:

But switching from Metal to Aqua (or from Aqua to Metal) requires lots of minor changes (only activating or deactivating this looks horrible, I’ve tried this already :wink:)

Just had a look at the latest benchmarks and indexing of plain text/SimpleText/RTF/RTFD documents is actually 5.2 times faster than Panther (no benchmarks for Tiger yet but Apple claims that Tiger’s indexing is 3 times faster than the one of Panther, have to verify this as soon as there’s some time left to install the developer preview of Tiger).

And the performance of indexing is actually quite important as indices have to be updated quite often.

I don’t care (much) if apps have an aqua or metal look.  What matters more are the interface elements, like what I described here.

From what little I’ve looked at it Spotlight indexing has been mysteriously inconsistent.  For instance, the results for my Panther boot drive indexed differently than a clone of it.

DEVONthink indexing is consistent and reliable, even if on a smaller scale.

I’ve been wanting to ask about the whole issue with Spotlight as well.  I agree that DT has a lot of functionality beyond just search, and I expect to continue using DT with Tiger (especially once DT can do searches with complex boolean logic!!).  What I’m wondering is how DT might benefit from and integrate with Spotlight.  For example, from what I’ve read, Spotlight is document-centric … in other words, the only kind of object it can return in response to a search is files.  (Mail messages will be exploded into individual files in Tiger, which is why the search works on emails.  Don’t know how they’re doing iCal events.)  Since DT keeps all its data in a few big database files, it seems like there will be no way for system-wide Spotlight searches to find DT contents, which would suck.  Do you foresee any way of making this work?

Conversely, it would be ultra-cool if DT would tap into some Spotlight APIs so that searches done from within DT would trigger a system-wide Spotlight search in parallel.  The results from the Spotlight search – which would include things like emails, calendar entries, and other files anywhere on the computer – could perhaps show up either in the same results list as DT’s internal results (marked somehow to indicated that they’re external), or perhaps in a separate pane next to the DT results to avoid confusion.  Could that also be extended so that relevant emails, calendar items, etc would appear in DT smart folders?  Ideally, when you set up the smart folder you’d be able to choose which among the categories of Spotlight items (email, etc) you’d like to have included in that smart folder.  

A tangential question: any chance that future releases of DT will be able to read/import the individual email message files that Tiger will use?  That way, if the above suggestion were implemented, the non-DT emails that show up in a DT smart folder or search (thanks to Spotlight integration) could be viewed within DT, like in the notepad view.

Okay, that’s all for now.  Perhaps this thread should be moved to Requests and Suggestions.

I’ve noticed cached messages of IMAP account mailboxes (at least) are saved as individual files with Apple Mail in 10.3 (at least).

Probably a similar issue integrating Spotlight with SQLite databases on 10.4.

Found this while searching for more discussion of that topic.  Reminded me of Core Data again, too.

Plenty of interesting food for discussion here but maybe it’s a bit early to move this into the realm of requests/suggestions since neither DT 2.0 or Tiger are available. :slight_smile:

I use DT to discover relationships between things as well as just finding things.  The “Classify” and “See Also” buttons work together to record and explore these relationships.  I don’t have the Tiger pre-release so I haven’t seen if Spotlight has anything similar.  I use DT as a information structure database as well as a scrapbook for my thoughts.

In addition to providing information structuring, DT also presents a unified interface.  While it could be argued that the Finder does something similar, a single app can provide a better user experience to access information than multiple applications can running under the Finder.  DT can form links between documents in a way that would be difficult to implement in multiple applications running some kind of linking toolkit.

Version 2.0 will provide all search operators of DEVONagent, including AND, OR, NEAR, BEFORE, AFTER etc.

I guess this limitation will apply to many applications storing lots of contents in one file - even Stickies will be affected :slight_smile: Apple has currently only bypassed the problem by modifying Mail & Addressbook but actually a more flexible Spotlight interface would be welcome.

We’ll definitely try to use Spotlight but probably in a different way, e.g. to index file formats which Spotlight supports but DT does not (and therefore you could use DT to perform all your searches). But it’s not yet clear if that will be possible (no time available yet to install the Tiger preview :slight_smile:)

DT 2.0 should be able to import both mbox files and Tiger mail files.

Microsoft says Longhorn to ship in 2006 without WinFS is a current Ars Technica thread that’s mentioned Spotlight and other issues relating to searching, metadata, etc., including some interesting “bigger picture” context and observations.

It’s worth noting that this seems to be the season for contextual search of your computer’s data. With the announcement of Spotlight, the introduction of Google Desktop for Windows, and the announced search features of Longhorn, it appears that everyone is in the game now that Devon has had to themselves for awhile. The most notable difference is that Devon works on a specific database, while the other programs are system-wide. I wonder if this is going to push Devon to consider a system-wide approach as well. Also, it seems this is going to make the other organizational features of Devonthink more important as means of differentiating themselves from Spotlight, Google Desktop and Longhorn. I’ll be curious to see how it all pans out.

I’m not convinced though that Tiger will be doing full text search of documents. For example, I downloaded 350 MB of developer docs from Apple’s developer library. I dragged them into DT and it chugged along for about 45 minutes, building an index of all significant words. Now I can find any document containing the key phrases.

I also have this for the OS X Server documentation, the question “I want to set up Windows sharing… was that in the Windows Services guide or the User Management guide?” can be answered very quickly by searching on the right key phrase.

I believe Spotlight focuses more on finding files based on meta information (where the Spotlight plug-ins are searching), which would mean in tags, EXIF data, etc. I don’t think it’s going to index every word in every file, so DT should always have an advantage there. That’s not to say that DT couldn’t use or extend a Spotlight plug-in to improve indexing and searching, should it be more effective than what’s currently being used.

bless you.