Spotlight Commentary

[NeverTheLess]

I have a question concerning the Spotlight Commentary field.

If I have a file in Devonthink and go to “Info”, I see a field called Spotlight Commentary. I think it’s obvious what this field does. So I enter a term that I want this document to show up for when I search in spotlight.

Now here’s what I don’t understand: When I go back to the Finder and get the Finder “Info” of said file, there also is a field called “Spotlight Commentary” but it is empty.

Wouldn’t it make more sense to have Devonthink edit this specific field on the finder level and make it show up identical? Or am I misunderstanding something here?

[korm]

It depends on whether the document is indexed, or internal to a DEVONthink database:

Indexed Spotlight comments and tags are written immediately to the file, and are searchable in Spotlight and, in the case of tags, with any Open Meta tag utility (e.g., Punakea, Leap, etc.)

Internal Spotlight comments and tags are not written to the file unless the file is exported. Thus, the Spotlight Comments and Open Meta tags are not searchable until then.

Changing the behavior for internal documents to be the same as for indexed documents has been requested before, and you’ll see more dialog on that if you search the forum.

[Bill_DeVille]

If you Import-captured the Finder file, a copy of it is made in your database. If then you add content to Spotlight Comments in the Info field of the file in the database, it does not affect the external copy that may still exist in the Finder. If you were to export the database copy to the Finder using File > Export > Files & Folders, then examine the Info panel of that copy, the Spotlight Comments would contain the text you had entered in the database.

If you Index-captured the Finder copy, then add text to the Spotlight Comments field in the database, the added text will be visible when you open the file’s Info panel from the Finder.

[Greg_Jones]

Just to expand a little further on what korm and Bill have said, DEVONthink makes documents contained in the database searchable in Spotlight by creating a Spotlight metadata cache file. That’s why database documents from a Spotlight search show the document name, but the icon and the document type is listed as a DEVONthink document. The actual documents are not searched, thus the thinking behind why the comments do not need to be written to the document unless it is exported.

[NeverTheLess]

Thanks for your very fast replies. I only now had the time to read them carefully and have understood the differnce between Importing and Indexing.

My follow-up question is less technically but aimed more at the practically use-cases of those two import options.

I’m brand new to DEVONtools in general and am just making my first little steps. Now if I want to index all the pdfs / eBooks I have, I assume Indexing is what I should choose since there is no benefit of having a duplicate copy of each single file in DEVONthink itself, right?

But what are the use cases in which that would be helpful? Since if a file is already on the harddrive, why should I clutter the harddrive and create a very large database-file, by capture-importing all the files to DEVONthink?

Maybe a stupid question but atm I can’t think of a use case where this is useful.

[Greg_Jones]

But what are the use cases in which that would be helpful? Since if a file is already on the harddrive, why should I clutter the harddrive and create a very large database-file, by capture-importing all the files to DEVONthink?

I suppose I would answer this by reversing the question-why keep the file on the hard drive (in the Finder/filesystem) if it has been captured via import into the DEVONthink database? I don’t view DEVONthink as a Finder replacement, but because DEVONthink does allow me to view, organize, search, and make connections among my data that is not possible with the Finder, I don’t see any reason to keep duplicates in the Finder.

In other words, there are good reasons to have indexed files in a database and good reasons to have (imported) files in a self-contained database, but no good reason that I can think of to have documents in the Finder and the same documents imported into the database. Space considerations aside, sooner or later you’ll encounter a situation where the document(s) are changed one place and not in the other, which is not a good thing.

[korm]

The question “do I index or do I import” has no clear answer, IMO. Users find benefits in both. A critical difference, as user tshanno recently pointed out, is that if you reorganize your indexed files by reorganizing your folders in Finder, then DEVONthink will likely loose track of the files and you’ll need to start over again. But whether that situation arises is completely up to you.

If a user is unsure whether to index or import, I always advise starting with indexing, and see how that works for you. It you don’t like the results, then select all your indexed documents, use the Move Into Database command from the contextual menu, and from that point forward all your files have been imported. (With the advantage of not losing the tags, labels, and links that you built while those files were indexed.)

Once your originals in the filesystem are moved into the database and no longer indexed, it’s advisable to be sure you have a good back up, in a secure location (preferably off site) of the original folders and files, and then delete the original folders and files to free up the space.

[NeverTheLess]

Thank you two for elaborating on this. Interesting discussion. And since I’m just starting I will go with Indexing for the near future.

Importing is (imo) only useful when the files then are deleted from the finder. One thing to note is: there’s no easy way back. Then the user is relying on DEVONthink a lot. So your suggestion of an offsite backup of the files sounds like a good idea.

[KP1]

@NeverTheLess

Importing is (imo) only useful when the files then are deleted from the finder.

True. Unless you need two copies of a file for some reason.

@NeverTheLess

One thing to note is: there’s no easy way back.

I have to disagree with you here. It is very simple to export files and groups from DT to the finder. Just use File>Export>Files and Folders. Of course, if you need ongoing access to files within DT from another program, and the “Open with…” command isn’t sufficient for you, indexing would be helpful.

[Bill_DeVille]

@NeverTheLess

Importing is (imo) only useful when the files then are deleted from the finder. One thing to note is: there’s no easy way back. Then the user is relying on DEVONthink a lot. So your suggestion of an offsite backup of the files sounds like a good idea.

If you have an Import-captured database, there’s always an “easy way back” by selecting all database content and exporting it to the Finder.

A comprehensive backup strategy for important data should consider such Bad Things That Can and Do Happen to People as hard drive crashes, theft of computer equipment and fires, floods, earthquakes, tornados, etc. (but I don’t try to cover nuclear war or asteroid strikes). Those considerations for protecting data apply whether DEVONthink databases are Imported or Indexed.

The decision to work with Indexed or Imported (self-contained) databases is up to the user’s preferences and workflows - so either can be appropriate for a given user.

I started using DEVONthink some nine years ago, so some of my databases contain files dating back to my initial database. I prefer Importing any important document file to a database instead of Indexing it. Over the years I’ve migrated my databases over a succession of Macs.

My backup strategy emphasizes redundancy and speed of recovery should anything happen to a computer or storage drive. The set of most commonly used DT Pro Office databases is present on more than one computer, as one form of backup. I use Time Machine for backups. And I create database archives (the smallest possible complete backups using commands built into DT Pro and Office) when significant changes have been made to a database, which I periodically copy to a portable hard drive that’s stored in a safety deposit box at my bank.

Currently, online backup is not a feasible option for me. I live in a log cabin in the woods and hills of Brown County, Indiana. My primary broadband access is via a satellite service, which has high latency and is very slow compared to cable or fiber optic service in less rural areas. Worse, the Internet connection drops out in bad weather either in my local area or at the distant satellite uplink station located in Texas. I have a backup access via a MobileMe and a (weak) Verizon wireless signal, but that’s even slower, and has a 5 GB/month cap - but it lets me check email and do some Web browsing.

So it would take at least a couple of days to upload to an online host my primary set of databases that I frequently have open, and I would likely receive warnings from my satellite ISP in progress and get access speed throttled down. Obviously, If I had to depend on recovery from online storage, recovery would be excruciatingly slow. That’s why I depend on recovery, if necessary, from that portable hard drive kept at my bank.

I’m looking forward to the new Sync procedure for databases that is currently under development. But given my database sizes and Internet access conditions, I’ll have to emphasize sync via local wired or wireless networks rather than cloud sync. In any case, I’ve got a couple of databases that I would never put up on the cloud even if encrypted, for security reasons.

[korm]

@KP

@NeverTheLess

One thing to note is: there’s no easy way back.

I have to disagree with you here. It is very simple to export files and groups from DT to the finder. Just use File>Export>Files and Folders. Of course, if you need ongoing access to files within DT from another program, and the “Open with…” command isn’t sufficient for you, indexing would be helpful.

Except … if you have made use of replicants, labels, annotations, or any work that relies on x-devonthink-item links (the “Item Link”) for documents inside a database, the replicants and labels will not survive an export, and the links might not survive if they refer to documents that were exported and then deleted from the database. Links to documents still in the database can survive.

[KP1]

@Bill_DeVille

A comprehensive backup strategy for important data should consider such Bad Things That Can and Do Happen to People as hard drive crashes, theft of computer equipment and fires, floods, earthquakes, tornados, etc. (but I don’t try to cover nuclear war or asteroid strikes).

…or the zombie apocalypse and the robot uprising.

@korm

Except … if you have made use of replicants, labels, annotations, or any work that relies on x-devonthink-item links (the “Item Link”) for documents inside a database, the replicants and labels will not survive an export, and the links might not survive if they refer to documents that were exported and then deleted from the database. Links to documents still in the database can survive.

Yep, I wasn’t clear. I was assuming the poster was talking about getting data out of DT if moving to another program. In that case, it would not be expected that DT metadata such as links within the database would survive.

[tshanno]

@korm

Except … if you have made use of replicants, labels, annotations, or any work that relies on x-devonthink-item links (the “Item Link”) for documents inside a database, the replicants and labels will not survive an export, and the links might not survive if they refer to documents that were exported and then deleted from the database. Links to documents still in the database can survive.

I always assumed DT exports replicants as duplicates of the same file. Otherwise how does it decide what’s a replicant and what’s the original?

Tom S.

[Greg_Jones]

I always assumed DT exports replicants as duplicates of the same file. Otherwise how does it decide what’s a replicant and what’s the original?

There is an occurrence of the replicant that is identified as the ‘master’ in the database, although it is transparent to the user. If you bring up the search window and search for a document that has replicants, select the document in the search results and type Command-R to reveal the document. That document is the ‘master’. If you delete that occurrence, then another occurrence of the replicant becomes the ‘master’.

[korm]

@tshanno

I always assumed DT exports replicants as duplicates of the same file. Otherwise how does it decide what’s a replicant and what’s the original?

When groups and documents are exported, replicants are also exported - as individual documents. So, they lose their affinity with one another. After export, changing one instance doesn’t change all instances of a given document, unlike the behavior of replicants where changing one instance changes every instance.

It takes more verbiage to describe what happens than to just test it out for oneself and see from experience what happens.

[tshanno]

Thanks to both of you. Yes, I should have just tried it.

Tom S.