tags, metadata, and smart groups

DEVONthink Tips
1

(Excuse the longish post, but I was so excited by the ability to create “tags” in DevonThink that I wanted to provide a summary.)

When I first used DevonThink, one of my fundamental questions had to do with metadata and tagging. Sites such as del.icio.us had spoiled me. I wanted some visual representation of the various topics contained in my database, some way of perusing the database by keyword or metadata.

While the concordance is an exceptionally useful tool for browsing through all the words contained in the database, it is somewhat broader than a keywords search. And while DevonThink is unparalleled in its ability both to locate specific information quickly and to suggest related information (See Also and Classify), I was looking for a more casual way of browsing the main categories in my database or doing searches that would bring up all items I had tagged “bookreviews” and “Marx” (for instance).

My first attempts at tagging involved replicating items to subject groups. This method, however, was a little cumbersome and time consuming; I ended up having to use the contextual menu a lot, since the “Move To” button would remove the item from its original (and permanent) location in my database. At the same time, this tagging system was highly unstable, since one accidental click of the “Move To” button would erase all other instances of an item (and thus eliminate its “tags”). Likewise, the “Ungroup” command raised the deadly possibility of losing all my filing work, since the “tags” were not actually attached to the item.

Since then, posts on this forum alerted me to the possibility of using comments and smart groups to create tags. The results have been nothing short of spectacular! Now I can have the best of all possible worlds: both DevonThink’s powerful organizational and search tools AND the simplicity and flexibility of a Web 2.0 tagging structure. Meanwhile, the basic underlying structure of my database (organized by source and/or date) remains very stable.

I know this all may seem fairly basic to many of the readers of this forum, but I wanted to take a moment to present my own method of tagging in DevonThink, which is wholly indebted to scripts and advice already posted here:

devon-technologies.com/phpBB … php?t=1465

devon-technologies.com/phpBB … ght=#13171

Here’s my basic method. Though it requires a bit more manual maintenance than web-based tagging systems, it nonetheless provides most of their features–and some others besides. (I would welcome any suggestions on how to tweak this method):

1)Everytime I create a new note or clipping in the database, I enter single-word tags in the comment field in the info box (e.g., history illness nineteenthcentury bookreviews). [There are also scripts for adding tags to multiple items, but I haven’t had time to check these out yet.]

2)Just to be on the safe side, I keep a list of all my tags in a sheet (which I can sort alphabetically). This might be overkill, but it provides a quick reference for all the keywords in the database and helps to prevent inconsistent labelling.

3)Using the Smart Groups [Comments] search script provided in the links above, I create a new Smart Group for each new keyword. With Quicksilver, I can create a new folder with just a couple of keystrokes. From now on, everytime I open this folder (filed in a group called “tags”), it will contain all of the items labelled with that particular tag (e.g., “Iraq”).

[One caveat: The Smart Group doesn’t update unless you click on it or open it, so the number of items in a group is only as accurate as the last time you accessed the group. Does anyone know of a script to trigger all the scripts associated with Smart Groups, so that one can get an up-to-date representation of their contents?]

4)And now for the really good stuff: Once you’ve built up your system of Smart Groups, you can use the classify button to suggest tags or keywords for each new item, as the classify button will show groups with related items. Think of this as something similar to the “recommended tags” feature on del.icio.us.

  1. Even better, if you have the “auto links” feature turned on, every time one of your keywords appears in a document, it will link to the relevant tag (i.e., smart group). Sometimes (especially with mashed keywords) you’ll have to create aliases in the info box of the Smart Group to make this work properly. (E.g., the keyword “nineteenthcentury” might need aliases such as “nineteenth,” “19th,” etc.)

This feature provides just one more way of discovering fortuitous links within my database. Since I now do all my writing in DevonThink, I’m never more than a click away from relevant sources, whether through auto-links, Classify, or See Also. In my opinion, the addition of tagging makes DevonThink by far the most robust and flexible personal information management software ever!

I now also have a way of browsing my notes by topic (something I do to generate ideas, especially when I’m not always sure what I’m looking for). Also, I can do keyword searches by searching only the comments field of my database.

I’m not sure what type of tagging or metadata capabilities will be included in DevonThink 2.0. My only concern is that my current metadata will not become obsolete. If there is a new keyword or tagging feature, will I be able to transfer my keywords in comments to that?

2

Hi ‘mdl,’ thanks for the long post. I too use a similar system for tagging. The only thing I would add is that you can use something like textpander or some such program to list tags and abbreviations. I can either type in my abbreviation or select it from a menu bar menu (which can get rather long with many tags) so my tags are consistent (so there are no typos!). It would be nice to be able to pull up a menu of keywords within DT for selection, but this works and eliminates the possibility of mistyping (which I am wont to do on occasion!).

I also use tags to tag not just individual files but entire folders, so I don’t necessarily have to tag each item within it.

Your linking and use of the classify button are really good suggestions. I too do most of my writing now in DT. I occasionally experiment with other writing-based programs. But keeping my writing projects in the same db as the research material I use has proven much more workable. I do a LOT of research for each project, and DTs ability to create webs of connection between all that material is critical to my work. I don’t know how I could ever manage all the information I collect without it.

So, using DTs scripting capabilities and finding ways to create tags and smart folders have been pretty critical for me as well for organizing and managing all this information! And DT can be a pretty workable writing program as well. The only real feature it lacks for me is the kind of quick outlining fuction Mori has, where I can press a tab key or a menu button and move an item around in an outlining structure. But, there are workarounds there too, if not as elegant.

Thanks again for taking the time to post your tagging system! It was helpful.

Alexandria

3

Alexandria,

Thanks for recommending textpander. It works much better than my previous method of keeping track of bookmarks, so I think I can scrap the internal list of keywords altogether. Do you happen to know if there’s any way to print Texpander’s list of words? It would be nice to have a paper copy of my keywords.

I would agree that DevonThink’s outliner is not very well suited to quick brainstorming and organization. I tend to go back and forth between OmniOutliner and DevonThink, brainstorming in the former and writing in the latter. An Omni outline exported in OPML format becomes a nice hierarchy of groups (folders) when imported into DevonThink. It is then easy to use these groups to organize one’s writing, either filing previous drafts and paragraphs into the appropriate place or writing from within the folders. This method is works especially well if the writing is broken up into paragraph-size chunks, which can be moved around quite easily and eventually merged into a single document.

I’d be interested in hearing more about how others use DevonThink for writing.

Matt

4

See the “Usage Scenario” thread, especially from a year or two ago, when we had much discussion about writing methods. Bottom line: everyone is different, but the DT products are flexible and adaptive.

5

An update to my original method of keeping track of tags and metadata:

I now keep the list of my keywords/tags in a rich text document instead of a sheet (one tag per line). Any time I enter new tags in the comments field of an item, I copy these to the tag list. (The “Words” button allows me to see if there are any duplicates.)

Using the Word Service package provided on the Devon Technologies website, I can sort these keywords alphabetically. This then provides me with a quick overview of tags for which smart groups exist, since these tags will appear as a link to the smart group.

As a solution, this is still more manual than I would like, but it nonetheless provides a way of keeping track of tags and gaining a quick overview of my notes. Here’s hoping that Version 2.0 will provide indexing of metadata. :slight_smile:

6

I love tags!

An example of tags is found in Notae, a nice little notepad app. Check out this link on Macupdate.com: macupdate.com/info.php/id/20405

You’ll see a screenshot with Notae’s tags. I would like to see that in DT. Notae comes with a Spotlight-enabled search function.

7

I am also looking for a way to have the Database update the smart groups.

Unlike what someone stated in a previous post on smart groups, it is not only the number of files in the smart folder that is updated when you click on that group; rather files are NOT transferred to that group until you click on it. This means that if one tags a file, and then fails to click on the smart group using that tag, the file will be lost if it is then deleted, since it never “made it” into the smart group it was tagged for.

In order to avert having to manually hunt through hundreds of groups to click on them in order to include the tagged file into its proper smart group, it would be useful to have a way to manually update the smart group … kind of like a “synchronize” for smart groups.

Is there any way to do this through AppleScript? Or is this a feature that will have to wait for v2.0?

8

talazem, if I understood your post you are manually saving items into existing smart groups. Is that correct?

Although I’m not entirely certain, as I haven’t experimented to see what happens, that may not work. The idea behind a smart group is that it contains the replicants of the results of a search query. Each time one clicks to open the smart group, it refreshes its contents.

I can think of a couple of possibilities. [1] refreshing might empty and then refill the group; [2] a new replicant might replace a previous replicant. And when one adds a comment in the Comment field of a replicant’s Info panel, does that comment get added to all instances of the replicant?

As I don’t do manual tagging or use smart groups as you are using them, those haven’t become live questions for me. But I would encourage you to experiment before getting an unpleasant surprise. Let us know what you find out. Your approach may work.

Comment: Personally, I find that I make fewer mistakes when I use the contextual menu options to move or replicate items, and get the operation done more quickly than by going to the Actions button.

9

Bill, I think you may have misunderstood me. I am not manually placing them; that quite obviously defeats the whole purpose of using tags with smart groups.

Let’s say I import an archive from Safari into DT. I have the script set to go into my *Inbox group. From there, I tag the archive. I then want to delete the archive from the inbox.

Now, without a way of autmatically updating the smart groups, I have to manually go to that smart group, and open it in order for the smart group to then import a replicate of that archive into it.

This is a problem because (a) it is manual, and I may have many archives or files in my inbox that I want to quickly be shipped off to their respective groups, and it is a pain to have to manually open the groups in order for the replicates to move into the groups; and (b) if I forget to do this, and I then delete the original file/archive from my *Inbox, the replicate will never get placed into its smart folder, and I’ll have lost the file.

Maybe a script could be written that would update all of the contents of the smart groups? Again, it is not just a matter of the number of files in the group being updated; rather, it is a matter of the files themselves being replicated to each of the tagged smart groups.

BTW, if anyone is wondering why such an effort of tagging is even necessary, my answer is: ease and safety. “Ease” in that searching for key words is at times more efficient than relying on the AI-aspects of DT (though they are extremely useful); and “safety” in that if I ever have to export my files out of DT to the Finder, or to another computer or operating system that doesn’t have DT on it, my categorization of my files will not be lost. Merely replicating files doesn’t guarantee that the categorization will survive a mass export of files, or the loss of DT environment. However, replicating through tags means that one will always now what basic categories, or groups, or folders (call them what you will) the files belong to.

In Bill’s terms, I have found tagging to be the best long-term information management “belts-and-suspenders” safety approach to organizing my information. In the past, I have been burnt several times by keeping files on MS Windows (was using Onfolio there), then when making the move to Mac, losing all the “metadata” of the categories and folders. Even within DT, when updating from 1.0.2 to a previous beta of 1.1, I lost all my replicants in what Christian later informed me was a bug. This bug caused me to lose all my replicants, and thus my organization of my material. I realized that if I had tagged, and not relied merely on manual replication, I could have quickly reconstructed my group/folder hierarchy with the smart groups, and all would have been easily saved.

In short, I think the power of DT’s AI features and replicants are best aided and accompanied with tags; and, vice versa, tagging is best utilized with the other tools offered by DT. If the upcoming v2.0, if it includes more robust smart groups and queries, will then have the best of both worlds; or rather, it will probably create a new world of info management that hasn’t existed in commercial KM software before.

10

Hi, talazem. No disagreements here.

It’s probably unlikely there will be any changes until true smart groups arrive, so your caution about not deleting an item until it’s smart group has been triggered is valid. In fact, it would probably be safest to move such items into a ‘holding’ group rather than deleting them. You can open the Info panel of such a holding group and tell DT Pro not to use its contents for Classification purposes.

11

The discussion here and elsewhere in these forums, as well as on 43Folders, has inspired me to try doing something like this. But before I go ahead I want to set up a faceted tagging system (please correct me if I’m misusing the term), and I think there would be value in setting up some quasi-standard practices. These would probably be even more valuable in del.icio.us and the like than in DT.

Imagine you find an article on learning environments that you want to read later. You might tag this learningenvironments (or learning_environments, or LearningEnvironments), and also @ToRead. The @ signifies that the article itself doesn’t discuss the ToRead concept but there is a ToRead action associated with the article. With the spread of GTD this seems to be becoming standard usage. But what if you decide you want your students to read the article? Giving it a forstudents tag doesn’t make much sense, because then searches for articles actually discussing things universities should do for students will be contaminated. And @ForStudents, which I have been using, doesn’t really make conceptual sense, unless I reinterpret it as “I must direct the students’ attention to this”. If we want to set up a collection for a particular group of students, such as EnvironmentalScience3a, the problem becomes clearer.

So what I’m wondering is: how many facets to tagging systems are there, which ones should we recognize explicitly, and what practices do people advocate in this regard?

P.S. The other issue alluded to, i.e. whether to tag something LearningEnvironments, learning_environments, or learningenvironments, is also quite important, I think. With comma-separated systems, even learning environments would be acceptable, though I know that many, including the
CiteULike developer, think that alternative is crazy.

12

Hi, mdl,

I have Word Service installed, along with many other Services, but I can’t find any Sort Alphabetically or similar command in my Services menu, nor any commands that start with Word. Could you give a pointer as to where to find this function?

Thanks,
Rick

13

Rickl,

Services > Format > Sort Lines Ascending.

14

Got it. Thanks!

15

Bill wrote:

This is exactly what I am doing. I am using the comments field for my action-related items (i.e. to do’s, calls, pending, etc.). Then, I create a smart group based on the letter(s) in the comment field. For example, “a” means “action,” so I have a smart folder named “a.” If a document that I have tagged as “a” is already in a group (i.e. “Newsletter,” then a replicant is created in the “a” smart folder. However, there are some miscellaneous action items that don’t belong to any particular grouping, i.e. “redo fax form.” I place these directly in the “a” smart group. Yes, I could create a miscellaneous group, but I find that that creates an unnecessary step.

Bill cautions against doing this. So far (knock on wood ), I haven’t noticed any problems, but I certainly don’t want to invite trouble down the road. I’m wondering if anyone else does this and if so, have you run into any problems. Does anyone know what potentially could happen by creating documents directly within smart groups?

Karen

16

I tested that quite awhile ago and it seemed to work fine for awhile although later that database contained many corrupted items which had been replicants and/or in groups with attached scripts (including smart groups) immediately after moving it OS X 10.3 to 10.4. I didn’t take time to try isolating the cause (fortunately only that one database was affected) but it was enough of a bummer to keep me from using smart groups at all ever since.

I still don’t fully trust attaching several kinds of scripts to groups because of uncertainty of interactions that might occur (sooner or later) with items/content in them, e.g. adding/deleting of items to smart groups like you’re asking about. A specific current issue I have is not being able to remove an attached script without selecting the group and triggering it to run. For instance, is there any way to detach the Synchronize script from a group without it accessing the external folder it references?

17

sjk wrote:

Thank you, sjk, for your helpful response. I have spent a while pondering what you have said (thus the late reply) and have consequently moved non-replicants out of the smart groups.

I do a daily backup of my files to an external drive. Before installing the next major upgrade, I will make a copy of the DT database and delete all the smart groups from the copy. I can then check my data after the upgrade and if I find corruption, I can use the copy without the smart groups.

Thanks again for your reply!

Karen

18

Finally taking time to catch up on the past month’s forum postings …

And here’s my very belated “you’re welcome”. :slight_smile:

19

This is a bit too late to be of much use, but I have an Applescript called ‘Trigger Attached Script’ in the ‘Shell Add-ons’ available in the academy (http://www.devon-technologies.com/support/academy/dt_scripts.html.

Regarding the use of a sheet for tags: a small Applescript could be written using the ‘choose from list’ dialog, which allow multiple selection. This would allow you to select a number of records, choose one or more tags from a list box derived from the sheet, and add those tags to the Comment of all selected records.

–Eric