Make an Annotation with Links, Notes, Tags v2

Inspired by korm’s work to build a better interface for the annotation template using keyboard maestro, here is an advanced annotation template that doesn’t require km. Like the most recent versions it uses tags to allow multiple annotations to a single document.

Annotation with date.jpg

This script has a few advantages over previous scripts:
a) You can edit the cited text at the time of making the annotation
b) You can choose to open the annotation document immediately after doing the annotation.
c) You can set the creation date of the annotation. If the annotation box check box is ticked the default creation date of the annotation is the same as that of the document.
d) The notes field can be copied to the spotlight comment field.
e) It cleans up the cited text by removing any non alpha-numeric text.

(c and d are especially useful for creating chronologies)

Just unzip and drop the script bundle into your devonthink scripts directory (taking note of korm’s advice below if you have a problem).

Credits:
The core of the code is korm’s code used in his keyboard maestro script.
The gui is built with Carsten Blüm’s Pashua (http://www.bluem.net/en/mac/pashua/), and the Pashua runtime is included in the bundle.

[edit]
v1.1 Search link works properly; the document being annotated is the frontmost document rather than the selected document; c & d added.
Quick annotate with group tagging v1.1.scptd.zip (264 KB)

Revised v2 above

Updated

Thanks to everyone for this thread. I (naively?) thought that the ability to take, and sort, and link notes about documents was the main rationale for Devonthink as a research tool. But until I found this thread, I was unable to figure out how to do it. The manual has 238 mentions of “link,” but I could not find a coherent explanation of how to create cross-links among master/related documents. (e.g. a tutorial. See below for a partial answers.)

In any case, I think with these threads I will be able to figure this out, with “only” another hour or two of effort. Since I’m just a trial user of DT, it may make the difference between buying it, and giving up.

Now I suggest, echoing earlier comment, that this information be pulled into a sticky thread, and ultimately into a PDF tutorial. For other newbies, here are the threads I’ve found that are key.
Removed Macro for Keyboard Maestro
QuoteHighlight&Annotate script
posting.php?mode=reply&f=20&t=18885 (this thread)
manually cross-linking & adding rich text notes to items?
Mystified by Annotation - is there a plain-language tutorial A useful tutorial on how to use all this.
Related:
A useful discussion of how to structure documents in a DB and use “see also” to find links. Someone else's use of AI

Finally, the bibliography manager Bookends has some of this capability. Every referenced document can have an unlimited number of “notes” which are tied to that document. The notes can be tagged individually. And when you highlight a passage in a PDF, it automatically creates a note containing the highlighted text. Of course, Bookends is specialized and much less flexible.
(All errors and omissions in this message are mine. I’m just starting to figure this out. If someone who does understand these issues would summarize the essentials, we would all be grateful.)

@ R^2_is_misleading: Stick with it. DT is complex because of its freeform nature. When it all clicks together I haven’t found any other piece of software which remotely compares.

How annotations work

An annotation contains several links to the page in the document that is being annotated. (If its a pdf, otherwise just to the document)
a) In the Url field;
b) In the Hyperlink in the document called Page;
c) In the Hyperlink in the document called Text.

The annotation can also be reached under any of the Tag groups which reference it.

Annotation_Document.jpg

All the annotations to an original document can be reached by clicking on the URL link at the top of the document. This brings up a separate window with a list of all the annotations to the document. (technically its the tag group hidden under the tag group annotations but you should never have to worry or look in that tag group)

linking tag group.jpg

I hope this is helpful.
Fredriko

(There are some interesting posts on the bookend forum on integrating bookends annotations with DT https://www.sonnysoftware.com/phpBB3/viewtopic.php?f=6&t=3465

THIS VERSION HAS BEEN DEPRECATED
There is a later version at http://forum.devontechnologies.com/viewtopic.php?f=20&t=21707

This is a revised and enhanced version of the previous annotation template. This template adds auto-completion for tagging and provision for sub-group tagging.

Annotation tagging with sub issues.jpg

Changes and usage

  • The date fields have now been separated and placed near the top so they can been easily tabbed between. This is the creation date field for the annotation. By default it is the same date as the document being annotated. The date format is year, month, day
  • The annotation prefix field has been removed. The ability to rename the annotation tag name has also been removed. I have never once needed to reach an annotation this way so it appears redundant.
  • The “Annotation Tags” field offers a drop down box and auto-complete suggestion for the first tag. You can still enter further tags separated by semi-colons but it won’t attempt to auto-complete subsequent tags.
  • Two specific tag sub-groups are provided for. In this case “Issues” and “People” but their names can be changed in the first lines of the script. Three auto-complete tag fields are provided for each sub group. These fields will only auto-complete on other tags in the sub group. Sub-group tags do not show up under the “Annotation Tags” field auto-complete. (Look under the “Tags” group to see how these are structured.
  • There are four checkboxes:
    [list][*] Open annotation - This opens the annotation immediately after the dialog for further editing.
  • Notes in comment - Notes will also be placed in the spotlight comment field.
  • Set annotation date - If this is not checked then the creation date of the annotation will be the date the annotation is made. If it is checked the creation date of the annotation date will be that set in the Date boxes.
  • Set annotation location- By default all annotations are stored in a folder called “annotations” in the inbox. If you want to bring up the group selector to choose where to store the annotation, this box should be checked.
    [/*:m][/list:u]
    Just unzip and drop the script bundle into your devonthink scripts directory (The ‘open scripts folder’ menu item).

If you want to produce a spreadsheet of the annotations and source documents then this thread http://forum.devontechnologies.com/viewtopic.php?f=20&t=19534 will be helpful.

There is also a variation on this script which clips whole pages or series of pages as pdfs. Useful for annotating graphical documents. viewtopic.php?f=20&t=19827

Notes on Gatekeeper: If the mac prevents you from running this script bundle, read this http://support.apple.com/kb/HT5290 on how to change your security settings or try Korm’s advice earlier in this thread.

Notes on launch speed: If this script is activated from DT’s script menu I have seen it take up to thirty seconds to launch. I have no idea why. If you are having this problem I recommend placing the script in the user script’s folder and using Fastscripts (http://www.red-sweater.com/fastscripts/) to attach a keystroke to it and launch it instantaneously.

Frederiko

(Note: For me personally this template offers some of the functionality of Lexisnexis Casemap (a windows litigation preparation package) but with the greater flexibility of DT.

[edit v2.1 2014/09/27] Addition of dropdown menu for label and autocomplete tags are now in alphabetical order. Minor change to template to add reference to source document and link to page

[edit 2014/11/3] If the script isnt working for you its because your date/time format is different to that the script was written for. A fix is coming.
Annotation template with specific issue tagging v2.1.scptd.zip (284 KB)

Frederiko - thanks again - this just keeps getting better! :slight_smile:

Could I maybe confirm some things on your usage, since this would make things easier on getting my head around how I could utilise it…

I see, based on the previous screengrabs, that you use the 3 fields under the [Issues] and [People] to generate new tag groups. You then presumably place the relevant pdf’s/rtf’s into these groups - so you’re basically using Tags as Groups (I think that’s the right way of terming it)…

So if you’re working on your master document, you in effect start “building” a folder/group structure around that master document, driving it through the creation of Tags? Is that more-or-less how you do things?

Hello Cassady,

Tags and Folders are different

The annotation file must always be stored in a proper folder and not in a tag group (although its possible to do so, it has unintended consequences). Tag groups should only ever contain replicants (and they are not true replicants because deleting the annotation file will also remove it from all the tags it is associated with). The annotation template takes care of this so you shouldn’t have to worry about this occuring.

The only way you should ever add a tag to a document is by adding the tag to the documents tag list (the strip at the bottom of the document) or in the information panel.

Tags are for metadata and Folders are for documents

The default is to store the annotation file in a folder called ‘annotations’ in the Inbox but you can store it anywhere that works for you. For some types of documents I like to store the annotation in the same folder as the document being annotated.

Folders and tags are two completely different organisational structures. For me folders represent ‘physical bundles’ of document whether they be transcripts of hearings, or documentary evidence received from one of the parties to the litigation. Annotations are also ‘physical’ but more akin to Post-it Notes or note cards. Tags are a way of classifying the information in the ‘physical bundles’ and narrowing it through annotations into more abstract and precise categories. Tags are also a way of pulling related information from different documents into the same logical category. Tags are akin to the index in a book.

The way I use the tags is best illustrated with the following four examples:

a) I want to find all annotations (or source documents) that concern ‘John Smith’ AND the issue of ‘misrepresentation’. I can use advanced search or a smart group to find all annotations that are the intersection of these two tags.

It doesn’t matter where these annotations (or documents with the same tags) are stored in the folder structure. A search on one or more of the tags will find the annotations whereever they are.

In this example I might turn up a passage from an evidence transcript at trial by Mary W about the representations she made to John Smith and an extract from a statement she gave to the police on a different occasion a year earlier about the same thing. These two pieces of information may be hundreds of pages apart and in different folders in documents received from different people. Previously I might have overlooked the linkage.

b) The reason for the tag sub-groups is that I found groups of logically related pieces information (in my case, People & Issues) were being lost in my tag cloud. When I am annotating the 100th document I can no longer remember whether I tagged the person I saw in document 1 as ‘Smith’, ‘J Smith’ or ‘John Smith’. Having separate tag sub groups not only allows for auto-complete just on a subset of the tags, and thereby aids consistency, but also reminds me constantly that its an important field that I probably should complete. It’s so easy to get lazy when you have thousands of pages to work through and cross-reference.

c) Tags make it easy to extract just a portion of evidence. Before I go to trial I can give John Smith just a subset of the documents that are relevant to him together with a list of pages he should pay specific attention to. In a similar vein I may want an overview of just what evidence I have on the issue of misrepresentation and whether it needs to be bolstered by looking for further evidence. Looking in the tag group ‘Issues’ shows all the issues in the case that I have identified, and burrowing down further to the ‘misrepresentation’ tag shows all the annotations and documents that deal with the question of misrepresentation. I will generally export a summary of these annotations and their source documents to numbers for printing so they can be discussed in a strategy conference.

d) A witness may refer to a particular document. Clicking on the url link brings up a window with all the other annotations I have made to that document which will probably include what other witnesses have said about the same document. Opening an annotation will show me my notes to the cited passage and the tags at the bottom of the document can take me to the tag groups containing all the other related referenced people and issues.

The examples I have given are quite specific to what I do, but I hope it will stimulate people to think about how they use DT and modify the workflow to suit them.

I hope this provides some clarity.

Frederiko

Many thanks Frederiko, it does!

I’ve been playing around with it over the course of the past few weeks - and the possibilities are almost endless! :smiley:

This looks like a great script! It’s seriously impressive (esp. to those of us who has no real knowledge of scripting), and we’re very grateful for @korm for first creating the original script! Once again, amazing work and the DTP community is thankful for your generous help…

I’m having some problems operating this new function, and I’d be grateful for any help… I downloaded and installed the latest script version that @Frederiko created (i.e., Annotation template with specific issue tagging v2.1.scptd.zip), and have tried to run it, using the steps outlined above. The very first time I selected text from a PDF document and ran the script it seemed like the script worked, but only partly…

For instance, after I filled out all of the information from the dialogue box, the script created an annotation document (as it should), but that document remained stuck inside the same folder as the original document – whenever I clicked on the annotation file (hoping to add on more to it) it just wouldn’t open. So, I tried to start over: I deleted that newly created annotation file in the trash, emptied the trash, and again following the steps outline above. But now, every time I run the script, the new annotated document automatically ends up in the trash – that is, the Mac trash, not the DTP trash. (I followed the steps that @korm suggested to fix Pashua, but it didn’t have any effect.)

I’m sure I’m committing some elementary error here… Anyway, I would appreciate any help you could offer. Thanks!

@jprint174

It’s hard to tell from your description what it is about your setup that is causing the problem. Could I ask you to open a new database, put in a few documents and annotate them. At least this would tell me whether the problem lies in the setup of your computer or is specific to a particular database.

If the problem persists I will pm you a special version of the script that writes a log file of the important variables and maybe we can work it out from that.

Frederiko

Revised v2 above not requiring additional S/W etc.

I really appreciate your help with this… Ok, I followed all of the steps, but unfortunately the problem persists.

I followed all of @korms steps, ran Verify & Repair severals times, cleared out all of my orphans, emptied the trash, quit, rebooted, etc. I even deleted the old Annotation script, and reinstalled it. And I followed all of @Frederiko’s steps as well. In fact, I was wondering if the problem might be because I have so many groups and files stored in one particular DB that I use, so I was happy to create a fresh DB and use the script within it. (I’m now wondering if I should create a new DB to accommodate an upcoming project that will be data heavy - and create a separate set of tags for it as well - just in case it would improve functionality and mitigate future problems that could come from a bloated DB.)

Anyway, I’m sad to report that after I went through all of the steps creating the annotation files w/ this script, those files kept going straight into the Mac’s trash (not the DTP trash). By the way, as I’m sure you know, the script automatically created the Issues and People tag categories (for the parent DB tag directory - not that I have a separate tagging system in any of my groups). When I first ran the script, it did say that I need to detail a value or location of the “Annotation Tag” (or something to that effect…I can’t recall what exactly the dialogue box said). Is that helpful? As part of that, were the specific steps that I should have taken before selecting text and running the script (e.g., tag creation or management)? My sense is that one didn’t have to do much preparation w/ @Frederiko’s latest script, but maybe I overlooked something.

I’d be glad to take whatever other steps you could suggest to fix this problem and ensure the script works. Many thanks again for all of your help!

Revised v2 above

Thanks very much for your message. Just to rule things out… You said “the number of groups and documents you have in a database is not a factor.” This might be a sidebar issue, but does it matter one way or there other - in terms of better performance and reducing glitches - to to break up files/groups by putting them in separate DB’s? (Maybe there’s an added organizational benefit by breaking things up into separate DB’s…dunno.) And does it matter where the tags are located vis-a-vis groups in a DB? As I said, I orignially had tags set up in the parent DB tag directory (forgive me if there’s another description for this location), and was wondering if it might make any difference if I move tags within particular groups. (I know this requires changing the DB properties.) I’ve tended to leave things like tags where DTP originally creates them, but again, maybe it’s advisable to move them elsewhere (per a better organizational set up).

Anyway, after I went through all of your clean up steps, I cleared the DTP log so I don’t see anything there. I opened Console, and am looking through it now, but don’t see anything that clearly identifies script-related errors. I’m not very familiar w/ Console, however, so I’m probably overlooking something.

Here’s one thing… After I deleted the script and reinstalled it, I notice the new script still had some of the old values (e.g., tags that I set up) left over from the original time that I used it. Would it be possible that there’s something left over in the script (some preferences perhaps?) that got corrupted? Just troubleshooting here…

No, I’m not running Keyboard Maestro - I only used it once, long ago. I’m not sure what else I could be running that would have an effect of DTP as far as trashing the files. By the way, the files appear in the trash w/ file names like: “Pashua_238868488798” Is that helpful?

Thank you again, and please let me know if I can provide any more info that could help.

Revised v2 above

@korm Thanks for your reply. I’ll answer your second question first. To clarify, I created tags – within the People and Issues tags section – from within the dialogue box that emerges when executing @Federiko’s script. After I followed you steps to clean and repair my DTP DB’s, I deleted that scripted, quit DTP, rebooted, etc. Eventually, I re-installed the script, and when I did I noticed that the tags within the script’s dialogue box had the same tags that I had just made – an I thought they would be deleted when I trashed the original script. So, I assumed that maybe there was some residual elements of the script (or something that I created as part of operating the script) that were still in my DTP, and might be related to the problems that I’m facing. Does that make sense?

As far as I can tell, I have been following all of @Federiko’s instructions. But hey, it’s certainly possible that I’ve overlooked something. I’ve tried to carefull follow each step.

Thanks for the info re: the DB’s organizatonal set up not being a factor as it relates to this issues. By the way, I spent quite a bit of time looking through forum search results for helpful posts re: DB size and organization (esp. the latter). I’ll continue to look through them, but I’d be grateful for any suggested posts that would be particularly helpful. Thanks again!

I’ve been revisiting this fantastic script, and have quick questions about how to use it vis-a vis my workflow and organizational set up. I have one main DB that contains most of my research for all of my projects in one location. (I’m working on one massive book project, and six relatively smaller research projects.)

I’ve been debating whether to keep everything in one DB or to split things up – mostly to better streamline the kind of granular tagging that this encourages. For now, it seems best to keep everything in one DB since some of the topics are overlapping (and tags are supposed to be for “abstract concepts,” right?). So, I’m wondering: (a) If this organizational set up seems like a sound approach – or if others would advise something different, and (b) if keeping the all-in-one research DB seems best, if one should consider creating further sub-group tags to differentiate between these different projects – just to achieve better organizational order?

Finally, one quick functional question… What’s the best ways to located tagged documents through selecting a certain combination of tags? For instance, if I had tags for “Europe,” “Dogs,” “Portugese Water Dogs,” “Research Later,” and wanted to find all research topics relevant to Portugese water dogs and research later, how could I best find all tagged notes with associated the “Portugese Water Dogs” and “Research Later”? (A colleague suggested Scrivner is an excellent app for this, but it doesn’t have tagging that allow for a kind of aliases.)

Thanks again!

Damn. Wish I’d known this before using Highlights to annotate a doctoral thesis and a full-length book for a project I’m working on. Something like this capability should be baked into Devonthink. Thanks, Frederiko & Korm, for a lovely piece of scripting.

Actually, I think you might enhance your workflow by using a Highlights-created text as a reference file, as you’re deciding which parts of your annotated PDFs you want to tag w/ this script. Just a thought… That’s how I plan to use both of them together!