Make an Annotation with Links, Notes, Tags v2

[Frederiko]

THIS VERSION HAS BEEN DEPRECATED
There is a later version at http://forum.devontechnologies.com/viewtopic.php?f=20&t=21707

This is a revised and enhanced version of the previous annotation template. This template adds auto-completion for tagging and provision for sub-group tagging.

Changes and usage

• The date fields have now been separated and placed near the top so they can been easily tabbed between. This is the creation date field for the annotation. By default it is the same date as the document being annotated. The date format is year, month, day
• The annotation prefix field has been removed. The ability to rename the annotation tag name has also been removed. I have never once needed to reach an annotation this way so it appears redundant.
• The “Annotation Tags” field offers a drop down box and auto-complete suggestion for the first tag. You can still enter further tags separated by semi-colons but it won’t attempt to auto-complete subsequent tags.
• Two specific tag sub-groups are provided for. In this case “Issues” and “People” but their names can be changed in the first lines of the script. Three auto-complete tag fields are provided for each sub group. These fields will only auto-complete on other tags in the sub group. Sub-group tags do not show up under the “Annotation Tags” field auto-complete. (Look under the “Tags” group to see how these are structured.
• There are four checkboxes:
  [list][*] Open annotation - This opens the annotation immediately after the dialog for further editing.
• Notes in comment - Notes will also be placed in the spotlight comment field.
• Set annotation date - If this is not checked then the creation date of the annotation will be the date the annotation is made. If it is checked the creation date of the annotation date will be that set in the Date boxes.
• Set annotation location- By default all annotations are stored in a folder called “annotations” in the inbox. If you want to bring up the group selector to choose where to store the annotation, this box should be checked.
  [/*:m][/list:u]
  Just unzip and drop the script bundle into your devonthink scripts directory (The ‘open scripts folder’ menu item).

If you want to produce a spreadsheet of the annotations and source documents then this thread http://forum.devontechnologies.com/viewtopic.php?f=20&t=19534 will be helpful.

There is also a variation on this script which clips whole pages or series of pages as pdfs. Useful for annotating graphical documents. viewtopic.php?f=20&t=19827

Notes on Gatekeeper: If the mac prevents you from running this script bundle, read this http://support.apple.com/kb/HT5290 on how to change your security settings or try Korm’s advice earlier in this thread.

Notes on launch speed: If this script is activated from DT’s script menu I have seen it take up to thirty seconds to launch. I have no idea why. If you are having this problem I recommend placing the script in the user script’s folder and using Fastscripts (http://www.red-sweater.com/fastscripts/) to attach a keystroke to it and launch it instantaneously.

Frederiko

(Note: For me personally this template offers some of the functionality of Lexisnexis Casemap (a windows litigation preparation package) but with the greater flexibility of DT.

[edit v2.1 2014/09/27] Addition of dropdown menu for label and autocomplete tags are now in alphabetical order. Minor change to template to add reference to source document and link to page

[edit 2014/11/3] If the script isnt working for you its because your date/time format is different to that the script was written for. A fix is coming.
Annotation template with specific issue tagging v2.1.scptd.zip (284 KB)

[Cassady]

@Frederiko

This is a revised and enhanced version of the previous annotation template. This template adds auto-completion for tagging and provision for sub-group tagging.

[attachment=0]Specific annotation.jpg[/attachment]

…

Just unzip and drop the script bundle into your devonthink scripts directory (The ‘open scripts folder’ menu item).

If you want to produce a spreadsheet of the annotations and source documents then this thread [url]Export an index of annotations to a numbers spreadsheet] will be helpful.

Notes on Gatekeeper: If the mac prevents you from running this script bundle, read this http://support.apple.com/kb/HT5290 on how to change your security settings or try Korm’s advice earlier in this thread.

Frederiko

(Note: For me personally this template offers some of the functionality of Lexisnexis Casemap (a windows litigation preparation package) but with the greater flexibility of DT.

Frederiko - thanks again - this just keeps getting better!

Could I maybe confirm some things on your usage, since this would make things easier on getting my head around how I could utilise it…

I see, based on the previous screengrabs, that you use the 3 fields under the [Issues] and [People] to generate new tag groups. You then presumably place the relevant pdf’s/rtf’s into these groups - so you’re basically using Tags as Groups (I think that’s the right way of terming it)…

So if you’re working on your master document, you in effect start “building” a folder/group structure around that master document, driving it through the creation of Tags? Is that more-or-less how you do things?

[Frederiko]

Hello Cassady,

Tags and Folders are different

The annotation file must always be stored in a proper folder and not in a tag group (although its possible to do so, it has unintended consequences). Tag groups should only ever contain replicants (and they are not true replicants because deleting the annotation file will also remove it from all the tags it is associated with). The annotation template takes care of this so you shouldn’t have to worry about this occuring.

The only way you should ever add a tag to a document is by adding the tag to the documents tag list (the strip at the bottom of the document) or in the information panel.

Tags are for metadata and Folders are for documents

The default is to store the annotation file in a folder called ‘annotations’ in the Inbox but you can store it anywhere that works for you. For some types of documents I like to store the annotation in the same folder as the document being annotated.

Folders and tags are two completely different organisational structures. For me folders represent ‘physical bundles’ of document whether they be transcripts of hearings, or documentary evidence received from one of the parties to the litigation. Annotations are also ‘physical’ but more akin to Post-it Notes or note cards. Tags are a way of classifying the information in the ‘physical bundles’ and narrowing it through annotations into more abstract and precise categories. Tags are also a way of pulling related information from different documents into the same logical category. Tags are akin to the index in a book.

The way I use the tags is best illustrated with the following four examples:

a) I want to find all annotations (or source documents) that concern ‘John Smith’ AND the issue of ‘misrepresentation’. I can use advanced search or a smart group to find all annotations that are the intersection of these two tags.

It doesn’t matter where these annotations (or documents with the same tags) are stored in the folder structure. A search on one or more of the tags will find the annotations whereever they are.

In this example I might turn up a passage from an evidence transcript at trial by Mary W about the representations she made to John Smith and an extract from a statement she gave to the police on a different occasion a year earlier about the same thing. These two pieces of information may be hundreds of pages apart and in different folders in documents received from different people. Previously I might have overlooked the linkage.

b) The reason for the tag sub-groups is that I found groups of logically related pieces information (in my case, People & Issues) were being lost in my tag cloud. When I am annotating the 100th document I can no longer remember whether I tagged the person I saw in document 1 as ‘Smith’, ‘J Smith’ or ‘John Smith’. Having separate tag sub groups not only allows for auto-complete just on a subset of the tags, and thereby aids consistency, but also reminds me constantly that its an important field that I probably should complete. It’s so easy to get lazy when you have thousands of pages to work through and cross-reference.

c) Tags make it easy to extract just a portion of evidence. Before I go to trial I can give John Smith just a subset of the documents that are relevant to him together with a list of pages he should pay specific attention to. In a similar vein I may want an overview of just what evidence I have on the issue of misrepresentation and whether it needs to be bolstered by looking for further evidence. Looking in the tag group ‘Issues’ shows all the issues in the case that I have identified, and burrowing down further to the ‘misrepresentation’ tag shows all the annotations and documents that deal with the question of misrepresentation. I will generally export a summary of these annotations and their source documents to numbers for printing so they can be discussed in a strategy conference.

d) A witness may refer to a particular document. Clicking on the url link brings up a window with all the other annotations I have made to that document which will probably include what other witnesses have said about the same document. Opening an annotation will show me my notes to the cited passage and the tags at the bottom of the document can take me to the tag groups containing all the other related referenced people and issues.

The examples I have given are quite specific to what I do, but I hope it will stimulate people to think about how they use DT and modify the workflow to suit them.

I hope this provides some clarity.

Frederiko

[Cassady]

@Frederiko

Hello Cassady,

…

I hope this provides some clarity.

Frederiko

Many thanks Frederiko, it does!

I’ve been playing around with it over the course of the past few weeks - and the possibilities are almost endless!

[jprint714]

This looks like a great script! It’s seriously impressive (esp. to those of us who has no real knowledge of scripting), and we’re very grateful for @korm for first creating the original script! Once again, amazing work and the DTP community is thankful for your generous help…

I’m having some problems operating this new function, and I’d be grateful for any help… I downloaded and installed the latest script version that @Frederiko created (i.e., Annotation template with specific issue tagging v2.1.scptd.zip), and have tried to run it, using the steps outlined above. The very first time I selected text from a PDF document and ran the script it seemed like the script worked, but only partly…

For instance, after I filled out all of the information from the dialogue box, the script created an annotation document (as it should), but that document remained stuck inside the same folder as the original document – whenever I clicked on the annotation file (hoping to add on more to it) it just wouldn’t open. So, I tried to start over: I deleted that newly created annotation file in the trash, emptied the trash, and again following the steps outline above. But now, every time I run the script, the new annotated document automatically ends up in the trash – that is, the Mac trash, not the DTP trash. (I followed the steps that @korm suggested to fix Pashua, but it didn’t have any effect.)

I’m sure I’m committing some elementary error here… Anyway, I would appreciate any help you could offer. Thanks!

[Frederiko]

@jprint174

It’s hard to tell from your description what it is about your setup that is causing the problem. Could I ask you to open a new database, put in a few documents and annotate them. At least this would tell me whether the problem lies in the setup of your computer or is specific to a particular database.

If the problem persists I will pm you a special version of the script that writes a log file of the important variables and maybe we can work it out from that.

Frederiko

[korm]

Revised v2 above not requiring additional S/W etc.

[jprint714]

I really appreciate your help with this… Ok, I followed all of the steps, but unfortunately the problem persists.

I followed all of @korms steps, ran Verify & Repair severals times, cleared out all of my orphans, emptied the trash, quit, rebooted, etc. I even deleted the old Annotation script, and reinstalled it. And I followed all of @Frederiko’s steps as well. In fact, I was wondering if the problem might be because I have so many groups and files stored in one particular DB that I use, so I was happy to create a fresh DB and use the script within it. (I’m now wondering if I should create a new DB to accommodate an upcoming project that will be data heavy - and create a separate set of tags for it as well - just in case it would improve functionality and mitigate future problems that could come from a bloated DB.)

Anyway, I’m sad to report that after I went through all of the steps creating the annotation files w/ this script, those files kept going straight into the Mac’s trash (not the DTP trash). By the way, as I’m sure you know, the script automatically created the Issues and People tag categories (for the parent DB tag directory - not that I have a separate tagging system in any of my groups). When I first ran the script, it did say that I need to detail a value or location of the “Annotation Tag” (or something to that effect…I can’t recall what exactly the dialogue box said). Is that helpful? As part of that, were the specific steps that I should have taken before selecting text and running the script (e.g., tag creation or management)? My sense is that one didn’t have to do much preparation w/ @Frederiko’s latest script, but maybe I overlooked something.

I’d be glad to take whatever other steps you could suggest to fix this problem and ensure the script works. Many thanks again for all of your help!

[korm]

Revised v2 above

[jprint714]

Thanks very much for your message. Just to rule things out… You said “the number of groups and documents you have in a database is not a factor.” This might be a sidebar issue, but does it matter one way or there other - in terms of better performance and reducing glitches - to to break up files/groups by putting them in separate DB’s? (Maybe there’s an added organizational benefit by breaking things up into separate DB’s…dunno.) And does it matter where the tags are located vis-a-vis groups in a DB? As I said, I orignially had tags set up in the parent DB tag directory (forgive me if there’s another description for this location), and was wondering if it might make any difference if I move tags within particular groups. (I know this requires changing the DB properties.) I’ve tended to leave things like tags where DTP originally creates them, but again, maybe it’s advisable to move them elsewhere (per a better organizational set up).

Anyway, after I went through all of your clean up steps, I cleared the DTP log so I don’t see anything there. I opened Console, and am looking through it now, but don’t see anything that clearly identifies script-related errors. I’m not very familiar w/ Console, however, so I’m probably overlooking something.

Here’s one thing… After I deleted the script and reinstalled it, I notice the new script still had some of the old values (e.g., tags that I set up) left over from the original time that I used it. Would it be possible that there’s something left over in the script (some preferences perhaps?) that got corrupted? Just troubleshooting here…

No, I’m not running Keyboard Maestro - I only used it once, long ago. I’m not sure what else I could be running that would have an effect of DTP as far as trashing the files. By the way, the files appear in the trash w/ file names like: “Pashua_238868488798” Is that helpful?

Thank you again, and please let me know if I can provide any more info that could help.

[korm]

Revised v2 above

[jprint714]

@korm Thanks for your reply. I’ll answer your second question first. To clarify, I created tags – within the People and Issues tags section – from within the dialogue box that emerges when executing @Federiko’s script. After I followed you steps to clean and repair my DTP DB’s, I deleted that scripted, quit DTP, rebooted, etc. Eventually, I re-installed the script, and when I did I noticed that the tags within the script’s dialogue box had the same tags that I had just made – an I thought they would be deleted when I trashed the original script. So, I assumed that maybe there was some residual elements of the script (or something that I created as part of operating the script) that were still in my DTP, and might be related to the problems that I’m facing. Does that make sense?

As far as I can tell, I have been following all of @Federiko’s instructions. But hey, it’s certainly possible that I’ve overlooked something. I’ve tried to carefull follow each step.

Thanks for the info re: the DB’s organizatonal set up not being a factor as it relates to this issues. By the way, I spent quite a bit of time looking through forum search results for helpful posts re: DB size and organization (esp. the latter). I’ll continue to look through them, but I’d be grateful for any suggested posts that would be particularly helpful. Thanks again!

[jprint714]

I’ve been revisiting this fantastic script, and have quick questions about how to use it vis-a vis my workflow and organizational set up. I have one main DB that contains most of my research for all of my projects in one location. (I’m working on one massive book project, and six relatively smaller research projects.)

I’ve been debating whether to keep everything in one DB or to split things up – mostly to better streamline the kind of granular tagging that this encourages. For now, it seems best to keep everything in one DB since some of the topics are overlapping (and tags are supposed to be for “abstract concepts,” right?). So, I’m wondering: (a) If this organizational set up seems like a sound approach – or if others would advise something different, and (b) if keeping the all-in-one research DB seems best, if one should consider creating further sub-group tags to differentiate between these different projects – just to achieve better organizational order?

Finally, one quick functional question… What’s the best ways to located tagged documents through selecting a certain combination of tags? For instance, if I had tags for “Europe,” “Dogs,” “Portugese Water Dogs,” “Research Later,” and wanted to find all research topics relevant to Portugese water dogs and research later, how could I best find all tagged notes with associated the “Portugese Water Dogs” and “Research Later”? (A colleague suggested Scrivner is an excellent app for this, but it doesn’t have tagging that allow for a kind of aliases.)

Thanks again!

[amcawood]

Damn. Wish I’d known this before using Highlights to annotate a doctoral thesis and a full-length book for a project I’m working on. Something like this capability should be baked into Devonthink. Thanks, Frederiko & Korm, for a lovely piece of scripting.

[jprint714]

@amcawood

d**n. Wish I’d known this before using Highlights to annotate a doctoral thesis and a full-length book for a project I’m working on. Something like this capability should be baked into Devonthink. Thanks, Frederiko & Korm, for a lovely piece of scripting.

Actually, I think you might enhance your workflow by using a Highlights-created text as a reference file, as you’re deciding which parts of your annotated PDFs you want to tag w/ this script. Just a thought… That’s how I plan to use both of them together!

[amcawood]

That’s what I’ve done in retrospect

[jprint714]

Just wondering if there’s a way to de-select the default checked box for “Set annotation date”? I’m not sure I understand why it’s the default option… I guess that if one were to instantly create tags for a document the moment that it comes out it would be useful – is that the point? In my case, I often annotate documents that are sometimes a bit dated, so that feature doesn’t benefit me. Unless I’m missing some point, which is entirely possible!

Also, is there another, easier way to select multiple tags under the main Annotations tags? Like you @Frederiko, I also have a “followup” tag (as you have in your illustrated example) along w/ a few other more specific research-related tags, so…I would love to figure out how to include these various tags, apart from having to write out each tag name, separated by a semi-colon (IF possible!).

Would still love any kind of input from my last post, if that’s possible! Many thanks…

[Frederiko]

Finally, one quick functional question… What’s the best ways to locate tagged documents through selecting a certain combination of tags? For instance, if I had tags for “Europe,” “Dogs,” “Portugese Water Dogs,” “Research Later,” and wanted to find all research topics relevant to Portugese water dogs and research later

Smart groups or advanced search, but maybe I am not understanding the questions properly.

Just wondering if there’s a way to de-select the default checked box for “Set annotation date”? I’m not sure I understand why it’s the default option… I guess that if one were to instantly create tags for a document the moment that it comes out it would be useful – is that the point? In my case, I often annotate documents that are sometimes a bit dated, so that feature doesn’t benefit me. Unless I’m missing some point, which is entirely possible!

In my workflow the dates of document are crucial, and I want my annotations to have the same dates as the documents that are referred to. That way a smart group for a particular time period also shows the annotations for the same time period.

Its easy to change. Use “Show package contents” on the script file to open the directory showing the files in the script bundle. Find the file called “main.scpt”, open it in Script Editor and change the line “adjustDate.default = 1” to “adjustDate.default = 0”

You can also change the name of the sub group tags by editing these lines:

property groupTag1 : "Issues"
property groupTag2 : "People"

Also, is there another, easier way to select multiple tags under the main Annotations tags? Like you @Frederiko, I also have a “followup” tag (as you have in your illustrated example) along w/ a few other more specific research-related tags, so…I would love to figure out how to include these various tags, apart from having to write out each tag name, separated by a semi-colon (IF possible!).

No its not possible. Autocomplete only works on the first tag. The only reasonable alternative way would be to have several tag fields in the same way as sub group tags are handled.

For various reasons, including the type of documents that I work with and I was finding that that my annotations lacked sufficient context when I returned to them, I have moved to almost exclusively using the version of the script that clips a whole page or more from the pdf. I can then markup the clipped page with the highlight and pdf annotation tools. That way I see the annotated passage in context and the full document is a one click away. Its definitely a more useful approach if you are working with documents, such as historical manuscripts or diagrams, where ocr is not very useful and where one or two lines of description do not fully convey the complexity of the annotation.

Frederiko

[jprint714]

Finally, one quick functional question… What’s the best ways to locate tagged documents through selecting a certain combination of tags? For instance, if I had tags for “Europe,” “Dogs,” “Portugese Water Dogs,” “Research Later,” and wanted to find all research topics relevant to Portugese water dogs and research later

Smart groups or advanced search, but maybe I am not understanding the questions properly.

So basically, I was just wondering how best to run an advanced search – one that would be based on just select tags (to find files based on convergence of select People / Issues). Make sense?

Just wondering if there’s a way to de-select the default checked box for “Set annotation date”? I’m not sure I understand why it’s the default option… I guess that if one were to instantly create tags for a document the moment that it comes out it would be useful – is that the point? In my case, I often annotate documents that are sometimes a bit dated, so that feature doesn’t benefit me. Unless I’m missing some point, which is entirely possible!

In my workflow the dates of document are crucial, and I want my annotations to have the same dates as the documents that are referred to. That way a smart group for a particular time period also shows the annotations for the same time period.

I hear you, and I totally appreciate that. For me, I typically save (or convert) my files as PDF so that I can annotate them. So, the date of the original document (i.e., it’s publication date) doesn’t necessarily match the date the document was converted / saved as a PDF…and as wonderful as your script is, I don’t think it would be able to make that distinction, correct?

Anyway, I thank you for your suggested fixes! That’s really helpful. I assume that there are particular reasons why the dates of document are crucial to your workflow…

You can also change the name of the sub group tags by editing these lines:

property groupTag1 : "Issues"
property groupTag2 : "People"

That’s also super useful… Is there any easy way to permanently keep the “None” tag from re-emerging?

Also, is there another, easier way to select multiple tags under the main Annotations tags? Like you @Frederiko, I also have a “followup” tag (as you have in your illustrated example) along w/ a few other more specific research-related tags, so…I would love to figure out how to include these various tags, apart from having to write out each tag name, separated by a semi-colon (IF possible!).

No its not possible. Autocomplete only works on the first tag. The only reasonable alternative way would be to have several tag fields in the same way as sub group tags are handled.

Gotcha. That would be useful, but I assume that would mean overhauling the script to accommodate more tag fields, right…? I’m not suggesting that – just wondering.

For various reasons, including the type of documents that I work with and I was finding that that my annotations lacked sufficient context when I returned to them, I have moved to almost exclusively using the version of the script that clips a whole page or more from the pdf. I can then markup the clipped page with the highlight and pdf annotation tools. That way I see the annotated passage in context and the full document is a one click away. Its definitely a more useful approach if you are working with documents, such as historical manuscripts or diagrams, where ocr is not very useful and where one or two lines of description do not fully convey the complexity of the annotation.

I’m not sure which version of the script this refers to…and if it suggests I’ve been using the wrong version! I hope not. I think the version I’m using comes from this part of the post:

[url]Put up Example page]

Are you referring to some other script? If so, does this other script you’re referring to still comport with the ‘one thought - one notecard’ approach (as Bill de Ville described it? I ask because it seems like you’re suggesting a move from a more granular tagging system (i.e., the one here) to one that includes using large chunks of text (in order to provide a fuller context). Is that correct?

[NZT-48]

i’ve just tried using this script and like it very much. i’m wondering if there’s a way to fix the way that the script captures quotes. i work as a reporter and have a fast-pace work environment, so it’s a bit of work to change every quote and apostrophe. thx.