Questions about rethinking Annotation, Tags, and Notes

[korm]

TagNotate is pretty simple to get up and run and looks like it has promise beyond the confines of the iPad app. The UI is very pleasant, easy to understand, and enjoyable to use. Though, for now the app’s value remains mainly within the proprietary app. On the upside, the authors have prepared the way for integration into other apps. From a tagging/annotation workflow perspective – this app makes the linkage very easy. The abilty to AND / OR tags into a kind of on-the-fly smart group is useful, and then the annotations can be collected and exported. The tags are included as string in an additional line of text at the bottom of the annotation.

Some comments:

[size=125]Pro[/size]

• Each notation is stored in that PDF’s metadata in the “Author” element. Each notation has a UID (not sure if the domain is unique to that file or genuinely unique), and the “tags” are stored in a set. 3rd parties could take advantage of this feature. Unfortunately, DEVONthink (as of 2.8.2 on Yosemite) does not display the contents the PDF-specific metadata, including the “Author” element, for PDF’s notated with TagNotate. The contents of “Author” can be viewed in Preview, Acrobat, etc., on Yosemite. Example:

{"uid":"D62F3673-689C-4873-BB42-FB12AA1F7CEB","tags":["Cicero","Ovid","Cato","Virgil"]}

I did not yet test to see if TagNotate overwrites existing contents in “Author”, which would be a problem especially for academic users.

• Simple interface – better design than many iPad PDF programs.
• Clipboard is a good feature for collecting annotation info to copy into other documents.

[size=125]Con / Suggested Improvements[/size]

• I dislike the privacy notice and the fact that so-called non-identifiable information is sent to the mothership with no opt-out. Why the authors are collecting the tags that users create is mysterious. This is 2014. Opt-outs are essential. Over here, I work with numerous customer-sensitive documents and would be unable to use TagNotate because of it’s non-conformance with common corporate and governmental security policies. It’s a bizarre design choice that is not essential for supporting the app’s mission.
• Not updated for iOS 8 – this matters because two of the strongest features – UID-identifiable annotations and the clipboard mechanism – would play very well in the iOS ecosystem. Also, there’s been no update since May 2014. This is forever in iOS land and possibly a red flag. I discount the promise of a desktop version – lots of developers promise new platforms or versions for years on end. (Ahem, we know who )
• Annotation types are limited to highlights, free-form drawing, text, and notes. More of a personal preference, but the absence of boxes and stamps is odd. OTOH, TagNotate recognizes these latter types and does not seem to mess them up.
• No apparent use of call-back URLs – which would be important for tighter integration of TagNotate with other apps on the iPad – including DTTG. The bones of this feature (the UID) is present.
• Clipboard copies of annoations + tags is plain text. Would be a nice-to-have if the export were in Markdown format with a link back to the annotation.
• It would be useful to have tag sets, in three senses:
  [*]if I’m tagging news articles I want a different set of tags than if I’m tagging information theory articles
• I want to import/export a tag set for use elsewhere – perhaps as an OPML list
• Hierarchical tags
  [/*:m]

[Nhaps]

Korm, thanks for the time spent in reviewing TagNotate. I am glad that DT staff is taking an interest in intra-tagging functionality. I have contacted the TagNotate developer and I hope that he will be able to join us in this thread.

[jprint714]

Yes, agreed – thanks @korm for your review!

While we’re on the topic of using external apps to make annotation, tagging, and notes work better, why not use the Highlights app that @korm suggested in this post: [url]"Highlights" app - export annotations to DEVONthink, etc.] ?

Or does it lack some feature that doesn’t address what we’re seeking?

By the way, I traded a few emails w/ one of the folks at Highlighter, and he said they’ll look into a way to include links to the PDF source files for future editions (which he promised will be out soon).

If Highlighters is a viable options, maybe there’s some way to integrate it w/ the kinds of tagging options we’re also seeking.

Thoughts?

[Nhaps]

jprint714, here is my take on:

Highlights app for mac. It overlapps with Skim which means tagging is out of the picture for now, but it has advantages. I asked Highlights developer to include page link on the righ panel so we can quickly locate a quotation in context back on the pdf on the left side. To my surprise they accepted it, and it’s slated for next release.

TagNotate for iPad. I will be waiting for a response from the developer of TagNotate to korm’s review above. The issue at stake here is the potential of intra-document tagging. While this is not yet materialized we should continue encouraging DT developers and volunteers to improve on the scripts are you requested above. Truth be said, both apps have complementary features, and a promising future. But TagNotate is more innovative.

[jprint714]

@Nhaps, thanks for your reply… I haven’t tried TagNote yet, but I’ll look into it.

For now, process involves:

-Saving docs and article on webpages as PDFs
-Saving those PDFs in DropBox folders (organized by topics)
-Syncing my DropBox folders within iPad’s GoodReader app
-Reading and marking up my PDFs in GoodReader
*Eventually I route the marked up PDFs back to DTP where I…well, where I’d like to create tags for the marked up PDFs, create annotated files that would include quotes and links to the marked up bits, and then (eventually)…compile memos that consist of quoted and linked bits from annotated files.

As I understand it, if I take my annotated PDFs from GoodReader, and open them through the Highlights app, Highlights will auto-generate a Markdown document based on of selected annotations and notes I created in GoodReader. And…they say they might soon be able to create a version that includes a Markdown document with links to those annotations.

So, what does TagNote have that Highlights doesn’t offer – or what do they have that might be a better solution to my process? (By the way, I’m not tethered to my process – it’s just the best system I think I divined – so I’m happy to consider an alternate process.)

Thanks!

[Nhaps]

@jprint714. My workflow differs from yours, I don’t use GoodReader, and I am still learning how to use TagNotate.

But here is the main advantage that Highlights does not have: search and filter your global (all pdfs) or single (pdf) annotations by tags (underlining, highlighting, comments). TagNotate’s forte is smart listing your annotations on-the-fly by tags. You can search by page number, by tag or tags. By clicking the quotations you are taken immediately to the location in the pdf and back to the annotation again. Instructions are easy, accessible at the bottom.

Highlight is faster because underlining generates the quotation automatically, but gives you only bright that nasty bright yellow. TagNotate is slower to create but gives you more options plus one or more tags for each annotation.

What does this do for your research? Well, for now it offers a tighter integration between the whole pdf and your annotations. It equips you to see the whole article from different angles through the selection of tags. In this process you come to get a better handle on the material.

How do you transfer your notes to mac? Well, there is a clipboard option within TagNotate, which you can use to copy to Notes through iCloud or any other way. Tags are included as text file along with quotations. Or you can save the PDF to dropbox and the all annotations will be visible in Preview and Skim. All this for $2.99? You gotta be kidding me, get outa here.

From Skim you can explode to RTF files in DevonThink, or combine into one RTF. As korm said, all tags are located in the “author” metadata field and unfortunately DT cannot read them yet. So for now I am using TagNotate to digest my pdfs through the benefit of intradocument tagging. And if I need to paste a quotation it’s right there on DT. (The only problem then is basically having to reproduce the tags in Devonthink, for each exploded RTF, which I am not doing btw)

[jprint714]

Thanks very much for your reply. I guess I don’t quite see the benefit of go through all the trouble of tagging in the app if it’s not transferrable to DTP’s tagging system – and my tags folders will finally reside in my DTP databases. Maybe I’m missing something… Again, I thank you for you thoughtful reply.

[korm]

@jprint714

Thanks very much for your reply. I guess I don’t quite see the benefit of go through all the trouble of tagging in the app if it’s not transferrable to DTP’s tagging system – and my tags folders will finally reside in my DTP databases. Maybe I’m missing something… Again, I thank you for you thoughtful reply.

Even if TagNotate tags were “transferrable” to DEVONthink’s “tagging system”, there is no “tagging system” on the desktop – in DEVONthink or otherwise – that can use annotation/text-associated tags in the TagNotate sense. The question isn’t transfer – indeed, the tag data is transferred by TagAnnotate because it is embedded in the “Author” property – but utility.

OTOH, it’s a simple matter to produce a report of tag usage and annotations and export from TagAnnotate to somewhere else.

[jprint714]

@korm

Even if TagNotate tags were “transferrable” to DEVONthink’s “tagging system”, there is no “tagging system” on the desktop – in DEVONthink or otherwise – that can use annotation/text-associated tags in the sense TagNotate sense. The question isn’t transfer – indeed, the tag data is transferred by TagAnnotate because it is embedded in the “Author” property – but utility.

OTOH, it’s a simple matter to produce a report of tag usage and annotations and export from TagAnnotate to somewhere else.

Thanks @korm… I think I follow that process, and generally understand how one could transfer TagNotate’s tagging system (though it seems a bit convoluted). As understand it, TagNotate’s main advantage is the ways in which tags can be applied and used w/in the app – both globally and on individual documents. All of that sounds very handy, but…it seems like TagNotate is also a bit limited (for the reasons that @korm mentioned in the critique above), and so far its seems that TagNote’s utility is mostly confined to its iPad app functions, correct? Is that a fair assessment?

I suppose part of what give me pause about using it is that, and also…it seems like the folks from the Highlights app have said that they might soon be able to create a version that includes a Markdown document – with links to annotations, that could be readable in DTP. I do see the benefits of TagNotate’s tagging system (if I understand them properly), but if Highlights developers are right, it seems like their process could provide many of the same benefits (thought lacking TagNotate’s tag functions) and save a whole lot of energy through generating transferable Markdown annotated docs w/ links for DTP.

Please let me know if I’m missing something here or if this sounds about right.

By the way, @korm, I know you’ve preferred to stick to the technical aspects of this (which I really appreciate, as you know), but I keep wondering if you have a different work process for annotating, note-taking, and tags. I mention this because it would sure be interesting to hear how you approach things differently given your strong insight into DTP, and how to marry it w/ other apps for these kinds of functions. Thanks!

[korm]

IMO, Highlights and TagNotate both offer interesting new twists on PDF annotation, but to compare them is somewhat like “I love my Mini Cooper because it reminds me of my Chevy Tahoe”. The only thing at the interesection of a Highlights and TagAnnote Venn diagram is “PDF”, otherwise they are completely different feature sets and, more importantly, approaches to the technology of annotation.

To accomplish its magic TagNotate uses a portion of the PDF standard in a non-standard way. The PDF standard provides an “Author” property for annotations – this property is intended to be used for storing idenfiiers (names, initials, or the like) of individual reviewers of a PDF document. TagNotate uses “Author” to store a string containing both the unique ID for that annotation and the set of tags relevant to that annotation. “Non-Standard” means that this approach is not part of the accepted standard for PDFs so that whatever TagNotate does to your document is at risk of being overwritten by some other software. In other words, it is fragile. Interesting, but fragile.

If Highlights were to incorporate TagNotate’s approach it means the developer of Highlights would choose a non-standard approach (and thuse potentially reduce the size of its market). Never say never – but these are the aspects one needs to be aware of when choosing one of these softwares.

Also, btw, Highlights produces Markdown natively – discussed in the original post I made here about Hightlights.

@jprint714

wondering if you have a different work process for annotating, note-taking, and tags

Nothing of general interest that I haven’t already discussed over the years.

[Bill_DeVille]

I concur with korm’s point that TagNotate’s use of the Author field of PDF document properties is fragile, as it can be overwritten in editing by other applications to the surprise of the user. It would also interfere with the use of that field for its intended purpose, for many users. For that matter, there exist examples of PDFs in which TagNotate’s approach is blocked.

I’m not very interested in an annotation method that is limited to only a few of the filetypes in my databases. I prefer approaches that can work with any document, regardless of its filetype.

When DEVONthink adopted the OpenMeta tagging system, we and other developers were somewhat concerned about the fragility of that system. OpenMeta tags use a section of code in files that had been reserved by Apple, and not explicitly made available for modification by third-part developers. The advantages of OpenMeta tags is that they work for a wide range of filetypes, OpenMeta tags created in DEVONthink can be recognized by some other applications and tags created using other applications in that way can be recognized by DEVONthink. The danger was that with any update or upgrade of OS X, Apple could take over the area of code that holds the tags for another purpose. So far, taking that risk has provided benefit to users, and my guess is that Apple won’t kill the tags, at least without providing migration of tags to an alternative approach. The tagging system didn’t conflict with other uses of files, unlike TagNotate’s approach, and isn’t limited to a subset of documents by filetype characteristics.

Impressive scholarly research was done long before computer tools were available. QDA approaches were used in analysis of Shakespeare’s writings (including some alleged not to have been written by him), in Newton’s alchemical writing, in analysis of the Bible, the Talmud & c. In Samuel Johnson’s Dictionary he defined Lexicographer as “a harmless drudge”. Not as impressive but certainly emphasizing drudgery, I worked on a multivolume bibliography using thousands of handwritten notes from library research, followed by a series of syllabi for graduate courses about science, technology and public policy that were funded by the National Science Foundation. Having been impressed at the age of 13 or 14 by Vannevar Bush’s concept of the Memex and watching since then the development of computer technology, you have no idea how much I was wishing for computer assistance while doing all that drudge work!

Computer tools are now widely available and used in research. They can allow for more time spent in thinking and less in sheer mechanical drudgery. Unfortunately, the old Yiddish saying that ninety-something percent of everything is drek still holds concerning the product of research. Nothing substitutes for genius.

I often wish I had genius.

Adapt and use tools, including computer software, to get work done. I love DEVONthink because it provides a set of tools that I can often kludge to do a job that it wasn’t specifically designed to do. Often, with a bit of drudgery, I use a network of linked notes in such kludges.

[jprint714]

@korm

IMO, Highlights and TagNotate both offer interesting new twists on PDF annotation, but to compare them is somewhat like “I love my Mini Cooper because it reminds me of my Chevy Tahoe”. The only thing at the interesection of a Highlights and TagAnnote Venn diagram is “PDF”, otherwise they are completely different feature sets and, more importantly, approaches to the technology of annotation.
…
If Highlights were to incorporate TagNotate’s approach it means the developer of Highlights would choose a non-standard approach (and thuse potentially reduce the size of its market). Never say never – but these are the aspects one needs to be aware of when choosing one of these softwares.

I hear you, @korm… That’s my sense of it as well. While there’s not much overlapping area between these two apps in a Venn diagram, and (as you rightly point out) they do represent different approaches to annotation, it sure would be nice to have both features – and…have text links for Highlights and TagNotate-created tags (or tagging features in another iPad app) that could be transferable to DTP.

For me, that would represent the best of both worlds; I’d use TagNotate’s tags for notable annotations (e.g., a topic that’s esp. relevant to a current work issue), and then rely on Highlights’ captured annotations and text links for for other areas that I’s mark up (which could be important but not necessarily tag-worthy, or when I’m racing through an article and just want to quickly highlight area w/o tagging everything). And if these feature could work w/in DTP, then we’d really be cooking w/ gas! Maybe this is a pipe dream, but it does seem like a great solution, no?

@korm

Also, btw, Highlights produces Markdown natively – discussed in the original post I made here about Hightlights.

Yes, I certainly noticed that as I was exploring Highlights. Since our discussion, I’ve started setting up a text expander app (seems many people suggest aText these days). I’ve been reading up on Markdown, and trying to understand how it would help me w/ my work. I’ve been using TextEdit or some of the DTP templates (e.g., the Phone Notes template) when I write up notes during interviews, or transcribe them, or for other assorted note taking. I don’t write code (if that wasn’t glaringly obvious!), which seems to be what Markdown is partly known for. Anyway, I’m trying to better understand its benefits for my uses… Is there a particular Markdown app that you could suggest for me, given the kinds of things I’d use it for? Thanks!

@jprint714

wondering if you have a different work process for annotating, note-taking, and tags

@korm

Nothing of general interest that I haven’t already discussed over the years.

Dang! Ok, no worries at all, @korm… I just find I learn a ton from you in your posts, and it almost always helps my work in some ways. In that way I was thinking that if I understood your cradle-to-grave approach to reading, annotating, filing and storing stuff, I might be able appropriate a different, more efficient work process – that’s all. I outlined how I work here: [url]Put up Example page] but expect there’s probably a better, cleaner way of doing this… Again, no worries – you’ve already been incredibly generous. Was just curious…

[korm]

@jprint714

Is there a particular Markdown app that you could suggest for me, given the kinds of things I’d use it for?

I prefer to use either Sublime Text (v2 not v3) or Ulysses III for editing and Marked for preview/formatted export. Sublime Text is rather technical, but the plugins and extensions are very useful for many tasks. Ulysses III is elegant and pleasurable to use. I’ve bought or tried numerous dedicated Markdown editors and discarded them all but Ulysses III. DEVONthink’s internal Markdown implementation is the least featured of all Markdown editors.

[jprint714]

Thanks for the suggestions, @korm!

Ulysses III looks great - very elegant (as you say). A little on the expensive side, though. Anyway, thanks for another great suggestion…!

[jprint714]

Quick follow up…

While I really appreciate TagNotate’s functions, I agree that it does have some notable limitations – and is, indeed, fragile (esp. given the way in which tags are located in the author section of PDFs). So, can anyone suggest another iPad annotation app that can also implement tags? I’ve been using GoodReader, which has good file management w/ DropBox and decent annotation functions, but maybe there’s a better alternative – and one that also includes tags. Suggestions? If not, can anyone propose a work-around solution that basically works similar to TagNotate?

Thanks!