Feature: Annotation overlays

I have a feature request for DTP that would help me exceedingly with some cross-referencing work I’m doing.

The problem is this: I have tens of books imported into my database, full of material I want to cross-reference and annotate. For every entry, there might be: footnotes, index entries, annotations (my own notes added to the text), hyperlinks to parts of other books, and markers to show where pages are divided in the original text.

In addition, To facilitate the power of “See Also” I’ve also chopped these books into coherent subsections, each typically ranging from 50-500 words, but sometimes exceeding that amount. (This is where the real power of DTP shines, in that a lot of the cross-referencing is automatic now. But I still need more).

To add all this extra material on top of the base text, I use text in square brackets. If inline, the bracket is a footnote; if it contains a link, it’s a cross-reference; if it’s a quoted paragraph, it’s an annotation, etc.

This works somewhat well, so long as I don’t put the annotations between words – which breaks phrase searching. I either have to annotate after each sentence, or after each paragraph. And the fact that the annotations are embedded in the text means “See Also” can use the annotation text to facilitate AI searching.

However, the problem with this system is that it gets UGLY fast. It’s one thing if there’s just a few annotations, but once you get ten or twenty annotations in a 100 word entry, it starts becoming a forest that you can’t tell apart from the trees.

So, here’s my feature request, after thinking about this problem for a long while:

Add the concept of delta "overlays" to DTP.

Here’s how it works: If you’re viewing an article you want to overlay, the first requirement is that you lock it. Once locked, you can select “Data -> Create Overlay”. This causes a tab bar to appear above the article, with the first tab being “Original”, and the second being “Overlay 1”. The second tab is now selected by default.

Each overlay has a separate color, with the usual Apple default of red first, then orange, yellow, green, blue, and purple. But of course, the overlay’s name and color are both afterwards configurable.

(As a quick optimization, if you attempt to create an overlay for an unlocked document, a dialog appears asking if it’s OK to lock the document before creating the first overlay. A document with overlays cannot be unlocked either, at least not until the last overlay has been deleted).

Now, what does an overlay do? Each one separately records additions and deletions made to the base document. It displays additions (only) in the overlay’s color. For example, I could create a Footnotes overlay, type in some footnotes after a particular sentence (which would appear in the overlay’s color), and then when I select that overlay, I would see all the document’s footnotes in red, “overlayed” on top of the base text in black. But if I select the “Original” tab, I would see no overlays, just the base, unmodified text.

For any one of my many cross-referenced documents, I might have six overlay tabs working for me:

1. Original (base) document
2. Footnotes (red)
3. Index entries (orange)
4. Annotations (yellow)
5. Cross-references (green)
6. Original page markers (blue)

I’m only able to select one tab at a time. This is because one overlay might potentially delete text that another one modifies.

All textual additions contained within an overlay are also added as words to the database. If these words are found during a search, the search pane display the document’s name in the color of the overlay. (This only happens, though, if the hit occurs only within a single overlay, and not in the base document). This is a clue to me that the hit occurred within an overlay, and if I’ve been consistent with my coloring, it tells me which overlay it occurred in.

Let’s say I now select a document from the search pane containing hits in an overlay. I choose “Find” to locate the hits, with the first hit occurring in the base document. So far, everything looks normal. But now when I hit Cmd-G to locate the next hit, it’s in an overlay. (Whether an attempt is made to find overlay hits in sequence, or only after all base document hits are found, is up to the developers). In order to display the overlay hit to me, DTP automatically selects that overlay, and highlights the colored text in the display window; and so on for all other hits that might occur in other overlays within the same base document.

I call this feature overlays because it reminds me of transparency overlays made for a paper document. It’s also reminiscent of Photoshop’s “adjustment layers”, in that it lets you build up a series of creative adjustments to a static base document.

Since I use DTP primarily for collecting and indexing static text, I’d find such an ability to annotate locked documents like this invaluable. It would also allow me to make overlays for translations of a base text, but deleting the document’s text within the overlay and then pasting in the translation. Imagine having a database of the Bible, with greek and latin overlays for each section of each chapter? You could search in greek, and quickly switch to english or latin to find the correspondent translation!

This solution also makes it possible at any time to recover the pure, unmodified base text, to facilitate re-reading of the entry after a great deal of time has passed. Fully annotated text is great in the middle of a research project, but for pure reading it’s somewhat miserable.

And lastly, since I have not encountered this feature yet in any text editing systems, it would give DTP users a “killer app” to brag about! I’d love to see DTP being hailed as the ultimate text researcher’s tool.

That’s a great idea;
I think that you might be able to partially create your delta overlays system with current software; by annotating PDFs in Preview (Bill originally mentioned this feature) using Preview’s yellow sticky notes to create a numbered, catagory place holder in the margins of your PDFs (unfortunately this tool has no colour or font choices). For instance an Annotation marker could be A12, or a Footnote could be F7 etc. That would refer to ‘note streams’ in six separate documents, or a single document divided by colours or into 5-6 sections (either as DT notes or word processor files). Once Annotated the original document and the notes supplement could be opened side by side.

Most of my data is not in PDF form, and creating them as such in order to use Preview (an external tool) to edit them would be too much work to actually happen… :slight_smile:

I dig this. I’d like it if you could define your own uses and number of layers, though. Like I said in the other thread, I’d like to use something like this for a version control system.