There’s no standard for annotations in ePub. If one app provides for that function, there’s no guarantee another understands the stuff.
2 Likes
I am trying to move away from locked in system such as Amazon uses. So I’m not sure what your comment has to do with me looking for an epub reader and feeling that DTTG could fulfill this.
1 Like
Agreed. Seems that devonthink makes use of annotation files linked to main document. I could envision a way to read an epub without changing but allowing a text document or something linked to the file as a work around. Would stay with Devonthink philosophy of standard exportable files, yet still provide ability to mark up.
Sure. But whatever system you use, annotations in ePub will lock you in with it. PDF is, in fact, slightly better in that regard. But then it does not reflow. As so often, there’s no silver bullet.
1 Like
Agreed file formats lock you in to that file format. But epub seems to be accepted ebook standard now, it’s unfortunate that kindle seems to be the only one not to support it. (They now at least convert epub to their format, but not direct support). It is drm free and easily portable similar to pdf, and reflow for mobile is great.
That’s not what I meant (nor what I said). Rather: If you annotate your ePubs with one software, you’ll be locked to that software with your annotations.
For the rest: I’m fully aware of the (dis)advantages of ePub and Kindle formats. And I stopped using Amazon products and services long ago, anyway. Although I found their Kindle reader superior to the alternatives for other formats.
see Your Kindle can finally read EPUB files! – Take Control Books written by the Take Control folks.
I’ve not pursued this, but apparently can read ePUB in Kindle.
1 Like
I agree the kindle reader is awesome, formats not so much.
And sorry I misunderstood. With all the fantastic stuff DT can do I would think there has to be some sort of solution that pulls out highlights to a text file of some sort could be an option. Would allow things to be portable. (Although I suppose would need to be linked to the epub somehow, and keeping it within the epub would obviously commit you to the software).
I have been looking into this for the past few months, and it seems that the kindle does not actually read epub. The send to kindle now supports epub as an input format (which it then converts). This is why I have been actively looking for other options that support epub. I started using Apple Books, but with ios16 they removed the ability to export annotations except for 1 by 1. Have been using YOMU reader on iOS some but do not love it.
One would think reading an epub and exporting annotations wouldn’t be the hardest thing ever but apparently it is.
I think the issue @chrillek is highlighting isn’t with the summarised highlights document, which would just be markdown (or another option). The problem they’re flagging is that there is no accepted standard for annotating epub files. This means that if you highlight an epub in Software A, you cannot then see those highlights in Software B. Each app basically figures out its own way of annotating an epub, and other apps can’t read it. So in your request, the highlights in your epub would only ever be visible in DT, where you did your reading. That’s no different to your kindle highlights being trapped in Kindle, and is presumably one of the issues you were trying to avoid by reading in DT in the first place?
This is still somewhat an issue for PDF: although PDF does now have accepted standards, not all apps use them (DT does, which is why highlights added to a PDF in DT will show in some other apps). As an example, GoodNotes doesn’t use the accepted standard for PDF annotating, nor do some e-ink devices. (GoodNotes is great, but all users should be aware of that limitation just in case!)
It’s the same with other non-propriety formats too of course. There’s lots of different markdown standards now. I don’t imagine that problem will get solved any time soon, it’s not like there’s an official board of people who decide these things. It’s more like this…
2 Likes
(ePub should’ve been like PDF in this regard, but I suspect the problem was/is capitalism and all the publishing houses and digital reading tech companies not playing nicely together. It’s in Amazon’s interests to lock everyone into their own software, and they take up such a large market share and wield such influence it probably derailed the proliferation of a single open standard via epub.)
1 Like
I understand this all.
Now practically I’m an end user, I’d like a way to read an epub and get my highlights out. Nothing does this well, and given everything DT does it seems like this might be a natural extension was my point. I don’t particularly care how, but if any app did this I would be using it now.
Not trying to be argumentative, just hopeful this could be an added feature (and it’s not as if I’m the first to ever ask for this).
If what you’ve been used to doing with highlights could be done instead with deep links, you might be able to build a semi-acceptable DT workflow using Hookmark. But I haven’t found a way to create the links from within DT – you have to use the Open with… command (or ⌘⌥O, assuming Books is set as the default app for .epub) to open the epub in Books and create the link there – and the resulting ibooks://assetid/ link doesn’t work for me when clicked on in a DT document, though it does when pasted into other apps. (I could be missing something here…)
Skim.app also uses a propietary annotating method.
Not to contradict you in your assessment of capitalism, but… ePub is just glorified HTML, and that doesn’t know about bookmarks.
Now, HTML would show for highlighting using the mark element. But that would require modifying the underlying HTML, which would require editing functionality in the reader. Not to mention DRM problems…
So, obviously e-books are not up to par with analog ones.
1 Like
I generally avoid highlighting and “annotate” outside of the book or article. Specifically, I prefer to take notes in a separate text file (imaginatively named “reading notes” with the author and title). There are obvious disadvantages to this, but one of the not-so-obvious advantages is the simplicity: I can check my notes (handwritten or text) without having to look for or look through the original book / article, so I avoid the highlighting issue altogether. This tends to pay off more in the long run (ten or twenty years) after accumulating a database of interconnected files (a kind of digital zettelkasten). Your mileage may vary.
I enjoy using the Kindle (e-ink with long battery life and a convenient ecosystem) and have read something on it pretty much every day for a decade now, but the lock-in and lack of page numbers make it rather irritating for research. In practice, fiction and non-fiction indirectly related to my research interests / non-academic makes up the bulk of my Kindle usage. I use PDFs (on iOS and OSX) for almost everything else, and epub only as a last resort. DT is a wonderful place to manage my PDFs and annotations, so my answer to the original query would be a qualified “yes.”
I wouldn’t mind if DT had more support for the kind of highlighting stuff that has been mentioned, assuming time spent developing the features was not too onerous, but like so many other fancy features in apps, I wouldn’t use it even if it was there, because the added complexity in my workflow would be of little benefit to me. For a similar reason, Amazon’s “scribe” Kindle with the stylus looks great in theory (I enjoy using the Kindle and I annotate), but I suspect it would be a poor fit for my workflow, especially when you consider how it handles annotations. Yikes.
1 Like
Thanks, really useful details! I’m developing a similar system at the moment, still trying to figure things out.
My calibre is installed on my nas using docker and use KyBook for epubs on my iPad and DT for pdfs. KyBook can export annotations too. Let me ask you a couple things
How do you manage multiple copies of your books in calibre using tags for clean and annotated versions?all in the same folder?
On DT do you put your md annotations in a certain way or you just drop them in a book folder, anything else you add on those files to link book and annotations better?
I’m considering adding an ebook reader to the mix to go easier on my eyes, I presume that would be ok as long as I am able to extract the annotations on a separate file and add it in DT, any experience using ebook readers in this way?
Many thanks
I do love talking about books!
My books are stored in the Calibre folder. Calibre creates a folder per author, and a subfolder per title. I only actually store the PDFs in the Calibre folders now, as I’ve tweaked my system slightly. So in my Calibre folder, I have the “clean” PDF that I created, and if I’ve read the book it has the final annotated copy which is the same file name with “(ANNOTATED)” added to the end.
I’ve moved the original Kindle format files to a separate folder so they’re grouped together but outside the rest of my book system.
I do, because file naming conventions are the best 
Book notes have the following format:
Surname, first name - title (my notes)
I add a full citation at the top of the note so I can copy and paste the citation when needed.
They are also all tagged “book note”, so that I can find all book notes if I feel like it (sometimes useful if I’m looking for something).
I don’t keep the book in DT except in very rare cases (when a book is so valuable I want it to always surface in relevant searches). Once I’ve read it, annotated it and made my annotations md, the annotated book is stored in my Calibre folder and removed from my database.
I have one main database where nearly everything goes (I have a couple of smaller ones for very specific isolated things, but they are supplementary to the main one). I maintain a folder/group structure based on topic, and book notes are filed in the relevant topic folder. This is so that they surface in the right place when I am looking for something relevant. If a book has lots of good info that is across multiple topics, I will often break it down into separate notes. When that occurs I follow a different naming convention:
Surname, first name - topic (my notes)
They are still tagged “book note” and still include the full citation. I actually find this quite handy, because if an author has ended up with a lot of notes, it means the book is very valuable to me.
This system doesn’t 100% work, because some books are random books I’ve read that don’t fit into my neatly organised folder structure. I maintain a random/misc folder/group, where I stick random files that don’t fit anywhere else. In this I have a “Random book notes” subfolder/subgroup specifically to collect book notes that don’t fit anywhere else.
It’s dangerous to have a random/misc folder, so I do keep an eye on it and if items along a topic start to gather, it means I’ve found a new interest in something and I create a folder for it. As an example, my database is mostly science stuff, so notes on fictional literature that I want to keep have nowhere to go and accumulate in the random books notes folder.
Run a search for “remarkable” in the forum, as it’s been discussed a few times recently (I’m just using “Remarkable” as a lazy search term - the discussions have been about e-readers in general). Some e-readers don’t create annotation layers in a cross-compatible format, and some also don’t allow for easy export, so that’s something to bear in mind.
I would love to use an e-ink screen, but am unwilling to interfere with my current workflow unless it blends perfectly (I.e. I want a cross-compatible annotations layer and a simple export function). What I really need is Apple to come out with an e-ink device…
1 Like
That sounds like a very good reason to have a random/misc folder. I have a scrapbook database that is for everything that I can’t immediately think of where it should go. At least once it’s led to a new database of its own subject matter based on observing an emerging theme.
Thanks!! really throughly explanation! Let me ask a couple of questions
When annotating PDFs I would like to add images from the PDF to that annotation file, reading technical stuff with diagrams and so on. Highlight app is able to add images I think DEVONthink is not able to annotate images and export annotated text and images in a .md file, is that right? Also, just trying highlights app in iPad and despite it says pro version is able to open directly PDFs within DEVONthink database I only can open them when importing to the app hence creating another duplicate of the pdf file.
When archiving annotated PDFs in calibre, you just copy the file into the existing folder adding (annotated) to the filename, how does calibre display this? Two entries in its data base? Can I access these two different versions from calibre interface or just browsing to the folder?
Thanks again!!