Thoughts on DEVONthink (All in One tool)

Nicely expressed. Going a little farther in that direction, I find no problem with all the research material in DT and nothing but the manuscript in a pure editing environment.

Particularly when there is more than one document to a project. For instance, some scam at a government regulator. It’s going to take lots and lots of letters before anyone pays attention.

That’s a case where I would store the documents in the DT database with my research and evidence. I get the research in an environment I like and I get to pick the environment I want to write in.

1 Like

A way I sometimes describe it to people familiar with the Scrivener workspace (but not DT) is that DT is Scrivener for reading, and Scrivener is DT for writing – but reading is often writing, and writing is often reading. So for example I always write reports, minutes, and the like in DT, because DT excels at writing around existing documents. But I always read screenplays in Scrivener, because it handles Fountain files so well.

I’ve lately started making a lot of use of a Scrivener-like workspace within DT, where I make my working Markdown document an Annotation file to all the different documents I need to reference, by setting their URL property to the Markdown document’s item link (either by a Smart Rule, or by pasting it manually into the URL column in list view or the URL field in the Generic inspector tab), and with item links to all the source documents in the Markdown file. (Just drop them all into the Markdown file from the list pane for absolute links, or for DT item links select all in the list pane, Copy Item Link, and paste into the Markdown file.) Then open any of your source documents in its own window, and drag the Inspector pane to maximum width. You now have your working Markdown document on the right in the Annotations pane, and can use the embedded item links (particularly in WYSIWYG view) to call up different source documents in the Preview pane on the left. I started out doing it to mark stacks of student essays, but it’s also great for all those other Scrivener-like situations where you’re writing with a lot of different reference materials that need to be in view.

1 Like

Interestingly, there seems to be a general consensus that Scrivener isn’t really suitable as a research tool. @MsLogica would rather have only 10 documents in Scrivener than 100.

I can assure you that this is a dramatic underestimation of Scrivener’s capabilities :slight_smile:

I created a test project with over 20,000 documents. RTF, PDF (2 to 1,000 pages, averaging maybe 10 pages), and web pages.

Scrivener works just fine. It doesn’t crash, navigating between documents is just as fast, creating new documents is no problem, searching through them isn’t either, and importing web pages works as expected.

There’s only one feature that no longer works: searching the entire project. Once you have 5,000 documents, searching becomes agonizingly slow. At least with my document structure. The exact number may vary.

Scrivener’s only problem is the search function. Unfortunately, this makes the entire app unusable for large projects—at least for me—because I want to keep all my documents in the same project.

I’ve been in touch with a user who has over 100,000 documents in a single project. Everything works normally for him too, except for the search function. His workaround is to divide his documents into a large number of folders—probably no more than 2,000 documents per folder—and he only ever searches within a specific folder at a time.

So, Scrivener is perhaps the best writing app out there. And Scrivener could serve as a “repository” for a great amount of research material in every respect, if only the search function could be improved. I don’t know if that’s feasible. Maybe in version 4. Or maybe it’s already been improved in the new writing app. That would make me very happy :slight_smile:

I wouldn’t say it “isn’t suitable,” I would just say that DT is better.

Just as I would say that DT is a reasonable writing environment, but Scrivener is better.

5 Likes

That’s cool! I didn’t realize the URL would function as an annotation link. This works across databases, too.

Thanks for a great tip.

2 Likes

Could you elaborate a little? i.e. in this marking scenario which is the file that is your working doc that is linked to, and what the student papers function as in this workflow.

I’m not in education per se, but it sounds close to a thing I might have to do soon, with a nascent intern/mentor program, which on second thought is a kind of education system.

1 Like

Working document is a Markdown file, though the system would also work with Rich Text; and the student essays are in either an indexed folder or imported Group. (It doesn’t matter which.) Then I do the following:

  1. Copy the item link of the Markdown file.
  2. Set that item link as the URL property of each essay in the folder, either by a Smart Rule as below, or by checking URL in View => List Columns… and pasting manually into that column for each essay in the list view. (Or you can just paste into the URL field in the Generic tab in Properties, which can be useful if you have late single entries to add.) This sets the Markdown file as the Annotation file for each of the essays.
  3. Now select all the essays in list view, choose Copy Item Link, and paste into the Markdown file. This gives you links in your Markdown file to each of the essays.
  4. Optionally, replace “[” with “# [” to turn the linked essay titles to Markdown headers. As well as showing in your chosen heading style in the document, they’ll appear as items in the Markdown file’s Table of Contents. This is particularly neat in WYSIWYG view for links (Settings => Files => Markdown and check “WYSIWYG images & links”). I write my overall feedback on each essay under the header for each.
  5. Open any essay in a separate window (or open the Markdown file in a separate window and click on the link to any essay); maximise the window to fill the screen; drag the Inspector pane’s separator leftwards to maximum width or your preferred optimum; and select the Inspector’s Annotations tab.

You’ll then have your essay in the left-hand (Preview) pane, and your comments document in the right-hand (Annotations Inspector) pane; and can move between essays by clicking on the appropriate link in the Inspector pane. You can also include links to other documents you may need to call up as you go.

Sometimes it’s more convenient to write comments directly into the online marking system (Turnitin or whatever) and then paste them into the Markdown file for my records. But doing it in DT has the advantage that you can move very rapidly between different essays (“Ah, I linked to a reference in my feedback on that earlier essay that would be relevant here as well; now where was it…?”).

I think I’m only scratching the surface of what you can do with this setup, which is particularly useful when you have a folder full of documents you need to write notes about one by one, but can also be used for any kind of writing where you have an assortment of source documents to call up Scrivener-style. Just set your working MD or RTF file to be those documents’ Annotation file via the URL trick, and do your writing in the resized Inspector pane.

6 Likes

I’m sitting reading your post a couple of hundred yards from the Welsh border[1], and I’m intrigued by your research area! Are you aiming for publication? If so, please let us know when the work is published! It sounds fascinating…

Diolch yn fawr!

[1] So close that at night my bedroom window is lit by the searchlights on the watchtowers, as the border guards continue their relentless struggle against the bara brith smugglers, who still trudge, as their forefathers have always trudged, along the age old track from Penarlâg, through the bottom of my back garden, on their way to the underground markets of Eccleston.

2 Likes

Now, that’s an angle I had not considered. Bara Brith smuggling! An ancient practice worthy of further investigation.

Sadly, my focus is on Wigmore during the 13th and 14th centuries. I’m interested in looking at the lives of ordinary people then, not the high and mighty. So I’m engaged in writing a series of short stories, with as much definite historical facts included but that’s just the background for the tales of the individuals. And I chose Wigmore because it was a Marcher Lordship, where things were done differently form the ‘traditional’ medieval peasant setting: open fields, lord of the manor, etc., all told so well by others.

The database grows with customs, clothing, coroners reports, recipes, Welsh campaigns, the names of vicars, local quarries etc., etc., . I would like to eventually publish, but I’m not certain when.

I promise if it happens I’ll let you know, assuming you’re still interested or haven’t yet been imprisoned for implications in the smuggling.

2 Likes

That sounds fascinating… I only really know the basics about the Marcher Lords – though I learnt a lot listening to the recent In Our Time episode on them. Did you hear it? It may be a bit basic for you, given what you’ve been researching, but I found it a really good introductory overview. (As most In Our Time episodes are…)

In Our Time - The Welsh Marches - BBC Sounds

No. I did not catch it. Sadly.

But… (to get back to this thread), this is why I really like DT (to do research on so many abstruse things that may or may not come together in any story but are needed for background) and Scrivener for putting the odds and ends together with characters. And I like them separate, thank you.

(But I do wish I had learnt more about bara brith smuggling in school.)

1 Like

Occasional IOT guest here can confirm that DT played a role in prep for the Oedipus one!

1 Like

With MD, having to switch between an editor and viewer introduces too much friction.

I’m honestly surprised there aren’t more WYSIWYG Markdown editors — or at least, not standards-compliant ones. I haven’t found one that conforms to either Commonmark or to John Gruber’s original loose spec. They get simple stuff like paragraph breaks wrong.

Similarly, I think it would be really useful if there was a WYSIWYG editor that just let you edit HTML5 and CSS, with menus or buttons for the various semantic HTML elements, and a separate place to edit the CSS for them and see the changes live. Basically, a word processor with a style system like Pages, but with HTML+CSS as its native format.

2 Likes

I use DT’s Formatted Note editor for basic editing (it’s HTML format)
When I need advanced features, I switch to a dedicated HTML editor

1 Like

One reason might be that WYSIWYG does not make a whole lot of sense with a markup language. The latter is about structure (or semantics, if you like). The former is about presentation.
And markdown itself doesn’t know anything about presentation. Only if it’s rendered (or converted, if you like) to HTML or PDF or whatever does presentation come into play. In the case of HTML, you still need CSS to define the WYS part.
And if you use another CSS, the IWYG part changes. Nothing is baked into the MD itself.
While the whole idea of WYSIWYG is to burn the presentation into the document.
If you want that, use Word, Pages, Quark or so. But don’t hold your breath waiting for an editor to do MD and WYSIWYG.

Such products did exist once. They disappeared because nowadays not so many people write HTML anymore. They use WP or static site generators or frameworks to produce the HTML.
For example, I use Hugo to create HTML from MD and then I craft the CSS with HI and the aid of the browser’s developer tools.
Also, because WYSIWYG is a vague concept in the times of hugely differing form factors, device capabilities etc.

1 Like

I will have heard that one (IOT is by far my favourite program on any medium…) but I shall listen to again with renewed interest, thanks! Do you keep all your research in DT?

It comes at a good time because last night I finished Emily Watson’s book on the joys/perils/pitfalls of translation, Crossing the Wine-Dark Sea, and I was thinking it was time to do some reading of the classic plays (in translation), and perhaps even opening the copy of Pharr I bought ten years ago and haven’t yet opened…

1 Like

Have you tried Typora on Mac. I think that is a good WYSIWYG Markdown Editor.

Thanks for the idea. It has been invaluable for meeting preparation when discussing several documents.

You can do this also with batch process:

For Markdown, I see a lot of people resisting it because they find it hard to deal with not seeing a visual representation of what they might get at the end of the process. I mean, that’s why DEVONthink has a styled Markdown editor, right? And others get very distracted by the fact that the markup remains visible while you’re writing.

By default, Pages has semantic styles — title, heading, subheading, footnote, caption, and so on. I set up the extras I need, write the document, and then mess with the styling at the end. I find that a much better way to work with a word processor, which is why I think it would be good if the software was more tailored to it. And there’s no reason why you couldn’t do all of that with HTML as the underlying format. You’d then get the benefit of HTML being far more likely to be readable in 20 years, and far more likely to be convertable between different pieces of software.

I didn’t say that it’s impossible with HTML. I just said that probably not many people write HTML nowadays, so there’s probably not a market for this kind of tools. And there’s the issue with different target devices requiring eg responsive images, media queries and all that.
Wysiwyg comes from print media where you have a physical page. HTML does not know this concept, much less MD. And a “styled” editor as DT offers is a far cry from WYSIWYG. I’m with you re “styled”, that is indicating the markup like bold, italic, headings etc.

You’re right about styles in Pages (and Word). And if people use them, they get the best of both worlds. Personally, I just don’t see many people do that.