Please help with advice on Devonthink workflow

and had assumed that Scrivener would be right for that application because it seems to offer native outlining and navigation options that seem intuitive.

Never assume such things, especially based on hype you hear.

The value of software, like vehicles, is how it suits and benefits YOU. It doesn’t matter if a million people say a particular car is the one every person should have. Until you test drive it and get a feel for yourself, that’s their opinion. The same applies to software, yes even our applications.

5 Likes

And Scrivener might be the right tool for you!
At the time I had a complex pipeline set up, writing in Markdown with LaTeX and using Scrivomatic ( GitHub - iandol/scrivomatic: A writing workflow using Scrivener's style system + Pandoc for output… ). It was elegant when it ran correctly, but needed a lot of fiddling to set up and keep running smoothly.
What made me switch was a few things, such as the whole project-as-a-single-file thing and the inelegant sync situation with the iOS version, which always seemed buggy. I ended up many times with multiple versions of the same project, not knowing which one was the current one or what the differences were. Also, Scrivener has a whole lot of functionality that I never needed and I felt intimidated by it. It felt like I was dealing with a lot of overhead that wasn’t needed by what I wanted to achieve.

With Obsidian, everything is markdown, I keep my vault is a git repo, so I have very granular versioning and history and use that to sync with the iOS version. There are a butt-load of plugins that let you do anything you can think of, and do it easily.
The Pandoc plugin lets me export in a variety of formats if I need to very quickly (I don’t use it that often). Usually, to publish on the web, I just copy a completed markdown file to a different git repo which I then commit and a pipeline using Jekyll and Vercel publishes it automatically.
I can use LaTeX when I want to, I use daily notes and templates. I make graphs with mermaid, it syncs with Zotero and Readwise.

There is a learning curve with both Scrivener and Obsidian and if you are working on specific big projects such as a thesis or a book, Scrivener certainly has a lot going for it and is certainly worth a look. For me and my use cases, Obsidian is a better fit.

@markmusselman Reading you original post again, I think that DT and Scrivener might be the combination you are looking for. As others have pointed out in this thread, Apple Notes and Hookmark are not needed.

Regarding editing PDFs:
%rant% I hate Adobe, mainly for their SW licensing strategy and the ridiculous DRM they put on some of the ebooks I buy (EPUB and PDF) which makes the ebooks almost unusable for me. (@Adobe: Just put a d*** watermark in the file and let me open it in the ebook reader of my choice!) So I try not to use any of their products, even if not is a free PDF reader. %rant_over%

You can use the DT built-in PDF reader (limited but working functionality), but I prefer the DTTG one with an Apple Pencil (also limited, but enough for me).

And of course you can use 3rd party PDF editors (but not all of them work well with the DT round-trip sharing function) IIRC PDF Expert is one that does not play nicely. I am not sure if there is a comprehensive list of which PDF Viewers/Editors behave well and which don’t. But you can just try out the one(s) you have or search this forum for pointers.

Thank you. You have given me lots to go in and I appreciate your time.

1 Like

I would not attempt to produce my thesis in Obsidian. Scrivener will require you to compile and produce the final version for submission in a full-fledged word processor like MS Word or Pages. Your director and committee may want to use MS Word’s comments and tracking to respond to drafts. Scrivener works well for many people for drafting long works, but it’s complicated as well as powwrful. Play with it for the free 30 days. Watch lots of the Literature and Latte videos.

You might want to use a reference manager which will work with Word or other word processors to handle formatting citations. Zotero is probably the most popular, but see also Mandeley and Bookends.

Find out how your readers want to respond to drafts.

2 Likes

My typical workflow is to download pdf files and other content into a specific database in DT3. I use the functionality of DT3 to read and annotate/highlight the pdf so that they are saved in DT3. I also have the same pdf files available through the DTTG app on my I-pad. This makes annotating documents convenient as they are in multiple apps. Sometimes I have used the function to download annotations to get a summary of my highlights.

I don’t make notes in DT3 very often. What I have started to do over the past year is to move my notes to Obsidian. Everything, notes and written works in Obsidian. This is my research depository with links to references by using Endnote. Everything is referenced in Obsidian with both a direct quotation from the original source, and a summary or paragraph written in my own words.

For writing an article, or when I wrote my dissertation, I use Scrivener. I wrote my dissertation in Scrivener as it worked best for moving sections around and dividing/combining other sections. I still use Scrivener for writing research articles using my notes from Obsidian and annotations from DT3. The feature of ‘compiling’ a document in Scrivener allows you to export the article/dissertation to Word which is what most institutions and publishers ask for.

Workflow Overview:

  • Search for articles and content.
  • Download all content (e.g., pdf, magazine articles) into a topic database in DT3 (e.g., leadership, teams, sensemaking).
  • Download all reference information in my reference management software (e.g., Endnote; Endnote works with Scrivener and Obsidian).
  • Read and annotate/highlight the articles downloaded (either by downloading to a unique database all relevant articles, or by running a content search in a larger combined database).
  • Transfer the annotations/highlights to unique notes in Obsidian with references.
  • After reading (annotations in DT3) and making notes (Obsidian), I can then begin to form an outline for my research article in Scrivener. I then continue to populate the outline and create new sections as I continue reading, annotating, and making notes. This process continues until all articles have been read and annotated, and all notes have been made in Obsidian and captured in the article in Scrivener.
  • Final organization of the article and complete finishing touches to make sure all sections work together and that the article flows from the beginning to the end is done in Scrivener.
  • Transfer to Word.
  • Final editing and formatting is done in Word before submitting. Once the final version is done in Word with all appropriate references linked from Endnote, all future revisions are made in Word.

Thats a quick overview of the workflow that I have developed over the years. For shorter articles, I typically skip Scrivener and write directly in Word (e.g., book chapters, blog posts). But for most everything else, I use the above workflow.

4 Likes

Something that doesn’t seem to have been mentioned in this thread is how one deals with footnotes. I wrote an 80,000-word doctoral thesis using Scrivener, and probably had footnotes running into the hundreds. Scrivener handles those quite well, but I’m not sure I would want to try it in Obsidian. No doubt there is some way of doing it (Obsidian is remarkably flexible) but Scrivener would be my choice for a longer text that demanded footnotes. I also found that Scrivener integrated well with Bookends.

2 Likes

I agree with everything you say, but… (I know you know this, and it’s just a typo, but for the benefit of the OP in case they don’t):

Play with it for the free 3 days

That’s 30 days and only days you open Scrivener count, so if you only use it every other day, you get 2 months before you have to buy…

(Sorry for intruding…)

2 Likes

Check out Mellel + Bookends with DT for thesis as for writing a plain text. But before that you have to do a lot of structuring work (reading and annotating, composing and endless re-composing of the logic of your article/dissertation and so on). For this type of work I recommend any comfortable OPML software. I use Omni Outliner for it. You can integrate it easily with reading, annotating and linking the PDFs in DT.

DEVONthink, Scrivener, and Bookends user here. Not a lawyer, and I’m note sure that I have much substantial to add to all the great suggestions provided by others already. But maybe you can still take something at least a little bit useful from the description of my workflow and my choice of apps.

First thing is the note format. I’d recommend to go for Markdown. It took me a while to get used to it but a number of reasons pro Markdown had convinced me.

As you probably know, Scrivener has a rich text editor. Of course you can write in Markdown in it—as you can do in every writing program, you just might need another app to render it—but that’s not even necessary for using Markdown notes taken in, say, DEVONthink. Scrivener has the great Import & Split function which imports Markdown, structured by headings if there are any, and with footnotes. The latter is really important as Apple never implemented footnotes in their RTF format but came up with RTFD instead which does not work with many apps (none outside the Mac universe), not even with the present version of their very own Pages.

While Obsidian has its own Markdown Footnotes “flavour” that does not work with Scrivener, DEVONthink’s do because they both support MultiMarkdown. MMD allows even two kinds of footnotes and Scrivener reflects this by turning on of them into endnotes. So if you use footnotes (and maybe endnotes) a lot you should use a note format that allows to transport them between apps.

Another thing I’d like to mention is the bibliography manager. Mine is Bookends. DEVONthink sits in the middle of my workflow. Everything I write and collect goes into it, some of it moves out, mostly into a Scrivener project. Sometimes I’m collecting material for a project that already does exist (like a monthly magazine), sometimes material collected in DT at first has to reach the critical mass to become a project at all.

What’s important for me is that, again, DEVONthink is in the middle, but not everything I write—actually the minority of it—is done in DEVONthink. And neither in Scrivener, at least not before the material is ripe to become a project. I’m building texts heavily on atomic notes. Very small ideas, lots are just rubbish, but I don’t judge them at the moment they come to my mind.

For jotting these notes I prefer a lightweight and customizable app that works on all my devices. For a while it was 1Writer because it can save its notes by default in an iCloud Drive folder I had indexed by DEVONthink. But I switched to Drafts although I need an extra step of exporting my notes to DT because of Drafts is not saving its notes openly.

It was the great scriptability of Drafts that make me switch and the comfort of many tiny details—widgets for the iPhone homescreen, even a life widget for the text most important at the moment. So good when you’re brain is set to a certain subject and the ideas keep floating in. And it works even on the Apple Watch, dictating and tagging for the later use in DEVONthink on your wrist!

(Sorry, I don’t want to sound like a commercial. I just got my first Apple Watch a few weeks ago and am so glad it worked out and I did not suffer from gadget fever.)

Anyway, back to what I wanted to talk about, the bibliography manager: When DEVONthink sits in the middle, Bookends kind of lurks above (or behind?) all apps. Whenever I quote something I use a Bookends reference, the temporary citation, to be exact. That means, a note taken on my phone, kept in DEVONthink, and finally moved into a Scrivener project, has an identifier that tells me the source of the quote. And in the final document compiled with Scrivener it will be replaced by the citation in the format I like or the editor or the publisher.

Even when I am reading paper books (love them) and quote them Bookends is there: When it’s a shorter quote I dictate it into Drafts on my phone, if it’s a longer one, I scan it into Drafts. And then I use my little Drafts action that holds the Bookends citations of the book(s) I’m reading at the moment, I just insert the page number, tap on the reference, and get a proper Markdown footnote with the reference tagged “excerpt” which triggers a Smart Rule in DEVONthink.

My point I wanted to make so verbosely is: I don’t now of any other program better to sit at the center of a multi-step and multi-app information/text workflow than DEVONthink, and I don’t know of any better writing program than Scrivener. The other apps involved—Bookends seems to be the best bibliography manager on the Mac for me, but it is Mac only which is a huge con for some—whichever they are, should be picked to fit with these two central programs.

6 Likes

So many generous replies - thank you all. I’ve clearly got some work to do - you are all so knowledgeable about options and comfortable with the workflows you’ve developed. I’ll eventually find what works best for me. Thanks again!

1 Like

I’m just wrapping my PhD and can’t add much to what other people have commented here with this exception: I have made great use of DT3’s ‘Workspaces’. A workspace is a themed collection of files you create and update as you wish by adding and subtracting files to your workspace. The workspace or collection is visible in the central working area (I forget the name for this) with tabbed entries at the top and the main file you have selected and are accessing to work on or read in the main working area. I have 23 workspaces (themed collections) at this time and I toggle between them as I wish. This suits me better than working with tabs because in a workspace I might be juxtaposing files from differing origins in my main organizational index. You access workspaces under the ‘Go’ heading-link on the main menu; workspaces is at the very bottom. One caveat: you need to hit ‘update’ manually when you make file changes in your workspace; DT3 does not do that automatically and I’m glad of this because sometimes I will just view another file momentarily before reverting to the cached workspace file. In each workspace of mine I have 5-9 files cached.

One other thing I’d share: I wanted to use Scrivener for my dissertation but ended up using Word because my committee members all wanted to reference and edit Word files and it became unwieldy trying to update my Scrivener file to work with their files. You might want to check out what the expectations are for your thesis template and preferred workflow of your committee members in advance!

Good luck!

6 Likes

May I ask you how you manage atomic notes?
You use Draft: do you index the notes in DT too?
Do you link the atomic notes each other? How?

I’m interested because I like the idea of a net of ideas linked each other.

I would add this: In other academic writing I had issues with Scrivener and Zotero when I tried to use them from which I concluded that Scrivener wasn’t particularly reliable for me to use for academic writing when I needed airtight citation management. If anyone has any tips I’m all ears.

My recommendation would be to use Bookends! It’s not perfect, but it is very powerful. And there are various scripts to be found in various forums that integrate Bookends and DEVONthink. Ryan Murphy wrote somewhere (perhaps on the Obsidian forum, perhaps on his own blog) about using scripts so that he could exploit Zotero’s online search to find material, which he then imported into Bookends. Bookends has recently introduced deep linking, meaning that you can copy a link to a specific place in a pdf and paste that link into a text file, so that you can later go straight to the location in the pdf if you need to. Combine that with Hookmark and DEVONthink, and you have a way of building quite a tight network of links to material.

Edit: I found Ryan Murphy’s blog post: https://fulcra.design/Posts/The-best-of-both-worlds-Use-Zotero-to-import-references-into/#:~:text=bookends/

3 Likes

What sort of issues? If ‘airtight’ means fully automated, I don’t have any ideas either. Scrivener simply doesn’t work that way.

I write all my academic outputs in Scrivener and add my own placeholder refs, e.g. {Maser 2023, 34-45}. There is no live link between Scriv and the bibliographical software, and I don’t really care for it. I simply keep adding my bibl data to Zotero for everything I cite alongside the writing. Zotero’s power comes in later, after I’ve exported a complete manuscript from Scrivener to Word – only then do I insert all the references with the Word-Zotero plugin. That gives me a file with lots of flexibility (e.g. if I’d need to change the citation style if article is rejected by one journal, and needs to be prepared for another). I have never successfully used Zotero’s RTF scan.

So my workflow does rely on manual labour, but it’s mostly not too bad in terms of time investment (unless you are working on a book, where you’d need a day or two to slot all the refs into place). I keep relying on it because Zotero keeps rigorous control of what you cite and where.

I got bitten a while back, when I received suggestions for revisions for an article that were so impactful that I opted to reimport the Word draft (where I do final edits before submission) BACK into Scrivener. This meant I had to insert the Zotero references twice: once for the original submission; and once for the re-submission. Irritating, yes, but working on the re-draft in Scrivener still gave me massive benefits. I sort of accept that academic writing and especially the editing and formatting processes include a degree of busywork that cannot be avoided.

Anyway – good luck with the final stages of your PhD. (Btw, I also fully agree that DTs workspaces can be very powerful!!)

4 Likes

This is a great power user tip :rocket: and I will need to try this out - thanks! My workflow is very similar to yours. I have just started a new book project. All resources live in DT; all planning, thinking, atomic notes, reading notes, small case studies, etc are written in Obsidian (Drafts on mobile for anything that occurs to me on the go). I have a big note-to-self stuck at the top of my Obsidian project note that says – ONLY TURN TO SCRIV FOR THE FULL WRITING-UP PROCESS’. I am trying more planning and outlining this time round. The first book I wrote took years and years, and I made my research discoveries as I slowly, laboriously, wrote and edited – it took a long, long time. I don’t have that luxury of time anymore (hello tenured job!), so hope that more extensive planning will lead to a much tighter writing experience. Scrivener therefore sits slightly differently in the research workflow.

I also write and draft shorter notes and so on in markdown, either in DT or in Obsidian. My solution for getting things into Scrivener tends to be DT’s ‘convert to rich text’ command. From there I drag whatever I need from DT/Obsidian into Scrivener and am away.

3 Likes

Thank you for sharing these insights on your workflow … it hadn’t occurred to me to use ‘placeholder’ references and then synch with Zotero later, post-export-conversion-into-Word. I’m definitely going to try that.

2 Likes

Really similar to what I do. I use OmniOutliner and Ulysses for writing.

Sorry for my late reply, it seems I had overlooked your posting.

I don’t index the notes taken in Drafts in DEVONthink. That is not possible due to the one (big) downside of Drafts: It keeps its notes in its own database which is not accessible from the outside.

I simply follow Drafts’ tag line “Where Text Starts” verbatim and use export actions to move them into DEVONthink. I tag them in either of the programs. Mostly manually but a Drafts action like the one for adding a Bookends reference sets the tag “excerpt” automatically. My main Smart Rule for the Global Inbox removes this tag and sets a label instead. Because I want to keep the (permanent) tags for content and not for the type of text/source.

I link the atomic notes in DEVONthink. After export all notes in Drafts’ get moved to its archive and normally won’t be used again.

This workflow is not ideal, but the fast note taking in Drafts for me pretty much is, so I went for it.

1 Like