DEVONthink academic workflow incl. Obsidian, PDF Expert, Zotero and Scrivener

Hi there, new DT Pro user here :wave: I’ve read a great deal of posts on this forum, searched the web and watched some YouTube videos but I still struggle to figure out how to use DT, and, most of all, how to integrate its usage with other apps, to best suit my needs.

I’m an academic, currently working on my PhD but also on some other academic papers. I’m looking for a simple workflow and I think I know exactly what I need, I’m just not quite sure how to make it all work. I use MacBookPro, where all my writing and searching for information sources is done, and iPad Pro, where I read, highlight and annotate my PDFs (which usually are the scientific papers I then want to cite in my own dissertation). Then, ideally, my workflow would include extracting the highlighted parts of the PDFs to save them for later and play around with them. And then of course writing based on my notes :smiling_face: Up until now I just manually copied parts of articles I wanted to cite in my own writing and pasted them into Word document where I organized them thematically and then used them for my own writing. With a project as big as a dissertation, that is not an effective way of working anymore (if it ever was). I need to automate things, but on the other hand I’m not the most tech-fluent person and I don’t desire to be: while I understand I’ll need to learn a great deal about the tools I’m trying to use, I don’t aim at creating a perfect system with all the bells and whistles. Good enough and simple is all I need.

That said, I was thinking about the following apps for different stages of my workflow, but the more I learn about these things, the more confused I am:

  1. DEVONthink Pro to organize and store (?) all my resources, mainly PDFs with scientific papers and books on my Mac, and DEVONthink to go to access them on my iPad.
  2. DEVONagent for advanced searches.
  3. PDF Expert for my iPad to read and annotate the PDFs - I already use it and it does a good enough job with annotating, I’m just not sure how well will it integrate in the next steps of the workflow.
  4. Obsidian for creating notes out of extracted PDFs annotations – both “atomic notes” and thematically organized longer notes consisting of multiple atomic notes. I know you can make notes in DT too but I just thought I would feel better if my notes weren’t mixed with my source PDF files.
  5. Scrinever for writing – using my thematically organized “summation” notes from Obsidian as a foundation for developing chapters.
  6. MS Word for editing.
  7. Zotero for citations.

My questions:

  1. I want my MacBook and iPad DT databases to be synced. I’ve read the iCloud is not ideal for that. Is syncing via Bonjour appropriate in my case? And is it better to store my files in the DT or on my Macbook’s hard drive and index them in DT in case of this particular conjunction of apps? I want PDF files to be accessible via DT both on Macbook and iPad but I also want them to be linked with Obsidian notes and for Zotero to use them as citation sources.
  2. I know I can open PDF files I access from DT to go in PDF Expert on my iPad but I don’t know what to do with them once I’m done annotating. I mean, where do I store annotated files? I want them to be accessible via my DT, so I want the original “clean” PDF file to be replaced by the annotated one. How is it possible when the reading and annotating is done on my iPad? It would perhaps be easier if the main storage repository for my files was an iCloud. But if the iCloud syncing is unstable and I want my files to be stored on my MacBook, what is an easy way to automatically make them “go” there from my PDF Expert on an iPad?
  3. I’m not sure how to link to source PDF files in DT while making a note in Obsidian. I know there are some YT videos on the topic so I’ll probably figure it out, but any recommendations are highly appreciated!
  4. Same goes for throwing Zotero in the mix.

I know that’s A LOT to unpack. I appreciate anyone reading and trying to help even with just one bit of an information. Any tutorials, blog posts or forum discussions recommendations are welcome (though I feel like I read this forum end to end lol). Thanks! :raised_hands:

2 Likes

Welcome!

Do run a search on the forum for “academic” or perhaps “researcher”. Workflows have been discussed many times and are always interesting to read about.

  1. “DT” isn’t a separate drive so this question is redundant. DT files are stored locally on your Mac, and synced to your iPad.

  2. I’m ignoring the second part of your question because we’ve agreed in question 1 you’re storing your files in DevonThink on your Mac. When you open a PDF from DTTG into PDF Expert changes should sync back automatically. With the caveat that it isn’t working quite properly at present but will be resolved (see here). This is fine for what you want - if you ever need a clean version of the PDF again, you can export a clean version from DT on your Mac without annotations.

  3. I think on both Mac and iPad you would use the DT item link (on Mac you certainly would, I’m just not sure how it works on DTTG and your question isn’t clear as to which device you’re using for this step). Are you a new user of Obsidian too? If so, do think about if you actually need/want this step. Plenty of people do use DT and Obsidian together, but equally plenty of people don’t. I just store my notes in DT alongside my research material. I tried Obsidian but it’s much easier just having all my work in one place.

  4. I will leave this to others, but you can also run a search in the forum for Zotero as various folk use it.

2 Likes

I have to say that I have found that often a solution that I assumed takes a different app is solvable in DEVONthink 3 itself. I would really recommend experimenting at least with that strategy. It is an experiment too to some extent. You never fully settle, it is like one’s actual work.

I would perhaps wonder if there is too many apps there.
For example you don’t need to mix, as you put it, your own notes with source PDFs. I know the feeling well that they should be separate. But you can do that in DEVONthink 3 itself. Just keep them in two different databases or even folders?
I used to do that but now I just put them all together, I do have a prefix I use for my ‘own’ notes and docs. That makes them easily sortable. I also date them in the title in a way that makes them sortable by date really. I think using too many different apps creates its own problems.

I don’t really write inside DEVONthink 3 much. Then I have a very very simple writing system with no frills and I bounce off DEVONthink 3, if I can put it that way, all the time.
In my opinion, other than a reference manager, I use Bookends which has integration with DEVONthink 3, I would stick to DEVONthink 3 for everything. I do use Ulysses for writing and then often move the notes or docs into DEVONthink 3 for posterity or whatever. I do, I realize make a lot of notes in “Finder Comment” which I then show in ‘view’. I find it totally adequate and can take any amount of text I have thrown at it, which is usually only a line or two.

1 Like

Per iCloud syncing. I may be the only person but I’ve not had any issues at all with syncing via CloudKit (not iCloud Drive) between DTTG and DT on the desktop. I have been testing DTTG/DT as a bookmarking replacement for raindrop.io and it’s worked flawlessly for that scenario.

That said that is not a high volume of stuff. But, I also use DT to archive my email on the desktop and it syncs to CloudKit as well. That is a high volume of stuff, especially on the first sync, and it was also flawless.

1 Like

Organize first based on the outcomes that you need to attain. Then, assemble and choose from the list of apps that allow you to get to the desired outcome. Not the other way around. Here is an example related to the scope of your question (with my selections in bold).

  • Write a long-form treatise / document → Scrivener, Word, LaTeX, OpenOffice
  • Manage a reference bibliography database → EndNote, Zotero, Papers3, Bookends, Mendeley
  • Review and annotate over PDFs → within bibliography database app (especially on iPad) OR using dedicated annotation apps (Highlights, PDFExpert, Goodnotes, …)
  • Organize, compile correlations among, and share various collections of annotation notes → Obsidian, DevonThink, Tinderbox, …
  • Organize and compile information on themes having file collections stretching back two or more decades → DevonThink, Tinderbox, …

One app does not have to do it all, nor should one app be forced to do something that another app does far better by whatever you deem as the best standards for efficient workflow. OTOH, before going too far to choose multiple apps, consider also that sometimes the highest friction is not working within any one app, it is in translating, moving, or coordinating a unit of information easily between any two or more apps. When stuck on this latter point, the best approach is to ask directed questions. For example, as related to DevonThink …

  • How well does a Zotero database of references integrate dynamically with DevonThink?
  • What is the most effective workflow to synchronize atomic notes stored in Obsidian to a DevonThink database?

Finally, you are not alone in your struggle to appreciate where and how to integrate DevonThink into your workflow. Take comfort (and care to realize) also that no one is pressing you to make an immediate switch to DevonThink for doing something specific just because they happen to have a good fit for doing that specific thing with DevonThink. In this regard, I still bounce in and out of Obsidian and DevonThink to this day, even after a good decade of work with the latter, and I only recently settled on Bookends versus Mendeley (and at one point Papers). Full comfort with what and how an app functions can take a bit of time to grow in you, especially for apps as deeply feature-loaded as DevonThink.

In all cases, allow time to explore, but keep sense of the where the road is supposed to land.


JJW

5 Likes

My two cents here are, to simplify your workflow (DT: macOS DEVONthink, DTTG iOS version):

  • Use DT/DTTG in as many tasks as you can accomplish.
  • Don’t use PDF Expert in iOS, and directly annotate inside DTTG. The annotations tools are good for normal annotations.
  • Don’t annotate anything in DT if you don’t want the PDF to go corrupt (to clarify, I mean in macOS only). It is not a DT issue but macOS itself, as the PDF framework is very buggy, and each new macOS version goes buggier. Don’t annotate as well with integrated macOS Preview app. Use PDF Expert or another tool in macOS that doesn’t use the integrated PDF Framework.
  • In DT, extract the annotations from DT itself. It will generate the file and my next step is simply store it in other database for further reference and/or searching, but you can export or index to another platform.
  • Annotation exportation from inside DT can be (more or less) automated: Stream annotations from your PDF reading sessions with DEVONthink - DEVONthink / Automation - DEVONtechnologies Community
  • If you scrap from web, you can automate and make some beautiful scraps in PDF: DEVONsave v3 (fulcra.design) (There are some discussions inside the forum about this, but no one directly presenting the last compatible version, then I put the original outside link).
3 Likes

Hey Marta :wave:

Welcome! I’m a fellow academic, and just chiming in because your workflow matches mine very closely indeed. You’ve already got some really great replies, and I will bracket my own by suggesting to go slow, and keep it simple.


If at the beginning of my academic writing and researching career someone had handed me my fully fledged workflow, I wouldn’t have known what to do with it. Also, having spent so much time building it myself over the years forced me to get comfortable with a suite of complex apps with endless permutations, tweaking things just so they suit what I want to get done (cf. @DrJJWMac 's sage advice: focus on your ‘desired outcome’). That said, I’m sure you have saved yourself some heart-ache by deep-diving into the DT forum, which is a goldmine for academic productivity and where I have probably learned the most (about DT and other apps, too).

Anyway, I’d also suggest to pair things back as much as you can. Spend time on the things that really matter (research, thinking, writing!), and build your workflow from the ground up as and when you need a new complication.


  1. Don’t use Devonagent just yet, and get comfy first with getting data into DT (via the clippers, import, drag and drop, etc. The DT Manual is wonderful on how to get data in, as is this free e-book: Take Control of DevonTHINK 3).
  2. Don’t use an external PDF reader on iPad (yet). I tried but couldn’t find any additional value. My needs for PDF reading are modest: highlight and annotate. Working with DT2G has the benefit I don’t need to ‘save anything back’ to the app. All annotations simply sync back to DT on my Mac. I’ve never had any trouble with the PDF framework that @rfog describes, but be warned!
  3. I use Obsidian for exactly the reasons you cite: keeping a knowledge base of reading notes alongside DT. You could do this all within DT, but in my view, there is less friction and Obsidian is designed as a note-taking app (DT isn’t, despite its immense capacities). I also keep atomic notes, but sort off – there is way too much fetishising of this vague idea of atomicity – I mean, good luck deciding on what is just ‘one’, ‘indivisible’ ‘idea’. But it is very valuable keeping both reading notes (that respect the flow of argument of your source), and extracted from there and in conversation with other notes, what you might term ‘concept’, ‘idea’, or ‘atomic’ notes that cut across your reading and thinking; and which may indeed, as you describe, belong in their own (arbitrary, user generated) ‘thematically organised’ hierarchies.

Your Qs:
1
I sync with Dropbox and it is rock-solid. Bonjour is fine if your machines are routinely in the same room. I would avoid indexing for now until you understand its opportunities and limitations exactly (because messing with indexed files = DT messing with your files stored in Finder).

Keep it simple with Zotero. It’s a brilliant tool. But I only use it as the repository for all citation metadata (NOT PDF storage and annotation), and to generate bibliographies at the end of the writing process. These apps clearly have overlapping capacities, but you don’t need to use them just because you can. I think it’s better to have one app do one major task: file storage (DT); bibliographical data management (Zotero); notes and thinking (Obsidian), even if all three apps allow some of these things to some degree. What is mean is, don’t duplicate these processes across different apps, but rather choose the app you like most for any one process.

Otherwise when I write, I keep my own shorthand place holders when I cite stuff (e.g. Cronon 1991, 22). I expand these only when I turn to Word to edit a draft for submission. That means, then, that I don’t do anything complicated like indexing a Zotero asset library (but you can if you wish).

3
@MsLogica has already given you good advice here. DT shines because every item has a unique location that you can link to it from anywhere. Simply right click any item in DT and ‘copy item link’; or hit ‘edit’ in the menu bar and do the same:

Screenshot 2023-09-26 at 11.18.04

You can paste that link wherever you want.

I get some extra mileage from writing my notes in markdown in Obsidian, and in my taskmanager (ToDoist), which as you may know offers an easy way to format links in this way:

[decriptive text here](link address here).

Here’s a real life example from my Obsidian from something I recently read and took some notes on:

[PDF in DT](x-devonthink-item://03CD1967-1F22-47BA-AD6B-F4C3EE11D9EB)

This is what this looks like in the wild:

The huge benefit is that I can just click the link from Obsidian and the relevant PDF gets opened by DT. It’s MAGIC and greatly speeds up my subsequent working with these notes. Obv you can do the same in-house in DT, which offer some very excellent ways of working with plain text, markdown, or RTF note taking.

Right – this is more than enough by way of info I bet.

Good luck, enjoy the apps, and your academic workflow journey.

7 Likes

Although the software I use is a bit different, maybe you can find something of worth in my writing workflow: https://wippp.com/blog/taekman-writing-workflow-2020

Marta,
Welcome. A reference manager is the only other app you need to complement DT in your academic workflow. Some people here use Bookends. I use Paperpile. You can index your pdf files to DT. You can do everything in DT, from note-making to writing your papers. Simplify your life and focus on your discoveries.

4 Likes

Hello Marta, I just completed my PhD in late August and my dissertation has been accepted into my university repository - yay!! My workflow did not differ substantially from what you outline or has been suggested above, with some variations. But I’ll add a couple of my own refinements:

  • Yes to storing files in DT3, including PDFs - clean and annotated. No need to duplicate storage elsewhere on your laptop though that is tempting.
    Pertinent to this, DT3 has a wonderful helping tool when storing annotated PDFs and that is the 'Summarize Annotations’ feature, found under the ‘Tools’ menu heading. Just single-click on the file of interest then go to Tools >Summarize Annotations and, voilà, a new entry (file) appears next to the file of interest and it has collated all your annotations. I would then parse through my list of collated annotations and ‘super-highlight’ those most important and sometimes re-float them to the top of the document. so you now have two reference files - the annotated PDF and a file with all the annotations captured in one file. it’s FN amazing, and so helpful!

  • in DT3 I also used a feature called ‘Workspaces’ found at the bottom under the ‘Go’ menu to create themed groups of files, e.g. Research, Conclusions, Sandbox, Data Analysis. When using this I would group together a whack of related files that show up as tabbed entries above the DT main writing field, and I could easily toggle between and update and delete files as I so wished. The files could be from different d-bases so this was very handy. One caveat is that you need to ‘update’ the workspace manually every time you make changes as DT does not save new groupings automatically. The ‘update’ feature is found as a sub-link to ‘workspaces.’ This feature was/is also so helpful. I now have 25 Workspaces, btw, most concerned with my PhD research and writing but some reflecting other projects, too. Overall, this was definitely easier than working with ‘tabbed’ entries.

  • I didn’t see this mentioned above but in the interests of my efficiencies I formatted my DT writing files (I don’t use Obsidian) to match my MS Word template and then it was an easy copy over to my Word files where I did my drafting and final writing. I tried using Scrivener and secondary writing files at the outset but I think it would’ve taken me an extra brain to keep track of this. Going from DT > Word, and using Zotero with Word worked flawlessly in the past three years, I’m delighted and relieved to say.

Good luck to you!!!

6 Likes

Hello! I have a very similar workflow to you in terms of Obsidian, Zotero and Scrivener. I am not an academic, but an independent artist and writer. However, my work is very research heavy, and I noticed as well that when I first began to use DEVONthink in the last year I spent a lot of time very frustrated that it didn’t work like Evernote, which I finally abandoned because it wouldn’t let me store documents locally, and with some documents containing multiple pdfs made from photographs of archival materials where many were hundreds of pages, I was really struggling waiting for it to download my files. There are a lot of folks here who’ve been encouraging and offered good advice. I’m going to share mostly about my structure, and workflow with research logs and Zotero.

Structure
I have a group named “Finding Aids, Notes, and Catalog Entries by Repository.” In that group are a series of groups named after the Archives and Libraries I research in. Inside a Archive grouping is a group labeled “Notes (Links to Scans of Holdings) - XXXXXXXX Library” Each note is written in Markdown and created from a template. It includes the research log number (more on this later), and a link to relevant Finding Aids and to PDFs of any scans I’ve made, or that Copy Services made. All of those pdfs are stored in a top level grouping called Resources. I try to make sure the Archive’s locator number is in the file name of any pdfs. What makes DT ideal for me, is that I can link to a particular page in a pdf or to particular text. To do this click on the page you want to link to, right click, select “Copy Page Link.” You can use Markdown to create a link with this app link. One thing that took me a while to figure out and I got some help here with is that when you past a link into a DT Markdown document it creates the link for you with the filename of the linked document as the linked text. Personally I wanted more control, and if you Paste and match style you have that to write your links. You can also paste the application link into Zotero. I’ve been putting pdfs into Zotero as links inside notes. I still have files that I’ve added to Zotero over the years, but am slowly moving over.

These applications links are also very helpful in that they can be added inside the Research (or wherever makes sense to you) folder and will open in DT.

My workflow may be different than others in that I don’t add anything to pdfs, but I want to be very easily able to show where in a pdf, say a transcription, relates to and this workflow is helping a lot.

I also ended up using a research log that is just a Numbers doc and stored in the project. I have it favorited and click the button to open in external editor at the start of every session.

Finally, I don’t use an iPad, but have a laptop and a desktop. I ended up creating a Sync Store on a portable harddrive and use that to go back and forth with. I do backup the harddrive approximately weekly to another harddrive. I also end up with it set to sync to both machines at shutdown of DT. The key here is making sure any external editors are closed before quitting DT, and then unmounting the drive before I shut down the machine.

I hope this helps, and I’m also happy to answer questions.

4 Likes

If you are ultimately going to do the writing in Scrivener, you might consider taking notes in Scrivener as well. You should definitely consider the pros and cons of adding a third application to the DevonThink - Scrivener workflow.

Experiment with DevonAgent using the specific sources that are important to you. It’s almost completely useless to me because one of the most important sources in my field is inaccessible to its web crawler. YMMV.

3 Likes