Can I improve my process for turning data into information?

DEVONthink Scenarios
1

I’m looking for advice on how I might handle a common workflow in DEVONthink differently. One of the residual effects of being a DT user is the constant, nagging suspicion that I’m not using it nearly as productively as I could.

My main purpose for DT is to store documents (PDFs, web archives, images, etc.), most of which are accompanied by a text file that contains my commentary on each document to give it context. I’m pretty sure this isn’t an unusual usage scenario, but I’ll give examples just in case:

  • My doctor hands me an after-visit summary. I scan that to PDF, put it in DT, then create a markdown file in which I write my own notes about the visit, and I include a link to the PDF in that text.

  • My furnace is not working properly. The repair person comes and cleans off a little sensor and shows me where that sensor is and how to clean it myself in the future. I scan the bill and take a photograph of the furnace that shows me where to locate the sensor, and I store those image files in DT. Then I create a markdown file in which I write the cleaning instructions, with links to the bill and the photo for my reference.

  • I’m doing research on an idea for a play. Over the course of several years (or longer), I download newspaper articles and scholarly papers in PDF format, read them, and write a summary of what I learned from each. I’ve probably also done some annotation — primarily highlighting — in the longer PDFs. Same process: I write my notes about each article in a markdown file and include a link to the source PDF for reference.

The way I handle that now is to create topic-specific databases (health, home repair, playwriting), and in each of those databases, I create two groups:

  • A group to hold all the markdown files, each of which is named with the date of entry and a brief description of the topic (e.g., “2019-03-10 Furnace sensor cleaning instructions”)

  • A group to hold all of those supporting documents and images

The end result is that I have a collection of markdown landing pages for each topic or document.

This works for me. But so far my interactions with DT are more about putting data into it. I have limited experience getting information out of it. I can find those furnace instructions every October without any difficulty. But what about when I want to review my research on that play idea? Are there DT-specific techniques that I’m not aware of that would help me beyond simply re-reading all of my notes?

Re-reading all of my notes is a perfectly useful and reasonable way to process that data. That’s how I’ve always done it. And I don’t expect DT to have some magical button that I click so that it can explain the meaning of my play to me before I write it. But I’m trying to be aware of the limits of my own imagination now that I’ve turned to DT to help me store and organize this stuff. I’m interested in hearing about other ways of managing that information or extracting it that I haven’t considered and that might serve me better in the long term.

Thanks for your suggestions.

1 Like
2

In my mind, this sums up the answer to your question. In fact, it sounds like you’re using DEVONthink very productively and in a nice focused manner.

If this works and makes sense to you, stick with it. Part of the DEVONthink experience is how personal it is. I organize things in ways that make sense to me but perhaps not to you (though there could be overlap).

It sounds a bit like you are looking for “a problem for your solution”. Have you actually stumbled on a problem here? You said re-reading your notes is “perfectly useful and reasonable”, and that says a lot. DEVONthink is great for gathering and organizing but the connections it makes between documents - even if sometimes surprising - are still algorithmic. It’s fantastic at delivering the “letter of the law”, but it’s up to us to decipher “the spirit”.

PS: I hope this is read as helpful, not dismissive.

3

I’d say that’s a residual effect of being alive in 2019. “Productivity” is a thing that machines are tuned to provide. Not us. We have too many websites, and books, and blogs, and podcasts preaching about “productivity”, or “flow” or “workflow”.

Sorry for the rant. I’m with Jim. If you enjoy what you are doing with DEVONthink (or any machine or software) then carry on. In fact what you explained that you do sounds pretty cool. Over time you might pick up a trick or two, or change something in your methods, for stop – it’s the incremental adaptations to working with your personal intellectual concerns that matter. Not big changes; or (ugh) “improving productivity”.

4

Not taken as dismissive at all. Thanks for taking the time to consider my request. The fact that the answer is basically “keep doing what you’re doing” just spares me from some unnecessary anxiety.

1 Like
5

Even as a person who could tell you a long, boring (but true) story about how David Allen’s system has dramatically changed my life for the better over the past decade or so, I share your disdain for Tayloristic attitudes about human performance.

Thanks for your encouragement. I’ll keep doing what I’m doing.

1 Like
6

One part of what I hear here is … What about when I only think that I know what I want but I know it is certainly in DT somewhere … What then?

One approach is to use keyword searches. This presumes that you know before you start the document that you want had those certain words in it at that time. Another approach is to use tag searches. This presumes that you have established a tag system before you start to catalogue your documents according to questions that you will ask some time later.

The other part of what I hear here is … What about when I just want to browse and catalogue stuff in a different way. This sounds like a place to think about how you would implement tags.

Otherwise, as others have said, if what you are doing works for you, no worries. Don’t make imaginary problems just to cause anxiety about whether you are missing out on other ways to use DevonThink.

–
JJW

7

Thanks for describing your work flow and with examples. Makes a ton of sense.

In my use case, I have gone back and forth between

  1. using DT the way @Tony2X described and
  2. putting related files (PDF. images, markdown) together in a Finder folder. Other times I just rename the PDF / image such that it can be retrieved in the future.

Option 2 is often times less hassle and renaming the file does the job. Hazel (another application) can move the file to a suitable folder.

What might I be missing out by not using DT instead for information storage and retrieval purposes?

8

Can you describe what “hassle” you are referring to?

Hazel (another application) can move the file to a suitable folder.

But then you’re still adding another application to the mix.

9

Probably i don’t use DTPO as well as I could but this is what I’d do.

Customise toolbar adding icons group, smart group, rich text. (I use icon and text for easy-on-the eye).

Create a group (as many as you like) and a smart group (as many as you like) and put smart group(s) inside the appropriate group. Rename group (s) to suit, ditto smart group (s).

Put all supporting documents and images in the appropriate group. Use the smart group(s) to find whatever you are searching for. Provided they are ocr’s, there is no need to rename pdfs, unless you want to sort on pdf name. I always ocr and rename pdfs to my requirements but as I say no need to rename.

I don’t use markdown (waste of time trying to learn as far as I’m concerned) and I rarely do annotation. But i do make notes.

Get into the habit of tagging each item. Also, in group, right click on the group for properties (or add icon to the toolbar) then in the tags box add the tags you want to use for anything in that group. Doing that saves time when tagging because any item added to that group will automatically be tagged with whatever tag(s) you have put in the group properties box.

Using your furnace example: a smart group search could be set up as:

Search in 'Database"
Any of the following are true.
(If you enter All of the following are true then the search would be restricted)

Click on All dropdown box and select Content then enter the search filters. Separate each word with a comma or if you use a phrase then put it in parentheses.
For example, furnace, repair, sensor, bill, “one day”

Add as many ‘Content’ lines as you like.
I tend to use several lines as it’s easier to delete/amend.

Provided you have tagged each item. The tag filter ‘is’ or ‘is not’ is very useful.

For photos and pdf, i use filter ‘Kind’.

Flagging is also useful. So is labelling to colour. Both are sort parameters,.

Date added isn’t necessarily your date.

I expect you would have to play around with the search filters until you get what you want.

The advantage of a smart group is that even if it might be inside a particular group the smart will search in any group regardless.

As for adding dates, I use Textexpander and have created abbreviations so I can prefix each entry with a date, eg 2019-03-13. Best format for sort is yyyy-mm-dd

Have fun!

1 Like
10

Thanks for the smart group/search tips. I’m just starting to try to use smart groups effectively.

That I do. I often find myself needing to add dates within the past week or the upcoming week, so I’ve got snippets for those dates as well. For example, I’ve got a “3date” snippet that expands the date three days ago (%@-3D%Y-%m-%d ), and a “date3” snippet that expands the date three days in the future (%@+3D%Y-%m-%d ).

11

For things like the furnace example I have a Keyboard Maestro macro that takes the highlighted DEVONthink (DT) item link and create an OmniFocus (OF) task. So OF can be set to remind me of periodic maintenance and in the note field will be the link to the DT group or item. The best part is I can be reminded and view the information on the iPhone, iPad, or Mac.

I also use markdown files, but I have opted to keep them in a related group with the images. If I need to refer to a pdf file, then I use the DT replicate feature to place it in my project group and also have it in my indexed group. I use Zotero and Zot File plugin to place all the PDFs in a cloud storage of my choice where DT will index that folder for me. I make the index content available to DT2G (ios), so I have it for reference on the go.

12

Would love to know what macro you’re using!

13

I used the one from DEVONthink and modified it slightly to skip the Due date (see text below if you want to paste it in to use).

I just used Keyboard Maestro in this case to associate a hot key while in DEVONthink to execute this AppleScript.

image
image.png967×776 102 KB

So when I select a group or item in DT and use the hot key, I get something like this in OF. Where the OF task has a prefix of Project: and in the Note field there is the DT link that gives me easy access to the DT info on either ios or macos.

image
image.png944×1101 79.9 KB

AppleScript to create OF task

-- Script to add a selected record to OmniFocus as a to do
-- Written by Eric Böhnisch-Volkmann, Version 1.0.2, Aug 10, 2010
-- © 2010-2017 DEVONtechnologies, LLC
-- Modified March 29, 2017 by Jim Neumann / BLUEFROG / DEVONtechnologies, LLC
--> Added Custom Date option
-- Modified by Mark Barton, Version 1.0.3, August 10, 2017 10:03 AM to always skip due date



-- Set properties
property pDaysIntoFuture : -1 -- Created to do will have a due date n days in the future
property pPrefix : "Project" -- Prefix for the created to do item
property pDelays : -2
property pDefaultDelay : "In one week"

-- Import helper library
tell application "Finder" to set pathToAdditions to ((path to application id "DNtp" as string) & "Contents:Resources:Template Script Additions.scpt") as alias
set helperLibrary to load script pathToAdditions

try
    -- Get the selection
    tell application id "DNtp" to set thisSelection to the selection
    
    -- Error handling
    if thisSelection is {} then error localized string "Please select a document or group, then try again."
    if (length of thisSelection) > 1 then error localized string "Please select only one document or group, then try again."
    
    -- Get and format the data we need
    set pLocalizedPrefix to localized string pPrefix
    tell application id "DNtp"
        set thisItem to first item of thisSelection
        set theSummary to (pLocalizedPrefix & ": " & name of thisItem) as string
        set theURL to (reference URL of thisItem) as string
    end tell
    
    
    -- Add new to do to OmniFocus
    try
        tell application "OmniFocus"
            tell default document to set newTask to make new inbox task with properties {name:theSummary, note:theURL}
            
        end tell
        
        -- Create new Inbox OmniFocus Window to verify right task was selected
        tell application "OmniFocus"
            tell default document
                make new document window with properties {perspective name:"Inbox"}
                activate "OmniFocus"
            end tell
        end tell
    on error errmsg
        display alert (localized string "OmniFocus is not available. You may need to update to OmniFocus Pro to use this script.")
    end try
    
on error errmsg
    
    display alert (localized string "Error when adding item to OmniFocus") message errmsg
    
end try
2 Likes
14

On ios you can just use the Share action from DT2G to create an OF task with the DT link.

15

Thank you for the post Tony2X! It’s great to hear that other people are thinking along the same lines!
Ever since I bought DevonThink I’ve been meaning to research how to automate your type of workflow. Maybe some of the expert can give us some pointers. This is my thinking:
I read a ton of research and articles a day at my work computer. After each read - I have comment/thoughts/summaries - right now, I write those comments in an email message and copy the article below and send to my e-mail account. The issue is it’s stuck there. Now, I know I can import these messages into DevonThink, but what I want to do is a step further and really added value. I want to create a Table of Contents/Glossary type file where I have the subject of the e-mail extracted as title, underneath to have my comments/thoughts and under a link to this email as reference. So essentially a script to run against a group of items which creates a summary with extracted comments (I guess they have to be defined by start/finish indicator) and then just a link to the overall file/email. The value add comes from - no nee to review 100s of annotation files - things are summarized in a single file with a link if further details are needed. From a thought perspective, it’s easier to organize in your brain the ideas in this “TOC” type file and from there to create action items. Anyways, I’m not married to the idea of e-mails with comments if there is another way to automate this workflow. Meaning I want to have a single running file with comments and links to underlying detail files but not have to create annotation files - for example I’m on my iphone/ipad - I read an article - I have comments which I want to attach to this article - I guess I can send the article to devonthink to go, write the comments in the comment section of the file, and later at home have a script to create this master TOC file, with all file comments extracted and link to a file for all files within a group.

Has anyone heard of this kind of workflow/scripts? I’m looking for something which is efficient - less clicks/filling after each read and having restrictions on the work computer - the only way out is email, with the iphone scenario it might be easier.

Any thoughts would be greatly appreciated!

Thanks!

16

the hassle I was referring to was to create a markdown document with links to other DT items (an image or a PDF). I have found it faster to just create a folder and link the files by putting them in the same folder.

17

According to your previous comment…

  1. putting related files (PDF. images, markdown) together in a Finder folder. Other times I just rename the PDF / image such that it can be retrieved in the future.

I’m not sure why you’re referring to using the Finder for this when the same linking can be accomplished with a group in DEVONthink. I do this every day with Markdown and links, all within DTPO.

18

I did’t mean to say that it is not possible to do so in DTPO. I meant that it is possible to do so in Finder with equal ease.

I am still trying to figure out how to use DTPO the right way. When I was using it, I wondered what I should save in DTPO and what in Finder (outside of DTPO). And so many helpful apps that I use such as Launchbar, Hazel etc depend on files being in the Finder.

So wanted to ask you all as to how you decide what to store in DTPO vs simply in the Finder, does it become a file system replacement and if so, does it not make it difficult to save things - with Finder can easily create and save folders in the Save As dialog whereas with DTPO, need to go to DTPO and then do the Folder creation etc.

19

DEVONthink is not a file system replacement. We already have a great file system, so why replace it?

DEVONthink is a focused place to store data – original source documents, your own creations, web bookmarks and web archives – whatever you need.

I don’t recommend buying DEVONthink and then asking “why do I have DEVONthink?” But, if you have projects or personal files that you want to organize, quickly search inside the files and search across a set of files or the database, then DEVONthink is a good option. If you just need a place to dump stuff randomly and maybe never look at again – it’s OK but why pay for that since you already have the macOS file system?

Academics use DEVONthink to organize class materials, work on research and papers. Writers use DEVONthink to collect information needed for the blog, article, or book they are preparing, Hobbyists use DEVONthink to organize genealogy information, info for their antiques and collectibles, or any of a hundred other purposes. Others use DEVONthink to organize bills and personal papers. See more ideas here.

But, don’t buy it if you have no idea why – or because someone on a “productivity” forum said “oh, you need this”, and you cannot figure out why.

2 Likes
20

That is true. Have purchased it already now :), just trying to figure out how I can leverage it to add value over and above being able to use Finder for the same tasks.

Even tried to use the “Index a folder” approach, but indexing is not reliable and of course, files can move around outside of DTPO interface.

I agree with all that you said about people use DEVONthink for organizing and searching for information, however, I have not been able to see the extra value over simply storing in Finder and using folders and Spotlight. Just to be clear, I am not trying to diminish its value, instead, I would like to be able to use DEVONthink for what it does better. Of course, what it does better - nested tags, search AI of some sort, has to be worth it when compared with some extra management overhead and the inability to use apps like Launchbar and Hazel.

And I think many people have figured it out, therefore asking.