Hi Kevin,
As a medical researcher I am in search of a solution of the same problem. Can you elaborate a bit more on your process? What script do you use to tidy up the notes? How do you tidy them? Are you willing to share?
All the best,
Johannes
Johannes
Sure. I can write this up in more detail at the weekend (and donât hesitate to bump this thread if I havenât). My AppleScript skills are very amateur but Iâll share what I have so far.
Kind regards
Kevin
Cheers, Kevin.
Looking forward to it! Thanks in advance and happy new year!
With apologies for the slight delay, here you go.
Background
As background, this workflow assumes:
- that original materials are stored in a (for example) â/Sourcesâ group and its sub-groups, for example:
/Sources/Articles/[ArticleName]/[ArticleName].pdf
This means that there is one canonical place to look for any article.
-
that feeding DT with short, one or two paragraph, extracts is useful (eg for the see also / auto-classify functionality, and more generally for classifying extracts into multiple groups )
-
that these extracts are stored in:
/Sources/Articles/[ArticleName]/Notes/ -
that each of these extracts is then replicated to one or more groups in a separate tree that is organised by themes / concepts / ideas.
Also note that this is currently only for extracting original materials from PDFs, rather than directly adding my own comments to PDFs. (It would be relatively easily extendable to that, but thatâs now how I prefer to work, at least for now.)
PDFs > Bookends
If a citation manager isnât important to your work, then this step is not necessary and doesnât affect the rest of this workflow.
Other citation managers are available but Bookends is a good, dedicated MacOS, app, with (now) very good AppleScript support.
Highlighting PDFs
In any PDF editor that highlights.
I use either native Bookends support - which has the advantage of being able to highlight and sync from an iPad - or Highlights.app.
Different colours of highlights can have different meanings which can be useful once you start to analyse your notes in DT.
For me:
Yellow: interesting extract
Blue: interesting extract that I think I disagree with
Green: To do (typically a reference to another article that I want to read)
(Note: Again, I donât tend to add my own comments / annotations at this stage.)
Bookends reference > DT3 including custom metadata
I then use a lightly modified version of a script I found here to create a DT3 record from a Bookends reference.
DT3 structure
I have a standard DT3 structure for articles and their notes. All are filed under:
/Sources/Articles/
With the structure:
/Sources/Articles/[ArticleName]/
/Sources/Articles/[ArticleName]/ArticleName.pdf
/Sources/Articles/[ArticleName]/Notes/ <-- Highlighted passages from the PDF live here
/Sources/Articles/[ArticleName]/ArticleName.pdf has several pieces of DT3 custom metadata such as:
CiteKey (the Bookends-generated BiBTex compatible citekey)
StartPage (eg 90 if the article runs from pages 90 - 120)
and so on.
I donât use DT3 as a citation manager, but having the citekey in the DT3 record means that if I want to use that record in what I write, I can put the citekey in a footnote directly, without having to go back to the citation manager.
Export highlighted passages
First, set the Highlights app export settings in preferences:
- Color: give the highlight colours appropriate names (for me: Extract, Query, ToDo);
- Customize the annotation header - I use Color Category and Page Number;
- Export - MarkDown and Export as Folder of Files
Open the highlighted PDF in Highlights.app then:
File > Export to > Devon-think
This creates a subfolder in the DT inbox with each highlighted passage as a separate note.
Manually move highlighted passages from subfolder in inbox to:
/Sources/Articles]/[ArticleName]/Notes/
âTidyâ the highlighted passages in â/Notes/â
In DT3, select:
/Sources/[ArticleName]/ArticleName.pdf
Then run this script. The script does the following:
- it takes every note in the /Notes/ subfolder
- gives each note some of the same metadata as the parent PDF, in particular the CiteKey
- gives each note a metadata record type of âExtractâ, âQueryâ or âTo Doâ.
- removes unnecessary information from the note, leaving only the quoted text
- If the record type is âTo Doâ then create an Omnifocus task with a rough and ready attempt at a Google Scholar URL.
The script is not neat and tidy, and has only minimal comments/documentation. Tidying it up is only my list of things to do⌠sometimeâŚ
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
set theTaskList to {}
tell application id "DNtp"
-- the original article / case should be selected
set theOriginalRecord to the selection
-- Get the metadata of the original article
set theMetaData to the custom meta data of item 1 of theOriginalRecord
set theAuthor to mdauthor of theMetaData
set theBookendsURL to mdbookendsurl of theMetaData
set theCiteKey to mdcitekey of theMetaData
set theDOI to mddoi of theMetaData
set theOffset to mdoffset of theMetaData
set theStartPage to mdstartpage of theMetaData
set theTitle to mdtitle of theMetaData
set theYear to mdyear of theMetaData
set theDTPReference to the reference URL of item 1 of theOriginalRecord
set thePath to the location of item 1 of theOriginalRecord
set theNotesPath to thePath & "Notes/"
set theNotesGroup to get record at theNotesPath
set theNotes to the children of theNotesGroup
set theStartPage to theStartPage - 1
repeat with theNote in theNotes
set theContent to the rich text of theNote
set theParagraphs to the paragraphs of theContent
set theParagraph to the fifth paragraph of theContent -- 5th para begins with "Extract" "Query" "To Do"
set theType to the characters 1 thru 4 of theParagraph as string
if theType is "Task" then add custom meta data "To Do" for "recordtype" to theNote
if theType is "Note" then add custom meta data "Extract" for "recordtype" to theNote
if theType is "Quer" then add custom meta data "Query" for "recordtype" to theNote
set thePage to paragraph 4 of theContent
set thePage to characters 6 thru -1 of thePage as string
-- Create DTP url to precise page. Need to do this before we mess up the page with the offset etc
set theDTPPageURL to theDTPReference & "?page=" & (thePage - 1)
set theExtract to paragraphs 6 thru -1 of theContent as string
set thePage to thePage + theOffset
set theExtractPage to theStartPage + thePage
--
add custom meta data theAuthor for "author" to theNote
add custom meta data theTitle for "title" to theNote
add custom meta data theYear for "year" to theNote
add custom meta data theExtractPage for "startpage" to theNote
add custom meta data theOffset for "offset" to 0
add custom meta data theCiteKey for "citekey" to theNote
add custom meta data theBookendsURL for "bookendsurl" to theNote
add custom meta data theDOI for "doi" to theNote
add custom meta data theDTPPageURL for "devonthinkurl" to theNote
if theType is "Note" then
set theRevisedNote to return & return & "Extract: " & return & return & theExtract & return & return & "Comment:" & return & return
else if theType is "Quer" then
set theRevisedNote to return & return & "Extract: " & return & return & theExtract & return & return & "Query:" & return & return
else if theType is "Task" then
set theRevisedNote to return & return & "To Do: " & return & return & theExtract & return & return
set theTaskList to theTaskList & theExtract
end if
set the plain text of theNote to theRevisedNote
set theEnd to the length of theExtract
if theEnd is greater than 100 then set theEnd to 100
set theNewName to characters 1 thru theEnd of theExtract as string
set theNewName to theCiteKey & " p." & theExtractPage & ": " & theNewName
set the name of theNote to theNewName
end repeat
end tell
tell application "OmniFocus"
repeat with theTask in theTaskList
set theQuery to "https://scholar.google.co.uk/scholar?hl=en&as_sdt=0%2C5&q=" & theTask
set the clipboard to theQuery
tell application "TextSoap"
cleanClipboard with "Clean text for use in websearch"
end tell
set theQuery to the clipboard
set the clipboard to theTask
tell application "TextSoap"
cleanClipboard with "Extract URLs by replacing"
end tell
set theURL to the clipboard
set theQuery to theQuery & return & theURL
set theTask to "Find: " & theTask
tell its document "OmniFocus"
tell its flattened project "[PROJECT NAME]"
tell its task "Articles to find and review"
make new task with properties {name:theTask, note:theQuery}
end tell
end tell
end tell
end repeat
end tell
Classify the notes
I then review the notes and replicate them into a separate folder tree of âIssuesâ.
Hope the above helps and happy to answer any questions.
And Happy New Year to all.
K.
3 Likes
wow. that is abslutely excellent and inspiring. will need a bit of time to work through that. cheers and happy new year!
It looks more intimidating than it is, really ! And once itâs set up, actually using it day to day is very simple (which was important to me).
Happy to discuss and help - as best I can - tweak this to suit your needs.
How to stitch notes together? Context: while reading an article I add sentences to dt3, how to make #2 then #3 add to the same note? Notes app does it.
Thanks for pointing this out. Never noticed this option.
Another option is to use the Take Rich Note service first and then the Append Rich Note service to append notes to the last taken note.
Iâm not able to find the " Append Rich Note service". Could you share the path or instructions?
Thanks in advance
Is the service enabled in System Preferences > Keyboard > Shortcuts > Services?
Yes it was enabled already. But I donât see it in the dropdown after selecting 2 Rich Text notes. Merge is offered.
The services are available in the Services menu of other applications and not meant for merging already existing notes but to take new ones & append to them.
Ok. Got it. Thanks again.