How to run Javascript on Markdown "summarize highlights" output?

I’m hoping for some help here with scripting the output of the “Summarize Highlights” feature with Markdown output. I would like to rearrange the text somewhat. I know a little javascript from working with Drafts and have put together the following script which achieves what I’m looking for in terms of text replacement:

let d = draft.content;
let regex = /(^## )(.*)\n(^\* )(.*)/gm;
let newDraft = d.replace(regex,'$4 ($2)');
draft.content = newDraft;
draft.update();

I’m wondering if anyone can help me port that over to DevonThink. I know that basically lines 2 & 3 are the key lines, but I don’t know several key things:

  • how do I grab the text from the DevonThink markdown file?
  • how do I then overwrite that text with this regex replacement and save?
  • how do I make this run automatically when creating a markdown highlights file?

As you can tell, I’ve not ever created a javascript file for the Mac, only so far working with web stuff or Drafts. I’ve got some experience editing applescripts, but I really don’t know that system well–I’ve more knowledge of javascript in general, just not how to use it for Mac scripting. Any help here or pointers to the right resources would be much appreciated!

How about an example of what you have in the document and what you’re expecting to change it to?

And why would this need to be run automatically when making a Summary document?

Same reason as expressed in this thread–I want to change what the output of summarize highlights looks like.

initial output:

# [ahmed interviews butlet.pdf](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F)

## [Page 1](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F?page=0)
* SA: I have been asked to interview you about Gender Trouble and how this book has shaped the field of Queer Studies. In the preface to the second edition of Gender Trouble, you write that ‘the life of the text has exceeded my intentions, and this is surely in part because of the changing context of its reception’. I really like this description. I like the idea that texts have lives other than the ones we give them as writers, and that these lives are partly about how texts are ‘picked up’.

## [Page 2](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F?page=1)
* Could you reflect a little on this textual moment, and also about the extent to which you have drawn on your experience in generating critical insights and why you would or would not make that explicit in the text?

## [Page 4](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F?page=3)
* There is, of course, vulnerability in taking the risk, knowing one will be called a name, or worse, but there is also a vulnerability in the name of which one acts, and which informs one’s very acting. After all, responding with indignation to injury means that one has been affected by that injury, and that being affected is not a fully passive condition. Vulnerability can be the condition of responsiveness – something affects us, or we find ourselves affected; we are moved to speak, to accept the terms by which we are addressed, or to refuse them, or, indeed, to skew them or queer them in direc- tions that, yes, as you have shown, deviate from what seem to be their original aims. Indeed, I would suggest that there are two dimensions of gender performa- tivity from the start: one is the unchosen or unwilled situation of gender assign- ment, one that we can come to deliberate about and change in time; the other is performative action that takes up the terms by which we have been addressed (and so a retaking, a taking over, or a refusal), the categories through which we have been formed, in order to begin the process of self-formation within and against its terms.


desired output:

# [ahmed interviews butler.pdf](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F)

SA: I have been asked to interview you about Gender Trouble and how this book has shaped the field of Queer Studies. In the preface to the second edition of Gender Trouble, you write that ‘the life of the text has exceeded my intentions, and this is surely in part because of the changing context of its reception’. I really like this description. I like the idea that texts have lives other than the ones we give them as writers, and that these lives are partly about how texts are ‘picked up’. ([Page 1](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F?page=0))

Could you reflect a little on this textual moment, and also about the extent to which you have drawn on your experience in generating critical insights and why you would or would not make that explicit in the text? ([Page 2](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F?page=1))

There is, of course, vulnerability in taking the risk, knowing one will-be called a name, or worse, but there is also a vulnerability in the name of which one acts, and which informs one’s very acting. After all, responding with indignation to injury means that one has been affected by that injury, and that being affected is not a fully passive condition. Vulnerability can be the condition of responsiveness – something affects us, or we find ourselves affected; we are moved to speak, to accept the terms by which we are addressed, or to refuse them, or, indeed, to skew them or queer them in directions that, yes, as you have shown, deviate from what seem to be their original aims. Indeed, I would suggest that there are two dimensions of gender performativity from the start: one is the unchosen or unwilled situation of gender assignment, one that we can come to deliberate about and change in time; the other is performative action that takes up the terms by which we have been addressed (and so a retaking, a taking over, or a refusal), the categories through which we have been formed, in order to begin the process of self-formation within and against its terms. ([Page 4](x-devonthink-item://C102E4EF-878E-4CCB-A158-277C466EB18F?page=3))

Like this… ?

exactly

Since the source is very well-formed, this is pretty trivial in pure AppleScript (and @cgrunenberg likes pure AppleScript solutions for their ease of understanding :slight_smile: ). Here is a teaching edition script that makes those modifications to the output of our Summarize Highlights command…

tell application id "DNtp"
	repeat with thisRecord in (selection as list)
		-- Loop through selected documents
		
		if (type of thisRecord as string) = "markdown" then
			-- Only process Markdown docs
			
			set newText to {}
			-- Set up an empty list to push the paragraphs into)
			
			set currentText to (plain text of thisRecord)
			-- Get the current text of the document
			
			repeat with thisLine in (paragraphs of currentText)
				-- Loop through the document, paragraph by paragraph
				
				if thisLine begins with "##" then
					-- If this is an H2 page link
					set cachedText to (characters 4 thru -1 of thisLine as string) -- Cache all but the control characters in a variable
					
				else if thisLine begins with "* " then
					-- If it's just a bulleted paragraph 
					
					copy ((characters 3 thru -1 of thisLine as string) & " (" & (cachedText) & ")  " & return) to end of newText
					-- Get all but the control characters and concatenate the cachedText.
					-- Then push it to the newText list
				else
					copy thisLine & return to end of newText
					-- Anything else, including the H1 line at the top, just push into the newText list
				end if
			end repeat
			
			set plain text of thisRecord to (newText as string)
			-- Set the text of the document to the new text
			
		end if
	end repeat
end tell
1 Like

Thanks so much, this is great.

Also: I picked up the idea somewhere that with the new rules in DT3, one could fire a script upon choosing “Summarize Highlights”, meaning that anytime I summarize highlights to markdown, the script you shared would run and replace the text in those ways. Is that correct? If so, how can I set this up?

No, there is no On Summarize event trigger in smart rules.

It is possible to target a location (and I always suggest being more specific than general in smart rules) and use the On Creation trigger.

  • Targeting a specific group here, though it could be a broader group, an Inbox, or a database. (Again, I shoot for safety first.)
  • Looking only for Markdown docs ending with Summary (again, specific).
  • On Creation run the script.
  • The sound is optional. I like it for debugging as I know it triggered.
  • You may want to add another action to change the files so they’re not continuously matched by the smart rule. this can be a move out of the targeted group, a change of name, etc.

Modify Markdown Summaries.dtSmartRule.zip (2.5 KB)

3 Likes

Great. Thanks for this mini tutorial!

You’re welcome. I hope it learns you something :wink: :stuck_out_tongue:

That’s true :slight_smile:

1 Like

Very much agree on avoiding Regex where possible ( not quite write-only, but getting there, and they add a second layer of complexity ).

But the ‘readability’ of AppleScript is not a function of its inherent properties – it just expresses how much time we’ve personally spent with it. (I can remember finding it very opaque, and hard to write, for quite a while, despite a 20 year background at the time, in languages like C, Pascal, Prolog and Lisp).

We probably shouldn’t kid ourselves, or ask others to drink the ‘English-like’ Kool-Aid, on that one :slight_smile:

Now that AppleScript’s development is sunsetted, and iOS is a bigger platform than macOS, perhaps it’s getting rather late in the day to push users back from JavaScript towards AppleScript ?

(I understand the impulse, we are all attached to the tools which happen to be familiar to us, but that doesn’t automatically make them the best choice – or the most responsible recommendation – for beginners).

1 Like

(I understand the impulse, we are all attached to the tools which happen to be familiar to us, but that doesn’t automatically make them the best choice – or the most responsible recommendation – for beginners).

JavaScript is familiar to you but terse and inexplicable to most laypeople.
And I would not advocate people move into JXA as a beginner, especially since it’s very poorly documented and makes a ton of assumptions about a person’s prior knowledge.

There are quite a few people who have taken a shine to AppleScript, just from posts on these forums, our built-in Help, and the scripts we’ve included with DEVONthink. And the fact that Script Debugger continues to sell and flourish speaks to the general health and interest in AppleScript as a primary automation language for Macs.

Interestingly, you are another example of people coming from higher-level languages who said they struggled with AppleScript’s syntax.

I, having no CS degree, taught myself out of necessity in the graphic arts/printing field, then later professionally AppleScripting in the corporate world (no, not a typo or exaggeration) and have always found AppleScript easy to grasp, easy to teach, and charming (despite it’s occasional idiosyncracies).

1 Like

I disagree (and incidentally, neither of us has any formal training in CS), I was very familiar with AppleScript long before I knew any JS at all.

I now find it:

  • Faster and easier to get things working in JS than in AppleScript,
  • much easier than AppleScript to get help with.

“very poorly documented”

Simply not true.

“it …makes a ton of assumptions about a person’s prior knowledge”

Not clear what the “it” refers to in that sentence … JS ? sorry – not clear what you have in mind …

people coming from higher-level languages who said they struggled with AppleScript’s syntax

I came from both lower level languages and higher level languages, and still found AppleScript precious and opaque – in particular, much harder to write than to read.

have “always found” AppleScript … XYZ

I think that’s the problem – long exposure to a language is very easily confused with the properties of the language itself. English speakers think English is easy and charming. Adult learners of it disagree.

I now find AppleScript perfectly legible and writable (I’ve contributed a fair amount of open source AppleScript in the past), but I don’t kid myself that it’s AppleScript itself which “is” easy. Like anything, it can become easier. (If you invest a very significant amount of time).

Meanwhile, even on macOS:

  • The Omni Applications have moved scripting development decisively away from AppleScript to JavaScript. See the discussion in their 2020 Road Map.

    • In particular where they point out that:

… AppleScript had some big caveats: it was generally easy to read, but it was a fairly esoteric language that many developers found less easy to write. It was able to accomplish some amazing things, much faster and more accurately than working by hand—but it was generally slow enough that you could watch it work, and if it had a lot of work to do you could be watching and waiting for a while. And it was only available for the Mac platform—which was totally fine in the ’00s when that was the only place our apps ran, but was a lot less useful in the ’10s when our customers were increasingly spending their time on mobile devices.

Perhaps we need to think carefully before telling customers that AppleScript “is” easy, as if learning to use it confidently and productively (without wasting hours of frustration) doesn’t need to represent a major investment of time.

The reality is that all languages require time to learn, and we have no evidence at all to show that JavaScript would require a different amount of time from AppleScript.

What we do have evidence of, however, is that JS is bound to be a better investment of that time.

  • AppleScript has a small and shrinking pool of applications, and no prospect of further development.
  • iOS is JavaScript-only (there can and will never be any AppleScript on it)
  • macOS has no AppleScript-only applications, but it does have JavaScript-only applications.
  • OmniGroup is moving away from AppleScript development to JS scripting development.
  • Web pages run on JavaScript – if you’re only going to learn one language, JS is the language to learn, and it’s the one with the richest range of learning materials.

I do understand the reflex of defending a technology that one has personally invested time in, and the vertiginous avoidance of stepping into a new one – I felt it myself at the transition from familiar use of AppleScript to first steps in JavaScript (which I originally made simply in order to be able to write scripts for TaskPaper).

But I think it’s only responsible to allow oneself to pause and reflect that the picture is now very different for beginners.

Compared to other languages it definitely is - just try to learn Czech :smiley:

1 Like

( Tho I understand CZ was a bit divided on that one … )

Having spent the best part of ten years teaching English to adult learners in another country, I have my doubts about that proposition (though I concede that Czech is not easy – I have friends who live there). The fact that huge numbers of native English speakers appear to be incapable of speaking or writing the language correctly only adds weight to the idea that it is not an easy language. But I have taken us off topic …

I’m tempted to dig into the 800+ pages of Sal Soghoian’s book on Apple Script, but I’m guessing someone in this thread could tweak BLUEFROG’s script for me so that I do something very similar with my Kindle highlights. I’d like to turn this…

Yellow highlight | Location: 1,121
For a time, we had come to believe that civilization moved in the other direction—making the impossible first possible and then stable and routine. With climate change, we are moving instead toward nature, and chaos, into a new realm unbounded by the analogy of any human experience.

Yellow highlight | Location: 1,131
On local golf courses, the West Coast’s wealthy still showed up for their tee times, swinging their clubs just yards from blazing fires in photographs that could not have been more perfectly staged to skewer the country’s indifferent plutocracy. The following year, Americans watched the Kardashians evacuate via Instagram stories, then read about the private firefighting forces they employed, the rest of the state reliant on conscripted convicts earning as little as a dollar a day.

…into this…

For a time, we had come to believe that civilization moved in the other direction—making the impossible first possible and then stable and routine. With climate change, we are moving instead toward nature, and chaos, into a new realm unbounded by the analogy of any human experience. (Location: 1,121)

On local golf courses, the West Coast’s wealthy still showed up for their tee times, swinging their clubs just yards from blazing fires in photographs that could not have been more perfectly staged to skewer the country’s indifferent plutocracy. The following year, Americans watched the Kardashians evacuate via Instagram stories, then read about the private firefighting forces they employed, the rest of the state reliant on conscripted convicts earning as little as a dollar a day. (Location: 1,131)

In other words, I’d like to
1. get rid of “Yellow highlight |” at the top of each passage,
2. create parentheses at the end of each passage,
3. relocate “Location: xxxx” from the top of each passage into the parentheses at the end of the passage.

I don’t need this to happen within DEVONthink, though no problem if that’s what you think is easiest. I was thinking I could copy and paste my highlights into a regular text file and then run the script on each file individually. Oh, and I have Keyboard Maestro, so if you think that’d be even better to use, please oh please send some instructions my way.

Thank you in advance for whatever help you can provide!

@BLUEFROG By chance, do you have a nice learning example for formatting RTF files?

I’m looking for an easy way to remove the indents from the Summarize Highlights RTF output (i.e., so that all the text is left-aligned and indent free). When compared to your markdown examples above, it would look more like the version on the right-hand side. However, I don’t really care if the links are moved to parentheses after the quote; they can stay above the text, like the default output, or move - either is fine. I just don’t like the indents because they’re more difficult to fix manually afterward.

Thanks for your help!

1 Like

Hey Jim, @BLUEFROG, Could you be so kind as to modify this code so that:

Instead of Page, it writes the name of the file in the paranthesis in the following format(semicolon)?

Right Now:

How I would love it to be:

1 Like