How to export the results list for the "See also & Classify" command

It’s fragile but sometimes it’s the only possible way to script.

And in these cases, often it’s best to not script it at all :wink:

Hey! It does work reliably :sunglasses: Someone wrote

and my personality says if something can be scripted then script it :stuck_out_tongue:

You don’t want @ulbrich to

  • select the see also records,
  • copy via menu,
  • paste the item links into a rich text document,
  • convert it to plain text document,
  • copy the converted names,

do you? :thinking:

At least the last 3 steps should be scriptable without UI scripting. The main disadvantage of UI scripting is IMHO that different preferences, a different window setup (e.g. no visible sidebar) or different localizations or minor updates of the app can easily break them.

My wish is create a list that I can share with others, as a suggestion of reading. Both way should be good, but I not an expert and, I really want to do that with 2.000 files.
Is there a way to select some files and create a subroutine to do that?

E.g.:

text01.md
text02.md
text03.md

output >>

text01-seealso.md
text02-seealso.md
text03-seealso.md

e.g. of text01.md

e.g. of text01-seealso.md

Captura de Tela 2020-10-06 às 06.48.52

Final result

Captura de Tela 2020-10-06 às 06.50.09

Do the others use DEVONthink and the same database? In that case the item links might be more useful. Or do the other users need access to the actual files?

No, I will just copy and past this list in a sample e-mail.

This is an example the way I do it.

  • I took an image os results
  • Then I wrote the titles (as @cgrunenberg suggest)
  • Then I wrote the first paragraph (to have an ideia about the context of this text)
  • Finally I wrote the last paragraph (that refers to biography)

I’m pretty sure your question how to get the names from the inspector and initially not mentioning that you’re dealing with a larger number of records led everybody in this thread in the wrong direction.

Luckily you’ve mentioned it now and luckily @cgrunenberg explained to me the differences between some AppleScript commands and what I saw in the See Also inspector today.

Maybe I’m missing something obvious, which is well possible, but I think I’ve found the way I would go if I wanted to share See Also results. In fact, I’ll use them in a similar way as your idea of adding the first and last paragraph to a result’s name seems is very nice. But it’s definitely not nice to do that manually.

First off, what you want can be done with AppleScript, I think. There are two commands that could do it, which one is the right depends on your setup.

DEVONthink can compare via

  • See Also inspector, which takes all available data into account and compares a record to all databases.

  • AppleScript command compare record, which compares records in a similar way as it is done in the See Also inspector. But it does not compare a record to all databases.

  • AppleScript command compare content, which can be used to compare to all databases, but it “only” compares text content.

I’m perfectly fine with the latter as for me it’s more important to compare to all databases.

See this diff, on the left See Also results, on the right compare content results

Back to

and

If it’s not important for you to share exactly the results of the See Also inspector then you could do everything with the script below, it

  • compares text content to all databases
  • sorts by score
  • cuts the results count at the value you set
  • optionally fetches first and last paragraph
  • writes the result names in a markdown record

From your captures it seems that you’re mainly using markdown which should work fine. I did however not had the time to longer test it with PDFs. From short tests it seems it doesn’t make much sense to “blindly” fetch PDF paragraphs. So I suggest, if you test it, to set property maxCharacterCount so low that it won’t exclude e.g. your markdown records but most PDFs. Links to them are always created, only their paragraphs would in the output markdown be replaced by a placeholder. Anyway there are comments.

Before I write on and on, let’s do it the other way round. Give it a try and afterwards I’ll tell you some things that possibly should be considered if you want to use it

-- Create Markdown See Also Suggestion List 

use AppleScript version "2.4"
use framework "Foundation"
use scripting additions

property maxSeeAlsoResults : 20
property removeSuffix : true -- remove suffix of suggested records
property includeParagraphs : true -- fetch the "first" and "last" paragraph. which paragraphs should act as "first" and "last" can be set below 
property skipMarkdownHeading : true -- skip the first non-blank line if it begins with # (independent of property firstParagraphOffset)

property firstParagraphOffset : 1 -- index of the paragraph which should act as the first. starting at 1. only lines which contain at least one word are counted in (independent of property addEmptyLines)
property lastParagraphOffset : 1 -- index of the paragraph which should act as the last. starting at 1. second last would be 2 (and so on)
property theParagraphPlaceholder : "<span style=\"color: red\">Here's something to do!</span>" -- this is used if a paragraph couldn't be fetched, e.g. when property firstParagraphOffset > count of paragraphs. change to your liking
property maxCharacterCount : 100000 -- skip fetching paragraphs of records whose character count is higher. for skipped paragraphs the placeholder is used. add these paragraphs manually afterwards by clicking the suggestion's name

property theOutputGroupUUID : "73964EA3-146B-4CA0-8C6A-456F21CD2C33" -- paste your output group's item link and remove "x-devonthink-item://"
property removeSelectedRecordSuffix : true -- remove suffix of selected record's name for usage in output record's name and heading
property openOutputRecord : true -- only available if one record is selected
property excludeFromSeeAlso : true -- exclude output record from see also
property excludeFromSearch : false -- exclude output record from search

tell application id "DNtp"
	try
		set theRecords to selection of think window 1
		if theRecords = {} then error "Please select some records"
		set theGroup to (get record with uuid theOutputGroupUUID)
		set theDatabases to databases
		
		show progress indicator "Creating See Also Suggestions List... " steps (count theRecords) as string with cancel button
		
		repeat with thisRecord in theRecords
			set thisRecordName to name of thisRecord
			step progress indicator thisRecordName
			set theContent to plain text of thisRecord
			set theSeeAlsoResults_record to {}
			
			repeat with thisDatabase in theDatabases
				set theSeeAlsoResults to compare content theContent to thisDatabase
				repeat with thisResult in theSeeAlsoResults
					set theName to name of thisResult as string
					if removeSuffix = true then set theName to my recordName(theName, filename of thisResult)
					set theName to ("[" & theName & "](" & (reference URL of thisResult) & ")") as string
					if includeParagraphs = true then
						set theCharacterCount to (character count of thisResult)
						if theCharacterCount > 0 and theCharacterCount ≤ maxCharacterCount then
							set end of theSeeAlsoResults_record to {name_:theName, score_:(score of thisResult), charactercount_:theCharacterCount, text_:(plain text of thisResult)}
						else
							set end of theSeeAlsoResults_record to {name_:theName, score_:(score of thisResult), charactercount_:theCharacterCount}
						end if
					else
						set end of theSeeAlsoResults_record to {name_:theName, score_:(score of thisResult), charactercount_:0}
					end if
				end repeat
			end repeat
			
			set theSeeAlsoResults_record_sorted to my sort(theSeeAlsoResults_record)
			if (count theSeeAlsoResults_record_sorted) > maxSeeAlsoResults then set theSeeAlsoResults_record_sorted to (items 1 thru maxSeeAlsoResults in theSeeAlsoResults_record_sorted)
			set theMarkdownText_list to {"<style> a {text-decoration: none;} </style>" & linefeed}
			
			repeat with thisResult in theSeeAlsoResults_record_sorted
				if includeParagraphs = true then
					set theCharacterCount to charactercount_ of thisResult
					if theCharacterCount > 0 and theCharacterCount ≤ maxCharacterCount then
						set theText to text_ of thisResult
						set theParagraphs to paragraphs of theText
						set theParagraphs_cleaned to {}
						repeat with thisParagraph in theParagraphs
							if words in thisParagraph as string > 0 then set end of theParagraphs_cleaned to thisParagraph as string
						end repeat
						if skipMarkdownHeading = true and (item 1 of theParagraphs_cleaned starts with "#") then set theParagraphs_cleaned to (items 2 thru -1 in theParagraphs_cleaned)
						try
							set firstParagraph to item firstParagraphOffset of theParagraphs_cleaned
						on error
							set firstParagraph to theParagraphPlaceholder
						end try
						try
							set lastParagraph to item -lastParagraphOffset of theParagraphs_cleaned
						on error
							set lastParagraph to theParagraphPlaceholder
						end try
						set end of theMarkdownText_list to "* " & name_ of thisResult & space & space & linefeed & firstParagraph & space & space & linefeed & lastParagraph & space & space & linefeed
					else if theCharacterCount > 0 and theCharacterCount > maxCharacterCount then
						set end of theMarkdownText_list to "* " & name_ of thisResult & space & space & linefeed & theParagraphPlaceholder & space & space & linefeed & theParagraphPlaceholder & space & space & linefeed
					end if
				else
					set end of theMarkdownText_list to "* " & name_ of thisResult & space & space & linefeed
				end if
			end repeat
			
			set theMarkdownText to my tid(theMarkdownText_list, linefeed)
			if removeSelectedRecordSuffix = true then set thisRecordName to my recordName(thisRecordName, filename of thisRecord)
			set theSeeAlsoListRecord to create record with {name:"See Also Suggestion List - " & "\"" & thisRecordName & "\"", type:markdown, plain text:("### See Also Suggestion List " & "[" & thisRecordName & "](" & (reference URL of thisRecord) as string) & ")" & linefeed & linefeed & theMarkdownText, exclude from see also:excludeFromSeeAlso, exclude from search:excludeFromSearch} in theGroup
			
		end repeat
		hide progress indicator
		
		if openOutputRecord = true and (count theRecords) = 1 then
			open window for record theSeeAlsoListRecord
			activate
		end if
		
	on error error_message number error_number
		hide progress indicator
		activate
		if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
		return
	end try
end tell

on recordName(theName, theFilename)
	set theSuffix to my getSuffix(theFilename)
	if theName ends with theSuffix and theName ≠ theSuffix then set theName to characters 1 thru -((length of theSuffix) + 2) in theName as string
	return theName
end recordName

on getSuffix(thePath)
	set revPath to reverse of characters in thePath as string
	set theSuffix to reverse of characters 1 thru ((offset of "." in revPath) - 1) in revPath as string
end getSuffix

on sort(theList)
	set anArray to current application's NSArray's arrayWithArray:theList
	set theDesc to current application's NSSortDescriptor's sortDescriptorWithKey:"score_" ascending:false selector:"compare:"
	set newList to (anArray's sortedArrayUsingDescriptors:{theDesc}) as list
end sort

on tid(theList, theDelimiter)
	set d to AppleScript's text item delimiters
	set AppleScript's text item delimiters to theDelimiter
	set theString to theList as text
	set AppleScript's text item delimiters to d
	return theString
end tid

@ulbrich If including first and last paragraph automatically doesn’t make sense with the current script because not all your your markdown records have the same structure I could add a list of text that will be skipped. In any case you could use it to get a list of the names and add the paragraphs manually which still makes the whole thing easier than your current way :slight_smile:

@cgrunenberg, do you think should be possible to do the following?

  • select a MD file
  • apply see&also command
  • Copy Item Link of these results
  • Then, for each item link
    a) copy the first paragraph
    b) copy the last paragraph

So if I select a file text1.md and I want the See&Also results of that, I wish a script that creates a new file calls text1-seealso.md, that contents the following text:

For text1.md the final results (in the actual database) of see also are:

text10.md
First paragraph of text10.md
Last paragraph of text10.md

text31.md
First paragraph of text31.md
Last paragraph of text31.md


As a final wish I would like to repeat it for all 2.000 MD files that I have in a select group.

  • Then, for each item link
    a) copy the first paragraph
    b) copy the last paragraph

What if the See Also results are not Markdown files?

@ulbrich The script I posted yesterday will do almost exactly what you want.

Have you tried it?

Does it not work for you?

My suggestion is that:

If all results is a markdown file, then create a text01-seealso.md that contents:
a) list of file name results
b) include first paragraph
c) include last paragraph

If the results show different kind of files, then create a text91-seealso.md with:
a) just a list with the file name results

@bluefrog I think that it is important to keep in mind what you want to do with these results.

I will give you an example. I really want to study in deep these 2000 files. These are articles familiar to people with the same background that I (in a community, that appreciates to discuss, religion, politic, literature, etc).

BUT, not all of then would like to read everything. Maybe you find a person that suggest to you: read this text, it is pretty good (e.g. Pacience in study). I have already this text, and I want to provide I quick list a SEE&ALSO results that indicates some topics that are connect with my friend suggestion.

That I can say: Hi my friend, it is very interesting, let’s read this subject together? This is a list that should be nice to have a look.

Then, both of us, can check an introduction of each text and a quick view of books that they are from.

We are talking about connection between people, interests, preferences, ideas and discussions, etc.

This is what makes me find DevonThink!

I would have to add 6 lines to this script

to do that

1 Like

@pete31 to be honest, I don’t have enough background to use scripts (not yet).

I really want to try, but I do not know how to give the first step.

If you help me, with a step-by-step guide, I really want try it.

All thinks that I know from MAC applications to optimise my database I learn here, with these group. Specially with support of @BLUEFROG, @cgrunenberg, @ngan and @Bernardo_V

I usually work with database that contents 2.000 to 20.000 files.

Topics that I really explore in deep:

For work:

  • Patents of 3D Printers (first one until 2018 - more that 20.000 files)
  • Market’s studies of 3D Printers (5.000 files)
  • Market’s country studies (2.000 files)

For hobby:

  • New Testament studies (2.000 files)
  • Storytelling for kids (2.000 files)
  • Gamification (1.000 files)

I started with a book of Steven Johnson. I have been gathering this material for more than 5 years and only now can I have good results, but without the use of automation any attempt is impracticable.

No problem. To do a first test run of the script you just need to

  • copy and paste the script in Script Editor.app
  • in DEVONthink select the group in which you want to create the markdown records
  • in DEVONthink select menu Edit > Copy Item Link
  • in Script Editor.app paste the link in property theOutputGroupUUID
  • remove “x-devonthink-item://” from the pasted link

Select some records and run the script.

This will give you an idea of what it does.

Of course. Read the first part of the script where the property variables are defined and my comments.

Then post the names of all property variables where you don’t understand what they do and I’ll explain them.

1 Like

It works perfect!

There are some questions that I have, but I will try a little bit more, BEFORE I ask.

I will keep in mind what @BLUEFROG said, so I will do it carefully

I used the first script, because the second show me a message error.

Ask whenever you want. If you ask now I could change it and you could directly try the changed script :slight_smile:

The second was just a quote of the real script.

1 Like

Ok, first of all, I think some “register” need to be reset before, run it.

If you see in my picture, the original name is “Em que perseveras”

But, in the first line of the new document shows:

See Also Suggestion list “Compromisso pessoal” >> It is another file

Second, should be nice, to create a new group and save these results there.

I think should be easy to localize.

E.g.

Original group: V5AA

See also results: V5AA\seealsoresults\

Hmm, don’t know what happened there. I’ll check.

I see in your picture that you’ve replicated the records in this group.

Because records are replicated I’m afraid it’s not possible to make sure that the script always “finds” the right group (e.g. group “V5AA”) to create the subgroup in as there’s no way to decide which parent group is the right one.

If you always use the script in a window in which you’ve selected the group in the sidebar then it’s possible to use the window’s root group instead of a selected record’s parent group. This way it would be possible to make sure to always create a subgroup in the right group. But you would have to make sure to always use it in a window like the one in picture …

I think it’s easier to use if you manually create one group which could then be used by the script to create subgroups in. You could then move or replicate those subgroups manually.

What do you think?

1 Like