Indexing help (or: macOS help viewer vs. DEVONthink)

[pete31]

@Hens

then it started to download

No, it indexed the files. You can see that on the small Finder symbol to the right of the record names. And that’s what

@BLUEFROG

indicate

does …

@Hens

I just wanted to see what would happen if I run this script lol (yes I know)

Why? Why oh why?

You have obviously no idea what the script is supposed to do, so why do you run it?

@Hens

I don’t see a group in Global inbox

Bluefrog made a little mistake and mixed up incoming group and inbox. So the group is in the database that was your current database before you quit DEVONthink. Because you did a force quit you’ll probably have to open it manually. The group is in this database’s root.

@Hens

Can I delete these “recently added” files?

NO!

Please wait until @bluefrog or @cgrunenberg tell you how to safely remove them.

[Hens]

I thought this was ascript to duplicate DevonThink 's Help’s menu to the Global inbox. well it freaked me out becasue I have never seen this red line under the DevonThink’s icon, thats why I stopped it.

Now I see where the group file is, the database I was in, there is 430 files there, obv Help’s partial files.

Thanks, we will see what he says.

[BLUEFROG]

You ran what script, mine or Pete’s?

[Hens]

yours lol

[BLUEFROG]

If you used mine - bearing in mind I issued more than one warning in this thread - you can delete the DEVONthink 3 Help group and empty the Trash. You should get the appropriate dialog about deleting indexed items and you’d obviously choose Only in Database.

[pete31]

Index DEVONthink help

This script indexes DEVONthink’s help.

It first duplicates the internal help to a new Finder folder, then indexes the duplicated help.

If you don’t like the indexed help anymore you can delete it without any risk.

How to index:

• Run the script from Script Editor.app.

• After 30 seconds you’ll find a new group in your global inbox.

• You can move this group or rename it and/or the records in DEVONthink (not in Finder).

• After a DEVONthink update delete the dated indexed help and run the script again.

How to delete:

• Select the indexed group

• Select menu Data > Show in Finder (but do nothing)

• Switch back to DEVONthink

• Move the indexed group to trash

• Empty the trash

• Switch back to Finder

• Move the folder to trash (and confirm the dialog)

That’s it.

Thanks Jim, you’ve built a great resource!

-- Index DEVONthink help

-- This script first duplicates DEVONthink's internal help to a new Finder folder. You can delete the indexed group (e.g. after an update) without any risk.
-- Run the script from Script Editor.app. After 30 seconds you'll find a new group in your global inbox.
-- You can move this group or rename it and/or the records in DEVONthink (not in the Finder folder).
-- How to delete: https://discourse.devontechnologies.com/t/indexing-help-or-macos-help-viewer-vs-devonthink/46475/20

set theSuperFolder_Path to POSIX path of (path to application support from user domain) & "DEVONthink 3/DEVONthink Help (indexed)/"
set theResources_Path_HFS to ((path to of application id "DNtp") & "Contents:Resources:DEVONthink.help:Contents:Resources:") as string
set createSuperFolder to do shell script "mkdir -p " & quoted form of theSuperFolder_Path

tell application "Finder"
	try
		try
			set theSuperFolder to folder (POSIX file theSuperFolder_Path as alias)
		on error
			error theSuperFolder_Path
		end try
		try
			set theResourcesFolder to folder theResources_Path_HFS as alias
		on error
			error theResources_Path_HFS
		end try
		
	on error error_message number error_number
		activate
		if the error_number is not -128 then display alert "Finder" message "Can't get folder: \"" & error_message & "\"" as warning
		return
	end try
end tell

tell application id "DNtp"
	try
		set theHelpFolder_Name to "DEVONthink Help - " & version as string
		
	on error error_message number error_number
		if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
		return
	end try
end tell

tell application "Finder"
	try
		try
			set theHelpFolder to make new folder at theSuperFolder with properties {name:theHelpFolder_Name}
		on error error_message number error_number
			if the error_number is -48 then
				error "Can't make folder. Folder already exists: \"" & theSuperFolder_Path & theHelpFolder_Name & "\""
			else
				error error_message number error_number
			end if
		end try
		
		duplicate folder theResources_Path_HFS to theHelpFolder
		set locked of entire contents of theHelpFolder to true
		set locked of theHelpFolder to true
		
	on error error_message number error_number
		activate
		if the error_number is not -128 then display alert "Finder" message error_message as warning
		return
	end try
end tell

tell application id "DNtp"
	try
		set theDatabase to inbox
		set theHelpGroup to indicate (theHelpFolder as alias) to root of theDatabase
		
		set thePath to path of theHelpGroup
		set locking of (contents of theDatabase whose path starts with thePath) to true
		set exclude from search of (contents of theDatabase whose path starts with thePath and type is not html) to true
		set exclude from see also of (contents of theDatabase whose path starts with thePath and type is not html) to true
		my excludeFrom(theHelpGroup)
		
		set theHTMLRecords to (contents of theDatabase whose path starts with thePath and type is html)
		
		repeat with thisHTMLRecord in theHTMLRecords
			set theSource to source of thisHTMLRecord as string
			try
				set thisTitle to (item 1 of (my tid(item 2 of (my tid(theSource, "<title>")), "</title>"))) as string
				if thisTitle ≠ "" then
					if thisTitle = "DEVONthink Documentation" then set thisTitle to "DEVONthink Documentation : " & (item 1 of (my tid(item 3 of (my tid(theSource, "<span class=\"topic\">")), "</span>"))) as string
					if thisTitle contains "&amp;" then set thisTitle to my replace_String(thisTitle, "&amp;", "&")
					if thisTitle ≠ "" then set name of thisHTMLRecord to theHelpFolder_Name & " - " & thisTitle
				end if
			end try
		end repeat
		
		try
			set theSmartGroup to create record with {type:smart group, search predicates:"kind:html", search group:theHelpGroup, name:theHelpFolder_Name} in theHelpGroup
			set theWindow to open window for record theHelpGroup
			set selection of theWindow to {get record at ((location of theHelpGroup as string) & (name of theHelpGroup as string) & "/Resources/" & theHelpFolder_Name & " - DEVONthink Help")}
			activate
		end try
		
	on error error_message number error_number
		activate
		if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
		return
	end try
end tell

on excludeFrom(theGroup)
	tell application id "DNtp"
		try
			set locking of theGroup to true
			set exclude from classification of theGroup to true
			
			set theChildGroups to children of theGroup whose type is group
			repeat with thisChildGroup in theChildGroups
				my excludeFrom(thisChildGroup)
			end repeat
			
		on error error_message number error_number
			error number -128
		end try
	end tell
end excludeFrom

on tid(theText, theDelimiter)
	set d to AppleScript's text item delimiters
	set AppleScript's text item delimiters to theDelimiter
	set TextItems to text items of theText
	set AppleScript's text item delimiters to d
	return TextItems
end tid

on replace_String(theText, oldString, newString)
	local ASTID, theText, oldString, newString, lst
	set ASTID to AppleScript's text item delimiters
	try
		considering case
			set AppleScript's text item delimiters to oldString
			set lst to every text item of theText
			set AppleScript's text item delimiters to newString
			set theText to lst as string
		end considering
		set AppleScript's text item delimiters to ASTID
		return theText
	on error eMsg number eNum
		set AppleScript's text item delimiters to ASTID
		error "Can't replaceString: " & eMsg number eNum
	end try
end replace_String

[Hens]

Thanks, I followed your instructions then put it in the trash but didn’t delete that group after clicking “Only database”

there is a little lock showing in the group.

[BLUEFROG]

Unlock it from the Data > Mark menu.

[Hens]

Thank you

[Hens]

This is much. better. thank you
So every time there is a new DevonThink version, you have to delete it

[pete31]

@Hens

So every time there is a new DevonThink version, you have to delete it

Assuming that you don’t want to look up something in an old version of DEVONthink’s help: yes

[mmynsted]

This is a really great idea. Having the help indexed is nice. I got tired of manually copying it and indexing it though. I found a way that I prefer.

Use the help bundle directly from DEVONthink

There is value in simply using the help bundle directly. To do this easily, the help bundle would need to be altered slightly. I will explain later. (See Why would the help bundle need to be modified?)

Without changing the help bundle we can still call into Apple help using its URL schema, shown here.

Here is how I use it

I added the following markdown links in my DEVONthink database. They work fine on my system, with the default help bundle (the help installed with DEVONthink).

NOTE: The links do not appear to work from the discourse page so to test them you will have to copy them locally.

Link to the DEVEONthink help

DEVONthink Help

 [ DEVONthink Help](help:openbook=DEVONthink%20Help)

The URL looks like this.

 help:openbook=DEVONthink%20Help

The result looks like this.

Link to search DEVONthink help

We will search for the words search prefix in the DEVONthink help.

Search for “search prefix” in DEVONthink Help

 [Search for "search prefix" in DEVONthink Help](help:search='search%20prefix'%20bookID='DEVONthink%20Help')

The URL looks like this.

 help:search='search%20prefix'%20bookID='DEVONthink%20Help'

The result looks like this.

Link directly to the search prefixes page in the appendix

The Search Prefixes appendix page

 [The _Search Prefixes_ appendix page](help:anchor='appendix-searchprefixes'%20bookID='DEVONthink%20Help')

The URL looks like this.

 help:anchor='appendix-searchprefixes'%20bookID='DEVONthink%20Help'

The result looks like this.

Why would the help bundle need to be modified?

The last link is the kind of thing that I find the most valuable. I want to link to a particular page. Often I want to link to a particular part of a page. The way this is done is with anchor links. Anchor links need to be defined, and used, in the help. The best that I can tell, they are not (yet). See the explanation from Apple here.

So if anchors are not defined, how did the last link work? I did a little investigation and made a lucky guess.

The last URL looks like.

help:anchor='appendix-searchprefixes'%20bookID='DEVONthink%20Help'`

If I capture the search prefixes link from the find page of the help I get this URL.

 file:///Applications/DEVONthink%203.app/Contents/Resources/DEVONthink.help/Contents/Resources/pgs/appendix-searchprefixes.html

This is not what I want because it is path dependent. The anchor link does not depend on the path because it uses the help book identifier, registered with macOS, and the page identifier rather than the page file name. DEVONthink’s help registration id, DEVONthink Help, would likely be more stable and static across editions and versions than the path.

To get the URL I opened the page with the link in the macOS help viewer. (Click help in DEVONthink)

Then I right clicked on the link and selected Copy Link.

When I opened the help bundle, with view package contents, I could see the layout of the help. Looking in the sources I did not seen any anchors I could use. Perhaps the index provides anchors for each page? The page I wanted was in the pgs path. That can be seen both by examining the package contents and be deconstructing the file-path link copied above.

 ...Contents/Resources/pgs/appendix-searchprefixes.html

So I just put the of the file as the anchor

 ...anchor='appendix-searchprefixes'

Proposal

In-page anchors

Please add in-page anchors to the DEVONthink help bundle and change the existing link URLs to use them. This way when we copy the links in the help we get anchor links rather than path-dependent links. This also permits us to deep link into particular parts of pages rather than just the page.

[pete31]

@mmynsted

I got tired of manually copying it and indexing it though.

This script indexes DEVONthink’s help and renames the records properly. There’s nothing more to do than running the script. Did you try it?

Linking to help pages is interesting, however it‘s relying on macOS‘s help viewer which in general is bad in searching. I don’t quite get why one would use the same bad app just in another way. This approach also misses the point of using DEVONthink‘s features, e.g. its search capabilities, See Also or replicating.

[mmynsted]

I want one consistent way that I may keep directly linked notes, Zettelkasten style, with any macOS help file. I do not want to maintain copies that I must update after an upgrade. Searchable, indexed, help files would be fantastic.

I need to try your script.

Does the script produce something that works the same as other indexed files in DEVONthink? Do Update Indexed Items and the Repair Database actions work the same for the help file? I wonder if it could use the API to find help files from the registry (perhaps when clicking on an application from the finder)?

[pete31]

@mmynsted

I want one consistent way that I may keep directly linked notes, Zettelkasten style, with any macOS help file. I do not want to maintain copies that I must update after an upgrade.

I see. I’ve indexed duplicates of each important app’s help bundle, however I do not link to them, I just use DEVONthink’s search to quickly find what I’m looking for. I don’t update them with every new app version but read the release notes instead. If there are new features I want to use or existing ones have changed I update the indexed help.

So we have different needs. My script does use a duplicate of DEVONthink’s help bundle and the indexed records need to be deleted after an update, so that’s not what you’re looking for.

@mmynsted

Searchable, indexed, help files would be fantastic.

Do you mean it would be useful to index the original help bundles from inside an app’s package? If so take a look at the script @BLUEFROG posted in this thread. Obviously it needs to be used with care as you would be dealing with original files.

[BLUEFROG]

I do not encourage my initial approach due to the use of the original files. It was really more academic than practical.

@pete31’s approach with duplicated and indexed files is safer and still useful for periods of time as the documentation is only updated after releases.

[mmynsted]

@pete31

Do you mean it would be useful to index the original help bundles from inside an app’s package? If so take a look at the script @BLUEFROG posted in this thread. Obviously it needs to be used with care as you would be dealing with original files.

Yes, exactly. I want to search and link help bundles in their original locations.

[pete31]

@mmynsted

I want to search and link help bundles in their original locations.

I understand it’s far more comfortable to index a help bundle once but I don’t think it’s a good idea. There’s always the risk of user error and then you might and up with a broken app. But of course it’s up to you.

Played around with your idea, especially I looked for anchor links in my indexed help bundles and other apps. Seems not a single app out of over 150 that I tested got them. Are you aware of an app that got anchor links? I’d like to verify the script I used with an app that definitively got anchors.

---

Not sure if it’s useful at all, however while playing around I came up with a script that can search in one app’s help or in help of all apps that are registered with a help bundle. It obviously doesn’t make sense to use the script as it is, however if one puts it in e.g. an Alfred workflow it could be useful to someone.

-- Search in macOS's help viewer

use AppleScript version "2.4"
use framework "Foundation"
use framework "AppKit"
use scripting additions

property searchAllHelpBundles : false

set theQuery to "search prefixes"
set theApp_Name to "DEVONthink 3"

try
	if not searchAllHelpBundles then
		set theApp_Path to POSIX path of (path to application theApp_Name)
		set theBundle to current application's NSBundle's bundleWithPath:theApp_Path
		set theBundle_HelpBookName to (theBundle's objectForInfoDictionaryKey:"CFBundleHelpBookName")
		set theHelpManager to current application's NSHelpManager's sharedHelpManager
		theHelpManager's findString:theQuery inBook:theBundle_HelpBookName -- search one help bundle
	else
		set theHelpManager to current application's NSHelpManager's sharedHelpManager
		theHelpManager's findString:theQuery inBook:(missing value) -- search all help bundles
	end if
on error error_message number error_number
	if the error_number is not -128 then display alert "Oh no" message error_message as warning
	return
end try

[mmynsted]

@pete31

Not sure if it’s useful at all, however while playing around I came up with a script that can search in one app’s help or in help of all apps that are registered with a help bundle.

Wow, thank you! I will check it out.

@pete31

Played around with your idea, especially I looked for anchor links in my indexed help bundles and other apps. Seems not a single app out of over 150 that I tested got them. Are you aware of an app that got anchor links? I’d like to verify the script I used with an app that definitively got anchors.

The Skim app has useful anchor links defined. (I think I have it installed from macports or homebrew.) Here is a working link example.

 [Finding text](help:anchor='Finding-text-in-a-PDF-file'%20bookID='Skim%20Help')

The links are defined in the pages so one could do in-page linking but are not really used with help URLs in the help document. This is disappointing.

Here is an example URL from where the help uses a file link where they could be using an anchor link.

file:///Applications/Skim.app/Contents/Resources/Skim.help/Contents/Resources/en.lproj/SkimHelp_48.html#Automating-tasks-with-AppleScript

@pete31

I understand it’s far more comfortable to index a help bundle once but I don’t think it’s a good idea. There’s always the risk of user error and then you might and up with a broken app.

What specifically might break? Would we be at risk breaking the app with the help or DEVONthink, or both? If the macOS help viewer can not resolve or understand a link, it simply gives an error saying as much.

I also notice that not all apps actually use help bundles for their help. An example is Scrivener, which uses PDF. I still plan to be able to search the help, in-place, for selected applications, this simply adds complication. I will try to solve this for apps that use help bundles first, then move on to on-off cases.

I think between what you and @BLUEFROG have built, I should be most of the way to my goal.

[mmynsted]

@BLUEFROG

Generally speaking, it is inadvisable to mess about in the internals of an application or database. I was merely responding to your suggestion with my own approach.
This is also more academic than suggested, as YOU SHOULD NOT MODIFY THE INDEXED ITEMS for any reason.
You should also NEVER use Move into Database.

Hang on a second…

Okay. I think I understand the issue with indexing the help in-place. While it is searchable by DEVONthink, it is also editable. Edits would damage the help.

It looks like it would be possible to mark something as locked but I suspect that does not prevent the item from being editable from DEVONthink. There would need to be some kind of enforced read-only attribute, to make it impossible to edit from DEVONthink.

EDIT:

I just read through the thread again. Making a copy of the help makes sense now. I am not giving up on in-place permanently, but to do it safely it appears that DEVONthink would need to be altered.

If I am not going to read the help bundles in-place, then I no longer care about keeping them as help bundles. PDF files would work better for me.

I will see about converting them from help bundles to PDF. (I do not think your script is doing this, but I am not sure.) I should be able to alter the links to be relative during that conversion. Then when the links are used (or a search is performed) they will open in the PDF the same way that they work with other PDFs in DEVONthink.