Weird AppleScript issue with search

I’m trying to be ambitious and do some automation with dt.

I want to have a smart group that identifies a bunch of items in the database and also have an item that contains:

<!--- Begin X-SmartGroup: x-devonthink-item://15F405B0-91B3-4E75-9C85-F077E715FBA8 -->
 <!--- End X-SmartGroup -->

where the x-DEVONthink-item link is the smart group.

The script that I am writing will search the whole database for documents that have the above sequence and insert between the begin / end block links to all of the documents in the smart group.

The problem I have is that with AppleScript, I can get this document with

set recordsToUpdate to (search "Begin X-SmartGroup: ") in current database

but if I add even a leading -, it doesn’t find any documents. e.g.,

set recordsToUpdate to (search "- Begin X-SmartGroup: ") in current database

will fail.

I looked in the AppleScript dictionary for DT3 and there doesn’t seem to be anything there about the search string and special characters, but it appears the - is special. If I change it to +, it works.

Where can I find info on this?

The “-” is probably not indexed. It’s not about AppleScript or scripting, but about the search in DT. This expression
text: "Begin X-Smartgroup" finds the text.

I don’t quite understand what you’re trying to achieve here: All your items identified by the smart group (implying that there’s exactly one smart group that can identify an item) contain a marker pointing to the smart group in which they are contained?

  • That will work only for textual formats
  • The documents will not update if your smart group condition changes
  • You’re replicating functionality: The point of a smart group is to collect documents according to criteria, without modifying them afaict
  • What happens if a document is targeted by more than one smart group?

So every document in the smart group will not only contain a link to the smart group but also to all documents in the smart group?
Again – what is the ultimate goal here? And how do you plan to update all documents in the smart group if one document leaves or enters it?

Okay, so search only looks for what is indexed, not arbitrary text. Which explains why searching for the leading comment marker also didn’t work. Is there a raw search way to do it or do I first find records that contain the “Begin X-SmartGroup:” string and then validate the text around it?

I understand your point about smart groups. In particular, my use case is that I’d like to have a root document that has dynamic content from potentially multiple smart groups embedded within it - or have a document that provides context around what the smart group represents in a browsable way (not via a comment or notes field in the smart group). I’m toying with the idea of a browsable view of portions of my database by browsing the rendered (md) content.

It is possible that a particular document could appear in multiple smart groups.

As far as updating goes, I’m thinking it can be triggered by a smart rule - when a smart group changes, go through all documents that have the embedding and update the embedding.

The method is sound but perhaps way too complicated. Searching "x" NEXT "smartgroup" NEXT "x" should be sufficient to cover all the documents with this particular syntax.

I’d suggest starting from the incoming references of your smart groups, especially if you are not going to deal with a lot of them.

Otherwise, performance would potentially be abysmal.

I doubt that. At least I don’t see a trigger that might fire on “smart group content changed”. And AFAIK, smart groups do not necessarily update in real time.

Perhaps an annotation file might be better suited – for whatever it is that you want to achieve.

Isn’t that one of the use cases of a smart group? I can’t shake the impression that you’re trying to duplicate something that is already there.

Ah, this will require more investigation. Could be a smart rule that fires off periodically and processes the documents (maybe they are tagged)

I don’t think I’m trying to duplicate what is already there, rather, I’m rendering the smart group in a different way, as dynamic content in another document.

Consider a document in the database that looks like:

Areas to Explore

Here are some topics that I’d like to explore more.

Programming Projects

Here are some programming projects to try out:

In this example, “Areas to Explore” and “Programming Projects” are two smart groups that find other documents by a tag. My idea is that using the markers described at the top, I can have the above note change contents when the smart groups change and I can browse from here.

I know I can just have a link to the smart groups and browse that way, but that requires a bunch of clicking in and out that I want to avoid. I want an overview page that I can use as a launch.

It is not dynamic. Nor will it become so with the current tools available in DT, IMO.

And that would have to be updated whenever you add something to or remove it from a smart group. Thus doing what a smart group does anyway.

If your smart groups find documents by a tag, you could simply generate your overview document by avoiding the smart groups completely:

(() => {
  const app = Application("DEVONthink 3");
  const db = app.databases["myDatabase"];
  const records = app.search('tag:myTag', {in: db.root});
  const referenceLinks = records.map(r => `${[r.name()](r.referenceURL())}`);
  const targetRecord = app.getRecordWithUuid('x-devonthink-item:UUID-of-Overview');
  let txt = targetRecord.plainText();
  txt.replace(/startMarker1.*endMarker1/,`startMarker1\n${referenceLinks.join('\n')}startMarker1`);
})()

This JavaScript code (untested!) searches for all records with the tag “myTag” in the database “myDatabase”. It then updates the MD file pointed to by x-devonthink-item:UUID-Of-Overview by replacing everything between “startMarker1” and “endMarker1” with links pointing to the matching records.
If you want to have that dynamic, you’d have to use a smart rule that is triggered by a change in tags (“After tagging”, I guess) and make the rule run this or similar code.

If such an approach saves you any time compared to “a bunch of clicking”, I don’t know.

Smart groups do not contain anything. And they also do no respond to any smart rule event triggers nor can they be targeted by a smart rule. Lastly, their dates do not change. The addition, creation, and modification dates are all the same, regardless if their matches change.

Are you looking for something like this…

1 Like

That’s the general idea - getting the list from the tags would be fine and would eliminate needing a smart group. A sorted list with the document names as links would be a starting point.

My concept is predicated on using a smart group as the controller for the content. It keeps things in check and tidy IMO.
And those document names are links. Text decorations on links are :nauseated_face: :smiley:


This document is updating on its own.

smart-group-listing

That’s it - how do I build that? What does the raw Tagged Files.md look like to achieve that?

With WYSIWYG off…

And yes, I have in-document styling because it’s my habit when developing as it’s much faster to iterate styling.
Here it is unstyled…

Functional but not as nice to look at.

That content looks static - is it being generated from a smart group then? How is it generated / updated?

Yes, it is generated from a specific smart group.
It uses a triggered script, a less-spoken of but still useful type of script…

on triggered(theFile)
	tell application id "DNtp"
		set recURL to "x-devonthink-item://1B4A9511-CAA3-4BF9-960B-F56610221389" -- Change this to the item link of the targeted smart group.
		set smartGroup to get record with uuid recURL
		if (exists smartGroup) then
			set {childCount, recName} to {count children, name} of smartGroup
			if (childCount ≥ 1) then
				set childList to {"# [" & name of smartGroup & "](" & recURL & "): " & childCount & linefeed & "---|  " & linefeed as string}
				
				repeat with theChild in (children of smartGroup)
					tell theChild
						set the end of childList to ("[" & (name without extension) & "](" & (reference URL) & ")| " & linefeed as string)
					end tell
				end repeat
				set end of childList to (linefeed & "<style type=\"text/css\">body{margin:2.5rem;}a{text-decoration: none;line-height:1.5rem;}h1, h1 a{color:orange;}td{color:#AAA; padding: .8rem !important;}table{margin-left:0 !important;border-color:orange;}</style>")
				set plain text of theFile to (childList as string)
			end if
		else
			log message "Smart group not found."
			return
		end if
	end tell
end triggered
1 Like

@BLUEFROG Thanks a lot for that! I’ve combined my original idea with your mechanism and come up with this:

I want to add in the ability to have a tag rather than a smart group.

I don’t have the count at the top like yours, but the script is generic as it gets the smartgroup or tag from the embedded text.

It’s a total hack job right now - prototyped to working and not cleaned up at all.

I can post the hack job now or when I’ve cleaned it up

One thing I did notice is that if you’ve already gone to that document and the script has triggered, if the script changes and you leave / come back, the new script isn’t picked up. You need to quit DT3 and restart to pick up the script update

You’re welcome.

DT caches some scripts. I don’t remember when and which ones, though.

Totally get that - caching is challenging, specially when you aren’t in control of the update operation, which is the case here.