OK, that’s an approach, it’s true. I guess I was hoping for a direct way that didn’t require searching the text of the document 
Thank you for the extremely fast reply and code.
If I’m not mistaken, source will return an HTML version of the document (converting if needed), and get links of returns all the URLs of all the links in the HTML source. (Or at least, that’s what happens when I test it.) Is the hoped-for advantage in this approach that it provides a common way of handling many different document types? Otherwise, it seems like it would be computationally more expensive. The HTML of a document many be much larger than the Markdown source, if one is using custom CSS and extensions like MathJax.
Is there a technical reason why DEVONthink record objects don’t expose a property for the link from an annotation document to what it annotates? The record has an annotation property for the forward direction, and based on past discussions in this forum, it sounds like record objects do have an internal back pointer to the document to which an annotation refers. Could this be made accessible programmatically?
If there’s no technical reason why it can’t be done, I’d be happy to open a ticket to make the request.
For the time being, I found another approach that I feel may be more robust than trying to pull URLs out of the text of the document. The code below assumes that the selection is one or more annotation documents. It loops over each given annotation document, looks at its incoming links, reads the annotation property of each linking document, and compares the UUIDs of the linked annotation document to the current annotation document. This works because an annotation document only refers to a single source.
on performSmartRule(selected_records)
tell application id "DNtp"
repeat with this_record in selected_records
-- We assume that these are annotation records already. We
-- need to get the thing they're annotating.
repeat with _incoming in incoming references of this_record
if (exists annotation of _incoming) then
set other_annot to get annotation of _incoming
if uuid of other_annot = uuid of this_record then
-- do stuff here...
end if
end if
end repeat
end repeat
end tell
end performSmartRule