Files in feeds not recognized as duplicates

I have multiple feeds from the same website that sometimes send the same .md files, so I want to delete duplicates as they arrive. (I don’t care if the process accidentally deletes a few files that aren’t duplicates.) I’ve set up the appropriate smart rule, but…

The problem is that DT doesn’t recognize the files as duplicates – even with Preferences set to less strict recognition. Two of these files are attached. Anyone know how to make DT see these as duplicates? I set up a smart group and the script provided in DT to delete similar contents DOES recognize the similar files and can delete them, but I need to manually select the files to check. I’ve tried all sorts of adaptations of the script but I can’t get it to work automatically.

I’d love your suggestions on this!
Feed Duplicates.zip (10.3 KB)

They’re not the same.
One has 1565 words; the other 1573.

1 Like

Ah. Then is there a way to automate DT’s Delete Similar Contents script?

Call it from a smart rule, perhaps?

You can’t make it a smart rule because you have to manually select the items you want to compare first. I can’t find a way to select all automatically. I used some piece of script to do that but got an error message from DT saying this operation is not allowed.

The used script and the error message would of course be useful.

Is it possible to curate your feeds to avoid this situation?

In this case, sadly, no. The site sends news out on a dozen feeds, plus has a Just In feed that overlaps them all to a random degree.

I tried to insert this script I found somewhere online into your script Delete Similar Contents:

tell application id "DNtp"
	try
		tell application "System Events"
			keystroke "a" using command down
			display dialog "Select all"
			delay 1
		end tell
		set this_selection to the selection
	end try
end tell

That’s not quite how it was inserted, but it resulted in an error from DT that said something like: DT does not allow keystrokes using other applications.

I’m sure it’s all clear now! I thought if I could find a way to highlight the files in a Smart Group, then I could adapt your script to run as a Smart Rule. Or is there an easier way? Thanks.

However, we do not suggest using UI Scripting as it’s fragile and tends ot be problematic.

What are two feed URLs causing an issue?

And especially smart rules shouldn’t depend on the user interface or user interaction, ideally they’re performed silently in the background.

So is there a way to automatically delete similar contents files in a smart group? Here are some of the feeds that overlap:

https://thehill.com/news/feed/
https://thehill.com/news/administration/feed/
https://thehill.com/news/campaign/feed/
https://thehill.com/policy/international/feed/

Thanks!

Only via a smart rule which is triggered on news and performs a script to filter them.