Script to set feed thumbnails to favicon

Currently, DT3 use the same generic icons as thumbnails for all feeds and posts in them. The appended script instead uses the favicon of the feed’s website, if that can be retrieved as
[http/https]://domain/favicon.ico
That works for a lot of feeds but not all of them. So I’d be grateful for any amendments.

(() => {

  let app = Application("DEVONthink 3");
  app.includeStandardAdditions = true;

  let dbName = "Feedly"; /* NOTE: Adjust this to your database name */
  let db = app.databases.whose({name: dbName})[0].root();
  let feeds = app.search("kind:feed", {in: db});
  feeds.forEach(f => {
	  let feedURL = f.url();
	  let URLmatch = feedURL.match(/(https?:\/\/)([^/]+)\//);
	  let domain = URLmatch[2];
	  let fullPath = URLmatch[1]  + URLmatch[2] + "/favicon.ico";
	  f.thumbnail = fullPath;
	  f. children().forEach(c => {
	    c.thumbnail = fullPath;
		})
    })
 })()

1 Like

Here’s a revision for smart rules that supports bookmarks & feeds and tries to retrieve the icon from the HTML source:

on performSmartRule(theRecords)
	tell application id "DNtp"
		if (count of theRecords) > 0 then
			show progress indicator "Download Favicons…" steps (count of theRecords) with cancel button
			repeat with theRecord in theRecords
				step progress indicator ((name of theRecord) as string)
				set theType to type of theRecord
				if theType is bookmark or theType is feed then
					try
						set theURL to URL of theRecord
						
						if theType is feed then
							set slashes to 0
							if theURL begins with "feed://" then set theURL to "https" & (characters 5 thru -1 of theURL)
							repeat with i from 1 to length of theURL
								if character i of theURL is "/" then
									set slashes to slashes + 1
									if slashes is greater than 2 then
										set theURL to (characters 1 thru i of theURL) as string
										exit repeat
									end if
								end if
							end repeat
						end if
						
						set theFavicon to theURL & "favicon.ico"
						set theHTML to download markup from theURL
						set theLinks to get embedded images of theHTML base URL theURL
						repeat with theLink in theLinks
							if theLink ends with ".ico" or theLink contains "/favicon" then
								set theFavicon to theLink
								exit repeat
							end if
						end repeat
						set thumbnail of theRecord to theFavicon
					end try
				end if
				if cancelled progress then exit repeat
			end repeat
			hide progress indicator
		end if
	end tell
end performSmartRule
4 Likes

Way cool.
Just some questions:

Does this work for the “posts” in a feed, too (if that’s the right terminology – I mean the HTML pages in a feed)?

Wouldn’t it be better to replace “feed://” with “http://” and leave it to the website to switch to https? That way the script works also for non-https sites

I guess that the repeat with i from 1 to length of the URL loop gets you the [protocol://domain] part of the URL?

It should support any item with a valid URL after removing this condition:

if theType is bookmark or theType is feed then

Are there still any websites out there using http://? :slight_smile: Most browsers started to block such sites already.

Exactly.

Hi guys

this looks very useful for thousands of my bookmarks. Can anyone write a sentence or two on how to use it?
How do you launch the script it a rule? do i use the execute script?


and if so do i need to save the script in the scripts folder?

and @chrillek, is your script javascript? How would one run it on thousands of bookmarks?

thx so much
Z

Don’t plan on “run[ning] it on thousands of bookmarks”.
Especially when dealing with network requests, you should work in smaller batches (which is good advice in general).

1 Like

You do. It’s all explained in the documentation. Look for automation, smart rules, external script.

Yes, my script is JavaScript. I’m more comfortable with it (and getting the base URL is actually less verbous then in AppleScript ;-). But it doesn’t matter if you use a JS or an AppleScript version of the script, they should do the same (though the one I posted earlier is some way behind what Christan posted, so you might go with his version).

As to “how do I run”: Mark one or two records and then select “Rules” and the rule from bottom of the popup menu.
And heed Bluefrog’s advice: All these network requests take time. Run it in batches

I have the impression that this script works fine as long as the favicon can be found in the html element of the page (or if there is one available at the standard location). However, when it is specified in a link element of the documnent’s head part, it doesn’t work.

Probably because download markup only downloads the element (or does it?) and embedded images is fetching img elements exclusively?

And I was wrong again - it actually downloads the whole HTML file. I amended my script, and it now finds more favicons. Though not all, yet…

function performSmartRule(feeds) {

  let app = Application("DEVONthink 3");
  app.includeStandardAdditions = true;
    
  var feedURL, URLmatch, httpPath, favIcon, HTML, embImages;
  
  feeds.forEach(f => {
	  feedURL = f.url();
	  URLmatch = feedURL.match(/(https?|feed)(:\/\/[^/]+\/)/);
	  if (URLmatch[1] === "feed") {
	     httpPath = "http" + URLmatch[2];
	  } else { 
	     httpPath = URLmatch[1] + URLmatch[2];
      }
	  favIcon = httpPath + "favicon.ico";
	  
	  let favTry = getIcon(app, httpPath);
	  if (!favTry) {
	    favTry = getIcon(app, httpPath.replace(/(feeds?|rss)\./,""));
	  }
	
  	  favIcon = favTry ? favTry : favIcon;
	  f.thumbnail = favIcon;
	  f. children().forEach(c => {
	    c.thumbnail = favIcon;
		})
    })
	}

  function getIcon(app, url) {
     let HTML = app.downloadMarkupFrom(url);
	let favIcon = null;
	  embImages = app.getEmbeddedImagesOf(HTML, { baseURL: url});
	  favIcon = embImages.find(img => (img.match(/\.ico$/) || img.match(/\/favicon/) || img.match(/icon/,"g")));
    return favIcon;
  }

Actually this shouldn’t cause any issues, the script uses progress indication, can be canceled and uses very little CPU time.

Hi Christian,

as you suggested, I use the script as external in a smart rule is triggered by “on create” (see screenshot). However, no new posts got favicons. Do you have any ideas on that?
Thanks a lot in advance

For posts you would have to use “On News”, for feeds after creating them “On Creation”.

Hi all

for some reason i cant get it to work for me. I tried both @chrillek js script and @cgrunenberg applescript methods with no success

this is what i have so far

i then stand on a DT bookmark and try to execute the rule

but nothing happens. Can any one guide me towards what im screwing up :slight_smile:

thx!

Z

Very nice - thank you

Not a big deal but… is there an easy way to adjust the favicons for dark mode?

What’s the URL of the bookmark?

No.

I’m running into weird problems with the JS version of the script: It runs ok once if I use “Apply rule” from the context menu, but terminates with an error -1708 (as can be seen in DT3’s protocol window). After that, it throws the same error without even doing anything (I peppered the code with logMessage calls).

If I run the exact same script in the script editor with the current selection (i.e. the same html file as in DT3), it works ok and doesn’t throw an error. Supposedly, this is some kink in JXA, but maybe I’m doing something stupid here?

A screenshot of the complete log might be useful.

DEVONthink 3 caches scripts to speed up their execution.

That’s the log:

11:29:34: 1: https://travel-dealz.de/deal/le-club-gold-status/?utm_source=feed&utm_medium=rss&utm_campaign=rss-feed	
11:29:34: match: 3	
11:29:34: getIcon https://travel-dealz.de/	
11:29:34: found: undefined	
11:29:34: getIcon https://travel-dealz.de/	
11:29:34: found: undefined	
11:29:34: 3: https://travel-dealz.de/favicon.ico	
11:29:34: 4: f.type - html	
11:29:34: ~/Library/Application Scripts/com.devon-technologies.think3/Smart Rules/Favicon.scpt	on performSmartRule (Fehler -1708)
11:30:49: 1: https://travel-dealz.de/deal/le-club-gold-status/?utm_source=feed&utm_medium=rss&utm_campaign=rss-feed	
11:30:49: match: 3	
11:30:49: getIcon https://travel-dealz.de/	
11:30:49: found: undefined	
11:30:49: getIcon https://travel-dealz.de/	
11:30:49: found: undefined	
11:30:49: 3: https://travel-dealz.de/favicon.ico	
11:30:49: 4: f.type - html	

The first part, up to and including the error message, comes from the rule being run in DT3
The rest is from the rule being run directly in script editor on the current selection.
I’m aware of the fact that DT3 is caching external scripts. What actually happens, is this:

  • I start DT3
  • I select a record and run the rule on it
  • all the messages are logged to the protocol window, including the final error message
  • I run the rule again on the same selected record
  • no messages are logged to the protocol window except for the final error message

So it seems that the script is somewhat “tarnished” after being run for the first time.
Since I don’t want to spam the forum, I don’t post the script here right now. Please let me know if I should do that or send it via personal message (or not at all :wink:

What’s the URL of the itme? Is the script above the latest version?