find dead links

devananda · September 22, 2018, 11:23pm

Is there a way to find (and thus delete) dead links?

I have a bookmark file in Devonthink and it is dead (the website no longer exists). There are many such files in DTPro and I am wondering if there is a way to search for bookmarks (or any link) that is dead?

Thank you.

cgrunenberg · September 24, 2018, 12:28pm

It’s not possible to search for them but a script can verify them. The following script replicates invalidate bookmarks to a group named “Invalid URLs”. In addition, both invalid and updated bookmarks are logged to Windows > Log. However, depending on the number of bookmarks this might require some time.


tell application id "DNtp"
	set theRecords to (contents of current database whose type is bookmark)
	if (count of theRecords) > 0 then
		show progress indicator "Checking Links…" steps (count of theRecords) with cancel button
		try
			repeat with theRecord in theRecords
				set theURL to (URL of theRecord)
				step progress indicator ((name of theRecord) as string)
				if theURL begins with "http" and theURL contains "://" then
					set theStatus to 0
					download URL theURL method "HEAD"
					try
						set theResponse to last downloaded response
						set theStatus to |http-status| of theResponse
						if theStatus ≤ 0 or (theStatus ≥ 400 and theStatus is not 403 and theStatus is not 405 and (theStatus is not 404 or theURL does not contain "github.com/")) then error
						set theDownloadedURL to last downloaded URL
						if theURL is not equal to theDownloadedURL then
							set URL of theRecord to theDownloadedURL
							log message theURL info "Updated URL (" & theDownloadedURL & ")"
						end if
					on error
						log message theURL info "Invalid URL (HTTP " & (theStatus as string) & ")"
						set theGroup to create location "/Invalid URLs" in (database of theRecord)
						replicate record theRecord to theGroup
					end try
				end if
				if cancelled progress then exit repeat
			end repeat
		on error error_message number error_number
			if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
		end try
		hide progress indicator
	end if
end tell

devananda · September 24, 2018, 7:41pm

Thank you!

Now one more question if you have a moment.

How would I delete the original and the replicant?

By the way, did you create this recently or was this something you already had?

cgrunenberg · September 25, 2018, 7:35am

Just select the item and choose Data > Move All Instances to Trash (accessible while holding the Alt modifier key) or press its shortcut.

It’s based on a similar script I’ve been using for some time now.

TheWart · September 29, 2018, 5:07pm

Fantastic script! Thanks for sharing.

PasatDNA · December 17, 2019, 5:18pm

Ok, I’m on shaky ground here and I need a little guidance. I created the script and it is working. Where are the broken links? Where are they located or how do I identify them? Thanks

PasatDNA · December 17, 2019, 5:20pm

Never mind… I just found the “Invalid URL” group - thanks very much…

Silverstone · December 17, 2019, 6:04pm

Nice gem.
Thanks @cgrunenberg!
Made it a Smart Rule on a weekly basis and added flagging to broken bookmarks
A “musthave” script

cgrunenberg · December 18, 2019, 7:05am

DEVONthink 3 include already a similar script for smart rules, see action Execute Script > External > Check Links.

vishae · April 6, 2021, 2:21pm

I can’t seem to find this Check Links script in the Script menu. Is it still there?

cgrunenberg · April 6, 2021, 2:26pm

It’s a script for smart rules (see Execute Script action) but not available via the Scripts menu.

vishae · April 8, 2021, 4:45am

Found it, thanks!

Although I’m not sure what it means when it says it’s updated the URL.

cgrunenberg · April 8, 2021, 6:32am

if the original URL gets redirected, then the script sets the destination URL of the redirects.

MacDav · April 8, 2021, 6:50am

I must be missing something - I don’t see it in my options ?

I’m using the latest version of DT.

cgrunenberg · April 8, 2021, 6:59am

My fault, it’s actually not part of the latest releases anymore as there’s now a “Check Bookmark” action. This action is a lot faster than the script and performed in the background.

MacDav · April 8, 2021, 7:19am

Confirmed - I can see it now. The help file needs to be updated to reflect this.

BLUEFROG · April 8, 2021, 4:36pm

Help > Documentation > Appendix > Smart Rule Events and Actions > Actions…

mksBelper · May 1, 2021, 7:15pm

Christian, Jim - thanks!

May I ask two questions related to this, please:

the help docs say of that Action …Check Bookmarks: Checks the URL of the matched items. Any issues with the URL are reported in the Log window or the Log popover…. what exactly does that do? Anything similar to the above-mentioned script’s ≥ 400 and 403 but neither 405 yet neither 404 nor containing 'github.com/' etc? I added ‘Display alert’ steps both before and after checking and they display almost instantaneously… does ‘Check Bookmarks’ actually perform HTTP validation?
is there a way to run such validation on webarchives?

Thanks!

BLUEFROG · May 1, 2021, 7:56pm

Yes, the script validates the URL by calling it and checking the response.
Yes, this will check the URL listed in the Info pane.

mksBelper · May 1, 2021, 8:32pm

Thanks, Jim! Great for #1. I should have known .

But for #2 (webarchives), I’m a little puzzled: I have a webarchive (in DT 3.7) whose correct URL is https://www.alamy.com. If I change the URL (in DT) to https://www.alamx.com (replace the final ‘y’ with an ‘x’) in Safari I get a 404 of course.

In DT I created a Smart Group (not necessary as I also do that in the Smart Rule itself) to look for any URLs containing ‘alamx’ (“Kind is Web Archive” + “URL contains alamx”).

Select the resulting item in the Info pane → right click → Apply Rules:

But nothing seems to happen.

I know the answer will be simple… .