Difference in behavior between convert menu & gear icon

Suppose I have a URL bookmark open in a window. DEVONthink displays a rendering of the web page at the location. There is a clear difference in how the PDF is produced between the following two methods:

  1. Select the menu item Tools ➜ Capture ➜ PDF
  2. Use the “gear” icon in the toolbar above the preview

Untitled

The difference is most apparent in pages that load content dynamically. If I have already scrolled to the bottom of a page in the window, and use method #2, DEVONthink captures the complete page as a PDF; if instead I use #1, the result is seemingly as if DEVONthink reopens the URL anew internally and then creates the PDF, because the PDF is often incomplete compared to method #2. (If I had to hazard a guess, I’d say method #1 does not seem to wait long enough for dynamically-loaded page content to finish loading before it generates the PDF.)

This leads to the following questions:

a) Is this difference in behavior a bug, or is it intentional?
b) Is there a way to get method #1 to render the already-loaded page in the window, instead of whatever it is doing in its current implementation?
c) Is there a way to invoke method #2 from AppleScript?

Question c is the most important from my perspective, as getting complete PDFs is currently difficult to automate without some way to get the behavior that results from using method #2.

The conversion (see Data > Convert) might indeed be different in case of dynamic web pages (especially those reacting to user input like scrolling etc.) and depending on the size of the window (e.g. single-page PDF documents). Method #2 can be scripted but it still requires the currently visible web page and your input, therefore I’m not sure if this would be any improvement at all.

Agreed, the 2nd method does need something. After working on this some more, I developed a Smart Rule that is triggered “On Clipping” and runs the following AppleScript:

# This uses heuristics to try to load page content before creating a
# full-page PDF snapshot of the page. Beware that it takes 10+ seconds
# to finish, during which time it has to be the frontmost application
# so that the page-down keystrokes apply to the correct window.

on performSmartRule(theRecords)
	tell application "System Events"
		set frontmostApplicationName to name of 1st process whose frontmost is true
	end tell
	tell application id "DNtp"
		# Need activate so that the key code commands below act on the
		# DEVONthink window and not the current Safari window.
		activate
		repeat with theRecord in theRecords
			set theWindow to open window for record theRecord
			delay 2
			tell application "System Events"
				# Some pages don't load images unless they come into view.
				# Page down a few times to try to trigger image loading.
				# This doesn't work for all pages, and an arbitrary no. of
				# times is not a general solution.  This is a hopeless 
				# situation, but maybe this will get _some_ images.
				repeat 3 times
					key code 121
					delay 1
				end repeat
				# Some pages don't load the full content until the user
				# reaches the bottom. There's no telling how much there
				# is, so this is another hopeless situation. Repeat this
				# a few times to try to load a decent amount.
				repeat 2 times
					key code 119
					delay 1
				end repeat
			end tell
			convert record theRecord to single page PDF document
			close theWindow
		end repeat
	end tell
	tell application frontmostApplicationName
		activate
	end tell
end performSmartRule

With respect to my original question, I think I have figured out how to invoke method #2 from AppleScript (it seems to be convert record ABC to single page PDF document, correct?). However, the script above is still quite suboptimal, and one of the worst things about it is that the page-down/page-end actions are done using system keystroke events, which in turn requires that the DEVONthink window with the page in question is the frontmost window.

What would make this better is a way to script page-down/page-end so that DEVONthink does not have to be the frontmost application. Is there a way to accomplish that?

As far as I remember that’s not possible via UI scripting.

Conversion to PDF using the AppleScript command convert record ABC to single page PDF document not work as hoped after all, and I am back to asking my original question but in a more focused way. Here is an attempt to provide a more reproducible case. First, save the following simple script in a file in ~/Library/Application Scripts/com.devon-technologies.think3/Menu/ so that it can be invoked from DEVONthink’s scripts menu:

tell application id "DNtp"
	set selected to the selection
	if selected is {} then error "Please select some contents."
	set selected to item 1 of selected
	set theWindow to open window for record selected
	delay 2
	convert record selected to single page PDF document
end tell

Next, do the following simple steps:

  1. Create a bookmark in DEVONthink for a site that uses logins. (A newspaper site, Twitter, etc.)

  2. Open the bookmark in a DEVONthink window. (I do this by right-clicking on the bookmark/web location item and selecting “Open”). DEVONthink will load the content at the web page’s URL.

  3. Log in if necessary (if you have not recently logged in to the site from within DEVONthink).

  4. Convert the page using the following 2 methods:

    a) Use the “gear” icon in the toolbar above the preview.
    b) Run the script above from DEVONthink script menu bar item.

The first method produces a PDF file from the logged in version of the page, but the second method does not, even though I can see in the DEVONthink window the full, logged-in version of the page.

So my question comes back to this: using AppleScript, can I reproduce the effect of using the “gear” icon to produce the PDF file?

A script could open the bookmark in its own window/tab, wait until it’s loaded (see loading property of window/tab), retrieve the single-page PDF data of (see PDF property of window/tab), then create a new PDF document using the retrieved data and finally close the window/tab again.

1 Like

@cgrunenberg Thank you! This was enough to point me in the right direction, and while trying to get the syntax right for creating a new record with PDF content, I searched around and discovered your posting from 2012 containing AppleScript code that helped me make things work.

An update for anyone who may stumble upon this in the future: the AppleScript code in the posting of June 25 upthread is out of date. A newer version, plus a companion bookmarklet and instructions, can be be found at https://github.com/mhucka/devonthink-hacks/tree/master/auto-convert-web-page-to-PDF