Automatic download

Please, what is the simplest way to automate a download of a password protected website : i need DTPO downloading every evening at 23:59 the web statistics log from my website and archiving it.
I guess that this is possible with the script “Create Offline Archive”, but as i didn’t work really with scripts right now i don’t know how to trigger it daily at 23:59 .
Sorry for this question, but perhaps it can be answered easily.

I tried a little bit with Automator, Open URL, print to Devonthink Pro, but this doesn’t work when the title of the browser window is not anymore the same```
on run {input, parameters}
– Save PDF to DEVONthink Pro
delay 4
set timeoutSeconds to 2.0
set uiScript to “click menu item “Save PDF to DEVONthink Pro” of menu 1 of menu button “PDF” of sheet 1 of window “Stats of mywebsite (2013-11)[/b] - urldetail” of application process “Safari””
my doWithTimeout(uiScript, timeoutSeconds)
return input
end run

Here if the "Stats of my..." changes.
Sorry for this greenhorn -question.

Try the script below. Change to your own specific URL parameters, etc. and anything else you wish to customize.

tell application id "com.devon-technologies.thinkpro2"
	try
		
		set theDate to (current date) as string
		
		-- set theURL to your own page address
		
		set theURL to "http://my.webpage.com"
		set theName to theDate & ": " & theURL
		
		set thePage to "failure"
		set thePage to create web document from theURL name theName in the incoming group
		
		if thePage is "failure" then error "No Download"
		
	on error error_message number error_number
		if the error_number is not -128 then
			set theMessage to "Error: " & (the error_number as string)
			set theInfo to "URL: " & thePage
			log message theMessage info theInfo
		end if
	end try
	
end tell

The core of the script is


tell application id "com.devon-technologies.thinkpro2"
set thePage to create web document from "http://my.webpage.com" in the incoming group
end tell

which is all you really need to tell DEVONthink to make an archive.

Create a calendar event with Automator.

Not having your parameters I cannot test this, and I won’t have time to debug or answer questions on this thread – but this should get you headed in the right direction.

Hi Korm,

:smiley: :smiley: :smiley: :smiley: :smiley: :smiley: , thank you so much, this is exactly what i needed, works fine!

Only wonder if it is possible to do easily the same thing recording a PDF, but is not so important.

Thanks again

For a PDF try

tell application id "com.devon-technologies.thinkpro2"
set thePage to create PDF document from "http://my.webpage.com" in the incoming group
end tell

Or plug the “…create PDF…” command into the longer script.

Thank you Korm.

Please, a last question, as i never worked with apple script :

Can i duplicate without any problem the “try /end try” (including the “try” and the “end try” routine in one script (one application)
in order to make one application for two, three ore more records?

Perhaps someone else would want to make that change? No time.

But … why not merely use DEVONthink’s Clip to DEVONthink? You’ll get farther faster and better than having all these custom scripts that do the same thing?

Is it possible to trigger DevonClip every day automatically , for specific adresses, via calendar?

Anyway , thanks for all your answers, i will learn more about Applescript on my own.

Sascha, it was easier than I thought. Here’s a method in a single script to get all of your stats pages as separate PDFs, which you can use as a script in an Automator calendar event:


-- add as many URLs to this set as you wish, separating with commas and putting the URL into quotations

property theURLS : {"http://www.mypage.com", "http://www.mypage.com", "http://www.mypage.com"}

tell application id "com.devon-technologies.thinkpro2"
	try
		
		set theDate to (current date) as string
		
		repeat with thisURL in theURLS
			
			set theName to theDate & ": " & thisURL
			
			set thePage to "failure"
			set thePage to create PDF document from thisURL name theName in the incoming group
			
			if thePage is "failure" then error "No Download"
			
		end repeat
		
	on error error_message number error_number
		if the error_number is not -128 then
			set theMessage to "Error: " & (the error_number as string)
			set theInfo to "URL: " & thePage
			log message theMessage info theInfo
		end if
	end try
	
end tell

The script will loop through all of the URLs in a set that you define here:


property theURLS : {"http://www.mypage.com", "http://www.mypage.com", "http://www.mypage.com"}

Replace my dummy URLs with the list of URLs. You can add more, or remove any you want. Just be sure to make it a list separated by commas with each URL in quotations. I suggest loading your URLs into the list, delete the dummy ones, and try running this from AppleScript editor or the menu before putting it into Automator – just to be sure it is capturing all of the URLs you want.

1000 thanks, korm, this works fine ! Really great. I think capturing like this as PDF is better than HTML or webarchives, i had the surprise that some “captures” in HTML remained dynamic and loaded again when they have been consulted.

Thanks again, i am happy :wink:

Hi , there seems to be a time out problem.

When i launch the script from the Finder, all Urls in the list are proceeded.
When i launch it triggered by a calendar entry when the calendar is open, all Urls in the list are proceeded.

When i launch it via calendar entry every day, only one URL is proceeded.

Has anyone an idea what i could do? Seems to be a timeout problem?

Sascha:

The script will timeout or fail if the site from which you are downloading is not available, or there is a network delay, or Calendar is closed and needs to launch, or any of a dozen other factors out of your control. The best bet is to create a separate calendar event with Automator for each URL you want to download, and adjust the URL list in the script accordingly for each event. I would space the events 3 - 5 minutes apart.

Alternately here’s a version that has a 60-second timeout for each PDF download – which you can make longer or shorter by changing


with timeout of 60 seconds

-- add as many URLs to this set as you wish, separating with commas and putting the URL into quotations
-- 20131122.2 added a 60 second timeout for each PDF creation event

property theURLS : {"http://www.mypage.com", "http://www.mypage.com", "http://www.mypage.com"}

tell application id "com.devon-technologies.thinkpro2"
	try
		set theDate to (current date) as string
		repeat with thisURL in theURLS
			set theName to theDate & ": " & thisURL
			with timeout of 60 seconds
				set thePage to "failure"
				set thePage to create PDF document from thisURL name theName in the incoming group
			end timeout
			if thePage is "failure" then error "No Download"
		end repeat
	on error error_message number error_number
		if the error_number is not -128 then
			set theMessage to "Error: " & (the error_number as string)
			set theInfo to "URL: " & thePage
			log message theMessage info theInfo
		end if
	end try
end tell