Can't export to a useable website

DEVONthink Tips
1

Yesterday I showed a colleague the web research that I had collected into a DEVONthink Pro Office database, and promised him that I would upload it to a web site where he could see it for himself. I made this promise based on having seen the menu item File>Export>As website… However, today I discovered that the exported things can’t be used as a web site because it lacks any way to navigate to what is there–there is no top-level entity or index. Perhaps I am missing something, but this export function seems to be useless for my need.

I browsed the DEVON forums and found the AppleScript offered at devon-technologies.com/phpBB … rt+website. I tried it and found that it does make a crude but useable index to the exported site (DTP folders are represented as text lines and folder contents are represented as links). However, saved web archives are not displayed properly, with many missing images. Again, this is not useful for my needs. Indeed, examination of the exported material reveals that files that were .webarchive in side DTP are exported as .html, thus loosing the required referenced material in order to display the page in its entirety. I realize that .webarchive is not a format that a website itself can understand, but it seems to be a deficiency of the DTP exporting function that the resources needed to display the page are not included nearby with appropriate links from the .html page.

I thought that I might find a workaround by exporting to OmniOutliner Pro since that program has an excellent export-to-website function, emulating its outlining functions with Javascript. However, this too failed because the .webarchive files stored within DTP are exported to OmniOutliner Pro as links to the internet rather than as links to the .webarchive files. This is not a deficiency of OmniOutliner Pro since it recognizes .webarchive files (which I tried to dragging one into an outline) and which it correctly opens into a web browser. (However, even OOP exports the .webarchive files as such and does not convert them into a format that is readable to non-Mac browsers.)

I tried exporting as a RTFD file but DTP crashed without completing the task.

I could export as Folders and Files but I’m guessing that my Windows-using colleague will experience trouble opening some files such as those ending with .html since Windows does not recognize four-letter file-name extensions (correct?). Also, doing so creates .webarchive files instead of .html files with a nearby folder of resources and I don’t think that .webarchive files can be read with Windows.

So–I am desperately looking for suggestions on how to make my data available to my colleague so that he can browse the data which contains .webarchive files within the originating DTP file. For what it’s worth, I also have Circus Ponies Notebook.

2

I would just like to be able to take a section of my database and export it to a usable static web site that I could walk down all the folders that I created, but do not seem to be able to do this either.

digi

3

WebArchive files are Mac-only and can’t be read by Windows users. That’s one of the reasons I rarely capture data in that file format. I usually capture rich text notes of selected text/images, so that I retain the images for an article. PDF versions of pages are also useful sometimes, although hyperlinks are lost in the ‘print as PDF’ routine.

I always create my own ‘index’ page for exporting with pre-designed navigational assistance. Just create a new rich text document. That rich text document goes into the top level of a new group. Then duplicate or replicate the groups and documents to be included in the Website export into that group. Now I set up static links from the ‘index’ rich text document to the material I want to export.

Finally, select the ‘index’ document and all of the subsidiary groups/documents to be included and press File > Export > As Website. Save the export to a Finder folder or to e.g. your .Mac account.

4

The only problem with doing it that way is that every time I update the contents I would have to update the index document. My goal here is to have access to all the documents that I keep on my mac when I do not have my mac. I do not have many web archives, as I prefer PDF’s and RTF’s, so just want the whole thing as a simple web site with simple navigation.

Digid

5

Have you tried the little script mentioned by the first poster in this thread?

Nothing fancy, but it’s automatically updated.

As my databases typically have many thousands of documents, I don’t have room on my .Mac public folder to post them on the Web, so work with subsets for distribution to others (often on CD or DVD). I usually do a fairly elaborate linking structure to make navigation intuitive for the user.

I also use the Web server built into DT Pro Office when I’ve got people searching a database via browsers on several computers.

6

To Bill Deville and others,

I’ve just beginning a new project in which, essentially, I will be creating a field guide with a picture and some annotations for each of several hundred items.

Right now each item is a DT document. Using the DT Pro text editor I’ve created a basic format for each item which I am happy with. The Table function was quite useful.

However, I can’t figure out good ways to transfer the “field guide” to other programs, aside from Text Edit, for final editing, formatting, and sharing – especially sharing with non-Mac computer users.

What does seem to work is to merge all the documents into one large RTFD file and print it from Text edit into a PDF file. But that is pretty much all that works.

Pages doesn’t seem to like RFTD very much – I lose the table formatting.

Exporting to Word keeps the table formationg, but I lose the images.

It would be nice to have the option of creating a weblsite with navigational tools. However, when I attempt to export to Web Pages using the method Deville mentioned in the Feb 27, 2007, post above, it almost works, but I lose the images.

This is the first time I’ve tried to use DT in this way, and my first time trying to create a “field guide” like this. Any assistance would be appreciated.

At this point I am asking myself if DT Pro is the right tool. For this particular project, might it be much simpler to move the Field Guide creation aspects of the project, to another software tools, such as Circus Ponies Notebook?

Many thanks,

Mitchell

7

Mitchell, here’s an example of what can be done by adding navigational links to a number of rich text documents, then exporting the material as a Web site.

This is a 2-year old early version of the DT Pro tutorial database, converted to HTML. Note that images on RTFD documents are included.

The URL is http://homepage.mac.com/wbdeville/DT_Tutorial_Export/Welcome%20to%20DEVONthink%20Pro.html.

I created a project group. At the top level I created the index page, in this case named Welcome. All the linked material was stored in subgroups. If I wish to publish existing rich text material already in my database, but don’t want to ‘mess up’ the originals with added navigational links, I would duplicate the content into a new project folder and work with the copies.

To publish as a Web site, I selected the Welcome page and all of the subgroups below it, then chose File > Export > As Website.

8

Bill,

Thanks for the speedy and helpful response.

When I realized that you could export images to the web and I couldn’t, I had the idea that maybe there is something wrong with my images.

Apparently there is. My files work fine with an jpg image I pulled off your site, and with jpgs I dragged from the finder.

The images I was using in my test files were originally from the web, but had been dragged and dropped through several programs. When I clicked on them and looked at them in Preview, I discovered that some of the images that don’t export are pict images and some are jpg images.

They appeared normal in the DT Pro document, and normal in preview, but they don’t export to a website. If I put one of the new “good” images next to one of the “bad” images I had been testing wiith, the “good” images appeared on the website, and the “bad” images were nowhere to be seen.

Do you have any idea why the export to Website works with some images and not others?

Is there anyway to tell the good from the bad ahead of time?

I hope this is clear.

Many thanks,

Mitchell

9

Sorry, can’t think of a way to tell the ‘good’ from ‘bad’ images by inspection.

Suggestion: If you hit an image that doesn’t export to HTML, you can always us the Selection tool in SnapNDrag to take a picture of the image, copy the new image to the clipboard and replace the old image.

10

I came to this thread because I want to export to web but also include images. I have text with pasted images but they don’t export :frowning: That’s sad because a text with images is so much more explainatory.

Is there anything that I’m missing? I’m using DevonThinkPro 1.53 on OSX 10.4.11 Rich text exports fine and retains formatting. When images are included, the text exports and leaves blank space where the graphics should be.

Martin

11

Just send me an example document and I’ll check this over here. Thanks!

12

Has anyone found a good solution for this. I would like to take one of my databases and make a standalone web site out of it. I do not want to have to manually update an index every-time I create a document. I want to take my database, and export it to something that looks like the 3 pane view, or even the a simple file system, with the comment field.

Dale

13

My suggestions are to learn to script, learn to program, or buy DTPO (which has a built-in webserver). What you’re talking about requires a heck of a lot of work – a Java applet or Javascript library for the folder hierarchy, more Javascript for the list of files, and something to turn the comments into something meaningful… which, of course, means bugging the developers to implement your favorite solution for how to do that, since everyone would want their comment field translated differently.

One way to do this and host from a remote server would be to pay someone who knows PHP to write a server application that would traverse a directory structure and serve the files, converting RTF to HTML and grabbing the comments through calls to OS X and so on on the fly, and put that in frames or whatever. There are a couple of PHP classes that do the RTF conversion. TXT is considerably easier. PHP could even create an index automatically that you wouldn’t have to update.

The best thing is probably just to buy DTPO 2. You can’t host DTPO’s webserver from a remote server (which might be an issue if you’re serving files to a large number of people off a weak internet connection), but other than that, DTPO should take care of almost everything you want. You could harass the developers to add a feature to the webserver to show the comment field which, frankly, probably would not happen for a couple months but might be appreciated by other users.

Other than that, the paths before you are largely fruitless, expensive, impossible, or just not very much fun.

14

Its been a while since that was true. Links in PDF pages created by Leopard by Print|Save as PDF results in links that are fully active. That makes PDF a better way in my opinion to archive web pages especially since the URL is also printed to the page. Also, unlike web archives, PDF’s can be read across platforms.

Thats why I have been begging for a way to batch convert links to PDF inside of DT but to no avail it seems. The RTFD often results in pages that are formatted to badly as to be useless.

Any chance of getting a script or someway to batch convert existing DT pages to PDF. It would also be nice to have the save to DT as PDF as an option along the existing HTML Page/Web Archive options in the DA browser for example.

15

DEVONagent can save paginated and unpaginated PDFs to DEVONthink. And with the Automator actions, you can open literally thousands of HTML files/webarchives and save each one as a PDF (unpaginated only) to DTP.

16

I realized stupidly that there is an existing script for Printing to DT as PDF so that part of what I need already exists. What I don’t get is how to use DA or Automator to batch convert existing DT files (Web archive, etc) to PDF files and with the URL’s intact. For example, suppose I have 500 links imported from DA to DT and want to store them as PDF?

What am I missing?

17

I can’t help very well right now because I don’t have DTP 1.X installed and DTP 2.0b5 doesn’t seem to have Automator actions (they’re updating them).

The Workflow (in Automator) is something like Get Selected Records (DTP) → Export Records (DTP) → Display Webpages [in new tabs] (DA) → Export Webpages [Type:PDF] → Import Items (DTP).

You would have to have Leopard, though, to preserve links. But that should work. I’ve never done it with weblocs or webarchives, but I’ve done it with a hell of a lot of HTML from mirrored websites and it has worked very, very well. I’d expect that weblocs and webarchives would work the same way, since they’re all going to end up loading an HTML page.

One caveat is that this will (I think) try to load all your non-HTML files too, and possibly throw errors and kill your workflow unless you work around that. You might have to add a Filter Items (Finder) thing in there to cut out everything that doesn’t have a webloc/webarchive/html extension.

18

This doesn’t solve the original poster’s problem (which is quite complex indeed), but here is a script that will batch convert the selected items to pdfs.

-erico


---make pdfs out of all selected records 
---by Eric Oberle

tell application "DEVONthink Pro"
	
	set cupos to selection of window 1
	if cupos is not {} then
		set first_rec to first item of cupos
		set import_location to current group
		
	else
		display dialog "Please select some records to convert to pdfs"
	end if
	
	repeat with this_item in cupos
		
		set the_Window to open window for record this_item
		
		repeat while loading of the_Window
			delay 1
		end repeat
		set the_pdf to get paginated PDF of window 1
		
		try
			set the_url to URL of this_item
			set the_name to name of this_item & "-pdf"
			set the_comment to comment of this_item
		end try
		set new_rec to create record with {name:the_name, type:picture, URL:the_url} in import_location
		set data of new_rec to the_pdf
		close the_Window
	end repeat
end tell
19

And now a separate post on the website synchronization issue.

It is my opinion that website generation is one of the reasons why DTpro should support a two-way file synchronization model for select file system folders. If there were a way to build a group in devonthink that had a “root” in the file system with two-way synchronization in devonthink (allowing for changes in the file system to come into devonthink and vice-versa), then it would be relatively trivial to write a bash or perl script that would use rsync to get the remote file system properly updated. (I think I have a script that does this one-way, but for most websites, you are going to want a two way sync to allow for quick fix hacking, group collaboration, etc)

Writing the algorithms to generate a website out of dtpro without two way synchronization would be considerably more difficult. One could, i suppose, create an export directory with applescript and then sync the file up as necessary. But I can’t imagine Dtpro being the right tool for this without an ability to dynamically keep itself up to date with a folder in the file system.

I wouldn’t of course want dtpro to always update the file system and mimic its organization. But the ability to have “virtual root” groups synchronized with the file system would make anything having to do with web sites or group collaboration a lot more feasible. Alas, I don’t see two-way synchronization coming anytime soon see http://www.devon-technologies.com/scripts/userforum/viewtopic.php?f=16&t=6722&p=30767&hilit=synchronization#p30767. Maybe someone else has a better idea on how to do this, but I think dreamweaver or coda are better tools for anything but the simplest, one-way dumps of data.

-erico

20

Clever script – I hadn’t thought of opening viewer windows and getting the PDF from there. I thought only that it was impossible to tell the main think window to view a given record and then gave up :stuck_out_tongue:

Yep. jwiegley (IIRC) and I would like to see support for an .dtBase2-internal “unmanaged” folder (similar to not allowing iTunes to manage your music files, and thus preserving paths and essentially providing a transparent gateway to the filesystem)… which would be a good place to put aliases to external folders. It would take some custom coding on the part of DEVONtechnologies, but it would seem to me to be a good addition. It would effectively enable DTP to run as a Finder replacement, manage your iTunes library from within DTP (not that I know why anyone would want to do that, but people are @#$%ing goofy :wink:), do website imports with fully browseable perfect mirrors, and some other neat tricks. Indexing the contents of the directory tree might be a little weird, but perhaps there could be a DTP-Spotlight gateway so as to keep the database small while still providing access within DTP to the referenced files. With, of course, some loss of flexibility. Or maybe people should just be prepared to suffer if they index several years of Microsoft Office source code.

Note that the majority of people who read your post will be unable to view the one you linked to…