Inconsistent behavior of URL command createHTML

Actually URL commands on the Mac are just intended for browser extensions and bookmarklets, therefore the source is optional but the URL required.

This would be plan B for me. I prefer to use the defined URL. My gut feeling is that the behavior is caused by the content of the HTML. There is no documentation (as far as I am aware off) that specifies what content the HTML can have. E.g. I found out that the semicolon makes it fail through trial & error.

It’s not the HTML, it is the fact that you pass the HTML in an URL. URLs must not contain certain characters, as described here:

and probably elsewhere. The obvious ones are ? and &, as those are part of the query. Similarly, # (introducing a HTML segment). With /, it’s more complicated: It is perfectly ok in the path segment of an URL or in the location parameter of your URL command. And you have to percent-encode the %, of course.

As I said before, I don’t really understand what Apple is doing with its encoding method, and the documentation of it is nearly useless. In JavaScript, I’d use encodeURIcomponent, which is aptly named and does exactly that: Encode the URI components. I’d pass title, location and source through it, and that’s that.

Re Swift, I found this

on the net, perhaps it helps. And there’s this

regarding NSURL initialization on iOS and “aligned OS” (see the “Important” set-in). Goodness, what are they smoking?

OTOH: If the encoding of the source parameter were broken, DT would probably behave even more strangely.

Meaning that DT creates the document by loading the data from the URL even if the source parameter is present?

I just want to make clear that the URL field works just fine. The application also supports saving a bookmark in DT and this uses the same logic and works perfectly. Also please note that when it goes “wrong” it creates a bookmark which indicates that the URL field is valid.

The main questions for me remain:

  • Why does it go wrong sometimes. My estimate is that is goes wrong in 30% of the URL calls.
  • Why does it always work well when I send the same command a second time. What has changed in DT between the 1st and 2nd request that makes that it changes behavior.

Just for clarity:

  • Going wrong means, it receives a URL and HTML and it creates a bookmark instead of an HTML document
  • Doing right means, it receives a URL and HTML and it creates an HTML document.

@BLUEFROG responded to that: Sometimes, download of an HTML document fails – network issues, server issues, moon phases. In those cases, DT saves the bookmark instead.

When you say URL here, do you mean the value of the location parameter or the URL command? I was talking about the latter. It’s all a bit confusing, with an URL containing an URL…

In any case

  • the whole command URL (x-devonthink://createHTML?…) must be valid
  • to ensure that, all query values (what follows the = up to the next & or to the end of the query) must be URL encoded.

I still think that passing the HTML in the source parameter is probably pointless, since DT downloads the HTML anyway, thus overwriting yours. And passing possibly large chunks of data might (!) cause timing issues, thus resulting in a failed download.

You can always run Wireshark or something like that to see what’s happening on the network and if/what DT downloads when.

Not in case of the createHTML command but in case of similar commands like createPDF.

So, for createHTML with a source parameter, DT will use its value to create the HTML, without loading the original document using the location parameter? Then this should always create a HTML document, while the OP gets bookmarks sometimes – those would be the last resort if the document couldn’t be loaded.

This is what I would think as well. I just did an experiment where I tried creating the document without a network connection. Three times in a row it created an empty webarchive. I then re-enabled my network and the next attempt did create the expected HTML document.

To me this proves that DT does make (need) a network connection although given that the HTML is provided it shouldn’t.

1 Like

Assuming that the URL and percent encoding are valid & complete.

Exactly:

  • either everything is encoded correctly, than it should always create a HTML document, since the source is provided
  • or something is not correctly encoded, than creating a HTML document should always fail.

But as @fredap says, it works sometimes and fails sometimes, with the same URL command. That makes very little sense to me.

Just want to finish this discussion. with a final observation. I just created an export out of FireFox using the Clip to Devonthink extension.

I created a Web archive using the extension and got a Web internet location instead. Doing it a second time (and yes the extension remembers what it did last so I am certain I did it right the first time) created a Web archive.

I have seen this behavior now twice.

So I want to reiterate that there is a bug in DT that sometimes creates a Web internet location instead of a Web archive.

1 Like