The parameters of createHTML and what they (should) mean

DEVONthink
1

I am using the command x-devonthink://createHTML and use three parameters being title, source and location.

According to the documentation the meaning of these parameters is:

Parameter Meaning
title The title of the item to be created (string).
source he HTML content for HTML documents (string, only used by createHTML and createFormattedNotes ).
location The URL linking to the content, e.g., the URL of the website for the bookmark (URL).

So far so good. For testing purposes I have the title New HTML and as the source I am using the following HTML:

<html lang="en">
<header>
  <meta charset="UTF-8">
  <title>New html</title>
</header>
<body>
   <p>This is a test HTML document created for DEVONthink.</p>
</body>

Then finally for the location I have two variants being:

  1. https://discourse.devontechnologies.com/t/how-to-use-a-url-command-outside-the-browser/74563/3
  2. https://www.example.com/test.html

The title, location and source are all URL encoded.

I call the URL using the format:

x-devonthink://createHTML?title=<encoded title>&source=<encoded source>&location<encoded location>

When the encoded location is the first URL, i.e. the URL pointing to devon technologies, the article that is created will have the content of the page the location is pointing at, so the content of the URL. The HTML that was send using the source parameter is ignored. The location is properly stored. The title is as expected.

Then I repeat this command with the second URL, being the example domain. The article that is created will have the HTML as specified in the source. The location is properly stored. The title is as expected

When I repeat the command without a location, I hear a beep and no explanation why it fails.

This leads me to a couple of questions:

  1. What exactly is source used for. Under what circumstances is it ignored?
  2. How do source and location relate?
  3. Why is the behavior different for the given URL’s.
  4. Why is the location mandatory if you save HTML. Not all HTML has a URL

BTW, I am using the latest 4.0.2 release.

For reference I am listing the complete URL’s below:

For URL 1, devon technologies

x-devonthink://createHTML?title=New%20html&location=https://discourse.devontechnologies.com/t/how-to-use-a-url-command-outside-the-browser/74563/3&source=%3Chtml%20lang=%22en%22%3E%0A%3Cheader%3E%0A%3Cmeta%20charset=%22UTF-8%22%3E%0A%3Ctitle%3ENew%20html%3C/title%3E%0A%3C/header%3E%0A%3Cbody%3E%0A%3Cp%3EThis%20is%20a%20test%20HTML%20document%20created%20for%20DEVONthink.%3C/p%3E%0A%3C/body%3E

For URL to example.com

x-devonthink://createHTML?title=New%20html&location=https://www.example.com/test.html&source=%3Chtml%20lang=%22en%22%3E%0A%3Cheader%3E%0A%3Cmeta%20charset=%22UTF-8%22%3E%0A%3Ctitle%3ENew%20html%3C/title%3E%0A%3C/header%3E%0A%3Cbody%3E%0A%3Cp%3EThis%20is%20a%20test%20HTML%20document%20created%20for%20DEVONthink.%3C/p%3E%0A%3C/body%3E
2

It’s always a good idea to search the forum first. Some of that has been discussed before:

And it seems that location is always required, even if it’s not a URL pointing to a valid server.

Also, the HTML in the second example is invalid (I didn’t check the first one). This is the decoded string:

<html lang="en">
<header>
<meta charset="UTF-8">
<title>New html</title>
</header>
<body>
<p>This is a test HTML document created for DEVONthink.</p>
</body>

As you can see, the html element is missing the closing tag. But even if it’s added, the URL command without location still only beeps. See above.

3

Thank you for thinking with me.

The same HTML was sent in both requests.

I added the missing </html> tag. This doesn’t make a difference.

One request copies the content of the remote URL and the other stores the provided HTML.

That is my point. I consider that a bug. If not it should be mentioned in the documentation that it is mandatory.

My main question remains why the different behavior for the two differents URL’s.

4

The URL handlers are only intended for clipping (via bookmarklets or browser extensions), they’re not recommended for automation.

5

Thank you for coming back to me. This is unfortunate, but also still leaves me with questions.

Suppose I’m writing a browser extension, and the extension has taken the HTML from the page, parsed it, cleaned it up and made a summary it wants to store.

I would then set the title and location, send the summary as text or source to DT and then DT would ignore my summary and store the HTML from the original page instead. That is the behavior I see. Shouldn’t the text or source content trump the content of the remote page?

My application isn’t a browser extension, but it faces the exact same problem the browser extension would.

6

Perhaps you can script what you want to achieve (which you unfortunately don’t tell). createRecordWith behaves as you expect it. That is also a browser extension cannot do, because you can’t call other apps’s scripting interfaces from within a browser (fortunately).

If you’d describe what you want to achieve, we could perhaps find another way.

7

That’s indeed the intended behaviour for clipping, the source is only used to improve the reliability (e.g. in case of paywalls).

8

It is not that I don’t want to, I just wanted to focus on the URL commands and not on my application. But you asked…

My application is an RSS reader build in Swift. It differs from other RSS readers by doing the following:

  1. Runs in the background and prefetches all content
  2. Parses the pages and removes all the fluff. So no ads, no “see also”, no menus, removes most of the JavaScript, no cookies prompt etc.
  3. When I read the RSS feed, I get presented a clean document. Occasionally I see an article I want to keep. That is where my question comes from.

At that point, I have an originating URL, a cleaned up HTML content and a title.

It seem like you are suggesting I should use AppleScript to store the information. This is something I could do but it harder than using the URL command.

9

I didn’t mention a language, and certainly not AppleScript. But you can use that or JavaScript. And I doubt that it’s “harder to use” than a URL command – we’re talking about three lines of code, basically:

function addToDT(title, location, source) {
Application("DEVONthink").createRecordWith({name:title, URL: location, 'record type': 'html', content:source}, {in: YourPreferredGroup});
}

I found this description, which is a bit dated but perhaps still helps:

There are others, younger, using the same approach

10

Thanks. I will look into this.

11

I am getting closer. I have created this method:

    func addToDT(title: String, location: String, source: String, group: String) {
        let scriptString = """
        Application("DEVONthink").createRecordWith({name:"\(title)", URL: "\(location)", 'record type': 'html', content:"\(source)"}, {in: "\(group)"});
        """
        
        let script = OSAScript.init(source: scriptString, language: OSALanguage.init(forName: "JavaScript"));
        var compileError : NSDictionary?
        script.compileAndReturnError(&compileError)
        if let compileError = compileError {
            print("compileError", compileError);
            return;
        }
        var scriptError : NSDictionary?
        let result = script.executeAndReturnError(&scriptError)
        if let scriptError = scriptError {
            print("scriptError", scriptError);
        }
        else if let result = result?.stringValue {
            print(result)
        }
    }

When I run it I get the following error:

scriptError {
    NSLocalizedDescription = "Error: Error: Application isn't running.";
    NSLocalizedFailureReason = "Error: Error: Application isn't running.";
    OSAScriptErrorBriefMessageKey = "Error: Error: Application isn't running.";
    OSAScriptErrorMessageKey = "Error: Error: Application isn't running.";
    OSAScriptErrorNumberKey = "-600";
    OSAScriptErrorRangeKey = "NSRange: {0, 0}";
}

It compiles and then complains that application isn’t running. I also tried DEVONthink 4 but that responds with the error “Application can’t be found” indicating that the name DEVONthink is good.

Obviously DT is running. Any thoughts?

12

One thing: you can’t pass in the group as a string. That’s not the cause of the error, through (I think).

I re-created the Swift approach in JavaScript, and that works just fine (i.e. no errors whatsoever):

ObjC.import('OSAKit');
(() => {
  function addToDT(title, location, source) {
    const scriptString = `Application("DEVONthink").createRecordWith({name: "${title}", URL: "${location}", 'record type': 'html',
      content: "${source}"})`;
    const JSLanguage = $.OSALanguage.languageForName($('JavaScript'));
    const script = $.OSAScript.alloc.initWithSourceLanguage($(scriptString), JSLanguage);
    const error = $();
    const compileError = script.compileAndReturnError(error);
    const result = script.executeAndReturnError(error);
  }
  addToDT('website title', 'https://bru6.de/jxa', '');
})()

So, the problem must be somewhere in your Swift code, I guess.

13

I believe I can rule out Swift as the problem. If I replace the line that set the variable scriptString with:

let scriptString = "Math.PI"

The application works and returns the value of pi.

So the problem must be in the statement:

let scriptString = """
        Application("DEVONthink").createRecordWith({name:"\(title)", URL: "\(location)", 'record type': 'html', content:"\(source)"}, {in: "\(group)"});
        """

When I change code to:

  let scriptString = """
  const app = Application("DEVONthink");
  """

No error. When I do something with the app:

  let scriptString = """
  const app = Application("DEVONthink");
  app.properties();
  """

I get the error that the app is not running.

14

I can’t help there, since I don’t speak Swift. It certainly looks ok to me.

Do other apps show the same error, like Application("Finder").name()?

You could try something like

const app = Application('DEVONthink');
app.launch(); // or app.activate() 
app.createRecordWith(…)

And I found this

(nine years old, though).
Perhaps rebooting your machine solves the issue? I know that it sounds weird, but I found some references on the Net were that resolved error -600.

It is, in any case, not something that is related to the OSAKit approach of executing JXA code – if that even works in JXA itself, it should work in Swift, too.

15

How hard can it be, right? :wink:

I rebooted to be certain and this did not resolve anything.

There was a reference to missing entitlements. I added the entitlement com.apple.security.temporary-exception.apple-events with the value com.devon-technologies.think and that has an effect.

The error is now:

scriptError {
    NSLocalizedDescription = "Error: Error: An error occurred.";
    NSLocalizedFailureReason = "Error: Error: An error occurred.";
    OSAScriptErrorBriefMessageKey = "Error: Error: An error occurred.";
    OSAScriptErrorMessageKey = "Error: Error: An error occurred.";
    OSAScriptErrorNumberKey = "-1743";
    OSAScriptErrorRangeKey = "NSRange: {0, 0}";
}

The error 1743 seems to indicate some additional permissions are needed. Most references seem to indicate that it means: Not authorized to send Apple events to System Events

I keep searching.

16

Good luck. I’m running the code via CodeRunner, and that just works. Probably, because in System prefs/Privacy & Security/Automation, I have allowed it to automate DT.

17

I have got it working.

It required three changes:

  1. The entitlement Apple Events needs to be enabled
  2. In the .plist file of the application the field Privacy - AppleEvents Sending Usage Description needs to be present. The presence of this field makes that when you run the application it will pop a prompt that will ask for permission to send events to DT.
  3. The entitlement App Sandbox needs to be NOT present

The HTML needs to be a preprocessed slightly like this:

html
   .replacingOccurrences(of: "\"", with: "\\\"")
   .replacingOccurrences(of: "\n", with: "\\\n")

@chrillek , thank you for your support.

18

You’re welcome. It was an interesting endeavor.