Request: implement SingleFile for web clipping

Okay, been posting here and there but I thought a thread would be better suited.

There’s some agreement on the forums that web clipping in DT is not ideal especially compared to the leader, Evernote. But EN uses proprietary technology that they have developed and researched for many years, and the DT team has better to do than spend time into developing such a custom solution.

Webarchives are the current solution if you want to preserve a web page’s formatting, but the format has many disadvantages:

  • Proprietary format not working outside of Safari
  • Will load external content if the code says so, leading to long-term archiving or even privacy problems
  • Not working behind paywalls

The SingleFile format would solve the issues from what I can see: it grabs media content and encodes it in base64 format in a single html page which can be rendered statically in anything that can view html. Best of all, it’s open source:

There are extensions for every browser save Safari and a CLI tool to save pages in that format. I believe it would be a tremendous addition to DEVONthink both on desktop and mobile, allowing for long-term archiving, with a standard format, needing minimal development time since all the tools already exist.

5 Likes

Interesting. However note the embedding of base64 data in the HTML could create a file larger than 10MB. Due to memory consumption issues in iOS, that is the size limit for HTML and formatted notes in DEVONthink To Go.

1 Like

Ah! Thanks for the precision!
Webarchives do not run into that issue then?

Many webarchives are loading content dynamically. And while I can’t expound on the technicalities, many webarchives are still linking data, not embedding the images.

1 Like

Thanks for the answer. Well, that’s all the more case for having SingleFile… and if that makes too large files for DTTG, that’s one more limitation of iOS.

I remember a few days ago when you reminded me to try SingleFile, and it really is impressive! It’s very elegant, very neat. @BLUEFROG mentions that SingleFile may has a problem with excessive memory consumption in iOS. But I work mostly on a Mac, after all, for now, the laptop is the real productivity tool. So the flaws don’t hide it!

1 Like

I’m glad you like it! First time I found it, I was hugely impressed what what we could do and how elegant the format was.

BTW, would you recommend using SingleFile, or SingleFileZ?

Depends on your use. I prefer SingleFile because it seems more readily openable, but if size is a concern then go for Z.

It is now possible to set up an automation to import the file (downloaded with SF) into DT3.

1 Like

Do you have a process for that? I would love to have this!

Forgive me, I don’t know how to invoke services directly with Applescript, and how to pass the filtered file to Applescript. Hopefully the following steps will inspire you, or my steps won’t work.

But if you just want to move the HTML file into DEVONthink folder, then things are much simpler. I’m stuck on how to invoke the “Add to DEVONthk” service.

1 Like

Thank you! I was thinking along those lines, but I was thinking maybe you had a way to create SingleFiles without resorting to another browser then Safari. (Wish there was at least an extension for that… getting outside to Firefox in order to save them is a pain.)

I see what you mean, I’m guessing you use Safari as your primary browser. My main browser is Chrome. So, for me, as soon as Chrome generates SingleFile, it automatically adds to DT, which achieves a similar experience to the DT Chrome extension.

1 Like

Yes, I do, which is my lamentation! I wish there would at least be a SingleFile extension for Safari - there’s one for every browser under the sun.

But people aren’t very enthusiastic about developing Safari extensions, and even if they are willing to do, they charge for them. I don’t use Safari just because of the lack of extensions.

1 Like

I totally understand you, it’s just that the integration with the system is so deep that it’s more convenient… except for that.

Have you ever used this extension called Save Page WE. It has a higher rating than SingleFile. Also this one, MaoXian web clipper

Sorry, no, as I’m mostly under Safari I just use the regular SingleFile on Firefox :slightly_smiling_face:

Might be worth looking at WARC, as used by the Internet Archive and Library of Congress.

Or MHTML, which I’m guessing might have more support in iOS to start with because it’s basically just an e-mail message.

1 Like