Watch out for "blob:" download URLs from item links in DT Server

NickLowe · July 30, 2023, 3:43pm

This is an apparent bug that I’m not sure I’ve fully understood, but report anyway as it can have quite alarming consequences in certain browsers. In a nutshell, attempts to download files accessed from item links can produce a seemingly malformed URL which if you’re lucky will simply fail, but if you’re unlucky (eg you’re using iCab Mobile) will not only crash the browser but cause it to crash on launch thereafter without some under-the-hood intervention.

To trigger this bug, you need to perform a fairly specific set of actions: you need to open an Annotation file (or other file with a DT item link); click/tap an item link (eg the one back to the original file in the Annotation file) to view it; click/tap the gearwheel and select Download Document, and then make the mistake of clicking/tapping on the apparently malformed URL beginning “http://blob:http://…” (Other kinds of file URL will sometimes begin with a “blob:” without the leading “http://”; these are also non-functional, but harmlessly so.)

In desktop browsers, you’ll just get a “Safari can’t open the page” or equivalent, but iCab Mobile will crash and become unusable, because on subsequent launch it’ll immediately attempt to reopen the tab that crashed it. The fix, which is easiest if you have XCode installed, is to go into iFunBox or iMazing, and in App File Sharing find the file iCab Mobile => Documents => .Users => admin => session =>session.plist. Double-click it to open it in XCode, and you’ll see an Item each for the tabs you had open at the time of the crash.

The last one will be the one that caused the crash, as you can verify from expanding it and checking that the URL field is the one with the “blob” string in it. Delete that Item using the minus icon that will appear when you hover, and save the file. iMazing will then ask you (possibly multiple times) if you want to reupload it; confirm that you do, and then you’ll be able to launch iCab with the offending tab gone (but the previous DT Server tab still open and working).

You can also simply delete the session.plist file, but that will remove all your open tabs (though you can restore them from History if needed). If you don’t have XCode installed, you’ll need to download the session.plist file, open it in a text editor, and delete the offending <dict>…</dict> entry; save; and overwrite the copy of the file on your device with the one you’ve just saved. I expect you can also do all this on-device with Filza if you’re jailbroken.

I haven’t figured out what the “blob:” element is there to do or what kinds of file URLs it appears in, but I hope there’s enough here for those who need to know to be able to make sense of it…

BLUEFROG · July 30, 2023, 4:06pm

So, the crashing part is actually an issue for iCab Mobile users, corect?

NickLowe · July 30, 2023, 4:22pm

Yes, that’s the only browser I’ve tried so far that it actually crashes – though that’s partly because I’m on iOS 13 and other browsers are too old!

BLUEFROG · July 30, 2023, 4:32pm

Time to shop for new hardware?

NickLowe · July 30, 2023, 4:56pm

Nah, it’s a choice! – I could be running iOS 16 on this device, but I rely on some old apps that wouldn’t like it… And the latest iCab is still compatible, so there’s no incentive to upgrade as it knocks spots off all other mobile browsers. Obviously I can’t run the latest DTTG, but my main databases are so massive that I make more use of Server anyway.

cgrunenberg · July 31, 2023, 6:37am

Does one of your items in the database have such blob URLs? So far it’s unclear whether it’s an issue of the database, of DEVONthink server or of iCab. Are you able to reproduce this using Safari or any other browser?

NickLowe · July 31, 2023, 10:40am

Yes, the blob URLs are a widespread feature when trying to download files from a link in an Annotation file, and both Safari and my Android browser (Neo Browser on a Boox tablet) exhibit them; the only thing that’s different about iCab Mobile is that it crashes rather than simply presenting an error. I could see what a wider range of browsers do it that’d be helpful, but it looks as if the thing that’s worth homing in on is the links that are generating them. I’ll poke around some more.

cgrunenberg · July 31, 2023, 10:50am

What’s the original URL of this link? And which file format does the annotation file use?

BLUEFROG · July 31, 2023, 1:36pm

A screencast would be useful, I think.

I think it being an Annotation file is incidental. I created an RTF file with a few item links pasted into it then fired up Server.

In a browser, long-pressing a link in the document gives a download option. This downloads to iCloud Drive. I didn’t see any issue in Safari and I just ponied up $3 for iCab Mobile and didn’t see any issue there.

Am I missing something?
Or doing something different?

PS: @aedwards, the file always downloads as view.. Is this addressable?

NickLowe · July 31, 2023, 1:38pm

I can’t remember what file I used for the screenshot, but it doesn’t particularly matter as I can now confirm that it seems to happen for all links back to original files from Annotation files, whether Markdown or RTF, and in all browsers on all platforms tested (MacOS, iOS, Android). Here are a few examples:

x-devonthink-item://9A04794F-0382-436A-B9AF-7476FB4DCA9C becomes http://blob:http://njl4.fritz.box:55754/d16cc433-eb02-441e-a9a3-036c72868da7

x-devonthink-item://DD9071DB-1B28-4DC2-91EE-B5DE17331625 becomes http://blob:http://njl4.fritz.box:55754/f386b887-09de-46c6-8010-c8b16d8ab59b

x-devonthink-item://EB2D47E6-1A2C-4E15-A8F3-F7118CD76CE1 becomes http://blob:http://njl4.fritz.box:55754/bd0d5af5-7323-4e96-8ec6-3c6322d89a74

I can’t make anything of this, but maybe it’ll make sense to you guys.

Incidentally, I’ve just found another MacOS browser that crashes whenever I try to access an Annotation file in DT Server – not this time when I go through the link and download, but when I click on it in the first place. It’s called DEVONagent Pro, and I highly recommend it in case you haven’t come across it. (Interestingly, it has no problem opening the Annotation file if I run a DA search for it – for non-Server users reading this who don’t know what I’m on about, a brilliant feature of DT Server is that it’s included in DA searches, so that you sometimes find the answer to your query was sitting there in your DT database all along – but similarly chokes if you then try to open the original document from the link back in the Annotation file.)

NickLowe · July 31, 2023, 1:46pm

Yes, I hadn’t known that about long presses, but can confirm that that works in all browsers on all platforms (including iCab Mobile, which you won’t regret stumping up for – its feature set eats all other iOS browsers for breakfast), and offers a useful workaround for the blob URL issue. But if you follow my original steps, you’ll get a blob URL and an error or crash.

Files downloading with the default title view.pdf is a widespread issue; I’d assumed it was due to a technical limitation, and I’ve got into the habit of simply renaming everything back after download, but it would certainly be great to have the original filenames preserved.

BLUEFROG · July 31, 2023, 2:04pm

Well, you’re definitely doing things I wouldn’t have thought of doing (though I got you on the long-press ). Goes to show how we all have different perspectives and ideas. A rich community and userbase indeed!

That being said, it seems unrelated to it being an Annotation file. It’s when jumping to another file via an item link in a Markdown document. It seems to not be affecting rich text here.

@aedwards:

The file from the link doesn’t reveal the document, so you’ll notice the download option is still referencing the Link List.rtf file.
Note the oddity of the PDF loading in the preview pane of the Split View for Markdown.

NickLowe · July 31, 2023, 2:35pm

Yes, that long-press tip was gold; Server is full of little discoveries like this, if also of strange little quirks like PDFs loading in Markdown split view that you just shrug and get used to. (“Forget it, Jake; it’s Server.”) It still feels like magic, though. Every time I fire it up I think “How is this even possible in a browser?”

I was sure I’d found at least one RTF Annotation document that blobbed, but I confirm that all the ones I’ve tried just now work fine and it seems just to be links in Markdown documents that are affected. (I’m not myself a great user of RTF over Markdown for Annotations and other documents with embedded links, which is why all my initial experiments were in MD.)

One reason for the slightly idiosyncratic workflow here is that (I think?) Server doesn’t offer an easy way of going straight to a known document except via a link; it can’t (I think?) do “name:” searches, and I still have an issue with some Groups not being browsable, which updates haven’t fixed for me. (I should probably get back on that ticket.)

BLUEFROG · July 31, 2023, 3:23pm

Glad to hear it’s providing such enjoyment While it’s really geared toward business, academia, and group collaboration, it’s still a very cool technology for the individual.

(I’m not myself a great user of RTF over Markdown for Annotations and other documents with embedded links, which is why all my initial experiments were in MD.)

I lean toward Markdown myself, but have come to (re)appreciate rich text in the past year or so. This is especially true with @aedwards’ new text editor in DEVONthink To Go!

it can’t (I think?) do “name:” searches

Indeed, it can!

Do note it’s appending the filetype as well.

NickLowe · July 31, 2023, 3:50pm

I’m a dolt. I can’t figure out what I was doing wrong, except that I did it again just now and got a set of nonsense hits before rerunning the search and getting exactly what I wanted.

Server is absolutely essential if you have massive databases that break DTTG and/or your iPad storage limit, but it’s also wonderful for untethering you from desktop in a wider range of ways than just having a synced copy on laptop or iOS, and across all platforms. I particularly love its handling of ePubs, with a navigable contents pane in permanent view. (It would be great to have this for PDFs.)

Going back to links in Markdown v RTF documents, a couple of quirks with the latter:

It doesn’t seem possible to use the gearwheel download option on links in RTF documents; you need to use the long-press trick to download the link, or else the file downloaded will be the RTF file rather than the linked file.
In the long-press version, trying to rename the saved file from “view.pdf” by choosing “Save As” rather than “Save” deletes the file suffix by default, so that you need to restore it manually to the file name.

BLUEFROG · July 31, 2023, 3:59pm

Development will have to assess the behaviors here. There’s plenty going on right now

NickLowe · July 31, 2023, 4:13pm

Thanks! Yes, none of this is a showstopper now that I have the long-press and name-search workarounds…

aedwards · July 31, 2023, 4:38pm

The actual file name is provided as an attribute to the blob object and the browser should use this when saving the file. The display of the blob url, instead of downloading the file appears to be due to a bug in a 3rd party library, will need some more time to determine whether there is a workaround o this.