Textbundle and textpack support on macOS and iOS

Native textbundle and textpack support on macOS and iOS, please! It would solve many problems with image-rich Markdown files.

I know, there is a open source Quick Look plugin, but firstly this plugin does not display referenced image assets (the main reason for textbundles!), plus Quick Look solutions don’t help on iOS.

3 Likes

This format would be a great modern replacement for the old-fashioned RTF format too!

I am using iThoughts for mindmapping on Mac/IOS. This tool has a complete markdown support and is able to export mindmaps in markdown as well as textpack. The thing is if I want to preserve the pictures in the mindmap, the way to go is via textpack.
From what i can see DT does not recognize textpack. Is there any plan to support this? Or maybe there is some other creative way to export mindmaps while preserving picture into markdown into DT?

1 Like

I can only second this!

There’s quite a list of notable apps supporting textpack: Bear, iThoughts, Marked 2, MindNode, Ulysses and more. So I think it certainly would be worth it for DEVONthink to support it too!

3 Likes

+1 :slightly_smiling_face:

3 Likes

Now we are in DEVONthink 4 (which is fantabulous!) and still no textbundle support… I guess Markdown with images is not a big deal for most.

Textbundle is basically a zipped folder containing MD and images. Not very different from a DT group containing MD and images. And code to convert _that _ into a Textbundle has been posted here before.

What exactly is the request for Textbundle support about? Importing them into DT? Exporting them from DT? Both are trivially solved with what DT offers already.

And textpack seems to be dead.

1 Like

… depends on your determination of “most”.

I think nobody really counts or quantifies here.

Plus, there are of course different approaches & schools around, with some sticking to script-augmenting DT for things it doesn’t have on-board solutions (though I don’t know of a script allowing auto-import of textbundle format)

But for sure, there are some (maybe even '“many”) who would like to have Textbundle. Just have to do a search here in forums, or check most recent thread on it.

Also noteworthy that team has in statements here in forums kept some implementation of textbundle afloat as possible in the future.

So, there is that.

Then, you will have to live w/ fact that DT has a culture of not counting votes, or opening up too much about the mechanisms of deliberation for such requests, too much. So, things are pretty much on teams discretion [edit; thx @Chrillek here], as scripts are a thing at the discretion [edited] of those who can code (or afford to learn it).

So, let’s hope together :grinning_face_with_smiling_eyes:

That should be fairly simple:

  • use doShellScript to unzip the bundle into a folder
  • Create a group in DT
  • Import everything from the folder but the Manifest into this group

15 LOCs tops, I guess.

Discretion, I suppose?

Interesting thought! :grinning_face_with_smiling_eyes: I am sure there are (fiddly) ways to export to (and import from!) TextBundle.

I think the big ticket item with proper support for TextBundle documents would be that a TextBundle document contains the images, so they are not individually visible in DEVONthink.

At the same time you could edit the document as cleanly as any Markdown document. The editor would be the Markdown editor with some additions to be able to manage the images in the document.

I would love it!

1 Like

Why does this matter?

Also, following the normal linking methods oft-advocated by me, having a group containing the Markdown documents and a subgroup containing images and/or resources provides the same functionality. In fact, it’s almost exactly the same sans a generic Markdown document and a JSON file as a manifest.

When using relative links, again a very normal HTML function, the group can be dragged and dropped to the Finder and still be functional with working links. To whit…

Markdown document group

Finder folder from dragged group

External Markdown document opened in external editor

1 Like

I can see this is a solution and if you zipped the folder/group to send to someone, they effectively would receive a working MD document when they unzipped.

I want to send MD documents to people. Unfortunately the md documents are often in a single group with a single ‘images’ or ‘assets’ sub-group with all the images in it. I understand for this to work I can’t necessarily use DT links but must use relative links (which I tend to do any way as md often fails outside of DT in other apps that can’t cope with the DT links - e.g. Marked2.).

Is there an easy way to create a zip of an md document together with the images it uses in a subfolder (so links still work when unzipped)? Or extracting everything into a group that can be exported and zipped? My md document folder has many md documents and the images subfolder has many images, so finding the images associated with a document and putting it all in its own group takes time.

This is not desperate - just wondering if it would be a viable workaround for sending an md file and its associated assets/images to someone.

Specifically for use with groups, Scripts > More Scripts > Archive Group

As @chrillek alluded to earlier in the thread, he previously shared a script that does pretty much that. (Unless I misunderstood something). The output is a TextBundle. I think it should be relatively easy to modify it so the output is a regular zip instead.

That can be scripted. Right now, I am traveling without a Mac, so can’t whip up anything. Perhaps someone else proposes a solution. If not, I’ll write up something when I’m back.

1 Like

I can very much follow your thinking, and also think it’s important to get to the core of the intent of why a lot of people ask for an onboard solution regarding TextBundle - and to read this in context.

This is a paradigmatic case where it’s more important to ‘read’ the intent, use scenarios and existing use constellations centered around DT than to get bogged down in technical questions or overly precise technical descriptions (especially when dealing with non-tech-centric users/customers) and translating everything into a purely technical import/export problem and loading this off to peripheral – script-dependent – solutions.

While nobody questions that it’s possible – and sometimes helpful – to import and export TextBundle through technical additions (scripting), this misses everything about a) the relevance/motivation of the request and b) the real reason TextBundle exists, and the realities of people using TextBundle.

TextBundle really is an exchange format. The impressive list provided in another thread by @jonmoore illustrates this: you find many venerable Mac apps there – and while none uses TextBundle as their basic work format, they nevertheless all provide an inbuilt(!) way to read it, honor/preserve the integrity of the format and – in some rare cases – even edit it. These cornerstone apps – or: their makers– recognize its critical purpose in exchange with other – also important – apps.

So while one can describe the format as ‘niche’ (maybe because no app uses it as ‘base format’…? :upside_down_face:), one can also see it as a neuralgic exchange format for media-enriched documents that has arisen from a specific use/problem situation re. interoperability between different apps in the speheres of markdown, mindmaps, and scriptwriting.

TextBundle exists to ensure data integrity in situations where the relevant unit is a rich document – text with associated images/media attachments, and not just the sum of its parts. Its official description can be paraphrased: ‘Primarily designed for the exchange of plain text files, such as Markdown or Fountain or mindmaps, in need of structured association of images or other attachments, between sandboxed applications.’

In the particular case of Markdown, which matters greatly for people wanting its benefits while working with media-rich texts, TextBundle incorporates images and media in a way that is portable and less prone to errors than using links to external folders. Everyone trying to use media-rich text documents within the Markdown universe knows it’s otherwise a PITA to deal with text-image combinations, especially across multiple apps or storage locations – even more when there is any kind of “roundtripping” such documents btw apps.

TextBundle serves structurally similar functions to OPML, or can maybe even more in case compared to the (proprietary) docx format or rtfd, as these latter two also maintain images and media attachments with their text bodies, preserving structural integrity.

While technically there can be scripted ways to import and export into/out of DT (porting means a process of friction, one should remember, though…), this doesn’t do justice to the intent of the format (an exchange format preserving a kind of document integrity and – more or less frictionless – “ease of use”), or the requests formulated here – whether in discussions around [photo-centered journaling](https://discourse.devontechnologies.com/t/advise-on-a-photo-centered-journal/), or at [multiple other places voiced in the forum](Search results for 'textbundle' - DEVONtechnologies Community ).
Scripted import/export simply breaks up (and on export reassembles) a data structure that is intended to be integral from the point of view of a non-technically inclined user.

A second perspective underlines the validity of the original requests and is crucial in understanding the real motivation of the requests: they come from people interested in “comfortably” writing and editing media-rich texts – things like visual journaling, or media-rich note-taking. While DT can be used for note-taking/journaling, this is, as we know, not its main intended or even recommended use. So, as documented in the forums, non-technically oriented writers/note-takers will – in the main – never use DT as their mainstay note-taker/journal editor. That is exactly in line with DT being primarily architectured, geared, and labelled(!) for intelligent data and file management - and not for editing/collating media rich documents. DT’s mainstay is to serve as a centralized location and organizational environment for all one’s documents, or interrelating them intelligently – without losing underlying document structures*.

For that very reason, the motif of the customer requests is – in the main – not to hand over or export a set of data structures from/to TextBundle in a technical way (though this is possibly helpful in some scenarios). Rather, it’s to leverage TextBundle as an easy-to-handle(!) transmission format that guarantees data-set integrity ‘in travel’ in cases around structured datasets (documents).

One can turn down that request, of course, and there will be some good reasons (missing resources; other priorities). But it doesn’t help the cause or increasing understanding of needs/uses, to always “silently” re-interpret (re-frame) the issues as a purely technical matter of ‘getting TextBundle data’ in and out of DT ‘in some way’… without acknowledging what really is on the line, and listening to use-cases and motivations of non-tech users.

It’s also important to see/acknowledge that DT is often and for a whole set of users not the ‘pivot for everything one does’ (even if some knowledgeable people here can use it like that), but really a hub for working with different and heterogenous documents, document formats, and document workflows which might involve DT – but do not make it the center of every document interaction. (With ‘document’ per se being carriers of data structure and integrity)

Embracing this legitimate perspective/framing – in which many ‘ordinary users’ would describe the issue and needs (see requests here in the forum) –, and opening up DT for TextBundle could also bring big intrinsic benefit to DT: it would align with, and indeed strengthen, its self-ascribed role as document manager and hub to interrelate and work with heterogeneous document formats, serving as a grand mediator and data collection and organization powerhouse.

4 Likes

Your post, @lerone, might be rather long, but it explains teh sitiation very well. Thanks for this!

I whole-heartedly concur with your core idea: Built-in TextBundle support in DEVONthink would be great for integral interoperability with apps like Bear and many others.

1 Like

After much battling (and it is probably not perfect by a long shot), I had created a javascript that creates a group of the documents as in your example. It works for me because all my images are always in an images group within the group that contains all the markdown documents. Links within my markdown documents are always relative (not DT links) so they work outside of DT.

For a selected markdown document it:

  1. Creates a group with the same names as the MD document
  2. Copies/duplicates the MD document to the group
  3. Creates an images subgroup to this new group
  4. Finds all links in the MD document with ‘images/xxxx.jpg’ and duplicates these to the images subgroup

I can then use the archive script to zip the and export the group for sending to other people.


(() => {
  const app = Application("DEVONthink");

  /* Loop over all selected markdown records */
  const records = app.selectedRecords();
  records.filter(r => r.type() === 'markdown').forEach(r => {

    /* get the text of the MD record, its name and group */
    let recordText = r.plainText();
    const recordName = r.name();
	const parentGroup = app.currentGroup();
	     
    /* get links to all images of inline or reference formt */
	const regexp = /images\/.+\..{3}/g;
    const imageLinks = recordText.match(regexp);
	
	/* Create a sub group with record's name */
	const subGroup = app.createLocation(recordName,{in:parentGroup});
	
	/* create an images folder in the subgroup */
	const imagesGroup = app.createLocation("images",{in:subGroup});
	   
	/* Copy the MD record into the subgroup and images into the images group */
	let newMD = app.duplicate({'record':r,'to':subGroup}); 
	for (const imageLink of imageLinks) {
	app.duplicate({'record':app.getRecordAt(imageLink, {in:parentGroup}), 'to': imagesGroup}); 
    }
  })
})()
1 Like

How do you handle newly created Markdown documents?

Not sure what you mean? I am modifying the script as it doesn’t check to see if there are images (no need to images subgroup if there aren’t any).