Transclusion to sections?

DEVONthink
1

Hi,

I am one of those who enjoy working with an Obsidian vault for journaling and developing thoughts.

Somehow Obsidian helped to free myself from the vast amount of data and options in Devonthink – which make Devonthink great though. And being able to work on the vault from within Devonthink with the vault indexed is great.

One option I became to value highly is transclusion of section in files with the syntax ![[filename-without-extension#section-name]].

Linking to sections with a hashtag is a widely used practice in HTML, Markdown etc., so I think it is safe to say, it is close to a standard.

I would love to see the option to link / transclude to sections in other Markdown files in the future.

2

I’m not sure what exactly you are asking for here. Firstly, linking to an anchor in an HTML document has nothing to do with transclusion. HTML does not know anything about transclusion.

Secondly, what is a “section” in a file? There’s a section element in HTML, but that is not related to anchors and links per se. There are no “sections” in a MD document (at least not that I know of). I guess that you might want to somehow™ link to a title element. While that is defined in (e.g.) Githubs MD dialect, it only works for in-document titles there, not across documents. I’ve not found any example for transclusion (!) of “sections” in MD dialects (except maybe that invented by Obsidian, that is). And I wouldn’t know what exactly it is that should be transcluded in that case (i.e. copied verbatim). The title and the following text before the next title? Before any next title or just one of the same level?

I think the specification is not very clear, and there’s no argument for inventing another MD thingy (and create another dialect in the process).

3

Thanks for taking care. I thought my sentences were clear though. I did not ask for transclusion in an HTML document or from an HTML document but just mentioned the hashtag as a common means to adress a section in a file.

Furthermore I am not asking for creating another MD thingy or dialect, but when formulating the feature request hinted to solutions that exist, that seem sensible to me and are related to similar usages in similar cases.

Furthermore, I think it is self explaining that, if a section of level h2 is adressed, the transclusion covers the h2 section including lower hierarchies.

Finally, because HTML uses the term “section” in a certain sense, I do not think that it is forbidden or stupid or whatsoever to use the term in a similar sense in a slightly different context. I cannot remember to have been educated in that way on the Pandoc mailing list or elsewhere.

It may well be that there is not usecase for you, but there are certainly for me. Summarizing parts of structurally identical documents in another document that only focusses on these parts e.g. are reasonable scenarios.

2 Likes
4

I can see a use case for linking to markdown headings. But:

Surely that is a little challenging? What happens with a document like this?

h1
h2
h3
h1
h2
h1

Transclusion might take the first h2 and h3 (if that were possible) but would then presumably trip up when it encounters the second h1.

I’m not arguing against you but merely indicating that it could be rather more complex than we might think.

Stephen

3 Likes
5

I’m not arguing against your use case. But I do not believe that „section transclusion“ is „widely used“. And if it’s not, there’s no generally agreed syntax or semantics. Which in turn means that DT would have to introduce a new piece of syntax and semantics that work only in itself and possibly Obsidian.

That’s not the way standards should go. In my opinion.

1 Like
6

Stephen,

Thanks for the considerate response.

Yes, I also think that there are challenges, and therefore I do not suggest every idea that comes to my mind, I think you understand my suggestion that way.

As for the limits of transclusion, I thought of it from the beginning of a section down to the beginning of another section of the same or a higher level. In your example, a transclusion starting at the first h2 would include the h3 part and finish with the second h2 part, and the second h2 would finish with the following h1 part.

Of course, other rules may apply like, transclusions only covering the part down to another section of whatever level starts e.g…

Maria

7

Got it. Think the same way often, but not in this case.

8

Yes, of course. However, having experienced one or two different markdown apps I tend more and more to the view that I much prefer to stick to apps which honour some form of standard markdown (although I accept there are even different flavours of that). That was a key reason I stopped trialling and experimenting with Obsidian.

Stephen

2 Likes
9

That is interesting. I have used MultiMarkdown and Pandoc, Github flavor, and I prefer to stick with standards that cover all of them. Therefore, after trying some plugins like dataview etc. in Obsidian, I have cut most of them off. I use Obsidian only as a part for in my Devonthink database because some features are really nice (for creating stuff and journaling), but I also wonder whether after all, Devonthink could take this over again. It is certainly alot more versatile and powerful.

Best,
Maria

2 Likes
10

Yes: I really like the way DEVONthink honours markdown standards and yet is so versatile. The only other markdown app I now use is Typora—which also seems quite strict about use of standards.

Stephen

1 Like
11

:+1:

12

Hard to say whether translusion at the section level would add value, from my perspective. I think it might, but would have to experiment (if it ever were to become a feature in DevonThink).

What I wanted to point out: It is possible to transclude items at multiple levels. So if document A contains transcluded contents from a document B, then adding a transclusion link to document A into another document would lead to it containing the contents from all three.

A workaround could therefore be to use the current transclusion functionality and be very “atomic”, i.e. focused and precise regarding the contents in each individual document, instead of using lots of headings and sections. Then using transclusion at multiple levels.

13

Yes, atomic notes would be a solution for the technical problem. This would mean to create groups or folders for every document that was one in the beginning. I do not think that the freedom of writing in a structured document will get lost with this approach.

To be more clear about the use case, I use the transcclusion of sections in structured descriptions of items or contexts, or parts of journals that will be summarized in a weekly summary. I can imagine of transclusion of sections from longer documents as well, could be very useful.

After all, developers may understand how much work such a development would be and how well an implementation can be adapted to potential standards.

14

Hi,

after the discussion from yesterday I would like to return the topic to my original intention:

Developing an option for transclusion of sections of Markdown documents in other markdown documents.

As an example for implementation I mentioned the implementation in Obsidian.

As an example for usage I mentioned transclusion of parts of structured documents in another document that collects only these bits of information, or just writing longer documents with citations from other longer documents.

Hoping that my intention did not get lost,
Maria

15

Amen.

Yes, I’d love if DevonThink could include heading transclusion, aka section transclusion. Obsidian has ANY-paragraph transclusion.

The syntax is simple:

[[TargetNote#This is a heading in TargetNote]]

And you can mark any paragraph for transclusion by appending to it a ^tag-such-as-this

[[2023-01-01#^tag-such-as-this]]

Here’s a link to the docs: LINK.

I think this might be the most useful addition to word processing since 1990. It’s great for research and note taking to have (a) the main document with all the parts you want to discuss, and (b) a note, which can neatly include the section it’s annotating.

The alternative is to break the original document into … atoms? One for each heading? Good grief.

The only reason I keep Obsidian running alongside DT is for this very feature.

16

And, of course, limited to Obsidian. If DT would ever offer that, I’d rather see them use something more standard-aligned, like the new ::target-text in HTML.

I don’t see any relation to text processing here. It’s more about hypertext and linking information.

I seem to remember that something like that was called “chapter”. Before 1990, probably :wink: And I also seem to remember that breaking longer publications in chapters was not so uncommon in the times of Framemaker et al.

1 Like
17

[quote="chrillek, post:16, topic:69562”]
I don’t see any relation to text processing here. It’s more about hypertext and linking information.
[/quote]

Touché. But as you must know, I was thinking of the text placement aspect of the word processing experience. But I will co-pedant you. Here’s the revision:

I think this might be the most useful addition to using a computer to present blocks of old/famous text alongside blocks of new/original text on a page since 1990.

Hypertext and linking are great, but sometimes, you just want the right text in the right place—without having to reinvent the “chapter.”

Maybe in the future, backlinks from the new note to the transcluded section in the original note will be accessible.

The times are changing. It feels like the softlink/alias/shortcut/transclusion function is extending its reach—not just into new spaces in information presentation, but even into how we mentally map relationships between ideas.

18

Personally, I think that breaking documents into smaller, more focussed thoughts can actually be a very powerful approach that in the end may result in a much better notes corpus.

  • Smaller notes allow you to tag each note individually with tags that are appropriate for the note’s actual content.
  • For a focussed note, you can properly use metadata to specify its bibliographic source, links, relations, comments, etc.
  • And, more importantly, you can assign representative titles to each note (which are like a core statement that can serve as the main entry point into the note‘s thought).
  • Most importantly, you can easily cross-link these notes using standard linking techniques.
  • And, of course, the currently available transclusion features will just work, so re-use is not an issue. You can combine and re-arrange these notes as you like.

Generally speaking, instead of improving software to better split our notes we should strive to improve software to better combine individual notes and work with them. I.e., offer something like a Scrivenings mode to view (and ideally even edit) & export a group of individual notes. Also offer to combine & arrange individual notes from, say, an outline (or a dedicated structure note / MOC / etc).

Personally, I’m convinced that allowing us to better handle a group of smaller notes would eventually be a much better approach.

4 Likes
19

I used to be a champion of atomism, even before PKM influencers made it a thing. Chopping things up in order for the machine to have more interstices to insert its superior connective tissue is precisely what I used to promote. But when I put that ideal into practice, it nearly destroyed me. And I’m still suffering from that lifestyle choice today.

In 2008, I had large swathes of text written for my dissertation. I read an article in NYT (or somewhere) by a guy who used DT to write his MLK biography. So I bought DT Pro and wrote a script that chopped up all my MS Word documents at their headings, made groups with the heading names, and placed the cut subsections inside, and converted them to RTF so I could edit them inside DT.

“Now all things are broken into topical bits!” I thought. “Now I can let DT’s AI work its magic and resynthesize everything better than it was before, like that MLK guy.”

It was a disaster, of course. The synthesis under the See Also function and the synthesis of contiguous paragraphs in good writing are different things.

So now I have PTSD about atomic notes and the whole Zettlekasten fad.

I know what you’re thinking. “You dumbass. The whole Tinderbox marketing crap about letting structure emerge organically blah blah is only meant for people whose work is already at the “jumble of disconnected bits” phase. You should have used DT’s AI relationship-finding See Also & Classify powers only at the early stage, and only if you’re someone who works by taking a small note, clearing your memory, writing another small note, clearing you memory again, and so on.”

Yeah, you’re probably right.

But consider this (with an eye to deriving a general principle, what we here in Tübingen call an allgemeines Prinzip): A single chapter file that contains 20 sections in their original order contains more information than a folder with 20 files. That information is their order.

Isn’t it better to access the text you need AND be able to see that accessed text in its original context?

P.S. — I think this whole atomism fantasy needs some serious research and debate. In practice, a strict folder hierarchy confronting me and coercing me into creating my notes in the right folder from the start is the most helpful thing the computer screen can do for me while I write. But I still keep trying the Zettlekasten/links-only thing anyway, because it sounds so exciting on paper.

2 Likes
20

My experience is similar. Once I’ve written something, I’ve done the synthesis that the atomic notes people are aiming for. That document can then be an entry point for further development, whether by me or someone else.

To get from the “jumble of disconnected bits” to the beginnings of an outline, I have yet to find a better tool than good old-fashioned paper. Usually a big sheet, where I use a different color pen for each source, cluster similar topics together, and draw lines and circles to my heart’s content.

3 Likes