Working with images with MD in 2022..would love advice from the DT community and pick your brains!

DEVONthink
1

Hi all

So I have been working with .md files for 4-5 years now and many a times have encountered frustrations with my files…either disappearing/broken images (due to md files moving/renaming), other applications not being able to read them etc, ect…

So I’m wondering how people deal with images in a intelligent and frictionless way as possible :smiley:

my main requirements would be:

  1. easy to add images (via drag and drop, paste etc)
  2. have them future proof as much as possible so that the images come/move along easily with my .md files and can never (or almost never) break
  3. compatibility with other apps: so I can write the .md in obsidian and move it to DT and it would “just work” ™ :slight_smile:

I know @BLUEFROG I’ve asked this before and you have 1 folder per 1 .md file and then a subfolder for associated images, but don’t you (and others) get annoyed with a million folders living alongside the md files?
also: dosent this make this less portable?

Honestly it seems like textbundle would solve all my issues but neither of my main apps I use (DT, obsidian etc seems t support it or want to support it…not clear why :)). is there a similar .md container/method that DT does/will support? seems much easier to deal with 1 file rather then many scattered files such as .md and all associated images?

would love to get any feedback from all the community here

thx so much!!

z

2

don’t you (and others) get annoyed with a million folders living alongside the md files?

No I don’t get annoyed by having many groups.

also: dosent this make this less portable?

No. It actually makes it more portable to use a group structure like I use. It’s the same basic structure used in web content for years. (Its also logically compartmentalized.)
With the structure I use, I can drag a group to the Finder and open it in a browser (or another Markdown editor) and have the links resolved.

I worked in graphic arts and printing for 32 years. This kind of data organization is critical there and I could pass off an entire job to someone in one group with its subgroups. Same logic applies here.

1 Like
3

gothcha @BLUEFROG

seems logical to me, and I’m very open to trying it out. wonder how this can work with application such as obsidian and others which seem to expect all your .md files to live on the same level. Any thoughts guys?

best

Z

4

also @BLUEFROG and all, I seems to now remember that the reason I never went into this method is that you have to manually create a group/folder each time you create a new .md file…or am I wrong?

I know you can create a template for the group+.md but can DT auto create a parent folder and move your .md into it when you create a new .md or capture from the web?

any tools/macros any one uses he can recommend?

best

Z

5

What are you using for a naming convention of the Markdown file?
Would you expect the group to be the same name?

PS: Having one group per document isn’t required. For example, you could have a group for a day and add documents and images to it. Or a weekly, monthly, every Friday group, etc.

6

What are you using for a naming convention of the Markdown file?

hmm no particular conventions. I just have subfolder for topics

image
image1092×805 132 KB

Would you expect the group to be the same name?

yes

PS: Having one group per document isn’t required .

interesting I am trying to follow your example and thought you had 1 folder per note. so your suggesting if I understand you that for each lowest level category (ie 1password in my example above) to have just 1 assets folder? is that still in your opinion robust in terms of probability and future proofing?

thx so much for your kind insight

Z

7

You’re welcome.

interesting I am trying to follow your example and thought you had 1 folder per note. so your suggesting if I understand you that for each lowest level category (ie 1password in my example above) to have just 1 assets folder?

As with many things DEVONthink (or computer)-related, the answer is: It depends.
Much of it depends on YOU and how you think about data organization.

My daily journaling certainly has one group per day, with subgroups containing images and resources, e.g., stylesheets, javascripts, etc. But I also do a lot of experimental things so isolating a stylesheet for example is more critical to me and why I don’t use a default stylesheet in my prefs except for support.

is that still in your opinion robust in terms of probability and future proofing?

The method I use has been around a loooong time and yes, it’s very portable.

But again, you need to consider how you think about the subject.
My comments in this case are descriptive, not prescriptive. If it makes sense to you and it’s efficient, by all means adopt it. :slight_smile:

8

But … Is this answer not just running in circles around the main question? Sure, the Web browser will resolve the links in the main .md file. But this is because the links are set up according to the fact that exactly that specific folder structure exists in the first place.

By comparison, when I create a long LaTeX document, I break each chapter into its own folder. I place a “figures” folder in each folder. And I place each figure into that folder. My LaTeX header has a directive function to reset the graphics path to the figures folder inside the chapter folder. I could just as well program the directive function to look for each image starting in its own folder. Beyond about two images, this approach would so clutter the internal layout of file content in my chapter folder that I would be … lost.

Which only says three things. First, folks have adapted ways to handle this particular approach. Second, they have stabilized/standardized the approach. And third, they have been doing this for a while.

I would hesitate to say that, with some thought, you could just as well make a main document structure where links resolved directly to documents in specifically-named sub-folders for the same class of document (e.g. a “graphics path” to figures approach), and that such a structure would also be … “very” … portable when it is self-consistent/reliant.

This is probably the best response.

In a nutshell for the OP …

The document storage structure that is used in DT is (very) portable because it is designed to be (strongly) self-reliant/self-consistent This does not mean that some other document storage structure does not exist that is also strongly self-reliant/self-consistent and thereby also (very) portable. As you have found, a potential downside to the document storage structure used in DT is that trying to parse through that structure to access specific document content can become easily overwhelming for mere mortals. But, computers do it all easily. So relax, pull up your favorite late-evening beverage, and set this worry aside.

–
JJW

1 Like
9

Unmanaged expectations

I have the impression that the “problems” arising with MD’s handling of images are related to misguided expectations. MD had (I think) not been designed to be a note-taking format. It was (and I might well be wrong there) designed to make writing semi-structured documents easier to write, especially easier than in HTML. And it was designed so that it can easily be rendered to valid HTML. I’m aware that MD can be rendered to nearly anything. But it has a strong penchant towards HTML: HTML elements can be included directly in MD and will be included in the resulting HTML – you can’t include PDF/PS commands in MD and expect them to be rendered in the final PDF/PS.

Markdown was invented to ease HTML production

So, HTML is the main target for MD rendering. HTML relies on links, i.e. pointers to information available not in the document but somewhere else. Yes, you can include images etc directly in HTML documents via data URIs, but nobody is doing that for real world cases and real images – the documents will become simply too large, and data URIs prevent lazy loading.

Usually, HTML is displayed by a user agent, vulgo a “browser”. Nobody is looking at raw HTML for long time, and there’s nothing there to be seen anyway, given that many things the user agent displays are links whose content a human can’t visualize.

MD can be used for note-taking, too

Regardless of all that, many people now use MD for note taking. It is easy, and the rendering results are immediately pleasing (most of the time :wink: ). But many users probably do not understand that what they see in that case is HTML. They see the images, and they think that they are “there” for some definition of “there”. Which they can’t be, because they’re just linked to, exactly as in HTML.

As you described, (La)TeX is better in that aspect. And a lot of people might be happier if they took their notes in LaTeX. However, they’d have to learn a markup language that is a lot (a lot!) more complicated than MD. So I see little chance for that.

MD is losing its character and advantage

So here we are: Many people like MD because it’s simple. And many users do not really understand nor care (and why should they, frankly) what MD was originally meant for. Vendors jump on the band waggon and add their own “extensions” to overcome perceived shortcomings of the format. Notably Obsidian, but also others (see the recent request for “admonitions” support). What was once a simple markup language is going to become a kind of stripped down HTML with more and more complexity, no body to govern over any kind of standard, and an increasing lack of interoperability.

Yes, one could do a lot of things to make MD “better”. Which will simply lead to a gazillion of dialects, each with its own application that can interpret it “correctly”. Maybe in the end, we’ll all simply write HTML again, because it has all these “necessary” features (and the some). Or (hope dies last) users will understand that MD was not made for wysiwyg layout (neither is HTML) and intelligent handling of external resources and switch over to other formats like Pages, Word, Star/LibreOffice, PDF.

8 Likes
10

thx @chrillek

really appreciate your detailed thoughts and explanation, very informative to me!

so a naive question? what should one use to create notes that are:

  1. future proof (any OS/App)
  2. good support to easily include images (preferably embedded )
  3. editable (which excludes PDF which I guess ticks the first 2 checkboxes)
  4. enjoyable to use and flexible :slight_smile:

to me the only thing that I could identify (and that DT support of course) is .md apart from the annoyance on image support which I understand is in inherit

thx again

Z

11

No-one can guarantee that. In my opinion, a clear text format (like MD, LaTex etc.) is the closest you can get, since simple text will always be readable. You may lose formatting, though. PDF is probably quite future-proof, too, given the amount of data available. But it is not a format you can directly use to take notes in.

Apart from technical considerations: How much information needs to be future proof? What was hot s*t today is outdated tomorrow, so maybe going the easiest way is best for most documents. I’m not talking about legal documents etc. of course – those love to be sitting in PDF files, preferably OCRd ones.

Define “good” and “easily”. Any answer to that will contradict my answer above, so no - you cannot have both. Pages, Word, Libre/OpenOffice, PDF all include images but are not particularly easy to use (too much UI, for my taste, too long to start up, over-featured for note taking).

“enjoyable” lies in the eye of the beholder. “flexible” fights with point 1 - whatever is “future proof” cannot be overly “flexible”. Imagine you use critical markup for comments etc. and the only critical markup processor for the Mac goes out of business.

Let me use a completely different perspective: If you really need some exotic markup syntax that is not available for your markup processor, you can probably write some JavaScript code and some CSS to simulate that (or have somebody write it for you). I’m afraid it has to be JavaScript, since the current browser developers refuse to implement AppleScript as an alternative. I posted an example for “check boxes” some time ago here, when DT didn’t support that yet.

You can also use JavaScript to embed images into your HTML, but than this HTML will be huge (but self-contained). You could do the same in MD (and I posted a script to do that with quicklook thumbnails here, too. you can find it elsewhere as well). But that will make your MD source completely useless, because it then contains huge blobs of near-binary data. No fun.

In my opinion, you’d have to figure out what your goal is.

  • If you just want to have MD files in DT that display nicely inside DT and on your machine: What is wrong with how it works currently?
  • If you want to pass around MD documents that the recipients can see as you do (well, provided they use the same CSS etc.): use a group containing the MD file and the images in DT and send them a zipped archive of that.
  • If you want to send wysiwyg documents to other people, convert the MD to PDF (and make sure to include your fonts with it …).
  • If you have other requirements, find another way to fulfill them.
1 Like
12

Not really.

Having used TeX/LaTeX for nearly four decades now, I would never recommend it to anyone for note taking. The overhead is too large, and the source is directly portable but narrowly parsable only to those who have a tex compiler or a tex-reader app in their in-built infrastructure.

Having dabbled in markdown over the last four years or so, I’d comfortably suggest it to anyone for advanced note taking. The advanced is purposeful. Markdown provides the advantages that it can be directed to a formatted output, that it can be parsed as such by anyone (due to the wide availability of tools to do so), and that it can provide hyperlinks to an extensive degree (something that Tex/LaTeX will not provide). Finally, markdown provides the benefit that it mostly stays out of the way while you compose.

My sole reason to present LaTeX was to document a counter example to the approach taken in DT whereby image files are (painstakingly) each encompassed in their own individual containers (folders) within the markdown tree structure. This approach in itself is not a benefit that makes the DT structure portable (counter to what seems to be implied). The approach is portable because DT keeps the entire referencing method to the tree structure entirely self-consistent. My LaTeX folders for my larger documents puts images all within one folder (not each within its own folder). My LaTeX folders are just as portable as any DT markdown folder with its each to its own folder because I force the self-consistency at the root level.

RTF, markdown, and LaTeX check off the first and third. You decide what it means to have the features: 2) “easily” include images and 4) enjoyable + flexible to use. I suggest the order will be markdown > RTF >> LaTeX as far as creating “notes” are concerned. By comparison, I suggest the order will be LaTeX >> markdown > RTF as far as creating documents that should end in well-structured, printed versions.

–
JJW

13

I appreciate this reply may not help everyone reading this thread but I offer it for what it’s worth.

As regulars here will know (sorry!) I have my life in the form of daily diary entries for the past 50+ years exported from Day One and imported into DEVONthink in the form of individual markdown records. I have added to many of those records (as I never did in Day One) DEVONthink item links to photos stored in an Assets folder for each relevant year of diary records. Thus I have a wonderful, searchable record of my life (of course, of interest to me only—but so much appreciated by me) in the form of those records and accompanying photos.

I take huge comfort from the fact that—although I’ve used DEVONthink items links for entries/records that were cross-referenced in Day One, although I’ve included markdown footnotes and quoted text and although I have, very occasionally, used markdown tables— the core of each record is essentially plain text. That, combined with the fact that DEVONthink does not, tentacle like, embrace things in any propriertary format, provides me with reassurance that my diary records:

  • are in the most sensible format—a compromise between formatting and universal readability (so far as that can ever be); and
  • as a result, will always be accessible in some way.

Before I used DEVONthink I had not the slightest idea that I would do all of this. Since I have done it…comfort, reassurance and absolutely brilliant searching (just try searching 18,000+ entries in Day One!).

Apologies for the extent to which this might be regarded as a little off-topic.

Stephen

5 Likes
14

I want to thank everyone here for the much detailed responses, will take a few days to think about the best way forward, seems like it will be mimicking @BLUEFROG method.

Maybe in the far future we will have some kind of widely agreed way of embedding images in md files :slight_smile:

thx again

Z

15

You’re welcome.

Do note:
Embedding allows for portability but suffers an increase in file size.
Linking requires lugging around resources or having access to them, but the file sizes stay wonderfully small (an admittedly biased perk) :slight_smile:

1 Like