I have a problem with file encoding regarding Markdown files. Scenario: I generate a new Markdown file in Devonthink (right click → new → Markdown-Text). I edit the file including German umlaute (äöüÄÖÜ). I export the file to the file system (export → document). Opening the file in Obsidian shows that all umlaute are not properly shown, which means the file encoding is not UTF-8. I checked the file encoding using the command ‘file -I filename’ in the macos terminal. It shows me that the exported Markdown file is not properly encoded. It should be UTF-8, however the file command says ‘charset=unknown’.
How can I make sure, that the markdown file is correctly encoded when exported from Devonthink?
Out of interest I just checked this (created new markdown document in DEVONthink as described, copied in the offending characters) and then exported to the file system. The file opens perfectly in Typora with all characters as expected.
Yes, I am able to open the file in other editors, such as Apple’s TextEdit or TextMate. They both show the umlaute correctly, however, these programs do not show me the file encoding type.
Interesting side effect: If I create a new markdown file in an indexed folder, the generated markdown file is encoded correctly in UTF-8 as indicated by the system ‘file -I filename’ command and Obsidian also displays it correctly.
This results in similar behavior. In fact I first exported multiple Markdown files using this menu option and stumbled over the incorrect file encoding.
Markdown is pure text (as you can verify by using cat in the terminal). It has no inherent text encoding. Or rather: I doubt that any software would use anything but UTF-8 nowadays.
Also, I never experienced any of this with md files in a lot of different programs.
So the interesting question is: what exactly do you see in Obsidian instead of umlauts?
This is what the markdown file looks like in Obsidian. The file was originally created in Devonthink and then exported to the file system. Bildschirmfoto 2022-01-25 um 14.23.12.pdf (18.3 KB)
Exporting the markdown file seems to change the file content regarding the special characters (umlaute), at least in my installation. In the attached screenshot you see a markdown file opened from within Devonthink (Open with…) in a hex editor (top). The same file exported from Devonthink to the file system and opened in the hex editor is shown in the bottom. As one can see both the file length and the file content are different. Obviously Obsidian has problems to display the desired content properly.
And this is the result of the ‘od -c filename’ command for the exported markdown file of my previous post:
goetz@lovebird Downloads % od -c MD-Datei_exportiert.md
0000000 # M D - D a t e i \n \n D e v o
0000020 n t h i n k : 212 232 237 200 205 206 247
0000037