My knowledge database is primarily based on plain text (specifically Markdown formatted files). The reason I use this methodology is that it keeps the majority of my database lean and I have no problem making use of an index only approach to my data - that way I can use my Markdown editor’s of choice across both desktop and mobile to get data into my DT database. I also have a huge PDF library hosted on OneDrive (ever since MS provided unlimited storage as part of their Office 365 package) and this is also indexed by DT and updates automatically in a seamless manner each time I relaunch DT.
I have approximately 5K Markdown notes that live in an nvALT folder and each note takes up a minimal amount of data as all of my image/rich resources reside on a separate web server (or increasingly direct links to OneNote or Flickr based images). This workflow works well for me as the project documentation I produce (based on my knowledge database) is created via Brett Terpestra’s Marked app (and sometimes via Ulysses on my IOS device) which is effectively converting a HTML representation of my documentation to a CSS styled PDF/DOCx on the fly.
But the major problem I’m having at the moment is that the only way to get my (20K plus) Pinboard bookmark data into DT is to import the bookmarks and then face the ‘Hobson’s Choice’ of converting to HTML, webarchive, rich note or PDF; which has a huge overhead as each bookmark ends up being on average 300k - 3Mb (depending on the format choice). HTML is the most efficient but that is a poor choice because the majority of the data is taken up by code markup. That same Pinboard data lives in HistoryHound (St Claire Software) and the index size for all 20k plus bookmarks is less than 100Mb (it only indexes the text content of each web page). On that basis I made decision to use HistoryHound over DT as my tool for searching my Pinboard bookmarks as HH also has DT like sophisticated search logic that helps my locate relevant content. The main thing that I’m missing using this approach is the smart concordance based search features of DT - primarily the ‘see also’ feature, which has consistently provided me with wonderfully unexpected links in my data over the years. I’ve yet to find another desktop tool that can do the same thing. None of this would be necessary if the full text search capabilities of Pinboard (part of Pinboard’s premium archival service) functioned well but alas it doesn’t.
The DT developers already provide a bookmarklet for converting individual web pages to Markdown (via Brett Terpestra’s Markdownifier web service) and then automatically adding the converted page to DT. My wish is that this type of conversion tool could be part of the main DT program for bulk converting existing bookmarks to Markdown formatted files. At the very least an Applescript could/should be provided to handle this requirement.
An alternative thought is that maybe DT’s developers could strike up a strategic partnership with St Clair - developers of HistoryHound so that DT can make use of HH’s index.
I’m aware that Devonthink wasn’t originally developed for plain text workflows and the developers believe it functions best as a monolithic database that captures and holds all your data but in this multi platform, multi device, cloud driven world, many people have moved away from this way of working. Overall I’d like to see the DT developers optimise their product for both plain text workflows and index only approaches. The eternal wait for a useful Devonthink To Go product line would surely benefit from this too.