I’d appreciate some advice on file handling. Here’s the situation:
I read most of the material I find online (most of that via RSS feeds) via Matter. I selectively highlight items of interest in my reading, and then sync those highlights to Obsidian on my Macs. Sadly Devonthink doesn’t have a sync service with Matter.
From Devonthink, I index the Obsidian-Matter sync file and bring all my Matter highlights into the same database as my other notes.
Here’s where it gets messy, literally. If I move all my Matter notes out of the index folder into the database, they still exist in Obsidian, and behind that, in Matter. The index folder constantly repopulates with notes that I already have in my database, along with any new highlight notes I make in Matter.
I often slice up my highlight notes into more atomic notes, and these new atomised notes are not flagged as being duplicated in my “Matter Sync” index folder. I’m prone to making new versions of these atomised notes months down the track before I realise I already have copies in my database.
I could delete all my highlights in Matter after every update, but I’d prefer not to do that. I like having that separate repository of highlights available to me, in the context of the article.
Any thoughts on a clean, straightforward way of incorporating Matter highlights into my database without duplication, and without touching the original stash in the Matter app?
If by “move” you mean “import” – yes, that’s what happens, I guess. Not surprisingly.
That’s, I’d say, what your Matter/Obsidian setup is responsible for.
Why do you import and index the same stuff? In my mind, indexing is exactly what you need here.
Not clear. Where do you “slice” – in the imported notes in DT, in your indexed folder? And why would you expect slices of something to be duplicates of the entire, intact object?
Matter does have an API, but they apparently have not released any documentation for it. The Obsidian plugin is open-source, and it should be technically feasible to modify the plugin to accommodate your own needs.
I would suggest referencing the highlights, using block refs, in your personal notes instead. Don’t touch the synced document if doing so would produce undesirable results.
I don’t have the technical skills to modify the Obsidian plugin, and at the moment I don’t have the time or inclination to learn those skills, so I’ll leave this in the “someday” basket. But an interesting possibility.
Not if I want to refactor notes, as mentioned below. Matter exports multiple highlights as a file per article, but one article may contain a number of concepts that I want to split out into independent notes with their own tags and links.
The indexed folder is regularly updated with the files that originate in Matter, so I move them out of indexing to work on them - tags, refactoring etc. When the indexed folder repopulates with all the content held in Matter, the files I have moved into the database now have duplicate content in the indexed folder. That duplicate content is not always flagged as “duplicate”, because within the database I have split up the original Matter file into multiple notes.
On consideration, the solution may be as simple as flagging any notes in the indexed file that I have copied into the database. Or I have to decide on Devonthink as a single point of truth, and periodically delete all exported highlights from the Matter app.
I posed the original question to the Devontech hive mind in case someone else was dealing with the same issue, but it seems like my love of the Matter-Devonthink sync is a personal matter, to be dealt with personally.
IMO Matter is among the over-hyped PKM tools. The open web has been in a decline for quite some time. It’s now long past the prime years of Pocket and Instapaper. Matter may or may not be offering a superior product compared to its predecessors. At the end of the day, though, they all follow the same business model: one that made more sense in a previous era, in which mobile browsing was a horrible experience and browsers didn’t have a “reading view” built-in.
They advertise their iOS app as Matter: 1% Smarter Every Day on the App Store. I consider this shockingly unprofessional deed as an implicit admission that a conventional and explanatory name like Matter: Read It Later would attract little attention. Many of us save entire articles, rather than solely their URLs, so we can have access to the content even if the host site goes down. Entrusting the data to another online site – be it Matter, Pocket or Instapaper – does not actually address the concern.
Instead of a measurable, quantifiable thing that exists independently out in the world, we suggest that intelligence is a label, pinned by humanity onto a bag stuffed with a jumble of independent traits that helped our ancestors thrive. Though people treat intelligence as a coherent whole, it remains ill-defined because it’s really a shifting array masquerading as one thing. We propose that it’s hard to empirically quantify intelligence because it exists only relative to our expectations – expectations that are human and, moreover, individual to particular humans. Because of this, much like Monty Python’s Spanish Inquisition, intelligence often turns up in the places we least expect it.
This plurality is precisely what we should anticipate: intelligence is not and never has been a single entity. Instead, it is a hominin-shaped heuristic, a way for us to easily perceive valued characteristics in other people. Like beauty, it lies in the eye of the beholder. And just as we cannot expect to automate the personal, shifting lens through which each of us sees beauty, a search for anything like artificial general intelligence (AGI) misses the point: nothing in intelligence makes sense except in the light of humanity, and our own evolved perceptions.
It has the original URL, so that’s what I go back to if I need to check for context.
Devonthink can do what Matter does, but not as nicely. That said, maybe I should consider replacing the friction of getting Matter highlights into DT with the small frictions of using DT as a read-it-later service.
I happen to have the very same article in my DT database. The article was extracted as markdown via a browser plugin called MarkDownload, and then imported into DT. A smart rule script is responsible for renaming, adding metadata, and cleaning up some clutter.
The result is a local and editable copy of the article. I can make cosmetic changes (removing newsletter prompts, highlighting pull quotes, etc.) if I wish to. Reference to select text within an article is accomplished through DT’s URL scheme: x-devonthink-item://.../?search=some%20text
That’s mostly a matter of personal preference. MarkDownload uses Readability.js by Mozilla for parsing. From my experience, it is less likely to take in clutter or omit paragraphs, especially on poorly designed sites.
No it doesn’t. On iOS I clip using a home-made Shortcut based on the Get Article using Safari Reader action.
No, although a GitHub search would probably give you something better than mine. It’s for the most part simple text manipulations, like removing paragraphs which begin with “Subscribe to”, “Follow us”, etc. The document itself is surely readable with or without such niceties.
Thank you for stepping through this. I’ll investigate. I think my main obstacle will lie with iOS, which is where I do a lot of my reflective reading (not my preference, just circumstance). DTTG isn’t as user-friendly as Matter for this read-it-later usage, but whether I can live with that remains to be seen.