Node Graph for Document Links

Notes > Anki Cards > Mind maps / Links to see connection between dispersed anki cards.

I really like what you’ve laid out here because it is closer to my own wish for a Tinderbox-type interface as a “view” within DT.

1 Like

This may have been said by others but . . . I am going to put this out there so somebody can correct me.

I was excited when I first saw this, but as I thought about it more I began to take the contrarian position.
It is important to know that items in DEVONthink can be related to each other in a complex tree. Beyond that I see no value in seeing such a graph of the contents.

One might ascribe weight or value to individual items/nodes by that item’s valency. Items that have more direct relationships with other items may be more valuable. I do not believe the same could be said for comparing tree depth. The direct relationships are already shown. The AI may find similarities with other items, that do not yet have a direct relationship. The AI’s findings are also already available.

When I am looking at a particular note or research paper, why should I care where it fits in some deep spaghetti bowl of relationships? The tree might be pretty but it when it comes to signal vs noise, it is noise.

Maybe it would be useful just because it would be pretty?

2 Likes

Thx for your views.

It made me think that the shortcomings you attribute to a network view hold just as well for a “hierarchical list” view: it is fascinating/pretty but most probably becomes meaningless when it displays too much information (aka too many items). And this whatever the visualization metaphor used.

That is why the discussion went on to discuss visualizing only parts of the whole.

2 Likes

Clearly the biggest obstacle here is the UX challenge in designing the interface in a way that unlocks new uses cases; at its most basic level (simply visualising the entire map at once in an undifferentiated manner) there is limited value. @benoit.pointet makes the valid point that scrolling a long hierarchical list of notes (as per DEVONthink and traditional note-taking apps) can be just as meaningless when one has a lot of notes.

I think there’s been a lot of interest in non-hierarchical models lately from a sense that top-down imposed levels of organisation do not reflect the highly-interconnected nature of knowledge. The node graph has appeal because the concept intuitively reflects the way in which knowledge is semantically interlinked, though the optimal UX mechanics will take time to work out. Hence, there is a bit of a UX playground going on with a number of apps now featuring this in a rather half-baked way (e.g. Roam and Obsidian), perhaps hoping to solve the design issues iteratively.

As it happens, while drafting this post, Obsidian was recently updated to include a ‘local graph’ feature that captures much of the below use cases that I suggest are enabled specifically by this. Here’s a screenshot of this from the Obsidian forum:

It’s possible to link this local graph view to a note pane and therefore have it automatically centre on the currently viewed note. When doing this, it enables the use cases described below:

  • Discover notes that have a meaningful connection to the present note, which the user might not be aware of
  • Improve the user’s ability to easily navigate through a users’ database of notes by navigating to closely-related notes. Currently, one can only navigate linearly by clicking on a wiki-link to a directly linked note. A graph view allows direct navigation in the local context, 2 connections away or more.
  • Facilitate creativity by enabling the user to see notes that are indirectly linked to a given note. With notes that are indirectly linked through intermediary notes, there is a high chance there may be a semantic connection that was previously unknown. E.g. where notes are linked A -> B -> C, there could be a direct link between A -> C that was not immediately obvious.
  • Allow for a new, organic bottom-up form of note organisation based on clusters of closely-linked notes. This reflects the semantically interrelated nature of knowledge (which a folder or tag structure cannot capture)
  • Identify notes that are not connected well to other notes, which could reflect gaps in knowledge that could be pursued

To facilitate this, I think there are certain properties that are important:

  • The graph should be centred around a specific note, which is the one that the user is currently viewing in a given DEVONthink window. Hence, the graph should exist as an optional view in a given DEVONthink window, such as in the Inspector (akin to Concordance).
  • As the user navigates to different notes in the window, the graph should update to centre on the current note.

Assumptions:

  • User is making extensive use of DEVONthinks’ already-excellent wikilinking/item-linking functionality
  • No AI or machine generated connections are involved - purely what the user has explicitly defined
2 Likes

Some script as a followup of this discussion:
https://discourse.devontechnologies.com/t/graph-view-a-network-visualization-script-for-dt/

I’m not sure I understand these abstractions when applied to a concrete corpus of documents – say PDFs gathered from by a researcher who uses PubMed ? Or is there an assumption that your script needs to process a set of notes with explicit links between them – links placed there in the course of note taking, for example?

The suggested demo script processes document IDs assigned by DEVONthink – I don’t see how it goes further than that to examine and evaluate the text of documents, or links within the documents.

It would be very helpful if you could provide a sample databases that we could run the script against and understand what it is doing and how to interpret the results.

I appreciate your significant labor in putting this together – and want to understand exactly what I am looking at, and how the automation produces it.

I want to have a library of plot and scene ideas, and then link them into an outline. The idea is to have a library of snippets that can be used, discarded, reused, and updated whether or not currently used in an outline.

It wouldn’t make sense to put every node in the DT database in the mind map. Creating nodes with descriptive titles that point to DT documents (or tags) makes a lot of sense, though.

Curiously, MindNode will only support one link per node. You can’t put links in notes associated with nodes, either.

Linking to a tag gets around that limitation.

If DT is going to sprout a mind map function, I think I’d like to see something like The Brain’s focused view. Show the group, and a configurable number of levels away. Not the whole database.

I’d say it would be marvellous to have something like this. Did you ever saw thebrain.com? I do not like the UI but it shows what could be achievable.

@benoit.pointet is really really close to having this working as a script - see Graph view : a network visualization script for DT - #18 by benoit.pointet and also see GitHub - benoitpointet/devonthink-graph-view

This was clearly an immense effort on his part - he is working on making it work with WIkiLinks in DT3 and then I think it will basically have achieved what you are suggesting.

This really only works if you adopt a note-taking workflow that focuses on making explicit links between notes, such as Zettelkasten. Generally, this would involve text or markdown files which contain links the user has made to other documents, in the form of either [[wikilinks]] or x-devonthink-item:// links within the text of the user’s notes.

A sample database that highlights the nature of this style of note-taking can be found here, which can be imported into DEVONthink (note: the links with a pipe ( “|” ) in them do not work in DEVONthink, as this database was not originally written by the author for DEVONthink (e.g. [[000 Index|Index]]):

The WikiLinks feature will also need to be enabled in DEVONthink:

@AlexEichler you may now try graph wiki links among a set of items.
See recent post of today on Graph view : a network visualization script for DT

The InfraNodus concept is external to DEVONthink, but is an interesting option if someone wanted to first export notes from DEVONthink.

2 Likes

I can give you a concrete use right now. I have a curated collection roughly 1000 research papers gathered over the last 30 years. I have moved nearly all of them into PDFs. I annotate those PDFs with both highlighted passages and personal notes. I’m still moving the various annotations out into separate files and linking but the general average is that I have around 30 items per PDF that are either a quote or a note. The quotes pieces and the highlighted notes have a lot more value to me when I can link them together. So I save them out as markdown and am using Obsidian to show those links.

A visual overview of what is linked and how many individual items are linked in any given area is critical to understanding the overall collection. It is also pointing me to areas where I need to do more literature review to understand the concepts. I can zoom in on specific nodes to see the links more effectively and I would love that capability in DEVONThink.

The graphical representation of the data is the primary beneft to me of using Obsidian.

Because a graphical representation of the sources and references used in a particular paper can be extremely helpful in deciding the value of that particular line of research, illustrate areas that need more work or review and also solidify your own thinking in your own related research. I can grasp the field beter from a node view than I can a hierarchy of folders and documents.

Both views are valuable but the node/link view offers things other viewing options cannot offier.

9 Likes

Still catching up on this thread, but it’s the search for these kind of “aha” moments as I do research for a book I’m writing that had me searching for whether there was some kind of functionality in DT like this that I hadn’t yet found. I’m a new user and DT is serving my needs in many ways, but one thing I’m still trying to learn how to do better is to identify connections I didn’t know I was looking for.

I have only read about half of this thread so far, but to the point I’ve reached it seems the pushback is on managing notes or organizing one’s understanding of information, in other words, in the product of one’s research or work. But what’s not explored as deeply in the thread is what I might term as the “raw material” of research. In my case, that’s surfacing relationships that may not easily present themselves, such as individuals involved in a subject of my book (about a particularly contentious use of law enforcement in response to Depression-era migration pressures) who may have been at the same events or used similar rhetoric, etc. If I were to see strong relationships, say, between a set of newspaper articles from the period and a group of archival finding aids that I hadn’t searched for, I’d know to prioritize my limited research time on those collections where those relationships are strongest. That’s particularly important right now when I can not physically access most libraries and archives where these materials exist, so I need to strategically make requests of the very few and very constrained librarians who can access some of these materials. Anything I can do to better target my requests helps that process.

I suspect another group that might benefit from a feature like this would be lawyers, paralegals and legal researchers (and maybe private investigators?) for whom surfacing relationships, however tenuous-seeming at first, may be helpful. And, since I also work as a journalist, I know reporting is another field that would benefit from a feature like this.

Anyhow, I have to return to the book writing, but I appreciate that this discussion is happening. I know developers have to think about how requested features will benefit an entire customer base, and I know I’m coming at this while still digesting this thread, let alone all the ways I can use Devonthink. I’m finding much value in the program as it is, but I felt it worth expressing why I am receptive to features like this.

3 Likes

A big +1 for this whole thread. As I’ve stated elsewhere, I have finally discovered (for myself) a legitimate use for node graphs. And that is the local node graph. The full size graphs that look pretty are sure neat, but I don’t fully understand yet how to use them. Conversely, the local graphs (at about 1-2 links deep) are tremendously valuable for both context and navigation. I’d love to see this built into DT3 someday.

6 Likes

I would be happy to see and read how the local node graph helps you, on the basis of an example.

It’s a local roadmap that I find helpful when reviewing any given note (currently all of this happens in Obsidian, just to be clear). I don’t know if I can explain it any better than: “I look at the local graph a lot and use it to navigate around closely connected notes.” The full size graph, while pretty, doesn’t lend itself to usability in my experience. Or rather, not regular usability.

I so much want to that a believe that a local graph can help in that regard.

I think what others are suggesting though is that this may be a solution in search of a problem and that the practical use case is elusive.

I am hoping to hear some specific uses cases. Such as “I study the historiography of XYZ, I was looking at my note on subject 1, the node graph pointed out to me that related concepts 2-4 were relevant, and as a result I added ABC to my current note.”

In my own work, theoretically local graphical nodes would be extremely helpful. But I have not yet found the use case that is not equally served by existing link technology.

I sure hope I am wrong though. The graphical nodes sure look cool.

I agree with this assessment.

It looks cool, for sure, but the utility isn’t there IMHO.