Node Graph for Document Links

A cursory search of the forum history shows that a number of users would be keen on having a node graph that automatically represents the link relations between documents, which would help navigating a “personal wiki” of notes, e.g. built using the Zettelkasten method.

Given that a lot of users have requested something similar to this, is this something in the roadmap? I also note that there have recently emerged a number of tools that offer this so clearly there is growing demand in the wider market for such functionality (e.g. Roam Research and Obsidian below)

Sample of forum posts from a quick search :
Visual display of database?
Feature request: Interactive mindmap for documents showing their relations - #30 by sgttwld
Mind Mapping? - #11 by benoit.pointet
Graphical display of links
Mind maps

This would look similar to the node graph in Concordance.

There is indeed already a Links inspector pane that could be a natural home for this
Screenshot 2020-05-31 at 11.24.36

Examples in other applications:

Roam research

Obsidian


https://obsidian.md/features#graph-view

8 Likes

Outside the option in the Convordance inspector, no there are no such plans currently.

Thanks for confirming that.

It is a bit of a shame, particularly there appears to be a lot of demand from users, and as well as that other applications are implementing this that are targeting a similar userbase as DEVONthink.

I hope it can be considered for development at some point down the line, even if there are other priorities?

5 Likes

I wonder if it would be possible to write something up to access the necessary data from DT and then generate the graph view outside of it, using some software or script.

Indeed, this type of graph view would be really really interesting to have in DT. It is perhaps the only feature this other apps have that is not possible to recreate within DT.

5 Likes

I had quite some thoughts on the topic.

There are several interesting sub-graphs to visualize:

  • links between items (in content or metas, explicit or inferred)
  • hierarchies (tags / groups structure)

  • And through the graph many properties (and their evolution) of items could be visualized.

Possible implementation outline:
An applescript crawls (part of) a DT DB and exports items as nodes and rels/links as edges in JSON, which then is made explorable as a graph in an HTML thx to a JS lib like sigma.js.
This would have the advantage to be generated and consumed directly in DT.
Scalability and speed would be the bottlenecks here.

Or export for external consumption like w gephi, would bring more graph display and analysis power, but coming w a higher learning curve.

5 Likes

How does the JSON output look like or do you have a link for me what is required by sigma.js?

@cgrunenberg look at http://sigmajs.org/ for samples of how the JSON would look like.

4 Likes

This looks really interesting. I am afraid you might have ruined the rest of my day, @benoit.pointet :wink: (will be hard to let go of reading about this)

1 Like

Same here. really hard to focus on something else. :slight_smile:
Will give it a try tonight: export something to JSON and make a minimal viz of it.

I wrote some mini json handlers for an Alfred Workflow. Perhaps it could save a few minutes in the process :wink:

It is part of this script among others:

1 Like

I’m looking forward for something like this.

I’ll be happy with a temporary workflow to access node graphs :smiley: —but, having it natively on DTP3 would be fantastic! :partying_face:

3 Likes

From my POV, it would be interesting to be able to query via devon search based on the connectedness index, e.g. add an additional argument to search connectedness_index > 3 looking for notes which are connected/linked to e.g. at least 3 other notes. I can imagine practical use of this feature.

1 Like

Well, given the links in the OP, that’s five requests in 16 years, so that might not actually be “a lot of demand”?

If you index the folder(s) that you’ve configured as Obsidian’s “vault(s)” then you can work with the files in DEVONthink, and use them also in Obsidian including the Obsidian graph view.

2 Likes

Some results:

  • 2322 Items, 3180 parent-child relationships generated in less than 2 mins, (limited to a depth of 4 edges from root of one of my biggest and broadest DT knowledge base)
  • that’s around 20 items parsed per second (note when run from script debugger it’s muuuuch slower)
  • the graph is still very messy: no distinction of groups and docs, no hovering fcts, etc.

Next possible devs:

  • make the graph explorable from within DT
  • improve graph interactions and readbility
  • faster crawling by abandoning recursion and moving to a queued crawling scheme

For those who want to play with it:

9 Likes

Many of us read others requests, and assume we don’t have to repost the same request.

It is obvious that some sort of visualization of links is both necessary and missing from DT3.

Look forward to testing your Obsidian idea, but this should be native.

4 Likes

I’d also add that there is a lot more than 5 threads out there. I simply copied the first few in the search and believed that was sufficient to make the point that it has been suggested numerous times already. The number of replies to this thread, and in the other threads, is quite clear in demonstrating a high level of user demand.

It’s great that people are coming up with workarounds (which again shows the demand for this) to prove the concept, but to make this a fluid way to trawl through one’s notes then I think it ought to be native. I thought, naively perhaps, that because there is a similar visualisation already present in Concordance that this would not be too much of a technical challenge to implement within DEVONthink.

The existing Edit > Insert > Item Link… feature makes it really easy to insert links to other documents, and an interactive Node Graph of these links would be a great user experience for navigating the database in a non-hierarchical way.

3 Likes

@benoit.pointet, thanks for sharing! I just tried it out and there is something hindering UTF-8. Accents and non-latin characters rendered as ?. I will take a look at it again latter tonight.

2 Likes

I’m not sure if development staff at DEVONthink make time and resource plans based on forums, but I’ll give my personal opinion that this is not worth the time – if there are better priorities serving more customers under consideration. The illustrations here are blobs, to my eyes. Easy to do or not, I would rather see ongoing improvements to the general UI, scripting infrastructure, etc., than pictures of blobs with minimal information content.

Don’t take that wrong. By not liking it, I’m not disparaging the concept – just expressing a point of view.

3 Likes

For example?

Don’t want to hijack this…I’ll DM you