Testing items through x-devonthink-item:// links

[davem]

Before I depart on some code I’m dreading writing, and as a form of sanity check: I’m using Tinderbox to aggregate text stored in DEVONthink. Nodes in an outline in TB have a bilateral link through x-DEVONthink-item:// links from TB.

I sometimes refactor this outline, and the linkage is befouled and befuddled en route. What I want to do is check (probably via AppleScript) if ilnks exist, and if they do, to check a little metadata (names, titles, etc) to see if they need updating.

Is this viable? I’m assuming I will be able to query DT through applescript for content based on x-DEVONthink-item:// links, and from there, work through their metadata? And presumably, some kind of exception would be raised if the item no longer existed or been changed outside of DT’s control?

(I’m not fishing for code, just fishing for a sanity check before I depart!)

[meowky]

@davem

Is this viable?

It is, and it should be relatively easy. The following handler example checks the URL metadata value on a given item link.

on check_url_of_item on this_item_link
	tell application id "DNtp"
		set theRecord to get record with uuid this_item_link
		if (theRecord is missing value) then
			return "item does not exist"
		else if (URL of theRecord is not "") then
			return "url of the item is: " & (URL of theRecord)
		else
			return "the item has no url"
		end if
	end tell
end check_url_of_item

[davem]

@meowky

get record with uuid

Oh, this is very nice. Thanks, @meowky, exactly what I’m looking for. Bonus as the associated metadata will be bundled in with theRecord.

[rkaplan]

@davem

I am curious about the project

I have long thought there may be some use for Tinderbox I am overlooking but haven’t found it

What is your goal in “aggregating text”?

Is there some reason why a DT script could not achieve the same aggregation?

[davem]

Quite probably, yes. But Tinderbox realises a whole set of other features that I find incredibly useful. I can make it comb through my text and look for parts I’ve marked out for review; I can add inline properties (summaries, complex word counters) to sections as needed; being able to show notes on a timeline view, extracting start and end dates from metadata, is incredibly useful.

Being able to work with a high-level nested outline is great for writing long documents, and something that few things do well for writing. The way I see these tools: Tinderbox is for sitting down at a Mac with, and taking a high-level view, and assembling an overall text; DEVONthink is great for storing the raw text, as well as my sources, and can find and surface things - it’s a memory, and synchronisation tool; I then can draft and edit in any tool that works with Markdown. iaWriter, DEVONthink itself, but also devices like Pomera DM250s.

(The project is a doctoral thesis).

[rkaplan]

Fair enough - For a doctoral thesis I can see where that can be really helpful.

What is your thesis about?

[uimike]

Dave, it is a bit off-topic, but given that you seriously use Tinderbox - do you use the map view a lot, or mostly the outline (or others). I have been planning to use TB with DT like yourself, but wanted to use the map view more. I love Tinderbox, but for me the map view renders kind of sluggishly, and I have a hard time peaking at what is inside containers in the map, which a bit defeats the purpose for me using the map view. If you want to share your experience a bit… For others here - sorry for the diversion

[davem]

Happy to - I propagandise both pieces of software in each other’s forums! TB is a weird and opinionated piece of software, the likes of which I’ve never seen elsewhere. I both love it, and wish it was completely rewritten, but realise that what makes it is that it’s the unique vision of its author. It’s what pushed me into the Apple ecosystem originally.

So for writing, no, I barely use the map view.

For thinking, I use it all the time. I remember positions of things more than anything else, so when I’m putting ideas together the spatial layout is really key for my sense of understanding. Trivial as they sound at first, the adornments in map view are hugely useful in clustering thinking and ideas, more than depth - I find depth in maps more useful for aggregating evidence around distinct notes.

In my writing, I scribble notes-to-self in the markdown surrounded by {} - so I have {todo:flesh this out}, {cite:find source}, {q:check structure}, and some agents in TB that scurry through my outline and pull classes of notes into other notes, linking back to them - so one with the prototype for this finds anything {q:...} and pulls the text out into child notes and links back, another looks for {xyz:...} notes, and so on. It means I can hammer words out in plain text, leave myself notes, and then have them all collated later. This I view through the map view, as the nested elements make visual sense then; but otherwise, yes, it gets too fiddly to see what’s inside them. I remember a much older TB handling this better, but visual emphasis shifted elsewhere.

I also use MindNode, but to be honest, it’s a very prescriptive visual layout engine. It makes very slick and mac-like diagrams, and the integration has its’ place, but I find TB much more fluid and open for building maps for thinking and associating supporting information around them.

Day-to-day, I’m mostly in outline view, with a separate tab and branch in timeline view, and other tabs with map views for document management and reviews (the {} discussed above).

And @rkaplan, without going too deep, it’s a critical data studies take on social media data used in research and policy making; that it’s both seductively easy to get, but intrinsically misleading when ‘sampled’.

[uimike]

Thank you, Dave. Your work is very cool, social media data ‘sampling’. By that I suppose you include ‘uneducated’ sampling - I wonder what the ‘confidence intervals’ for social media sampling could (ever) be ha ha.

I’m with you on TB, which I used for many years, but not so much lately. As yourself, I am very visual, and I’m finding that the inability to “scroll” inside the map containers kills me. And going deeper vs going wider doesn’t help either.

And yes, same here regards MindNode (and iThoughts, etc.) neat but kind of not the right tool for the way I think (which tends to be really scattered)

Being able to add prototypes and named connections (= edges) and being able to do some hard searches on attributes is great though. But perfect is the enemy of useful, so I reverted back to paper and pencil (just kidding)

[uimike]

All these years I’ve been looking for a pasteboard-like app that allowed me to easily connect nodes, enlarge containers to peak at the insides, maybe drill by holding the mouse (and having some tag list appearing next to it, and filters), and capable of great notes and outlines and links.
From the old Curio on, OmniGraffle, mind note apps, Tinderbox, really nothing does it the way I wanted. And the MIT data walls, connection galaxies, etc, didn’t do it as well.
But I am quite happy with Devonthink, just been using tags a lot, and trying to keep the hierarchies contained. DT’s search and other facilities are great for me. Yet, a map would be nice eh!

[rkaplan]

@davem

In my writing, I scribble notes-to-self in the markdown surrounded by {} - so I have {todo:flesh this out}, {cite:find source}, {q:check structure}, and some agents in TB that scurry through my outline and pull classes of notes into other notes, linking back to them - so one with the prototype for this finds anything {q:...} and pulls the text out into child notes and links back, another looks for {xyz:...} notes, and so on. It means I can hammer words out in plain text, leave myself notes, and then have them all collated later. This I view through the map view, as the nested elements make visual sense then; but otherwise, yes, it gets too fiddly to see what’s inside them. I remember a much older TB handling this better, but visual emphasis shifted elsewhere.

Very worthwhile topic - there is so much misunderstanding of statistics/sampling in the public.

Regarding Tinderbox - your workflow makes total sense when you are writing a document.
Is there any way to use Tinderbox to use similar notetaking and agents when reading a PDF rather than when writing a text document?

[davem]

Is there any way to use Tinderbox to use similar notetaking and agents when reading a PDF rather than when writing a text document?

Not that I’ve found. I was using LiquidText at one point, as I really liked how I could export a page of linked notes, but there was no way of getting that data in a structured form - it was literally encoded as box and text coordinates. I was manually recreating the layouts in Tinderbox, but coupled with some long-term unfixed Liquidtext bugs, I gave up in the end. I was spending more time recreating maps than reading.

My scripts are up on the TB forums, I think; however, they operate on larger texts, rather than in isolation - they’re not really adding anything beyond using your own buckets otherwise. If there’s a way of getting the text from PDF notes out and with links, it should work - I’ve just not found a sane way to achieve that step. PDF was never really designed for notes and annotations, and they’re implemented as fudges as far as I can tell.

[davem]

@uimike

But I am quite happy with Devonthink, just been using tags a lot, and trying to keep the hierarchies contained. DT’s search and other facilities are great for me. Yet, a map would be nice eh!

One thing I found quite useful was building maps in MindNode, where each node was a tag I was using in DEVONthink and had a link to the DT tag (group) link. They were neat for visual thinking, allowing a degree of reorganisation and annotation. I could use the previews of the maps to quickly jump to matching content in a DT window, sometimes exposing things I’d tagged but not seen in that context before. QuickLook nicely previewed MindNode maps and made links clickable, so it seemed like a natural extension of DT. However, this required a lot of disciplined tagging work (at one stage I’d hacked together a script to naïvely automate tagging, but it was not ideal and I’ve not updated it for DT3; I also felt DEVON Technologies advised against this use of tags). Some LLM magic in macOS may yet find a way, but LLM is looking like it’s only suitable for autocompletion tricks, rather than having viable RAG and domain-specific tools that clodhoppers like I can take advantage of.

[uimike]

Hm, thanks Dave - you did give me an idea - I use smart groups a lot, and definitely will try to see what happens in an iThoughts map (nodes support links as well), if I build constructs using tags and smartgroup Devonthink links.

[davem]

You could see where I was going with this abortive map from many years ago. The map with links is a very versatile tool for annotation; you could easily do the same in Tinderbox, but some elements of MindNode lent themselves to this kind of departure board model much better.

DT excels at document grouping, management, tagging and replication; it’s a perfect fit for this kind of discovery and analysis exercise alongside a more fluid and visual tool like MindNode, TB or iThoughts (which I used for a while, but went MindNode thanks to their outliner). It’s why I don’t care too much for the demands for nodal interfaces inside DT - I don’t think they automate well, and there’s so much more richness that can be achieved using two tools that are best in their areas, with a fluid coupling between them.

[uimike]

Heavy duty subject
I will try MindNode!
(very grateful)

[uimike]

I installed MindNode and like it a lot - thanks! I think it will be useful, with outline and better formatting capabiities.