Updated Stack script V1.0: Create stack of multiple cards/snippets for each document with backlink and Wiki-group link, cited text or blank note, comment, tagging by ordinary tag and group tag

[ngan]

Most of the “partial” discussions are here:DT3 Script: Create stack for each document for multiple cards/snippets with backlink, cited text or note, comment, and tagging & perhaps a synergy with the Bookend script so I won’t repeat myself.

The final version has two scrips, one for initial or re-configuration of the stack, another one is the main script. You probably don’t need to touch the two scripts.

EDITED 2019.07.29
Changes from v0.96 to v1.0

• ParentTag" is no longer necessary, constrained tag list is now chosen from “/Tags”
• manual entry as one of the options for naming the card
• use “current paragraph” as the back link for text-based source document
• up to 3 ordinary tag groups for constrained tagging
• group tag for tagging
• link of document-stack is added to the card
• WikiLinks can be added as additional default group links to the card
• new sorting algorithm to improve the performance on sorting long tag list
• a complimentary script “annotationV1” that creates replicants of the main annotation note in both inspector bar of the source document and in its document-stack.

Credits to (1) “stack” is a name mentioned by @Cali310 in OP and it does recall my good-old memory of using index card note-taking system in very young age, so I decided to use that name for the script. (2) @korm is the one who suggests me to make it easier for users to configure the script w/o needing to open the script. (3) @cgrunenberg suggests to re-consider using location as pointer of stack. Turns out (2) and (3) make the script much easier to use and make me aware of the possibility of local and global stack.

EDITED 2019.07.29
– Before runing this script, user needs to
– (1) create a group named “Stack” at the root of the Database.
– (2) create the two required custom meta data fields
– “mdcardnum”, datatype is “Integer”
– “mdstacklink”, datatype is “URL”

Updated Demo: 1st post for first time setup, 2nd posts shows how cards are created based on the configuration in the 1st post, and 3rd post for the v1.0 scripts.

(Database preparation before running the script “StackV1.0 Setup”)
A test database with a group “Stack” and one literature .
I also have two other groups, “Annotations” with an aliases of “AllAnnotations”, and “AllResearchNote” in the test database.

Create two custom metadata fields, the identifier’s name and data type must be exact.

I have copy some of the tag groups I use into this testDB

(Run “StackV1.0 Setup”)
You only need to run this script for first-time setup, or when u want to re-configure how the cards are created. I suggest to put the setup script under the script menu.

A reminder to make sure that you are initialising the stack in the correct database.

Choose Card Type, choose “C-MostlyCitedText” if you cite and take note most of the time.

Choose naming method for the card

Choose the unique card identifier counter. 3 means 1 to 999 card for each document stack

How to display the back-link

How to mark your cited text in source document. If you want custom highlight, u need to assign the colour to “shift-ctrl-7” in MacOS preference->keyboard->App Shortcuts.

Choose whether you want put a quick summary in the comment field at creation.

Choose which font to use in the card.

Choose Tagging method, most of the options are self-explanatory, the more special options are “C1” and “CA”.

If C1 or CA is chosen, choose up to three tag groups

Choose whether to tag with group tag

Choose whether to place WikiLink for additional group-links (a more flexible way to view other groups in place)

Where to show the newly created card on the screen at creation.

Resolution of your main monitor, for card position. Resolution info is auto-filled for one-monitor setup. You will have to change the data to the resolution of your main monitor for two monitor setup.

The config info is saved under the comment field of the group “Stack” - DON’T TOUCH IT unless you know how to read the main script.

END OF SETUP

EDITED 2019.07.29
After you test-run the script, if you want to reset the stack and test again, you need to (1) delete all document-stacks, and (2) empty all data in the two metadata fields “StackLink” and “IdxCard” (3) Empty the trash.

[ngan]

EDITED 2019.07.29: New Demo based on the configuration in Post#1

I attach the script “Stack V1.0” as a button on the toolbar. I select some text on my document and click on the button.

Enter a card name, or use the full name as default. A unique identifier will be added to the name.

Enter the comment.

The card is created. A list from three tag groups (separated by space) in the database is shown, multiple or no selection is allowed by cmd-click.

A card is created, displayed on the middle of the screen, and tagged. The first two links are WikiLinks, so you can switch the links later by search and replace, or by changing the name/aliases of the relevant groups in the database. The links for stack and source-document are “hard” item-link.

Another card creation, this time I haven’t selected any text. I am being asked whether to take blank note or to quit.

Blank note created

I created three cards in total and they are located in the document-stack (the doc-stack is created automatically when you create the first card). The doc-stack and the source document are cross-linked by the md field “StackLink”

The md filed “Idxcard” of the source document shows that 3 cards are created.

END OF DEMO

EDITED 2019.07.29:
The potential of stack is not about the multi-cards but about how to tag and connect specific ideas in a more targeted/precise manner. It is always more precise to tag a specific block of concept, facts, hypothesis, variables for analysis, and evidence. Similar snippets from different literature can be group together by smart group using tags and consolidate into a one-document summary. For example, I tag “R.WhatIs” and “T.Legitimacy” on multiple cards from multiple literature. I can group them into one smart group (tag is “R.WhatIs” and “T.Legitimacy”) to understand, merge the most relevant cards within and quote and write how a wide body of literature are conceptualising legitimacy. Stack is just a tool, it’s the workflow of ideas that capitalises its value.

[ngan]

THE SCRIPTS

In the process of writing this script and through the advice and discussions from/with a few forum members, I found that I can create two very different types of Top Stack.
(1) Local Stack. Each database can hold one top stack and gather all document-stacks from the source doc in the database.
(2) Global Stack. For advance user only. A centralised top stack can be placed in “A” chosen database. The top stack will/should be able to gather all document-stacks from all source documents in any database. The constrained list can be applied universally to all database (the chosen tags are duplicated to other database in the process of tagging). BUT, this approach is using uuid addressing method. The pros is you can name the stack anyway and place it anywhere you like. The cons is most DT programmers would say this is uncommon for user scripts, and is more risky for most users even they know scripting. AND, I havn’t fully tested the script on multiple database.

In conclusion: the default config, which you cannot choose in the setup process and definitely should not change in the two scripts, is “Local” stack.

I’ll be getting back to my work next week. So, enjoy at your own risk, take control to debug the scrips (I see none - at least for my own database), enhance the feature and level of user friendliness of the script, and share the codes to this forum. Make DT3 an interesting platform for all levels of users.

EDITED: If you are testing the script, and want to delete the document-stack/s and test again, make sure to clear the two custom meta data fields from each source document that you have the document-stack deleted. Otherwise, the script will always be looking for the document-stack link from the “StackLink” field of the source document, which is likely pointing to the document-stack in the trash (if u haven’t trashed the document-stack) or pointing to nowhere (if you have trashed the document-stack and u haven’t clear the md fields, the script will give u error).

Cheers

EDITED 2019.07.29

The scripts “StackV1.0 and StackV1.0 Setup”
StackV1.0.zip (3.5 MB)

The complementary script: “AnnotationV1”.
Basically, what the script does is (1) to create a new annotation note from the annotation.rtf template and generate the same links as stack (you need to change the script if you want to change the WikiLinks in this case), then put this main annotation note into the inspector bar of the source document, and replicate itself in the document stack. If doc-stack does not exist for the source-doc, the script will create a doc-stack and the stack will be shared by the stack script. (2) If an annotation note already exists, the script simply open the annotation in a document window.
AnnotationV1 (Public).scptd.zip (43.1 KB)

EDITED 2019.07.30
Another “AnnotationV2” that only works with StackV1. The main difference between V1 and V2 is on (2) if an annotation already exists, the script will open the annotation, create a doc-stack if it does not exist under the group Stack (lots of documents may already have linked annotation before using the Stack), and put a replicant of the existing annotation note in the stack.
AnnotationV2 (Public).scptd.zip (41.9 KB)

We all have different preference re how we want the main annotation note of a literature to look. I can only show you how I use AppleScript to integrate the main annotation in inspector bar into the document-stack. You will need to modify the annotation template and the script for populating your specific placeholders.

[ngan]

Some implications of the stack’s structure:

(1) Given the way DT is structured (the URL custom md field is common to all items once created) and how this script creates notes and link with document-stack, user can create card–stack for each card under the document-stack. So, you can have many level of stacks (e.g. Document-stack> Level 1 card-stack -> level 2 card-stack … level n card-stack) BUT all those levels will be compressed to one level of groups under the top level stack. I have no idea who’ll need multi-level stacks but only to suggest the possibility.

(2) Users can use the document-stack as note-archive system. If multiple annotation notes on the same document have been taken previously, those notes can be placed under the document-stack and become easier to re-call.

(3) Those who knows scripting will know how to use different name or location for the top stack and the parentTag by changing just two lines in each script for “local stack” setting.

[brookter]

I won’t have time to play with this for a bit, but I did want to say that it looks really interesting — thanks very much for taking the time to provided the scripts and document them so thoroughly!

[mbbntu]

@ngan

PatentTag

Just to let you know I think this is a typo – should be ParentTag, if I’ve understood correctly. I’m afraid I haven’t had time to look at things properly.

[ngan]

Thanks!

[Cali310]

@ngan I admire your diligence and imagination in both conceiving of and bringing this project to a reasonable conclusion within a relatively short timeframe.
If you’ll permit me some meta-comments on the project itself I would note the following:

1. The dialogic production of the script; you integrated ideas and code generated as part of the discussion on these forums
2. The public development of the script; you talked through the various iterations on the forum and responded to feedback
3. The exemplary explication of the script; you provided: a) the script b) screenshots c) verbal walkthrough
4. The open-ended implementation; you provided a degree of customization on the user’s part that allowed the script to have several possible outcomes

One might observe, finally, that these scripts are also a form of “discourse.” A dynamic mediation between the program and the user.

In my view this is a model project. Thank you again for sharing the development process and the script with us.

[ngan]

v0.96 updated in the 3rd post.
Just to add a few features to close all the gaps I could think of. I will freeze the design now. Hopefully we’ll be seeing the property “current line” in DT3 script dictionary by the time of its final release.

Enjoy

– V0.96 (Public) additions:
(1) two-way link between source document and document-stack. The source document and its stack can now calling each other by launching the item-link in their meta data field “StackLink”
(2) two additional user configurable options:
back link format: “SL”- back link in simple format with source document type and link info, “FL”- full document name is displayed as back link
cited-text highlight style: “U” - underline the cited text, “D”- highlight the cited text with default colour, “C” use “ctrl-shft-7” to assign a specific highlight colour. Users need to assign custom shortcut in MacOS.

[ngan]

I have updated the script to V1 and edited the content in 1st, 2nd, and 3rd post.

I try to code as clean as possible. The stack may not be your cup of tea, but the block of applescripts might still be of some value.

[ngan]

Some thoughts:

The potential of Stack is not about the multi-cards but about breaking down a document into specific ideas and tag them with a more targeted/precise manner. Such characteristic enhance our ability to connect/consolidate similar ideas. Stack is just a tool, it’s the workflow of ideas that capitalises its value.

(1) It is always more precise to tag a specific block of concept, facts, hypothesis, variables for analysis, and evidence (at least for me). Similar snippets from different literature can be group together by smart group using tags and/or can be consolidated into a one-document summary in later stage. For example, I tag “R.WhatIs” and “T.Legitimacy” on multiple cards from multiple literature on an ongoing basis, then I create smart group with predicate: tag is “R.WhatIs” and “T.Legitimacy” scope is Stack. I can scan the different views re one idea in one central location (the smart group). I can merge the most relevant cards into one document within the smart group, using it as a main reference to quote and write how a wide body of literature are conceptualising legitimacy.

(2) A stack can also be viewed as a “container” of ideas related to literature (or something). If group-tagging is enabled in the database, the user can group-tag cards from other doc-stacks or any items into a related document-stack. In a sense, it’s almost like a primitive form of the Hook app (mentioned in OP). You can establish a container for all misc bits about “a thing”. Obviously, DT3 can already do the same without this script. BUT, the the automatically created cross-linking mechanism between the sources and the cards (by md fields and links within each card), and with cards that contain more specific info than the whole document, may make the task easier. EDITED 2019.07.31: This is a “one-way” container concept. A document (or a thing/an item) has a container to hold every items (in the form of replicant) other than just cards, assigned by user, that are related to itself.

(3) As each card within a document-stack can also have its own stack, Stack can establish multi-level linkages (but presented as a flatten groups structure under the group “Stack”). An imaginative idea will be turning (1)+(2)+(3) into a primitive and limited form of Tinderbox or theBrain app. Personally, I never like apps that are based on, or try to mimic, cognition mapping. Given my minimal knowledge, most commercially available software can’t establish the organic/fuzzy nature, and the ad hoc creation and partial retention, of connected cognitive schemata like human brains. All schemata in s/w are created by non-flexible hard-links - which means it’s “extremely tiring” to maintain and update the linkage if such tasks are solely performed by human beings (perhaps that’s why we need AI). But it is a possibility. DT creates ad hoc linkage by classification and see also, but AFAIMA the linkage is established by word usage instead of concepts/interpretation. EDITED 2019.07.31: This is a “two-way” containers concept. If itemA is linked with itemB, itemB is in itemA-Stack and item-A is in itemB-Stack in the form of replicants.

Disclaimer1: I am just starting to change my workflow to test the above “theory” in (1) and some of (2), and probably never on (3), the experiment may be ending as another “hard landing” of good ideas with bad execution!
Disclaimer 2: In fact, I am just using a new bottle for the old and wise wine from the previous scripts “Annotate w links/Notes/tags/ V2 and V3” from @Korm and @Frederiko. BUT, I am trying to re-interpret the implications of cards/snippets perhaps both in a more specific context of research and research writing and a more elaborated context of connected-containers of ideas.

[ngan]

A poof of concept for using document-stack as the one-way or two-way container of relevant info.

Demo on two-way stack-link by using script “StackLink”:

The same database, two cards and a main annotation of a document are created in the doc-stack of P.339.

Let’s say I find DocA and DocB are related to Document “P.339”. I select DocA and DocB, I click on the button 'StackLinkV0.2" to start the connection.

A list is showing all available document within a pool of documents for linkage. A default group of documents can be set, or be chosen on ad-hoc basis by group selector (configure in the script). I choose P.339 as the designated document for connection.

Since I choose two-way connection as the option in the script, the doc-stacks for DocA and DocB are created automatically. You can see that DocA and DocB are replicated in the stack of P.339, and P.339 is replicated into the stack of DocA and DocB.

If a one-way connection is configured (DocA and DocB are relevant to P.339 but not necessarily the other way round), then no stack for DocA and DocB will be created.

Each doc-stack can be opened by launching the “StackLink” from the inspector bar of each document. Or, by a simple script (the button oStackV1.0) that avoids the hassle for switching the inspector bar.

In sum
(1) The concept of using stack and doc-stack as containers of all associated info is readily supported by the script dictionary of DT3. One-way (-- >) or two-way (< – > ) association between multiple documents or snippets (the cards in the doc-stack) can be established by Stack and StackLink.

(2) The current architecture of DT3 can do everything the script does, by ordinary tags, group tags or smart groups, BUT there are some potential issues/limitations (I could be wrong):
(2.1) The nature of tags and group tags are “cluster of info” or implicitly a “many-to-many” association. A collection of documents by group, tag, or group-tag means that the collection is related (a subject, a project, a concept, an idea, etc), but you might not knowing which/how a document is related to others.
(2.2) Item-links can be used to establish the doc/s-to-doc/s relationship. However, if there are multiple item-links, the current architecture of DT requires the user to copy and paste the links into a separate note (most likely the main annotation note). These tasks involve multiple steps.
(2.3) Depending on the usage, associations among documents may need higher precision. That’s why I start with the Stack project in producing snippets. One-way or two-way association between card/s-to-doc/s or card/s-card/s can establish a more precise connection (I think…)
(2.4) Tag and group-tag in DT are elegant and creative ways of creating a parallel metaphor to groups. BUT, there is the potential problem of endless expansion of tags (a scalability issue?). Imagine we use tags and group-tags for misc classifications as well as the above-mentioned tasks on hundreds or thousands of literature. Many tags will be created, AND imagine what will happen if we use auto-fill for normal tagging. Of course, these can be dealt with by discipline tag organization (such as grouping and naming), but AFAIMA it’s not easy to be disciplined at creating tags and/or tagging.
(2.5) Using doc-stack perhaps will have lesser issues. Doc-stacks are behind the scene, you won’t see or need to know their existence. When you need to callup the association, just open the stack (a doc-stack can be viewed as a collection of item-links), AND you can trace through the stack-links from one doc to another.

Just my 5 cents to sharing my thoughts. I have no intention to suggest others to follow the same methodology on info/research management.

The script on StackLink, and oStack (quicker way to open the doc-stack) are attached here. These scripts are just a proof of concept. You need to know AppleScript to change the configuration, such as one-way or two-way stack-linkage, and whether to use group selector or default location for the designated document for creating the stack link.

StackLinkV0.2.scpt.zip (485.1 KB)
oStackV1.0.scpt.zip (14.4 KB)

[rgthorpe2001]

I am in the early weeks of using Devonthink, and in the course of exploring it saw this script, which is in many ways exactly what I was looking for and what I wish I had had three decades ago when I was juggling 8 shoeboxes full of dissertation notes on note cards. I successfully set up Stack in one of my databases, and today have started testing it with a second. The issue that I have encountered is that, everytime I create a new card, it first generates an error message: “getUUID() has error Can’t get item 1 of {}.” However, when I click the OK button it proceeds forward with the normal dialogs and successfully creates the card. I’m not sure where to start in figuring out why this is happening, and would be grateful for any suggestions.

[ngan]

Please accept my apology. The script is meant to be a kick-starter for on-going development from other members, and it can be buggy. The script is quite complicated and can only be debugged on a user-by-user basis and only by those who know AppleScript well. I am using a version (after many iterations) that is highly customised for my own work at the moment and might consider posting a modified version during this summer holiday.

[rgthorpe2001]

No apology necessary at all - I appreciate all the work that went into this!

[samrose]

I love this script! It is what I have been looking for in my research for years. I was ecstatic to find it here, and I use it constantly. My only problem is that I am not a coder, and I am frequently running into this error “Can’t get «class DTco» of missing value” when I try to create a new stack from a selection. I appreciate that you can’t debug on a case by case basis, but I wanted to check in to see if you happened to have a more recent version, or if you could point me in any helpful directions?

Thank you in advance, and thank you for creating this wonderful set of scripts!