Table of content automatic update

Hi from London.

I didn’t want to ask a question that has been answered already but I have looked and not found an answer. Apologies if I missed something.

Create table of content smart rule that that would update table of content on hourly basis. I managed to pull it of with a script but cannot get it alphabetically sorted. (Johnny Decimal).

Someone smarter and more experienced please help.
Thank you

1 Like

Perhaps you could elaborate. Without the smart rule, information on the document format, and a sample TOC it’s difficult to help.
Complete sentences are also helpful for making others understand.

1 Like

I also use a script to create ToC notes
Sorting is handled within the script
Please provide your script code and examples of the names being sorted

I am trying to implement Johnny Decimal system with DEVONthink. New to both.

I have created folder structure (Screenshot attached) and now I want to create an index which is what table of contents would be.
The only thing is table would need to be updated automatically.
My first thought was smart rule (screenshot attached) with a script to create TOC.
I have tried couple of scripts I found online but they did not work with smart rule (Nor with script run as apple script nor from smart rules folder). Chat GPT got me so far as to create a script that worked with smart rule but TOC is not alphabetical. (screenshots of script and TOC attached.
I found a workaround with Keyboard Maestro (screenshot attached), but now I need to create another smart rule in DEVONthink which would be deleting the old TOC as not to have duplicates before the new one is created so I need another solution without KM.

@chrillek Hope that is elaborated enough :slight_smile:




So, you don’t create a table of contents in the traditional sense but a listing of file names with their URLs. I don’t see the point of KM here, though.

And if I understand you correctly, you want your “TOC” sorted by – well, what exactly? The full name of the file? The name, ignoring the leading decimal stuff?

I suggest something along the lines below. It’s written in JavaScript, as I don’t bother with AppleScript. But I’m sure it can be done in AS, too.
Anyway, the script creates a TOC sorted by record name, ignoring upper/lowercase. If you want to sort by name, ignoring the decimal stuff, the regular expression in the sort function must be changed.

const app = Application("DEVONthink 3");
const TOCName = 'Table of Contents';
const TOCGroup = app.currentGroup();

function performsmartrule(records) {

  /* Create an array of TOC entries from the records' names and URLs */
  const TOCEntries = records.map(r => `- [${r.name()}](${r.referenceURL()})`);
  
  /* Sort the array by record name */
  const sortedTOCEntries = TOCEntries.toSorted((a,b) => {
    aName = a.match(/\[(.*?)\]/)[1].toLowerCase();
	bName = b.match(/\[(.*?)\]/)[1].toLowerCase();
	return (aName < bName ? -1 : (aName === bName ? 0 : 1));
	});
  /* Create a markdown string with a heading and the sorted TOC
     entries line by line */
  const tocMD = `# Table of Contents
${sortedTOCEntries.join('\n')}`;

  /* Get or create the record to store the TOC */
  let targetRecord = TOCGroup.children[TOCName];

/* If the TOC record exists, the try block succeeds. Otherwise, 
   it fails, and the catch block creates the record
*/
  try {
    let ignore = targetRecord.name();
  } catch (e) {  
    targetRecord = app.createRecordWith({name: TOCName, type: 'markdown'}, {in: TOCGroup});
  }

   /* Set the content of the TOC record to the sorted list of TOC entries */
  targetRecord.plainText = tocMD; 
}
2 Likes

Welcome @vex

But…

  • What’s the actual point of this document? You’re going to end up with a massive and unmanageable list of links.
  • Why would you need this updated hourly?

I am trying to implement Johnny Decimal system with DEVONthink. New to both.

I would strongly recommend you slow down and step back for a moment. Being new to both systems then trying to create automations and some kind of manifest list is certainly overcomplicating things. You should not try automating things you can’t do manually. And you haven’t even determined if Johnny Decimal or DEVONthink is a good fit. (Also, I would not be relying on ChatGPT code.)

What do you do when things break or something unexpected happens? You’re stuck in the water. Wade, then splash about, then doggy paddle, do laps in a pool, etc.… before you commit to swimming across the English Channel! :slight_smile:

1 Like

I reviewed your script and see no code to sort your data

I have done an extensive research on both. Have used Johnny Decimal system before switched to PARA which never worked for me and decided to go all in and do it properly and automate it.
The point of the document if nothing else is easier overview of the whole inventory, index to use as main navigation hub, links to copy to separated notes and to AnyBox for contextual computing.
I can do it manually why couldn’t I? Copy deep link or use hookmark to create a single note of index. Or in this case just create a new Table of content manually?
I do not see your point of ruining the party, having an index update as I add data automatically is just a thing of beauty, so why not do it? :slight_smile:

That is because none of the later scripts that had data sorting in them worked. And there were plenty of them so I did not include them. I always got the -Error -1708 in Log.

Thank you! Really appreciate the help. Gonna try the script first thing in the morning when back at my mac and let you know how it went. :pray:
And well decimal in the filing system name I think is self explanatory for how should sorting go :slight_smile:

It’s not ruining anything. It’s managing expectations and suggesting you step back and look at the situation. Have you looked at smart groups? They could easily function in the way your document could. And your document wouldn’t provide any more efficient use in getting links to other apps, etc. A smart group would.

I think there is no point in creating a smart group in this scenario. If you take a look at my main folder I think a smart group would end up looking exactly the same? Or am I missing something? I have a folder (database with folders) sorted, the point is creating an index. A document with links to the folders. That can then be extracted and used from elsewhere for easier access. The way Johnny Decimal puts it on his website might be better explanation then I can provide here. He suggest having an index separated from the folder structure. He uses bear and each decimal has a separated note with deeplinks. I am not inteding to do the same but it is a good example of what I am trying to achieve.

A smart group doesn’t contain anything. It only shows matching items. So if the criterion was Kind is Any Document, it would only show documents. And if it was sorted by Name, it would follow your naming convention. You could also sort by Date Added/Created if you wanted to access newer or older items easily.

Yes, it’s a saved filtered search, I am aware of that. I see your point and smart groups can be very useful, but that is not what I am trying to achieve at this point.
After I am done with indexing Johnny Decimal I will use some of OmniFocuses logic with smart groups for managing project related stuff. As you said step by step. Without the basics (a system in place) no point doing much else. :slight_smile:

Maybe it’s just me
If I was using this script, and I want my data sorted,
I would add sort code to the script
You don’t seem to be interested in doing this

1 Like

To me, it looks over-engineered. Far too many groups. And what is the point in having all these groups if you then need an “index” of them? Which DT provides anyway in the form of a list that can be sorted.

1 Like

As I said, my knowledge of scripting isn’t the greatest. Without GPT, I couldn’t do it. That is why I am here asking for advice. That said, here are some scripts I have tried.

on sortRecordsNumerically(recordList)
tell application id “DNtp”
return (sort recordList by name)
end tell
end sortRecordsNumerically

on performSmartRule(theRecords)
tell application id “DNtp”
try
– Initialize the TOC content
set tocContent to “# Johnny Decimal Index\n\n”

        -- Process top-level records
        repeat with thisRecord in theRecords
            set tocContent to tocContent & my processRecord(thisRecord, 0) -- Start at depth 0
        end repeat

        -- Define the TOC name and destination group
        set tocName to "Johnny Decimal Index.md"
        set theGroup to current group
        set tocRecord to missing value

        -- Check if a TOC file already exists
        try
            set tocRecord to (child named tocName of theGroup)
        end try

        -- Create or update the TOC
        if tocRecord is missing value then
            create record with {name:tocName, type:markdown, content:tocContent} in theGroup
            log message "TOC created successfully."
        else
            set plain text of tocRecord to tocContent
            log message "TOC updated successfully."
        end if

    on error errMsg number errNum
        -- Log errors for debugging
        log message "Error " & errNum & ": " & errMsg
    end try
end tell

end performSmartRule

– Recursive function to process records and preserve hierarchy
on processRecord(thisRecord, depth)
tell application id “DNtp”
set indent to (my repeatString(" ", depth)) – Indentation for markdown hierarchy
set recordContent to “”

    -- Add record to TOC
    set theName to name of thisRecord
    set theURL to reference URL of thisRecord
    set recordContent to recordContent & indent & "- [" & theName & "](" & theURL & ")\n"
    
    -- Process children if it’s a group
    if type of thisRecord is group then
        set childrenRecords to my sortRecordsNumerically(children of thisRecord)
        repeat with childRecord in childrenRecords
            set recordContent to recordContent & my processRecord(childRecord, depth + 1)
        end repeat
    end if
    
    return recordContent
end tell

end processRecord

– Simplified sorting function for Johnny Decimal order
on sortRecordsNumerically(recordList)
set sortedList to {}
repeat with aRecord in recordList
set end of sortedList to {item:aRecord, key:(my extractNumber(name of aRecord))}
end repeat
set sortedList to my sortList(sortedList)
return (item of sortedList) – Return only the sorted records
end sortRecordsNumerically

– Extract the numeric prefix for sorting
on extractNumber(recordName)
set AppleScript’s text item delimiters to “.”
try
return (text item 1 of recordName) as integer
on error
return 999 – Default value if no numeric prefix exists
end try
end extractNumber

– Sort helper for alphanumeric lists
on sortList(aList)
set AppleScript’s text item delimiters to {“”}
set sortedList to (sort aList by key)
set AppleScript’s text item delimiters to {}
return sortedList
end sortList

– Utility function to repeat strings
on repeatString(theString, times)
set resultString to “”
repeat times times
set resultString to resultString & theString
end repeat
return resultString
end repeatString

on performSmartRule(theRecords)
tell application id “DNtp”
try
– Initialize the TOC content
set tocContent to “# Johnny Decimal Index\n\n”

        -- Process top-level records passed by the Smart Rule
        repeat with thisRecord in theRecords
            set tocContent to tocContent & my processRecord(thisRecord, 0) -- Start at depth 0
        end repeat

        -- Define the TOC name and destination group
        set tocName to "Johnny Decimal Index.md"
        set theGroup to current group
        set tocRecord to missing value

        -- Check if a TOC file already exists
        try
            set tocRecord to (child named tocName of theGroup)
        end try

        -- Create or update the TOC
        if tocRecord is missing value then
            create record with {name:tocName, type:markdown, content:tocContent} in theGroup
            log message "TOC created successfully."
        else
            set plain text of tocRecord to tocContent
            log message "TOC updated successfully."
        end if

    on error errMsg number errNum
        -- Log errors for debugging
        log message "Error " & errNum & ": " & errMsg
    end try
end tell

end performSmartRule

– Recursive function to process records and preserve hierarchy
on processRecord(thisRecord, depth)
tell application id “DNtp”
set indent to (my repeatString(" ", depth)) – Indentation for markdown hierarchy
set recordContent to “”

    -- Add record to TOC
    set theName to name of thisRecord
    set theURL to reference URL of thisRecord
    set recordContent to recordContent & indent & "- [" & theName & "](" & theURL & ")\n"
    
    -- Process children if it’s a group
    if type of thisRecord is group then
        set childrenRecords to my sortRecordsNumerically(children of thisRecord)
        repeat with childRecord in childrenRecords
            set recordContent to recordContent & my processRecord(childRecord, depth + 1)
        end repeat
    end if
    
    return recordContent
end tell

end processRecord

– Helper function: Sort records numerically by name
on sortRecordsNumerically(recordList)
tell application id “DNtp”
set sortedList to {}
repeat with aRecord in recordList
set end of sortedList to {item:aRecord, key:(name of aRecord as text)}
end repeat
set sortedList to my sortListNumerically(sortedList)
return (item of sortedList)
end tell
end sortRecordsNumerically

– Generic sorting function for Johnny Decimal order
on sortListNumerically(aList)
set AppleScript’s text item delimiters to {“”}
set sortedList to (sort aList by key)
set AppleScript’s text item delimiters to {}
return sortedList
end sortListNumerically

– Utility function to repeat strings
on repeatString(theString, times)
set resultString to “”
repeat times times
set resultString to resultString & theString
end repeat
return resultString
end repeatString

There is cca 30 more tries where that came from, if you’d like I can send them all to you.

As I said, Johnny Decimal website, all is there.
The key is having a searchable index separately (preferably individual notes) for navigation. You add files to the folders and add to notes indexes, that way you never have a duplicate and every little thing has got it’s place. With a bit of Text expander, Hazel, naming logic/ upfront thinking and smart sorting rules it can work magic. Also, decimals and naming rules make searching for what you’re easy and everything is consistent.
Might be just my OCD :person_shrugging:
Ofcourse general filing reference I do not intend to file that way, too much friction. I am going to use alphabetical approach by looser topics. When the time comes a topic becomes current, I’ll extract it from reference into a smaller research database.
I think it’s a solid plan.
Then link Projects from OmniFocus, related Johnny Decimal index (project support folder) from DEVONthink, Arc space, Obsidian and what not into AnyBox so everything is connected and one click away when you need it. Again, decimals and naming
21.11 PRJ001 - <> - OF
21.11 PRJ001 - <> - DT

The script you gave me works amazing, thank you, it does not sort folders tho, only files, GPT is useless for some reason with this. Any advice on that? Much appreciated and thanks :slight_smile:

DT is searchable. That’s the whole point of it.

I’m afraid I’m out of the age where i believed in magic.

A good searching algorithm makes finding easy. What you describe can be realized with folders in Finder – what do you even need DT for?

The interesting thing, in my mind, is that you don’t talk about what you want to achieve, with what kind of data you’re working, what your goal is. Only about a supposedly magic method that is solving all problems – but what are these problems?

Learn coding. ChatGPT is just magic, and that doesn’t work reliably.

2 Likes

Fair enough. Learning coding is one of the things I plan to use DEVONthink for. I never understood people who just find things to point out. Internet culture. It makes asking for advice when starting out that much more difficult not to feel stupid and get called out.
I see no harm in trying to create get organized and creating a system that will support bigger things like learning coding. Everyone was once learning. I appreciate your help with the script and hopefully will have a bit of your knowledge on this one day. (doubt so but still), :slight_smile: