Note - 2020.05.05: Latest version is posted here.
Completion of the previously mentioned proof-of-concept script Proof of concept: Merging a mixture of RTF/RTFD/MD files to MD and with links
What the script does:
- Merge markdown and RTF/RTFD formatted files into a merged file (MV) that is in markdown format.
- User can access and edit each original source file directly[almost] within the MV.
- The MV can be refreshed to reflect the latest change in the source files.
- If the MV is linked to one/more groups or tags, the MV can be refreshed to include the content of newly added files in those groups/tags.
- Possible applications: e.g.1 consolidate related notes by tags/groups and maintain dynamic linkage for new content and new files in tags and groups. e.g. 2 simple writings by breaking down the writings into chapters and sections see basic example here.
Demo:
- Some notes are selected. * I suggest attaching the script to a button.
Note:
- The selection doesn’t need to be file-only. If a selection includes both files and groups, all of the files in the groups and sub-groups will be merged into the MV.
- The script asks for options:
Section 1 is to choose whether to use the selected items or to choose the parent group of the selected files, or the smart groups in the database for the MV.
Section 2 is to choose the order of sections in the MV. Default means using the order of files that are displayed in DT.
Section 3 is to ask for adding tags, name, and separator to the end and beginning of each section. If the user needs direct editing of source files within the MV, the option of “Add [[Name]]” must be checked.
Note
- If the user wants to merge the files but maintaining the hierarchical order of information for items in groups and sub-groups, choose “Default”.
- If sorted by “Modification/Creation Date” and by “Name” is chosen, all files within the selection (files and files in groups/sub-groups) will be sorted all-together.
- User may want to link the MV to the parent group of the selection so that the MV will always include the future newly added files.
Note:
- The parent group of the selection is automatically listed as the first item in the dropdown list, the other groups are all smart groups in the current database.
- If the selection is the result of the search, there will be no parent group and only the smart groups are listed.
- The MV is created. Noted that there are both RTF and markdown files in the MV. The formats of the two types of files look similar without needing the user to convert the file format. The arrows point to the name/link of each section’s source notes.
- Click the link and the MV will switch to each source file (DT’s standard function), and the user can edit the source files directly. The images show that the format of one source file being in markdown and the other in RTF. The user can just click the back button to return to the MV.
P.S. Can MMD6 do this mixed format linkage?
- How to refresh the MV to relfect the latest change.
The MV file must be the frontmost document window and the user just need to click on the script button again.
Note: The user can change the name of the MV after its creation and place it anywhere in DT without affecting the MV’s refresh function.
-
Some hidden metadata is added to the heading of the MV.
-
If the parent group option is checked, or if the selection contains groups, the MV will always include the content of all files (md,rtf,rtfd) in those groups - after refresh.
Note: The groups/tags can have any type of documents (pdf, image, etc) but the MV will only merge the content of RTFD/RTD/markdown files.
The script
To run the script:
- You need to download “Dialog Toolkit Plus.scptd” and save it under “/Library/Script Libraries” The download link here Dialog Toolkit Plus.
- A group named “MergeView” must be created at the root of database. All the MVs are saved under "MergedView. But you can move the MV files around after the creation and they will still be refreshable.
use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions
use script "Dialog Toolkit Plus" version "1.1.0"
-- by ngan 2020.04.28
--v1b5 use QsortRecsByName,QsortRecsByMod,QsortRecsByCreate for speed
-- v1b4 add metadata at the heading of MV file
-- v1b3 working version: add selection dialog box and refresable merged view
property MVGpLocation : "/MergeView"
property MVNameFormat : "YMDHNS"
property sortByOptLabel : {"Default", "Modification Date", "Creation Date", "Name"}
-- don't change this
property rtfHeader : "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\" \"http://www.w3.org/TR/REC-html40/loose.dtd\">"
global useSelOrGp, theGps, theGpsUUID, exSubGp, sortBy, sortOrder, addTags, addName, addSeperator
global theRecords, theRecordsCount, theMVContent, theRecordName, eachRecordTags, theSeparator
global newMV, theMV, theMVName, theMVContent, theRTFSource
global rtfCount, mdCount
global gpPopupList, gpPopupLabel
-- Main --
tell application id "DNtp"
-- check whether the script is to create a new MV or refresh MV
if class of think window 1 is viewer window then
if (count of selection) is not 0 then
-- call dialog box to get parameters
set newMV to true
set theParameters to my getMVOpt()
if theParameters is "Cancel" then return
else
display alert "Select at least one item"
return
end if
else if class of think window 1 is document window then
set {theMV} to item 1 of {selection}
set theMVcomment to comment of theMV
if (theMVcomment is "") or (texts 1 thru 2 of theMVcomment is not "MV") then
display alert "This is not a merged file" giving up after 2
return
else
set newMV to false
set theParameters to my strToList(theMVcomment, ",")
-- get praameters from comment of the file
set useSelOrGp to "S"
set sortBy to theParameters's item 2
set sortOrder to theParameters's item 3
if theParameters's item 4 is "True" then set addTags to true
if theParameters's item 5 is "True" then set addName to true
if theParameters's item 6 is "True" then set addSeperator to true
set theGpsUUID to items 7 thru -1 of theParameters
set theGps to {}
repeat with each in theGpsUUID
set end of theGps to get record with uuid each
end repeat
end if
end if
-- prepare records for merged view
set theRecords to my getAllChildren(theGps)
set {rtfCount, mdCount} to {0, 0}
set theRecordsCount to length of theRecords
if sortBy is "N" then -- sort by name option is checked
my QsortRecsByName(theRecords, 1, theRecordsCount)
if sortOrder is "D" then set theRecords to reverse of theRecords
-- set theRecords to my sortRecsByName(theRecords, sortOrder)
else if sortBy is "M" then -- sort by modification date option is checked
my QsortRecsByMod(theRecords, 1, theRecordsCount)
if sortOrder is "D" then set theRecords to reverse of theRecords
--set theRecords to my sortRecsByMod(theRecords, sortOrder)
else if sortBy is "C" then -- sort by creation date option is checked
my QsortRecsByCreate(theRecords, 1, theRecordsCount)
if sortOrder is "D" then set theRecords to reverse of theRecords
--set theRecords to my sortRecsByCreate(theRecords, sortOrder)
end if
-- prepare contents
set theMVContent to ""
repeat with each in theRecords
-- Set name at section's beginning
if addName then
if sortBy is "N" or sortBy is "D" then
set theRecordName to "[[" & each's name & "]]"
else if sortBy is "M" then
set theRecordName to "[[" & each's name & "]]" & " Date Modified: " & my getDateString(each's modification date, "YMD", ".")
else if sortBy is "C" then
set theRecordName to "[[" & each's name & "]]" & " Date Created: " & my getDateString(each's creation date, "YMD", ".")
end if
else
set theRecordName to ""
end if
-- Set tags at section's ending
if addTags then
--get the tags of each record and convert them into a string of wikilink in [[]] format
set eachRecordTags to name of (parents of each whose tag type is ordinary tag)
repeat with i from 1 to length of eachRecordTags
set eachRecordTags's item i to "[[" & (eachRecordTags's item i) & "]] "
end repeat
if eachRecordTags is not {} then
set eachRecordTags to my listToStr(my sortlist(eachRecordTags), " ")
else
set eachRecordTags to ""
end if
else
set eachRecordTags to ""
end if
--Set separator
if addSeperator then
set theSeparator to "---"
else
set theSeparator to ""
end if
--prepare the content of theMV
if type of each is in {rtf, rtfd} then
set theRTFSource to my findAndReplaceInText(source of each, rtfHeader, "")
set theMVContent to theMVContent & theRecordName & return & theRTFSource & return & return & eachRecordTags & return & return & theSeparator & return & return
set rtfCount to rtfCount + 1
else if type of each is markdown then
set theMVContent to theMVContent & theRecordName & return & return & plain text of each & return & return & return & eachRecordTags & return & return & theSeparator & return & return & return
set mdCount to mdCount + 1
else
set theRecordsCount to theRecordsCount - 1
end if
end repeat
--add metadate to the heading of theMV
set theMVMetadata to "Last Refreshed: " & (my getDateString(current date, MVNameFormat, ".")) & return & "Total records: " & theRecordsCount & return & "RTF/RTFD files: " & rtfCount & return & "Markdown files: " & mdCount
set theMVContent to theMVMetadata & return & return & theMVContent
if newMV then
-- name and create theMV
set theMVName to "MV - " & (my getDateString(current date, MVNameFormat, "."))
set theMV to create record with {name:theMVName, source:theMVContent, type:markdown} in (get record at MVGpLocation)
-- save parameters to comment of MV
set comment of theMV to my listToStr({"MV", sortBy, sortOrder, addTags, addName, addSeperator, theGpsUUID}, ",")
open tab for record theMV
else
-- refresh theMV
set the plain text of theMV to theMVContent
display alert "The MV file is refreshed, check metadate for info"
end if
end tell
-- end of main program
--script specific handlers
on getMVOpt()
set {gpPopupList, gpPopupLabel} to my prepareGpsChoice()
-- "Dialog Toolkit Plus" coding
set accViewWidth to 300
set theTop to 8
set {theButtons, minWidth} to create buttons {"Cancel", "OK"} default button 2 given «class btns»:2
if minWidth > accViewWidth then set accViewWidth to minWidth
set {theRule3, theTop} to create rule (theTop - 8) rule width accViewWidth
set {addSeperatorOpt, theTop, newWidth} to create checkbox "Add seperator" bottom (theTop + 8) max width accViewWidth / 3 - 8 with initial state
set {addNameOpt, theTop, newWidth} to create checkbox "Add [[Name]]" bottom (theTop + 8) max width accViewWidth / 3 - 8 with initial state
set {addTagOpt, theTop, newWidth} to create checkbox "Add [[Tags]]" bottom (theTop + 8) max width accViewWidth / 3 - 8 with initial state
set {theRule2, theTop} to create rule (theTop + 12) rule width accViewWidth
set {sortOrderOpt, theTop, newWidth} to create checkbox "Descending (Uncheck = Ascending)" bottom (theTop + 8) max width accViewWidth / 3 - 8 with initial state
set {sortByOpt, theTop} to create matrix sortByOptLabel bottom (theTop + 8) max width accViewWidth initial choice 1 --without arranged vertically
set {theRule1, theTop} to create rule (theTop + 12) rule width accViewWidth
-- set {exSubGpOpt, theTop, newWidth} to create checkbox "Exclude sub-groups" bottom (theTop + 8) max width accViewWidth / 3 - 8 with initial state
set {gpPopup, theTop} to create popup gpPopupLabel bottom (theTop + 8) popup width accViewWidth initial choice 1
set {selOpt, theTop} to create matrix {"Use selection", "Use parent group or smart group"} bottom (theTop + 8) max width accViewWidth initial choice 1 --without arranged vertically
set {theRule0, theTop} to create rule (theTop + 12) rule width accViewWidth
set {boldLabel, theTop} to create label "For markdown, rtf, rtfd files" bottom theTop + 20 max width accViewWidth control size large size -- aligns center aligned -- with bold type
-- set allControls to {theRule3, addSeperatorOpt, addNameOpt, addTagOpt, theRule2, sortOrderOpt, sortByOpt, theRule1, exSubGpOpt, gpPopup, selOpt, theRule0, boldLabel}
set allControls to {theRule3, addSeperatorOpt, addNameOpt, addTagOpt, theRule2, sortOrderOpt, sortByOpt, theRule1, gpPopup, selOpt, theRule0, boldLabel}
set {buttonName, controlsResults} to display enhanced window "Merge View" acc view width accViewWidth acc view height theTop acc view controls allControls buttons theButtons with align cancel button
-- end of "Dialog Toolkit Plus" coding
if buttonName is not "Cancel" then
tell application id "DNtp"
if controlsResults's item 10 is "Use parent group or smart group" then
-- get the selected gps
set useSelOrGp to "G"
set theGps to item (my indexOfOneItem(controlsResults's item 9, gpPopupLabel)) of gpPopupList
else
-- get the selection
set useSelOrGp to "S"
set theGps to selection as list
end if
set theGpsUUID to {}
repeat with each in theGps
set end of theGpsUUID to (get uuid of each)
end repeat
set sortBy to character 1 of (controlsResults's item 7)
if controlsResults's item 6 then
set sortOrder to "D"
else
set sortOrder to "A"
end if
set addTags to controlsResults's item 4
set addName to controlsResults's item 3
set addSeperator to controlsResults's item 2
end tell
return {useSelOrGp, theGps, theGpsUUID, sortBy, sortOrder, addTags, addName, addSeperator}
else
return "Cancel"
end if
end getMVOpt
on prepareGpsChoice()
local lg, lgn, a, b
set lgn to {}
tell application id "DNtp"
set lg to every smart group of current database
-- set lg to my sortRecsByName(lg, "A")
set lgre to length of lg
my QsortRecsByName(lg, 1, lgre)
if name of current group is not equal to name of current database then set the beginning of lg to current group
if lg ≠ {} then
repeat with each in lg
set end of lgn to name of each & " «" & location of each & "»"
end repeat
return {lg, lgn}
else
return {{}, {"Parent is database and no smart group"}}
end if
end tell
end prepareGpsChoice
on getAllChildren(theseGps)
local l
set l to {}
tell application id "DNtp"
repeat with each in theseGps
if each's type is group or each's kind is "smart group" then
set l to l & my getAllChildren(children of each)
else
--«constant ctxt»
if each's type is in {rtfd, rtf, markdown} then set end of l to each's item 1
end if
end repeat
end tell
return l
end getAllChildren
on QsortRecsByMod(array, leftEnd, rightEnd)
-- based on Hoare's QuickSort Algorithm
-- modified by ngan for records sorting in DT
tell application id "DNtp"
script a
property l : array
end script
set {i, j} to {leftEnd, rightEnd}
set v to modification date of item ((leftEnd + rightEnd) div 2) of a's l -- pivot in the middle
repeat while (j > i)
repeat while (modification date of (item i of a's l) < v)
set i to i + 1
end repeat
repeat while (modification date of (item j of a's l) > v)
set j to j - 1
end repeat
if (not i > j) then
tell a's l to set {item i, item j} to {item j, item i} -- swap
set {i, j} to {i + 1, j - 1}
end if
end repeat
if (leftEnd < j) then my QsortRecsByMod(a's l, leftEnd, j)
if (rightEnd > i) then my QsortRecsByMod(a's l, i, rightEnd)
end tell
end QsortRecsByMod
on QsortRecsByName(array, leftEnd, rightEnd)
-- based on Hoare's QuickSort Algorithm
-- modified by ngan for records sorting in DT
tell application id "DNtp"
script a
property l : array
end script
set {i, j} to {leftEnd, rightEnd}
set v to name of item ((leftEnd + rightEnd) div 2) of a's l -- pivot in the middle
repeat while (j > i)
repeat while (name of (item i of a's l) < v)
set i to i + 1
end repeat
repeat while (name of (item j of a's l) > v)
set j to j - 1
end repeat
if (not i > j) then
tell a's l to set {item i, item j} to {item j, item i} -- swap
set {i, j} to {i + 1, j - 1}
end if
end repeat
if (leftEnd < j) then my QsortRecsByName(a's l, leftEnd, j)
if (rightEnd > i) then my QsortRecsByName(a's l, i, rightEnd)
end tell
end QsortRecsByName
on QsortRecsByCreate(array, leftEnd, rightEnd)
-- based on Hoare's QuickSort Algorithm
-- modified by ngan for records sorting in DT
tell application id "DNtp"
script a
property l : array
end script
set {i, j} to {leftEnd, rightEnd}
set v to creation date of item ((leftEnd + rightEnd) div 2) of a's l -- pivot in the middle
repeat while (j > i)
repeat while (creation date of (item i of a's l) < v)
set i to i + 1
end repeat
repeat while (creation date of (item j of a's l) > v)
set j to j - 1
end repeat
if (not i > j) then
tell a's l to set {item i, item j} to {item j, item i} -- swap
set {i, j} to {i + 1, j - 1}
end if
end repeat
if (leftEnd < j) then my QsortRecsByCreate(a's l, leftEnd, j)
if (rightEnd > i) then my QsortRecsByCreate(a's l, i, rightEnd)
end tell
end QsortRecsByCreate
-- utility handlers
on indexOfOneItem(theItem, theList)
-- credits Emmanuel Levy
set {oTIDs, AppleScript's text item delimiters} to {AppleScript's text item delimiters, return}
set theList to return & theList & return
set AppleScript's text item delimiters to oTIDs
try
-1 + (count (paragraphs of (text 1 thru (offset of (return & theItem & return) in theList) of theList)))
on error
0
end try
end indexOfOneItem
on getDateString(theDate, theDateFormat, theSeperator)
tell application id "DNtp"
local y, m, d, h, n, s, T
local lol, ds
set lol to {{"y", ""}, {"m", ""}, {"d", ""}, {"h", ""}, {"n", ""}, {"s", ""}}
set (lol's item 1)'s item 2 to get year of theDate
set (lol's item 2)'s item 2 to my padNum((get month of theDate as integer) as string, 2)
set (lol's item 3)'s item 2 to my padNum((get day of theDate) as string, 2)
set T to every word of (get time string of theDate)
set (lol's item 4)'s item 2 to T's item 1
set (lol's item 5)'s item 2 to T's item 2
set (lol's item 6)'s item 2 to T's item 3
end tell
set ds to {}
set theDateFormat to (every character of theDateFormat)
repeat with each in theDateFormat
set ds to ds & (my lolLookup(each as string, 1, 2, lol))'s item 2
end repeat
return my listToStr(ds, theSeperator)
end getDateString
on padNum(lngNum, lngDigits)
-- Credit houthakker
set strNum to lngNum as string
set lngGap to (lngDigits - (length of strNum))
repeat while lngGap > 0
set strNum to "0" & strNum
set lngGap to lngGap - 1
end repeat
strNum
end padNum
on lolLookup(lookupVal, lookUpPos, getValPos, theList)
--only for list of list with more than 1 items
local i, j, k
set j to lookUpPos
set k to getValPos
repeat with i from 1 to length of theList
if (item j of item i of theList) is equal to lookupVal then return {i, item k of item i of theList, item i of theList}
end repeat
return {0, {}, {}}
end lolLookup
on findAndReplaceInText(theText, theSearchString, theReplacementString)
set AppleScript's text item delimiters to theSearchString
set theTextItems to every text item of theText
set AppleScript's text item delimiters to theReplacementString
set theText to theTextItems as string
set AppleScript's text item delimiters to ""
return theText
end findAndReplaceInText
on listToStr(theList, d)
local thestr
set {tid, text item delimiters} to {text item delimiters, d}
set thestr to theList as text
set text item delimiters to tid
return thestr
end listToStr
on strToList(thestr, d)
local theList
set {tid, text item delimiters} to {text item delimiters, d}
set theList to every text item of thestr
set text item delimiters to tid
return theList
end strToList
on sortlist(theList)
set theIndexList to {}
set theSortedList to {}
repeat (length of theList) times
set theLowItem to ""
repeat with a from 1 to (length of theList)
if a is not in theIndexList then
set theCurrentItem to item a of theList as text
if theLowItem is "" then
set theLowItem to theCurrentItem
set theLowItemIndex to a
else if theCurrentItem comes before theLowItem then
set theLowItem to theCurrentItem
set theLowItemIndex to a
end if
end if
end repeat
set end of theSortedList to theLowItem
set end of theIndexList to theLowItemIndex
end repeat
return theSortedList
end sortlist
.
