Script: Finding the true parents for the lost children (misplaced indexed files and indexed groups)

I don’t know about the others, but this happened to me frequently during my early days of using DT:

I indexed a folder that contains many documents, I started to move the documents around to other groups by replication. The problem was, I used “move to” without knowing it sometimes. When I tried to check the number of documents in an indexed group versus the number of files in the linked macOS folder (usually at the year end), I always find mismatches. When that happened, I needed to compare thousands of filenames between a macOS folder with the names of documents in the DT’s indexed group to identify which indexed files in DT had been misplaced. This painful auditing process taught me remembering to always keep a replicant of the files in their original indexed group - but it still happens some times.

This script identifies all misplaced indexed files and indexed groups - files/sub-groups that have moved to other DT locations from their original indexed parent group/s - and offers users a choice to create a replicant of those files back to its original indexed group.

Demo:

  • I have a macOS folder named “Idx Gp 1” with three files.

  • The folder is indexed to DT but the three files within have moved/replicated to many different locations. Therefore, it seems that the “Idx Gp 1” in DT is empty but the “Idx Gp 1” macOS folder still conatins the three files.

  • I use search to find all indexed items in the database and user needs to select the groups/items to be checked by the script.

  • Note 1: The script can only find misplaced indexed files but not misplaced indexed groups. EDITED 2020.04.26 It turns out that the script can also handle misplaced indexed group, too.

  • Note 2: By design, the search and selection of indexed item is not automated. It is because some users may only want to audit the misplacement on some selected indexed files, and some users may have many thousands of indexed files and they don’t want to process the checks all at once.

  • A reminder that only items in one database are checked each time.

  • The report of misplaced files, MIF is the misplaced indexed file identified and TIP is the true indexed parent of the MIF. Three actions are offered: do nothing; make a replicant of the MIF back to its TIP; make replicants of all MIFs to a user-selected group for further evaluation.

  • Log message is generated to keep track of the replications.

REMINDER 1: In some occasions, user may have indexed files into DT directly instead of indexing a macOS folder. In such a situation, the directly indexed files do not have a true indexed parent group - which is correct - and will not be reported.
REMINDER 2: Use a test database to test the script function carefully. The script has quite a few alert boxes because of a process that deals with a lot of records needs to be careful.

The script

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

-- by ngan 2020.04.25
--v1b2 handle misplaced index files only


tell application id "DNtp"
	
	
	set theRecords to selection
	set theMisplacedRecords to {}
	set theDB to database of root of think window 1
	set theScriptName to name of me
	
	-- script only check misclocated files in single database
	if button returned of (display alert "This script can only run for files in single database" buttons {"Cancel", "Continue"} default button 2) is "Cancel" then return
	
	show progress indicator "Find misplaced files"
	
	repeat with i from 1 to length of theRecords
		step progress indicator "Checking file ..." & i & "/" & length of theRecords
		-- get the path of each indexed file
		set theRecordPath to path of theRecords's item i
		--get only the parent-as-group for each record.  if a parent group is also an indexed group, it will have a normal path in folder
		set theParentsPath to (path of parents of theRecords's item i whose tag type is not ordinary tag)
		set theParentsFullPath to {}
		-- Manipulate the path info to prepare for comparisons with the indexed file
		repeat with each in theParentsPath
			set end of theParentsFullPath to each & "/" & (filename of theRecords's item i)
		end repeat
		
		-- if path of an indexed file cannot be found in the path of any of its parents, the file is misplaced 
		if theRecordPath is not in theParentsFullPath then
			-- The true indexed parent should have the same folder path as the misplaced indexed file
			set theTrueParentPath to my findAndReplaceInText(theRecordPath, filename of theRecords's item i, "")
			try
				--Find the true indexed parent - UNLESS the file is indexed directly then theTrueParent will return {} 
				set theTrueParent to item 1 of (lookup records with path theTrueParentPath in current database)
				set end of theMisplacedRecords to {theRecords's item i, theTrueParent}
			end try
		end if
		
	end repeat
	
	
	step progress indicator "Reporting misplaced files ..."
	
	-- prepare the report for misplaced indexed files
	set numMisplacedRecords to length of theMisplacedRecords
	set lostChildReport to ""
	
	if numMisplacedRecords is 0 then
		-- No misplaced indexed file
		display alert "No dislocated indexed file"
		hide progress indicator
		return
	else
		-- prepare the list of misplaced file and suggests three actions
		repeat with each in theMisplacedRecords
			set lostChildReport to lostChildReport & "    MIF: " & (location of each's item 1 & name of each's item 1) & return & "    TIP: " & (location of each's item 2 & name of each's item 2) & return & return
		end repeat
		set theAction to button returned of (display alert "Total of " & numMisplacedRecords & " misplaced indexed files" & return & return & lostChildReport buttons {"Cancel", "Replicate to True Parent", "Replicate to a Group"} default button 1)
	end if
	
	
	if theAction is "Replicate to True Parent" then
		-- replicate the misplaced indexed file back to its indexed parent
		repeat with each in theMisplacedRecords
			replicate record each's item 1 to each's item 2
			log message "'" & (location of each's item 1 & name of each's item 1) & "' is replicated to " & (location of each's item 2 & name of each's item 2) & "'" info theScriptName
		end repeat
		display alert "check log message for replications" giving up after 3
		
	else if theAction is "Replicate to a Group" then
		-- replicate all misplaced indexed files  to an asisgned group for further evaluation 
		set theDestGp to display group selector for theDB
		repeat with each in theMisplacedRecords
			replicate record each's item 1 to theDestGp
			log message "'" & (location of each's item 1 & name of each's item 1) & "' is replicated to " & "'" & (location of theDestGp & name of theDestGp) & "'" info theScriptName
		end repeat
		display alert "check log message for replications" giving up after 3
		
	end if
	hide progress indicator
end tell

on findAndReplaceInText(theText, theSearchString, theReplacementString)
	set AppleScript's text item delimiters to theSearchString
	set theTextItems to every text item of theText
	set AppleScript's text item delimiters to theReplacementString
	set theText to theTextItems as string
	set AppleScript's text item delimiters to ""
	return theText
end findAndReplaceInText



A minor feature update:

The results now also show the list of stand-alone indexed file (SIF) - files that are indexed into DT directly.

Demo:

The Script
Please test the script carefully with a test database. The script won’t delete any items but may create replicants unnecessarily if there are bugs. P.S. I checked the script with 5-6k indexed files each in two databases and it seems ok.

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

-- by ngan 2020.04.25
--v1b3 also report stand-alone index files
--v1b2 handle misplaced index files and indexed sub-groups


tell application id "DNtp"
	
	
	set theRecords to selection
	set theMisplacedRecords to {}
	set theStandAloneRecords to {}
	set theDB to database of root of think window 1
	set theScriptName to name of me
	
	-- script only check misclocated files in single database
	if button returned of (display alert "This script can only run for files in single database" buttons {"Cancel", "Continue"} default button 2) is "Cancel" then return
	
	show progress indicator "Find misplaced files"
	
	repeat with i from 1 to length of theRecords
		step progress indicator "Checking file ..." & i & "/" & length of theRecords
		-- get the path of each indexed file
		set theRecordPath to path of theRecords's item i
		--get only the parent-as-group for each record.  if a parent group is also an indexed group, it will have a normal path in folder
		set theParentsPath to (path of parents of theRecords's item i whose tag type is not ordinary tag)
		set theParentsFullPath to {}
		-- Manipulate the path info to prepare for comparisons with the path of indexed file
		repeat with each in theParentsPath
			set end of theParentsFullPath to each & "/" & (filename of theRecords's item i)
		end repeat
		
		-- if path of an indexed file cannot be found in the path of any of its parents, the file is misplaced 
		if theRecordPath is not in theParentsFullPath then
			-- The true indexed parent should have the same folder path as the misplaced indexed file
			set theTrueParentPath to my findAndReplaceInText(theRecordPath, filename of theRecords's item i, "")
			try
				--Find the true indexed parent - UNLESS the file is indexed directly then theTrueParent will return {} 
				set theTrueParent to item 1 of (lookup records with path theTrueParentPath in current database)
				set end of theMisplacedRecords to {theRecords's item i, theTrueParent}
			on error
				-- if no indexed parent is found, the file is a stand-alone indexed file
				
				if type of theRecords's item i is not group then set end of theStandAloneRecords to {theRecords's item i}
			end try
		end if
		
	end repeat
	
	
	step progress indicator "Reporting misplaced files ..."
	
	-- prepare the report for misplaced indexed files
	set numMisplacedRecords to length of theMisplacedRecords
	set numOrphans to length of theStandAloneRecords
	set lostChildReport to ""
	set orphanReport to ""
	
	if {numMisplacedRecords, numOrphans} is {0, 0} then
		-- No misplaced indexed file
		display alert "No misplaced and no stand-alone indexed file"
		hide progress indicator
		return
	else
		-- prepare the list of misplaced file and suggests three actions
		if numMisplacedRecords > 0 then
			repeat with each in theMisplacedRecords
				set lostChildReport to lostChildReport & "    MIF: " & (location of each's item 1 & name of each's item 1) & return & "    TIP: " & (location of each's item 2 & name of each's item 2) & return & return
			end repeat
		end if
		
		if numOrphans > 0 then
			repeat with each in theStandAloneRecords
				
				set orphanReport to orphanReport & "    SIF: " & (location of each's item 1 & name of each's item 1) & return
				log message "'" & (location of each's item 1 & name of each's item 1) & "' is a stand-alone indexed file" info theScriptName
				
			end repeat
		end if
		
		set theAction to button returned of (display alert "Total of " & numMisplacedRecords & " misplaced indexed files" & return & return & lostChildReport & "Total of " & numOrphans & " stand-alone indexed files (For info only - see log messages)" & return & return & orphanReport buttons {"Cancel", "Replicate MIF to True Parent", "Replicate all MIFs to a Group"} default button 1)
	end if
	
	
	if theAction is "Replicate MIF to True Parent" then
		-- replicate the misplaced indexed file back to its indexed parent
		repeat with each in theMisplacedRecords
			replicate record each's item 1 to each's item 2
			log message "'" & (location of each's item 1 & name of each's item 1) & "' is replicated to '" & (location of each's item 2 & name of each's item 2) & "'" info theScriptName
		end repeat
		display alert "check log message for replications" giving up after 3
		
	else if theAction is "Replicate all MIFs to a Group" then
		-- replicate all misplaced indexed files  to an asisgned group for further evaluation 
		set theDestGp to display group selector for theDB
		repeat with each in theMisplacedRecords
			replicate record each's item 1 to theDestGp
			log message "'" & (location of each's item 1 & name of each's item 1) & "' is replicated to '" & (location of theDestGp & name of theDestGp) & "'" info theScriptName
		end repeat
		display alert "check log message for replications" giving up after 3
		
	end if
	hide progress indicator
end tell

on findAndReplaceInText(theText, theSearchString, theReplacementString)
	set AppleScript's text item delimiters to theSearchString
	set theTextItems to every text item of theText
	set AppleScript's text item delimiters to theReplacementString
	set theText to theTextItems as string
	set AppleScript's text item delimiters to ""
	return theText
end findAndReplaceInText