Does Smart Rule Script Execution Differ "Apply Script > Javascript" vs "Apply Script > External"

Hello,

I am creating a JXA script for a DEVONthink Smart Rule. Its purpose is to save YAML frontmatter values from Markdown files to custom metadata fields.

The script works perfectly when the JXA code is used directly within the Smart Rule via the ‘Apply Script > Javascript’ action.

However, when I save the exact same JXA code as an external SCPT file and use the ‘Apply Script > External’ option in the Smart Rule (selecting the SCPT file), the script fails.

The failure occurs when calling getCustomMetaData or addCustomMetaData functions. The error message is consistently “Can’t convert types”.

I can provide the code if needed.

Is this difference in behavior between running JXA via ‘Apply Script > Javascript’ and ‘Apply Script > External (SCPT)’ expected?

Thank you for any insights.

The internal execution modifies the code to work around another issue of JXA. Well, like already mentioned - it’s a bag full of bugs.

Thanks for the clear explanation
It’s unfortunate news because I preferred managing the script externally.
However, I’m glad I now understand the cause lies in the different execution methods.
I’ll stick to the embedded script…
Thanks again!

We’ll check if it’s (easily) possible to apply the workaround to external scripts too.

Embedded scripts have the advantage of not being cached by DT. If you change an external script, you have to restart DT to make it see the changes. With embedded scripts, that is not necessary.

1 Like

I agree, the caching issue requiring a DEVONthink restart for external script changes is very inconvenient. However, I prefer external scripts because I manage all my sources (.js, .applescript) in Git and use a shell script (sh) to compile the final .scpt files and deploy them to their appropriate locations automatically. So, this workflow leads me to prefer using external scripts.

…but speeds up especially smart rules a lot.

that does sound like it could be worth the hassle

Could you please post your code? The next beta will apply the workaround for internal JXA scripts also to external JXA scripts and improve the compatibility to external JXA scripts written for version 3.

As requested, I’m sharing the JXA Smart Rule script I currently use. This script updates DEVONthink custom metadata from the YAML frontmatter in Markdown files and employs a YAML hash to process only changed files.

Key dependencies are yq for YAML parsing (the YQ_PATH variable in the script needs to be set to your yq executable’s path) and my ShellHelper.scptd JXA library for running shell commands. The AppleScript for ShellHelper.scptd is provided below; it should be placed in ~/Library/Script Libraries/.

Please configure the METADATA_MAPPINGS, HASH_MD_IDENTIFIER, and RENEWAL_DATE_MD_IDENTIFIER constants in the JXA script. You will also need to pre-define these corresponding custom metadata fields in DEVONthink (Preferences > Data) with the data types as specified in the script’s METADATA_MAPPINGS.

To quickly test
First, in DEVONthink, set up a custom metadata field with the identifier price and select Integer as its type.

Next, ensure the METADATA_MAPPINGS in the JXA script includes an entry like \"price\": { id: \"price\", type: \"integer\" }.
Then, create a Markdown file with the following YAML frontmatter:

---
price: 27900
---

Finally, apply this JXA script (e.g., via a Smart Rule using ‘Apply Script > Javascript’) to the Markdown file. The price custom metadata field for that record should then be populated with the value 27900.

For debugging, setting DEBUG_MODE = true; at the top of the JXA script enables logging to /tmp/Update_Markdown_YAML_to_Metadata.log.

The JXA script is first, then the ShellHelper AppleScript.

JXA:

function performsmartrule(records) {
    'use strict';

    // 설정
    const DEBUG_MODE = false; // true 시 상세 로그 기록
    const LOG_FILE_NAME = "Update_Markdown_YAML_to_Metadata.log"; // 디버그 로그 파일명

    // DT 커스텀 메타데이터 ID
    const HASH_MD_IDENTIFIER = "YAMLHash"; // YAML 해시용 메타데이터 ID
    const RENEWAL_DATE_MD_IDENTIFIER = "YAMLUpdateDate"; // 최종 업데이트 날짜용 메타데이터 ID

    // YAML와 커스텀 메타데이터 매핑
    // 형식: "YAML키": { id: "DT메타데이터ID", type: "데이터타입" }
    // 데이터타입: "string", "integer", "date", "boolean", "real" 등
    const METADATA_MAPPINGS = {
        // 아래 목록을 필요에 따라 추가/수정/삭제
        "name":                     { id: "name",                     type: "string"  }, // 아이템명
        "originalName":             { id: "originalname",             type: "string"  }, // 아이템명/원문
        "category":                 { id: "category",                 type: "string"  }, // 분류
        "quantity":                 { id: "quantity",                 type: "integer" }, // 수량
        "price":                    { id: "price",                    type: "integer" }, // 구매 가격
        "purchaseDate":             { id: "purchasedate",             type: "date"    }, // 구매일
        "additionalPurchaseDates":  { id: "additionalPurchaseDates",  type: "string"  }, // 추가 구매일
        "vendor":                   { id: "vendor",                   type: "string"  }, // 구매처
        "condition":                { id: "condition",                type: "string"  }, // 상태 (새것, 중고 등)
        "isSold":                   { id: "issold",                   type: "boolean" }, // 판매 여부 (체크박스)
        "rating":                   { id: "rating",                   type: "integer" }, // 개인 평점 
        "notes":                    { id: "notes",                    type: "string"  }, // 추가 메모
        "warrantyExpiryDate":       { id: "warrantyexpirydate",       type: "date"    }, // 보증 만료일
        // --- 아래는 필요시 주석 해제 또는 다른 필드 추가 ---
        // "salePrice":     { id: "saleprice",     type: "integer" }, // 판매 가격
        // "saleDate":      { id: "saledate",      type: "date" },    // 판매일
        // "serialNumber":  { id: "serialnumber",  type: "string" },  // 시리얼 번호
    };

    const YQ_PATH = "/opt/homebrew/bin/yq"; // yq 실행 경로
    const SHELL_HELPER_ERROR_PREFIX = "SHELL_HELPER_ERROR:::"; // ShellHelper 오류 식별자
    // 설정 끝

    const DEVONthinkApp = Application("DEVONthink"); // DT 앱 객체

    try {
        ObjC.import('Foundation'); // Foundation 프레임워크
    } catch(e) {
        const errorMsg = `ObjC 프레임워크 로드 오류: ${e.message}`;
        console.log(errorMsg); try { DEVONthinkApp.logMessage(`[YAML Script CRITICAL] ${errorMsg}`); } catch (e2) {} return false;
    }

    function logToFile(message) { // 파일 로깅 함수
        if (!DEBUG_MODE) return;
        try {
            const logPath = "/tmp/" + LOG_FILE_NAME; const timestamp = new Date().toISOString(); const logEntry = `${timestamp} [JXA_YAML_Updater] - ${message}\n`;
            const nsLogEntry = $.NSString.alloc.initWithUTF8String(logEntry); const nsLogPath = $(logPath).stringByStandardizingPath;
            const fileManager = $.NSFileManager.defaultManager;
            if (!fileManager.fileExistsAtPath(nsLogPath.js)) { nsLogEntry.writeToFileAtomicallyEncodingError(nsLogPath, true, $.NSUTF8StringEncoding, $()); }
            else { const fileHandle = $.NSFileHandle.fileHandleForWritingAtPath(nsLogPath); if (fileHandle) { fileHandle.seekToEndOfFile; fileHandle.writeData(nsLogEntry.dataUsingEncoding($.NSUTF8StringEncoding)); fileHandle.closeFile; } else if (DEBUG_MODE) { console.log(`파일 로그: 핸들 열기 실패 - ${logPath}`); } }
        } catch (e) { if (DEBUG_MODE) { console.log(`파일 로그: 로깅 중 오류 (${message.substring(0,100)}...): ${e.message}`); }}
    }
    function isShellError(resultString) { return typeof resultString === 'string' && resultString.startsWith(SHELL_HELPER_ERROR_PREFIX); }
    function parseShellError(errorString) { if (!isShellError(errorString)) return errorString; return `쉘 실행 오류: ${errorString.substring(SHELL_HELPER_ERROR_PREFIX.length)}`; }

    function extractFrontmatterText(fullText) { // YAML 프론트매터 추출 함수
        if (typeof fullText !== 'string' || fullText.length === 0) return ""; let textToSearch = fullText;
        if (textToSearch.charCodeAt(0) === 0xFEFF) textToSearch = textToSearch.substring(1); else if (textToSearch.startsWith('\u00EF\u00BB\u00BF')) textToSearch = textToSearch.substring(3);
        const startMarker = "---"; const endMarker = "\n---"; if (!textToSearch.startsWith(startMarker)) { logToFile("프론트매터: '---'로 시작 안 함"); return ""; }
        const firstMarkerEndPos = textToSearch.indexOf('\n', startMarker.length); if (firstMarkerEndPos === -1) { logToFile("프론트매터: 첫 '---' 이후 개행 없음. 유효하지 않음."); return ""; }
        let endPosition = textToSearch.indexOf(endMarker, firstMarkerEndPos); if (endPosition === -1) { if (textToSearch.trimRight().endsWith(endMarker.trimLeft())) { endPosition = textToSearch.lastIndexOf(endMarker, textToSearch.length - endMarker.length -1 ); } }
        if (endPosition > 0 && endPosition > firstMarkerEndPos) return textToSearch.substring(0, endPosition + endMarker.length);
        logToFile("프론트매터: 유효한 종료 '---' 마커 없음."); return "";
    }
    function writeToTempFile(content, shellHelper, baseName = 'dt_yaml_fm_') { // 임시 파일 쓰기 함수 
        let tempFilePath = null;
        try {
            logToFile(`uuidgen (ShellHelper)으로 ${baseName} 임시 파일명 생성 중...`);
            const uuidResult = shellHelper.runShellCommand("uuidgen");
            if (isShellError(uuidResult) || uuidResult.trim() === "") throw new Error(`UUID 생성 실패: ${isShellError(uuidResult) ? parseShellError(uuidResult) : "uuidgen 결과 없음"}`);
            const uniqueName = uuidResult.trim().toLowerCase();
            const tempFileName = `${baseName}${uniqueName}.tmp`; // 기본 파일명 사용
            const nsTempDirectoryPath = ObjC.unwrap($.NSTemporaryDirectory());
            if (!nsTempDirectoryPath) throw new Error("임시 디렉토리 경로 가져오기 실패 (NSTemporaryDirectory null).");
            const nsTempDir = $(nsTempDirectoryPath).stringByStandardizingPath;
            const nsTempFilePath = nsTempDir.stringByAppendingPathComponent(tempFileName);
            tempFilePath = ObjC.unwrap(nsTempFilePath);
            logToFile(`임시 파일 경로: ${tempFilePath}`);
            const nsContent = $.NSString.alloc.initWithUTF8String(content);
            if (!nsContent) throw new Error("NSString 내용 생성 실패.");
            const errorPtr = $();
            const success = nsContent.writeToFileAtomicallyEncodingError(nsTempFilePath, true, $.NSUTF8StringEncoding, errorPtr);
            if (success) { logToFile(`임시 파일 쓰기 성공: ${tempFilePath}`); return tempFilePath; }
            else { const writeErrMsg = `임시 파일 쓰기 실패: ${tempFilePath}`; logToFile(`${writeErrMsg} (writeToFileAtomically... false 반환)`); throw new Error(writeErrMsg); }
        } catch(e) {
            logToFile(`writeToTempFile (${baseName}) 오류: ${e.message}${e.stack ? `\n스택: ${e.stack}` : ""}`);
            if (tempFilePath && shellHelper) deleteTempFile(tempFilePath, shellHelper);
            return null;
        }
    }
    function deleteTempFile(filePath, shellHelper) { // (임시 파일 삭제 함수 내용은 이전과 동일)
        if (!filePath || !shellHelper) return;
        try {
            const escapedFilePath = filePath.replace(/'/g, "'\\''"); const rmCommand = `rm -f '${escapedFilePath}'`;
            logToFile(`임시 파일 삭제 시도 (ShellHelper): ${rmCommand}`); const rmResult = shellHelper.runShellCommand(rmCommand);
            if (isShellError(rmResult)) logToFile(`경고: 임시 파일 삭제 실패 (${filePath}): ${parseShellError(rmResult)}`);
            else logToFile(`임시 파일 삭제 완료 (ShellHelper): ${filePath}`);
        } catch(e) { logToFile(`임시 파일 삭제 중 오류 (${filePath}): ${e.message}`); }
    }
    function calculateHash(textToHash, shellHelper) { // 해시 계산 (내부적으로 임시파일 사용)
        const tempFilePath = writeToTempFile(textToHash, shellHelper, 'dt_yaml_hash_'); // 해시용 임시 파일
        if (!tempFilePath) { logToFile("calculateHash: writeToTempFile 실패, 해시 계산 불가."); return null; }
        const escapedFilePath = tempFilePath.replace(/'/g, "'\\''");
        const hashCommand = `shasum -a 256 '${escapedFilePath}' | awk '{print $1}'`;
        logToFile(`임시 파일 해시 계산 중: ${tempFilePath}`);
        const hashResult = shellHelper.runShellCommand(hashCommand);
        deleteTempFile(tempFilePath, shellHelper);
        if (isShellError(hashResult)) { logToFile(`해시 계산 오류: ${parseShellError(hashResult)}`); return null; }
        const trimmedHash = hashResult.trim();
        if (trimmedHash === "" || trimmedHash.length !== 64) { logToFile(`해시 결과 무효 (길이 ${trimmedHash.length}): "${trimmedHash}"`); return null; }
        return trimmedHash;
    }

    logToFile(`스크립트 시작. 레코드 ${records ? records.length : 0}개 처리`);
    if (!records || records.length === 0) { logToFile("처리할 레코드 없음. 종료."); return true; }

    let ShellHelper;
    try {
        ShellHelper = Library("ShellHelper"); logToFile("ShellHelper 라이브러리 로드 성공.");
    } catch (e) {
        const criticalMsg = `치명적 오류: ShellHelper 로드 실패 (${e.message}). 중단.`; logToFile(criticalMsg);
        try { DEVONthinkApp.logMessage(`[YAML Script CRITICAL] ShellHelper 로드 실패: ${e.message}`); } catch (e2) {}
        try { Application.currentApplication().displayAlert("라이브러리 로드 실패", { message: `ShellHelper 라이브러리 로드 실패.\n오류: ${e.message}`, as: "critical" }); } catch (e3) {}
        return false;
    }

    records.forEach(record => { // 레코드 반복 처리
        const recordName = (typeof record.name === "function") ? record.name() : (record.id ? `ID ${record.id()}` : "이름없는 레코드");
        logToFile(`\n레코드 처리: "${recordName}" (경로: ${record.path ? record.path() : 'N/A'})`);
        let yamlTempFilePath = null; // <<< yq 처리를 위한 임시 파일 경로 변수 추가

        try { // 레코드별 오류 처리
            if (record.type() !== "markdown" && record.type() !== "txt") { logToFile(`건너뜀 (Markdown/TXT 아님): "${recordName}" (타입: ${record.type()})`); return; }
            const plainText = record.plainText();
            if (plainText === null || typeof plainText === 'undefined' || plainText.trim() === "") { logToFile(`건너뜀 (내용 없음): "${recordName}"`); return; }

            const frontmatterText = extractFrontmatterText(plainText); // 프론트매터 추출
            if (frontmatterText === "") { logToFile(`유효한 YAML 프론트매터 없음: "${recordName}". 건너뜀.`); return; }
            logToFile(`프론트매터 추출됨 (길이 ${frontmatterText.length}): "${recordName}"`);

            const currentHash = calculateHash(frontmatterText, ShellHelper); // 추출된 텍스트로 해시 계산
            if (!currentHash) { logToFile(`현재 해시 계산 실패: "${recordName}". 건너뜀.`); DEVONthinkApp.logMessage(`[YAML Script ERROR] 해시 계산 실패: "${recordName}"`); return; }
            logToFile(`현재 해시: ${currentHash} ("${recordName}")`);

            let storedHash = ""; // 저장된 해시 읽기
            try {
                const mdValue = DEVONthinkApp.getCustomMetaData({for: HASH_MD_IDENTIFIER, from: record});
                if (typeof mdValue === 'string') { storedHash = mdValue; }
                else if (mdValue !== undefined && mdValue !== null) { logToFile(`경고: 저장된 해시가 문자열 아님 (타입: ${typeof mdValue}). 무시.`); }
            } catch (e) {
                if (e.message && e.message.includes("Can't convert types")) { logToFile(`저장된 해시 읽기 중 'Can't convert types' 오류. 첫 실행 간주.`); }
                else { logToFile(`저장된 해시 읽기 실패 ("${HASH_MD_IDENTIFIER}", "${recordName}"). 오류: ${e.message}`); }
            }
            logToFile(`저장된 해시: ${storedHash || "(없음)"} ("${recordName}")`);

            if (currentHash !== storedHash) { // 해시 다를 시 업데이트
                logToFile(`해시 다름. 메타데이터 업데이트: "${recordName}"`);
                DEVONthinkApp.logMessage(`[YAML Script INFO] YAML 변경 감지, 업데이트 시작: "${recordName}"`);
                let metadataActuallyUpdatedByScript = false;

                // 추출된 프론트매터를 임시 파일에 쓰고 yq가 읽도록 함
                yamlTempFilePath = writeToTempFile(frontmatterText, ShellHelper, 'dt_yaml_content_'); // YAML 내용용 임시 파일 생성
                if (!yamlTempFilePath) { // 임시 파일 생성 실패 시
                    logToFile(`오류: yq 처리를 위한 임시 파일 생성 실패: "${recordName}". 업데이트 건너뜀.`);
                    DEVONthinkApp.logMessage(`[YAML Script ERROR] YAML 임시 파일 생성 실패: "${recordName}"`);
                } else { // 임시 파일 생성 성공 시
                    const escapedYamlTempFilePath = yamlTempFilePath.replace(/'/g, "'\\''"); // 경로 이스케이프

                    for (const yamlKey in METADATA_MAPPINGS) { // 매핑별 처리
                        if (Object.hasOwnProperty.call(METADATA_MAPPINGS, yamlKey)) {
                            const mappingConfig = METADATA_MAPPINGS[yamlKey];
                            const dtMetadataIdentifier = mappingConfig.id;
                            const expectedDtType = mappingConfig.type.toLowerCase();

                            // yq 명령어가 임시 파일을 읽도록 수정
                            const yqCommand = `${YQ_PATH} eval '.${yamlKey} // ""' '${escapedYamlTempFilePath}'`; // 파일 경로 변경됨
                            logToFile(`yq 값 가져오기 (키: "${yamlKey}"): ${yqCommand}`);
                            const yqOutputResult = ShellHelper.runShellCommand(yqCommand);
                            let extractedValueString = "";

                            if (isShellError(yqOutputResult)) {
                                logToFile(`경고: yq 명령어 실패 (키 "${yamlKey}"): ${parseShellError(yqOutputResult)}. "${dtMetadataIdentifier}" 값 "" 처리.`);
                                DEVONthinkApp.logMessage(`[YAML Script WARNING] yq 처리 실패 ("${recordName}", 키 "${yamlKey}"). 값 "" 처리.`);
                                extractedValueString = "";
                            } else {
                                extractedValueString = yqOutputResult.trim();
                                logToFile(`yq 결과 (키 "${yamlKey}"): "${extractedValueString}"`);
                            }

                            // 빈 값 / 타입 변환 처리: "" 사용
                            let finalValueToSet = "";
                            if (extractedValueString === "") {
                                finalValueToSet = "";
                                logToFile(`YAML 값 비어 있거나 키 없음 (키 "${yamlKey}"). 메타데이터를 ""로 설정.`);
                            } else {
                                finalValueToSet = extractedValueString;
                                // 문제 있으면 v3.1.3으로 돌아가자
                                if (expectedDtType === "date") { try { const d=new Date(extractedValueString); if(!isNaN(d.getTime())) finalValueToSet=d; else finalValueToSet=""; } catch(e){finalValueToSet=""}; if(finalValueToSet==="") logToFile(`경고: 날짜 형식 무효 ("${yamlKey}":"${extractedValueString}"). "" 설정.`);}
                                else if (expectedDtType === "integer") { const n=parseInt(extractedValueString,10); if(!isNaN(n)) finalValueToSet=n; else {logToFile(`경고: 정수 형식 무효 ("${yamlKey}":"${extractedValueString}"). "" 설정.`); finalValueToSet="";}}
                                else if (expectedDtType === "real") { const n=parseFloat(extractedValueString); if(!isNaN(n)) finalValueToSet=n; else {logToFile(`경고: 실수 형식 무효 ("${yamlKey}":"${extractedValueString}"). "" 설정.`); finalValueToSet="";}}
                                else if (expectedDtType === "boolean") { const l=extractedValueString.toLowerCase(); if(l==="true"||l==="yes"||l==="1") finalValueToSet=true; else if (l==="false"||l==="no"||l==="0") finalValueToSet=false; else {logToFile(`경고: 불리언 값 모호 ("${yamlKey}":"${extractedValueString}"). false 설정.`); finalValueToSet=false;}}
                            }

                            // 메타데이터 설정 시도
                            try {
                                const valueToLog = (finalValueToSet instanceof Date) ? finalValueToSet.toISOString() : finalValueToSet;
                                logToFile(`메타데이터 ID "${dtMetadataIdentifier}" 업데이트 시도 (타입: ${expectedDtType}, 값: ${valueToLog})`);
                                DEVONthinkApp.addCustomMetaData(finalValueToSet, {for: dtMetadataIdentifier, to: record, as: expectedDtType});
                            } catch(metaError) {
                                logToFile(`메타데이터 설정 오류 ("${dtMetadataIdentifier}"): ${metaError.message}`);
                                DEVONthinkApp.logMessage(`[YAML Script ERROR] 메타데이터 설정 오류 ("${recordName}", ID "${dtMetadataIdentifier}"): ${metaError.message}`);
                            }
                            metadataActuallyUpdatedByScript = true; // 처리 시도 플래그
                        }
                    } // 매핑 루프 끝

                    // YAML 내용용 임시 파일 삭제
                    deleteTempFile(yamlTempFilePath, ShellHelper); // 루프 끝난 후 임시 파일 삭제
                    yamlTempFilePath = null; // 경로 초기화

                } // if (yamlTempFilePath) 끝 (임시 파일 처리 성공 시)

                if (metadataActuallyUpdatedByScript) { // 업데이트 시도가 한 번이라도 있었으면 해시/갱신날짜 저장
                    logToFile(`저장된 해시 업데이트 (${currentHash}): "${recordName}"`);
                    DEVONthinkApp.addCustomMetaData(currentHash, {for: HASH_MD_IDENTIFIER, to: record, as: "string"});
                    const currentDate = new Date();
                    logToFile(`${RENEWAL_DATE_MD_IDENTIFIER} 업데이트 (${currentDate.toISOString()}): "${recordName}"`);
                    DEVONthinkApp.addCustomMetaData(currentDate, {for: RENEWAL_DATE_MD_IDENTIFIER, to: record, as: "date"});
                    logToFile(`메타데이터 업데이트 시도 완료: "${recordName}"`);
                    DEVONthinkApp.logMessage(`[YAML Script OK] 메타데이터 업데이트 완료: "${recordName}"`);
                } else { // 해시는 다르나 임시 파일 생성 실패 등 처리할 내용이 없었던 경우
                    logToFile(`해시 달랐으나, 업데이트 시도된 메타데이터 필드 없음. 해시/갱신날짜 미변경: "${recordName}"`);
                    DEVONthinkApp.logMessage(`[YAML Script INFO] 해시 변경, 실제 업데이트 내용 없어 메타데이터 미변경: "${recordName}"`);
                }
            } else { // 해시 같을 시
                logToFile(`해시 일치. 업데이트 불필요: "${recordName}"`);
            }
            logToFile(`--- 레코드 처리 완료: "${recordName}" ---`);
        } catch (recordError) { // 레코드 처리 예외
            const errorMsg = `!!!!!!!! 레코드 처리 중 심각한 오류 "${recordName}": ${recordError.message} !!!!!!!!\n스택: ${recordError.stack}`;
            logToFile(errorMsg);
            try { DEVONthinkApp.logMessage(`[YAML Script CRITICAL ERROR] ${recordName}: ${recordError.message}`); } catch (e2) {}
        } finally { // try-catch 블록의 finally 추가
            // 루프 내에서 오류 발생 시에도 임시 YAML 파일이 남아있을 수 있으므로 여기서 한번 더 확인 후 삭제
            if (yamlTempFilePath && ShellHelper) {
                 logToFile(`최종 정리: 남은 YAML 임시 파일 삭제 시도: ${yamlTempFilePath}`);
                 deleteTempFile(yamlTempFilePath, ShellHelper);
            }
        }
    }); // 레코드 루프 끝

    logToFile("----- 스크립트 실행 종료 -----");
    return true; 
}

ShellHelper.scptd:

use AppleScript version "2.4"
use scripting additions

-- --- 설정 ---
property debugMode : false
property logFileName : "shellhelper_debug.log"
-- --- 설정 끝 ---

-- --- 로그 파일에 메시지 추가하는 핸들러 ---
on logToFile(logMessage as text)
	if not debugMode then return
	
	try
		set logFilePath to "/tmp/" & logFileName
		set timestamp to do shell script "date '+%Y-%m-%dT%H:%M:%S%z'"
		set logEntry to timestamp & " [AS] - " & logMessage & return
		set fileRef to open for access file POSIX file logFilePath with write permission
		write logEntry to fileRef starting at eof
		close access fileRef
	on error errMsg number errNum
		log "[ShellHelper LOG ERROR] Failed to write to " & logFileName & ": (" & errNum & ") " & errMsg
	end try
end logToFile
-- --- 로깅 핸들러 끝 ---

-- --- 메인 핸들러 (JXA에서 'runShellCommand' 이름으로 호출) ---
on runShellCommand(commandString as text)
	my logToFile("runShellCommand received: " & commandString)
	
	try
		set commandResult to do shell script commandString
		
		try
			my logToFile("Shell command success. Result length: " & (count of commandResult))
		on error
			my logToFile("Shell command success. (Result length check failed)")
		end try
		
		return commandResult
		
	on error errMsg number errNum
		my logToFile("Shell command FAILED! Number: " & errNum & ", Message: " & errMsg)
		
		return "SHELL_HELPER_ERROR:::" & errNum & ":::" & errMsg
	end try
end runShellCommand
-- --- 메인 핸들러 끝

Any chance there could be some way (other than restarting DT) to force a refresh of the script cache? When iterating versions of a script that would be really helpful.

Is that an “expensive” operation for you which I assume is only occasionally needed? My DEVONthink shut-downs and start are normally quick.

Partly it is an issue because I have several large databases I keep open.

But I also run the DT Server so at any given time my staff could be downloading or uploading documents; restarting interrupts that.

I find your JS code hard to read (lack of newlines and un-bracketed if statements) and a bit over-engineered. There might not be anything wrong with it (syntactically), but it is so contrived that the logic is hard to follow.
Some things that I noticed follow below. Please feel free to ignore all of them.

  • You get the plainText attribute of a record and pass it to extractFrontmatterText
  • Where you immediately check if its parameter is of type string.
    Why? What else would it be if you pass in plainText?

Then in extractFontmatterText, you basically write your own parser to find everything between --- on the first line and ---\n somewhere later. Why not simply use

const match = txt.match(/^---(.*?)\n---/);

If match is not null, you know that the text starts with ---. And you know that \n--- occurs somewhere later. The whole frontmatter stuff will be available in match[1], too. Simply return match ? match[1] : null from that function and have the caller test for true instead of an empty string.

Now, your extracted frontmatter string looks like this:

price: 1234
name: My Name
quantity: 5
purchaseDate: 2021-22-07
...

There is no need to write that to a file which you then parse with yq. Simply split it into an array. If you feel fancy, you can create an object from that by splitting the array elements at the : to get properties and their values. In any case, it’s a piece of cake to look for the keys in your YAML data:

Object.keys(METADATA_MAPPINGS).forEach(key => {
  const mappingConfig = METADATA_MAPPINGS[key];
  const dtMetadataIdentifier = mappingConfig.id;
  const expectedDtType = mappingConfig.type; 
  /* And the types ARE ALREADY LOWERCASE. They won't become more lowercase than that. */
  const matchingYAML = arrayFrontmatterStrings.filter(line => line.beginsWith(`${key}:`));
  if (matchingYAML) {
    // process matchingYAML[0] 
  }
…
})

No need to run the same Shell command over and over again just to extract a simple line from a string. No need to check for shell errors.

And all these try… catch blocks? I do not see the point of those unless your catch does something useful. But you have an outer try that covers a gazillion lines of code – do you really think that calculateHash will fatally fail? If so, why do you return from the try instead of throwing an error? Or putting ObjC.import('Foundation') (which is not needed at all in the first place) in a try block? If this statement fails, there is more broken than you can catch anyway (and you’ll hear about that).

In order to calculate a hash of your record, you take the plainText, write it to a tmp file, pass that file on to shasum and filter its outpout through awk. All that with a lot of error handling etc. What it wrong with using a record’s contentHash attribute? It’s only SHA1, but for your purposes, that should suffice.

Alternatively NSString has a hash method returning an unsigned integer. Everything you can execute directly is better than having to call a shell command and worrying what can go wrong with it.

You build tmp file names that will never contain a single quote. But you still convert these names to others with single quotes escaped. A futile exercise, imo.

And WriteToTempFile contains a lot of useless code. Just build a JavaScript string containing the complete POSIX path of your file, preferably using a template string. For the UUID stuff, use

const uuid = $.NSUUID.alloc.init;
const uuidString = uuid.UUIDString;

instead of calling uuidgen via the shell. Append that to your filename as uuidString.js.
There’s also no need to normalize the return value of NSTemporaryDirectory, btw.

The shorter your code, the fewer the possibilities for errors. And the easier it is to understand and follow the logic.

3 Likes

@chrillek

I would love to see you write out a set of coding guideilnes simillar to the above and then implement them as .rules in Cursor. Then evaluate the resutling code created under those rules.

As we have discussed before, I think AI coding is particularly helpful for those who know how to code, not for those with no coding experience. Your insight into coding best practice is stunning as usual - but so few professional coders ever reach that level, no less those who are simply using code to help in other occupations which similarly demand ongoing study.

Interesting. I didn’t even know about Cursor before you mentioned it.
But I doubt that I can put anything into rules that easily. When I look at code, I can fairly quickly see what might be wrong with it. Much of that is common sense (write legible code, don’t cram tons of code in a single try block, eg). Some is based on „bad smell“, eg using shell commands in scripting. But most is just experience. I’ve been writing code for more than 40 years, in Fortran, Basic, Pascal, C, Perl, JavaScript and Lisp. And I read a lot of code as well as texts about coding. How (and why) would I put that into „rules“?

Eons ago the German car maker Volkswagen tried to develop an AI that would help speed up car repairs. All they needed to do was to put the experiences of the car repair people into that system.

It never took off. Simply, because the IT people didn’t know how to ask the repair people about their experience.

2 Likes

Thank you for taking the time to provide such detailed and very helpful feedback on my JXA script! I truly appreciate the insights and suggestions.

Your points on readability, YAML handling (especially the idea of direct JXA parsing for flat YAML), and using native UUIDs are excellent, and I will certainly work on incorporating them. Regarding the script’s current state, some of its complexity, and perhaps some less straightforward parts, are indeed traces of an iterative process where I was trying various approaches to solve issues encountered with external script execution (Apply Script > External), as has been discussed in this thread.

I will try to simplify the script based on your advice

I understand the challenge.

That’s exactly why I suggested you would be the man for the job.

Maybe instead of interviewing the repair people, the IT people could have taught the repair people how to prompt the AI system.

That was long before the time of prompts, in the
1980s. And they were not building an LLM, rather a neural network or something like that.

I’ll have a look at cursor later. Right now, I have to enjoy romantic Romanian cities :wink:

1 Like

Sounds like a taxonomy or an expert system.