I have an Alfred extension that I use for quickly querying DEVONthink databases, but it’s currently using AppleScript/Javascript to interface with DEVONthink.
Limitations are that it’s pretty slow and I have to run the search for each database individually, then merge the results manually and sort by score before displaying. So something like
const dbs = devonthink.databases();
const found = dbs.map((db) => doSearch(db, query));
let results = [];
for (const db of dbs) {
results = results.concat(doSearch(db, query));
}
results.sort(function(a, b) {
return b.score - a.score;
})
It works, but it’s not very performant.
Any ideas on how to make it faster? Any other ways to pull information out of DEVONthink for scripting?
Running the search for each database via AppleScript should be probably more or less as fast as running the same search across all databases in DEVONthink. But sorting of the results is of course a lot slower, therefore one workaround might be to limit the results to n items for each database.
Is it possible to do a direct query against all databases?
The sorting is heavy because each database query gives me the results sorted by score for that database only, and I need to combine those results after the fact. If DT can give me a full list that is already sorted by score and includes results from all databases, I could skip the sorting part
I remember faintly that DEVONthink exposes the metadata files somewhere that were previously able to be indexed with spotlight. I wonder if it’s possible to use that?
If you do it in JavaScript then using AppleScriptObjC might be faster. As it seems it’s not possible to sort DEVONthink records directly by a record’s score property, I’ve first build a new AppleScript record with some properties, e.g. the name to display in Alfred, the score for sorting and the reference URL or uuid to get the choosen result afterwards from DEVONthink.
I just started to look into AppleScriptObjC, so there might be a faster way to do it … If someone (@houthakker) knows if/how DEVONthink records can be sorted without first building an AppleScript record please share.
-- Search across all databases and sort results
use AppleScript version "2.7"
use framework "Foundation"
use scripting additions
tell application id "DNtp"
try
set theQuery to "name:test"
set theDatabases to databases
set theResults_record to {}
repeat with thisDatabase in theDatabases
set theResults to search theQuery in thisDatabase
repeat with thisResult in theResults
set end of theResults_record to {name_:(name of thisResult), refurl_:(reference URL of thisResult), score_:(score of thisResult)}
end repeat
end repeat
set theResults_record_sorted to my sort(theResults_record)
-- [format results for Alfred] -- don't know what's needed for Alfred
on error error_message number error_number
if the error_number is not -128 then display alert "DEVONthink" message error_message as warning
return
end try
end tell
on sort(TheList)
set anArray to current application's NSArray's arrayWithArray:TheList
set theDesc to current application's NSSortDescriptor's sortDescriptorWithKey:"score_" ascending:false selector:"compare:"
set newList to (anArray's sortedArrayUsingDescriptors:{theDesc}) as list
end sort
Forgive me I missed this – not always regularly checking here, as I always prefer JavaScript now, and there has sometimes seemed to be an active anxiety to preserve this as an AppleScript-only shop
But since you ask …
JS native sort is very fast (competitive development between browser suppliers) so no need at all for the baroque syntax-mix and slightly tricky type conversions of learning to call ObjC methods through the ‘ASObjC’ foreign function interface.
where sortOn fetches record.score() only once for each record – a standard technique sometimes connected with Perl and the name of Randal Schwartz, but more helpfully called the decorate-sort-undecorate pattern.
// sortOn :: Ord b => (a -> b) -> [a] -> [a]
const sortOn = f =>
// Equivalent to sortBy(comparing(f)), but with f(x)
// evaluated only once for each x in xs.
// ('Schwartzian' decorate-sort-undecorate).
xs => xs.map(
x => Tuple(f(x))(x)
)
.sort(comparing(fst))
.map(snd);
A few points:
DEVONthink’s search method searches across all databases by default, (if we supply no explicit ‘in’ value) so we don’t even have to iterate over the collection of databases.
For a descending sort, we can flip (a, b) comparison to (b, a). I’ve provided both a sortOn and a sortDescendingOn function below:
Full sample script, descending sort on `score`:
(() => {
'use strict';
// ---------------------- MAIN ----------------------
const main = () => {
const queryString = 'name:test';
const devon = Application('DEVONthink 3');
return sortDescendingOn(
x => x.score()
)(
devon.search(queryString)
)
// If we wanted to list scores and names,
// otherwise, comment out the following lines:
// .map(
// x => [
// x.score(), x.name()
// ]
// .map(str)
// .join(' -> '))
// .join('\n');
};
// ------------------- JS PRELUDE -------------------
// https://github.com/RobTrew/prelude-jxa
// Tuple (,) :: a -> b -> (a, b)
const Tuple = a =>
b => ({
type: 'Tuple',
'0': a,
'1': b,
length: 2
});
// comparing :: (a -> b) -> (a -> a -> Ordering)
const comparing = f =>
(x, y) => {
const
a = f(x),
b = f(y);
return a < b ? -1 : (a > b ? 1 : 0);
};
// flip :: (a -> b -> c) -> b -> a -> c
const flip = op =>
// The binary function op with
// its arguments reversed.
1 < op.length ? (
(a, b) => op(b, a)
) : (x => y => op(y)(x));
// fst :: (a, b) -> a
const fst = tpl =>
// First member of a pair.
tpl[0];
// snd :: (a, b) -> b
const snd = tpl =>
// Second member of a pair.
tpl[1];
// sortOn :: Ord b => (a -> b) -> [a] -> [a]
const sortOn = f =>
// Equivalent to sortBy(comparing(f)), but with f(x)
// evaluated only once for each x in xs.
// ('Schwartzian' decorate-sort-undecorate).
xs => xs.map(
x => Tuple(f(x))(x)
)
.sort(comparing(fst))
.map(snd);
// sortDescendingOn :: Ord b => (a -> b) -> [a] -> [a]
const sortDescendingOn = f =>
// Equivalent to sortBy(comparing(f)), but with f(x)
// evaluated only once for each x in xs.
// ('Schwartzian' decorate-sort-undecorate).
xs => xs.map(
x => Tuple(f(x))(x)
)
.sort(flip(comparing(fst)))
.map(snd);
// str :: a -> String
const str = x =>
Array.isArray(x) && x.every(
v => ('string' === typeof v) && (1 === v.length)
) ? (
x.join('')
) : x.toString();
// MAIN --
return main();
})();
Didn’t know that. Unfortunately I don’t know JavaScript, but your awesome answer sure will help the OP and other users
Yes, of course. I’ve mixed up compare content (which I used the day before and which needs a repeat to get all databases) and search. I wonder why @syntagm uses a repeat … maybe to only get results of some databases but not all?
Do you know if it’s possible at all in AppleScriptObjC to sort DEVONthink records by any property without the “workaround” of first building an AppleScript record (the one I used in the previous post)?
Thanks!
