Congratulations on transitioning the forum to Discourse… and with the entire post history intact! And the improved search system that it brings us has already borne fruit for me - and motivated this question:
And my question is, how can these output formats (RSS in particular) be accessed?
I can see from the web server’s logs that JSON is outputted as part of the response to the GET requests for reading the html for a browser, but I’m way out of my depth here so have no idea how to access the JSON data itself.
And my use-case is hoping to serve content from a DEVONthink database to an RSS client, so (for a given group UUID) how would I go about structuring a valid feed address that the web server would respond to appropriately?
This is absolutely fantastic - And not well known? I was only looking first for a way to read DT items in an RSS reader, but if the database search engine is available to be queried as an API then this opens up possibilities beyond anything I could have asked for…
Your suggestion for constructing query-based URLs works perfectly, and for format (and I’d love to know if there are other parameters beyond what the Simple Search interface can generate…), but on the original idea: Is there any way to just serve the contents of a given group?
I’ve now been able to return JSON data with a URL along the lines of http://[dtpo].home:52254/desktop/?action=children&id=[UUID]&format=JSON and JSON-feed compliant readers (DEVONthink itself! ) retrieve items from such an address, but with format=RSS the feed is empty.
( = Actually, the only feed reader I can get to work with these feeds so far is DEVONthink… ReadKit for example complains about invalid MIME types even with the search-based URL you suggest when format=RSS)
This is only an internal API for the web interface which might change anytime (and will in the future) and isn’t officially supported yet (contrary to the search URL). Its usage is definitely not recommended right now, I’m afraid.
Thanks for your clarification and apologies for only responding now: I have been mulling over the implications, though, and thinking of ways to restrict myself to the supported URL usage and still achieve what I planned.
I think it will be possible, but will just require some scripting of database content to get it to match what can be queried by the Simple Search interface. And I do respect the recommendation to avoid hooking on to an unsupported API, although will lodge here my hope that in the future it does become both supported and expanded - As its potential is so far-reaching!
One question on the supported URL format, though: Am I right in thinking there is no way to search via Tag?
It’s not present in the Simple Search interface, but this parameter alone would unlock much of the power held within a DEVONthink database. (Searches that served dynamic content from static queries would be more useful if they matched broad groupings)
And if searching via tags (or group) isn’t possible, then the only way to get items to match in a broad search would be to resort to adding manual data to group/classify items, but metadata fields are read-only (even via AppleScript?) so this would probably leave only the Comments field as the place for storing these ‘tags’… (All very un-DEVON-like! )
One question on the supported URL format, though: Am I right in thinking there is no way to search via Tag?
That’s correct. that’s not possible at this time.
Just a note…
but metadata fields are read-only (even via AppleScript?) so this would probably leave only the Comments field as the place for storing these ‘tags’… (All very un-DEVON-like! )
Comments are read-only. Spotlight Comments are not, and are often used for arbitrary metadata.
Another belated check-in on this idea to let you know I’m grateful for the information and am still working on thoughts about making proper use of this. I think that the web server is one of the most underrated gems within DEVONthink (and, thanks to the way in which web UIs seem to need to be reinvented every other week, probably also suffers more than the app as a whole from some users being blindsided by a ‘dated’ interface…), so anything that opens up this technology to new uses deserves some more attention.
So if I ever get round to implementing something I’ll be sure to post back here with the details. I will use Spotlight Comments for now as per your suggestion (though will be on the lookout for any future updates to metadata in DT ), but in the meantime can I just ask for clarification on:
By “metadata fields” I meant those which are written to by the app (pdf, mail, markup downloads, etc.) but are read-only in AppleScript - so the kMDItemComment pair is what you are referring to as ‘off limits’;
Spotlight Comments are those listed just as Comment in the AppleScript dictionary (and are writable), but in some testing I have found they do not seem to sync back to the Finder for me (and external modifications do not get picked up by DEVONthink) - Not a deal-breaker obviously, but is this expected behaviour, or should I be looking to do some mdutil maintenance my end before I start work which relies on them?
Anything listed under meta data of _someFile(which would include kMDItemComment) is read-only at this time.
Yes, comment of _someFile is the Spotlight Comments. Spotlight comments applied to imported items are not going to be found by Spotlight. Spotlight doesn’t index inside packages. If you applied Spotlight Comments to an indexed file out in the filesystem, e.g. your Desktop, Spotlight should find it.
Indexing is thankfully my preferred storage strategy anyway, so all checks out perfectly now that I actually test on something ‘real’…
But again the nuts-and-bolts explanation is helpful: I hadn’t got as far as searching for the items, and was instead just checking Get Info - and was a bit thrown by the fact that DT tags and Finder tags are mirrored perfectly even for items stored internally? Finder tags are just the old Labels with a new name I suppose, so I guess this would be because they’re Extended Attributes and not just part of the Spotlight index.
(But a quick xattr reminds me that so are ‘Spotlight’ Comments, so if Get Info is picking up Tags one way and Comments another then that’s something Apple would have to explain…)
This is also only out of interest, but if the standard DEVONthink storage strategy means that Spotlight won’t see these comments, what drove the decision to specify them as “Spotlight Comments” inside the application? I guess DT’s own published index for spotlight would mean they equate to the same thing when doing a Finder search? Either way knowing that I can maintain them from within DEVONthink or externally is going to be very useful.
All fascinating stuff(!) But more importantly now I’m starting to think they’re not such a bad place to be structuring my database from after all…
Spotlight Comments on the imported files aren’t detected by Spotlight directly. We use a metadata cache (as some other package-based applications do too) for Spotlight to use. Here’s a Spotlight search showing a hit, the DEVONthink icon, and a bit of the file path…
) retrieve items from such an address, but with 
)
)
), but in the meantime can I just ask for clarification on:
