In the Finder, in DEVONthink or did you use another app?
The Finder comment is added using Zowie.
Zotero data folder is indexed in DT.
- I add new reference in Zotero.
- I add PDF to the reference, Zotero renames it to match refence.
- Hazel is watching the same data folder and recognizes a new PDF, runs Zowie on that PDF and adds zotero: link to Finder comments
- DT smart rule sees new zotero: Finder comment and adds as URL
Works for the vast majority of PDFs that I add…
I wanted to mention (as the author of Zowie) that I’ve also noticed a few rare instances of Finder comments not showing up in DEVONthink. I have not been able to find a definite cause, but here is something that I noticed just a couple of days ago:
It seemed that if I went into the Finder, opened the file info panel, and did something to the Finder comments box that would cause an update (e.g., add a space and delete it again), then it caused DEVONthink to “notice” the comment – it would show up in the DEVONthink inspector.
That’s as far as I’ve gotten because the issue is so hard to study: I can’t make the situation arise deliberately (it’s so rare to begin with), and the tests I did to get this far eliminated the few cases of missing finder comments that I had at all. Based on this thin evidence, though, my suspicions are in the direction of something wonky happening with the macOS metadata system (that whole mdls thing).
What Zowie does to write the comments is to use AppleScript, and a 3 step process: read any existing comment, clear the existing comment, and finally, write the new comment. If anyone wants to see the code exactly, it’s here:
Sequence of events when writing updated comments: https://github.com/mhucka/zowie/blob/0603a6582bdca9c9e73f5b90b7dc916f758fd669/zowie/methods/findercomment.py#L128-L132
Let me try that when I get home–and you’re right, it’s insanely rare, and thus insanely infuriating! Knowing someone else is seeing it, I’ll pay closer attention when it happens and see if i can identify any additional data points…
It probably doesn’t help to solve your problem at all but some time ago I had a problem with comments too (but it was the other way around, i.e. they didn’t show up in Finder). Looking back I guess the quote below probably wasn’t the cause …
… but maybe you want to try what I did in the linked thread (i.e. setting only Finder comments via script) to see whether this could help track down your problem.
Making a change to the Finder comment fixes the issue.
Does indexing/importing a file to another database work as expected or also fail?
I wonder whether skipping the second step (clearing the existing comment) might make a difference.
I remember I had to add that step (clearing old comments) in order to get the Finder (or maybe mdls) to detect that there was a change to the comments. The step should be unnecessary, but I ended up having to include it. I wish I had written down the exact reasons!
If I have time to test it, I’ll report back.
- Moved into database: Finder Comment does not appear.
- Copied file and pasted into Global Inbox: Finder Comment appears.
a. Delete original file from DT and Zotero
b. Drag copied file to Zotero
c. File reappears in index folder–and Finder Comment appears.
I only have two PDFs left with this error–so willing to try anything else, but the pool is small.
Okay, so this narrows things down.
The metadata scheme on macOS is difficult to deal with in general. (This blog posting by Howard Oakley gives some sense of the situation.) There’s a mix of attributes on files/folders, a database (used by Spotlight), and background processes that are supposed to detect changes and update the database as well as notify other applications. The problem we’re seeing could be due to a lot of things, unfortunately.
There may be an alternative that avoids the Finder comments altogether. Zowie has an option to write the Zotero links into the
com.apple.metadata:kMDItemWhereFroms extended attribute (which is what corresponds to the URL field in DEVONthink), but owing to a limitation in DEVONthink at the time, this turned out to be impossible to make work, and so it was necessary to use an indirect approach: write to Finder comments and then use another process (e.g., a Smart Rule in DEVONthink) to copy the information from the comments to the URL field. This is what led to the current approach and what I’ve been suggesting people use. However, the DEVONthink developers, being the awesome bunch that they are, removed the original limitation (I think in 3.7.2 or thereabouts), so in theory it should be possible to make Zowie write the URL field value directly, without going through the step of writing a Finder comment. Unfortunately, in my testing this morning, it appears that changes to the
com.apple.metadata:kMDItemWhereFroms extended attribute are not picked up if a file has already been indexed by DEVONthink: in preliminary testing, it looks like the attribute value must exist prior to the file being imported. I’m still experimenting with this – maybe something else is going on.
For the time being, I think we’re stuck with the current approach and the occasional (thankfully rare) cases where the value doesn’t show up in the Finder comments.
So, some bad news on this front. Something may have changed for the worse in 3.8. If I search in my DEVONthink indexed folder of PDF files for files that have no Finder comments, it finds 800 out of ~4300 PDF files. Looking at them in the Finder, these files do have Finder comments. Here’s an example:
The thing is, DEVONthink must have been able to “see” those Finder comments in the past, because for about a year now, I’ve been using a smart rule to copy the Finder comments to the URL field, and these files do have values for the URL field. (The text I put in the Finder comments are URLs, specifically, Zotero select links.) The URL values could only have come from the Finder comments via this smart rule.
And now here’s where things get really weird. If I type in the Finder comment box of one of these files and simply add a space character at the end (i.e., after the URL), after a few seconds, the comment suddenly shows up in DEVONthink! And if I delete that space character … DEVONthink shows an empty comment again
I can do this repeatedly and watch it happen in real time.
While we can’t rule out the possibility that the Finder is doing something funky when it saves the metadata attribute, IMHO this behavior more likely points to something happening on the DEVONthink side.
Did you add the comments before/after the release?
Based on the date added fields of the files in DEVONthink, all of these were files before the 3.8 release.
I made a test just now, in which I added a new PDF file to the indexed folder and ran the program that adds the Finder comment. The text appeared in the Annotation inspector in DEVONthink almost right away.
Regarding the older files: I’ve run the File ▹ Update Indexed Items command multiple times on the highest folder containing all the subfolders where these files are located, but it doesn’t seem to change anything. Also, and more peculiarly, if I use the option in Zowie of overwriting existing comments (using the clear-then-set approach discussed above), the contents of the Finder comment still do not show up in DEVONthink.
(Update: if I hack Zowie to write the comment with a trailing space (meaning, simply modify the code to write
comment = comment + ' '), then the result does show up in DEVONthink.)
Looking at these problematic files in
xattr, I can’t say that I see any obvious difference between the ones where it works and the ones that show up with blank Finder comments. Here’s what it looks like in a problematic file (where the comment disappears from DEVONthink):
com.apple.metadata:kMDItemFinderComment: 00000000 62 70 6C 69 73 74 30 30 5F 10 26 7A 6F 74 65 72 |bplist00_.&zoter| 00000010 6F 3A 2F 2F 73 65 6C 65 63 74 2F 6C 69 62 72 61 |o://select/libra| 00000020 72 79 2F 69 74 65 6D 73 2F 52 53 53 4B 36 39 44 |ry/items/RSSK69D| 00000030 59 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00 |Y...............| 00000040 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 31 |.1| 00000052
And here’s a file I just added, where the Finder comments show up as expected:
com.apple.metadata:kMDItemFinderComment: 00000000 62 70 6C 69 73 74 30 30 5F 10 26 7A 6F 74 65 72 |bplist00_.&zoter| 00000010 6F 3A 2F 2F 73 65 6C 65 63 74 2F 6C 69 62 72 61 |o://select/libra| 00000020 72 79 2F 69 74 65 6D 73 2F 4A 53 34 5A 54 4A 58 |ry/items/JS4ZTJX| 00000030 41 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00 |A...............| 00000040 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 31 |.1| 00000052
This command checks the modification date of file contents and of file attributes. Only if it’s newer than the internal one the contents are indexed or updated tags/comments read.
Thanks for that info. With this in mind, I tried the following:
Ran the normal (i.e., without a trailing space) file comment code in my program to update the Finder comments of a PDF file in the indexed folder, then ran DT’s Update Indexed Items while watching the Annotation inspector in DT. No change – the comment never showed up.
Run the Unix command
touchon the file while watching the Annotation inspector in DT, then Update Indexed Items. No change – the comment never showed up.
Opened the PDF file in Preview and added some markup, and saved the file, and ran Update Indexed Items. Again, nothing changed in DT.
Not even the modification date?
Oh, I’m sorry, I meant in terms of the Finder comments. The mod date field in DT did indeed change.
“Nothing changed” in the comments field in the Annotation inspector:
Do the comments show up after restarting DEVONthink and then trying the first step again?