Some Finder Comments not Showing in DEVONthink

DEVONthink
1

I have a handful of indexed PDFs that have Finder comments that should be showing up in DT, but aren’t. For context, there are 842 files, and it’s only affecting 5 of them. I’ve tried restarting (DT and Mac), and updating indexed items. Any ideas?

Screen Shot 2021-09-28 at 11.17.02 AM
Screen Shot 2021-09-28 at 11.17.02 AM1328×312 78 KB

1 Like
2

Were these Finder comments added before or after indexing them?

3

After.

4

What operating system are you running?

5

In the Finder, in DEVONthink or did you use another app?

6

Catalina.

7

The Finder comment is added using Zowie.

Zotero data folder is indexed in DT.

  1. I add new reference in Zotero.
  2. I add PDF to the reference, Zotero renames it to match refence.
  3. Hazel is watching the same data folder and recognizes a new PDF, runs Zowie on that PDF and adds zotero: link to Finder comments
  4. DT smart rule sees new zotero: Finder comment and adds as URL

Works for the vast majority of PDFs that I add…

8

I wanted to mention (as the author of Zowie) that I’ve also noticed a few rare instances of Finder comments not showing up in DEVONthink. I have not been able to find a definite cause, but here is something that I noticed just a couple of days ago:

It seemed that if I went into the Finder, opened the file info panel, and did something to the Finder comments box that would cause an update (e.g., add a space and delete it again), then it caused DEVONthink to “notice” the comment – it would show up in the DEVONthink inspector.

That’s as far as I’ve gotten because the issue is so hard to study: I can’t make the situation arise deliberately (it’s so rare to begin with), and the tests I did to get this far eliminated the few cases of missing finder comments that I had at all. Based on this thin evidence, though, my suspicions are in the direction of something wonky happening with the macOS metadata system (that whole mdls thing).

What Zowie does to write the comments is to use AppleScript, and a 3 step process: read any existing comment, clear the existing comment, and finally, write the new comment. If anyone wants to see the code exactly, it’s here:

1 Like
9

Let me try that when I get home–and you’re right, it’s insanely rare, and thus insanely infuriating! Knowing someone else is seeing it, I’ll pay closer attention when it happens and see if i can identify any additional data points…

10

It probably doesn’t help to solve your problem at all but some time ago I had a problem with comments too (but it was the other way around, i.e. they didn’t show up in Finder). Looking back I guess the quote below probably wasn’t the cause …

… but maybe you want to try what I did in the linked thread (i.e. setting only Finder comments via script) to see whether this could help track down your problem.

2 Likes
11

Making a change to the Finder comment fixes the issue.

1 Like
12

Does indexing/importing a file to another database work as expected or also fail?

13

I wonder whether skipping the second step (clearing the existing comment) might make a difference.

14

I remember I had to add that step (clearing old comments) in order to get the Finder (or maybe mdls) to detect that there was a change to the comments. The step should be unnecessary, but I ended up having to include it. I wish I had written down the exact reasons!

If I have time to test it, I’ll report back.

15
  1. Moved into database: Finder Comment does not appear.
  2. Copied file and pasted into Global Inbox: Finder Comment appears.
    a. Delete original file from DT and Zotero
    b. Drag copied file to Zotero
    c. File reappears in index folder–and Finder Comment appears.

I only have two PDFs left with this error–so willing to try anything else, but the pool is small.

16

Okay, so this narrows things down.

The metadata scheme on macOS is difficult to deal with in general. (This blog posting by Howard Oakley gives some sense of the situation.) There’s a mix of attributes on files/folders, a database (used by Spotlight), and background processes that are supposed to detect changes and update the database as well as notify other applications. The problem we’re seeing could be due to a lot of things, unfortunately.

There may be an alternative that avoids the Finder comments altogether. Zowie has an option to write the Zotero links into the com.apple.metadata:kMDItemWhereFroms extended attribute (which is what corresponds to the URL field in DEVONthink), but owing to a limitation in DEVONthink at the time, this turned out to be impossible to make work, and so it was necessary to use an indirect approach: write to Finder comments and then use another process (e.g., a Smart Rule in DEVONthink) to copy the information from the comments to the URL field. This is what led to the current approach and what I’ve been suggesting people use. However, the DEVONthink developers, being the awesome bunch that they are, removed the original limitation (I think in 3.7.2 or thereabouts), so in theory it should be possible to make Zowie write the URL field value directly, without going through the step of writing a Finder comment. Unfortunately, in my testing this morning, it appears that changes to the com.apple.metadata:kMDItemWhereFroms extended attribute are not picked up if a file has already been indexed by DEVONthink: in preliminary testing, it looks like the attribute value must exist prior to the file being imported. I’m still experimenting with this – maybe something else is going on.

For the time being, I think we’re stuck with the current approach and the occasional (thankfully rare) cases where the value doesn’t show up in the Finder comments.

1 Like
17

So, some bad news on this front. Something may have changed for the worse in 3.8. If I search in my DEVONthink indexed folder of PDF files for files that have no Finder comments, it finds 800 out of ~4300 PDF files. Looking at them in the Finder, these files do have Finder comments. Here’s an example:

image
image1920×590 111 KB

image

The thing is, DEVONthink must have been able to “see” those Finder comments in the past, because for about a year now, I’ve been using a smart rule to copy the Finder comments to the URL field, and these files do have values for the URL field. (The text I put in the Finder comments are URLs, specifically, Zotero select links.) The URL values could only have come from the Finder comments via this smart rule.

And now here’s where things get really weird. If I type in the Finder comment box of one of these files and simply add a space character at the end (i.e., after the URL), after a few seconds, the comment suddenly shows up in DEVONthink! And if I delete that space character … DEVONthink shows an empty comment again :open_mouth:

I can do this repeatedly and watch it happen in real time.

While we can’t rule out the possibility that the Finder is doing something funky when it saves the metadata attribute, IMHO this behavior more likely points to something happening on the DEVONthink side.

18

Did you add the comments before/after the release?

19

Based on the date added fields of the files in DEVONthink, all of these were files before the 3.8 release.

I made a test just now, in which I added a new PDF file to the indexed folder and ran the program that adds the Finder comment. The text appeared in the Annotation inspector in DEVONthink almost right away.

Regarding the older files: I’ve run the File ▹ Update Indexed Items command multiple times on the highest folder containing all the subfolders where these files are located, but it doesn’t seem to change anything. Also, and more peculiarly, if I use the option in Zowie of overwriting existing comments (using the clear-then-set approach discussed above), the contents of the Finder comment still do not show up in DEVONthink.

(Update: if I hack Zowie to write the comment with a trailing space (meaning, simply modify the code to write comment = comment + ' '), then the result does show up in DEVONthink.)

Looking at these problematic files in xattr, I can’t say that I see any obvious difference between the ones where it works and the ones that show up with blank Finder comments. Here’s what it looks like in a problematic file (where the comment disappears from DEVONthink):

com.apple.metadata:kMDItemFinderComment:
00000000  62 70 6C 69 73 74 30 30 5F 10 26 7A 6F 74 65 72  |bplist00_.&zoter|
00000010  6F 3A 2F 2F 73 65 6C 65 63 74 2F 6C 69 62 72 61  |o://select/libra|
00000020  72 79 2F 69 74 65 6D 73 2F 52 53 53 4B 36 39 44  |ry/items/RSSK69D|
00000030  59 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00  |Y...............|
00000040  00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
00000050  00 31                                            |.1|
00000052

And here’s a file I just added, where the Finder comments show up as expected:

com.apple.metadata:kMDItemFinderComment:
00000000  62 70 6C 69 73 74 30 30 5F 10 26 7A 6F 74 65 72  |bplist00_.&zoter|
00000010  6F 3A 2F 2F 73 65 6C 65 63 74 2F 6C 69 62 72 61  |o://select/libra|
00000020  72 79 2F 69 74 65 6D 73 2F 4A 53 34 5A 54 4A 58  |ry/items/JS4ZTJX|
00000030  41 08 00 00 00 00 00 00 01 01 00 00 00 00 00 00  |A...............|
00000040  00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
00000050  00 31                                            |.1|
00000052
20

This command checks the modification date of file contents and of file attributes. Only if it’s newer than the internal one the contents are indexed or updated tags/comments read.