Some Finder Comments not Showing in DEVONthink

DEVONthink
21

Thanks for that info. With this in mind, I tried the following:

  • Ran the normal (i.e., without a trailing space) file comment code in my program to update the Finder comments of a PDF file in the indexed folder, then ran DT’s Update Indexed Items while watching the Annotation inspector in DT. No change – the comment never showed up.

  • Run the Unix command touch on the file while watching the Annotation inspector in DT, then Update Indexed Items. No change – the comment never showed up.

  • Opened the PDF file in Preview and added some markup, and saved the file, and ran Update Indexed Items. Again, nothing changed in DT.

22

Not even the modification date?

23

Oh, I’m sorry, I meant in terms of the Finder comments. The mod date field in DT did indeed change.

image

“Nothing changed” in the comments field in the Annotation inspector:

image
image548×648 23 KB

24

Do the comments show up after restarting DEVONthink and then trying the first step again?

25

Can confirm–dozens of PDFs with Finder Comments–that had been recognized in the past because they have the URL updated, not showing the Comment in DT, if I add a space, in realtime it shows up, and if I remove it, it disappears.

26

This morning I finally restarted DEVONthink and tested this. The behavior remains the same. Search shows 801 PDF files (as before). Updating the index does not change the result. If I pick a PDF file at random from the search results, go to it in the Finder, look at the Finder’s Get Info panel and the comment box within it, there is a comment.

  • If I then run Zowie (which uses AppleScript code underneath) to force-update the comment, still, nothing changes in the DEVONthink inspector view, even after minutes pass.

  • If I update the DEVONthink index, the DEVONthink annotation inspector still does not show anything in the comment box.

  • If I go to the Finder’s Get Info panel and add a trailing space in the comment box, after a few seconds, DEVONthink’s inspector shows the comment.

  • If I delete the trailing space from the Finder’s Get Info panel, after a couple of seconds the comment then disappears from DEVONthink’s inspector.

I tried some additional things:

  • Other characters (not just spaces) cause the same behavior. Inserting a character at the beginning of the comment in the Finder, instead of at the end of the comment text, also does the same.

  • If instead of adding characters, I delete a character from the existing comment string (the one that seems to be invisible to DEVONthink), the comment appears in the DEVONthink annotation inspector. (This renders the URL in the comment invalid, but for some reason, something about the change makes it visible to DEVONthink.)

  • If I delete the comment text completely from the Finder’s Get Info panel, DEVONthink still shows a comment. Even if I run Update Indexed Items, the comment is still visible in the annotation inspector.

This is DEVONthink 3.8 running on macOS 10.14.6. The indexed folder is on the local drive, and is not in Dropbox or iCloud or similar sync service, but it is in a folder managed by Zotero (but Zotero is not running at the time).

27

Hold on. I just remembered something: doesn’t DEVONthink have special handling for Finder comments that contain only URLs? I’m searching for the info now …

Update: I found it. @cgrunenberg you had replied to a support email of mine in December 2020, in answer to my reporting unexpected behavior involving Finder comments and the macOS “where from” file metadata attribute, and in this reply you had written the following:

this is actually not a new behaviour, DEVONthink does this for a long time as downloads in the past quite often added the URL to the Finder comments too. Therefore the comment is ignored while importing/indexing if it’s identical to the extended attribute.

Could this DEVONthink feature be implicated in the behavior we’re seeing here? These files all have the URL in the URL field, and that value is typically identical to the text in the Finder comment. And editing the value in the Finder comment would cause the value to be different from the URL field – which would explain why DEVONthink then shows the comment.

28

That’s indeed possible, Finder comments which are identical to the URL are still ignored (e.g. as they were often added by downloaded managers in the past). This makes me wonder why this redundant information (as URL and as comment) is actually necessary?

29

Thank you for the explanation and the confirmation. In answer to this question,

It’s true that it’s not actually necessary to have it in both places, but DEVONthink’s features for working with URLs (such as opening the URL in Safari via a keyboard shortcut, functions in AppleScript, etc.) are only available when a URL is present in the URL metadata field and not in the Finder comments. So, while one could leave the URL as a Finder comment and be done with it, IMHO, it’s not nearly as convenient to do so.

It would be possible to compensate by doing things like writing scripts or using Keyboard Maestro or do similar things to work with URLs in the Finder comments, but … DEVONthink has a URL field :slight_smile: . I don’t mean this in a snarky way, but it seems natural and logical to use it, even at the cost of duplicating a short bit of data in two places.

Now, going back to DEVONthink’s behavior in this situation, to summarize, what is happening is:

  1. an external process writes a URL as a Finder comment on a file
  2. a smart rule in DEVONthink is triggered by the addition of this comment, and copies the information to the URL metadata field
  3. DEVONthink, automatically detecting that the URL field and the Finder comment now both contain identical text, silently hides the Finder comment from the view in Inspector panel (but the Finder comment is left alone, and still exists)

This brings to mind some questions and comments:

  1. I looked through the documentation for mention of #3, and could not find it. Did I miss it? If not, could this behavior please be documented?

  2. Apparently I’m not the only person who finds this behavior at least unexpected, if not downright disturbing. (It’s hiding information from the user! This is … not normally considered a good thing.) Would it be possible to please add a preference, perhaps a hidden user setting, that lets the user control whether DEVONthink does this?

Finally, there remains a separate question about whether we can write the URL into the macOS “wherefroms” metadata field directly, instead of using the Finder comments. I’ll investigate that and report back.

1 Like
30

Actually the comment is ignored while indexing/importing or updating indexed items if it’s identical to the URL field. The comment is not hidden.

31

Oh, I get it now. OK, then that’s less disturbing. However, IMHO still unexpected (and apparently undocumented, unless I missed it). So, my two points, about documentation and having a preference, still stand.

32

I wonder whether the problem would be fixed if Zowie used either com.apple.metadata:kMDItemWhereFroms or the Finder comment but not both.

33

I wonder whether the problem would be fixed if Zowie used either com.apple.metadata:kMDItemWhereFroms or the Finder comment but not both.

It already has that ability (it’s one of the available methods), but there is a different issue that comes from writing into the wherefroms metadata attribute. I hope to have a detailed explanation of that shortly.

But again, I want to point out that the behavior we’ve been discussing in this thread is independent of Zowie. I think the issue merits discussion on its own. At the very least, if it’s not documented anywhere, it should be. Approximately a zillion times more people use DEVONthink than Zowie; the latter is largely inconsequential, but DEVONthink’s behavior here may catch a lot of other people unaware, even if they don’t report it.

34

So far I’m actually not aware of any similar reports which are not related to Zowie.

1 Like
35

OK, I tested it with DEVONthink 3.8 on a macOS 10.14 system and confirm that there’s a problem, at least in my environment.

If I write a com.apple.metadata:kMDItemWhereFroms attribute value with a URL to a PDF file, and then import that file into DEVONthink (by dragging it into a database from a Finder window), DEVONthink shows the value in the URL metadata field.

If I have an indexed folder of PDF files, pick a file in that folder, and write a com.apple.metadata:kMDItemWhereFroms attribute value to a file that is already indexed, DEVONthink does not show the URL in the URL metadata field. This remains true even if I run Update Indexed Items on the parent indexed folder, or on the individual file in question for that matter. I also tried restarting DEVONthink. I can verify that the attribute does exist on the file by going to a shell and doing this:

> xattr -l "/Users/mhucka/databases/zotero-bibliography/storage/H8WLLTPK/Library Futures Foundation 2021 — Report — Controlled Digital Lending - Unlocking the Library’s Full Potential.pdf"
com.apple.metadata:kMDItemWhereFroms:
00000000  62 70 6C 69 73 74 30 30 62 79 62 69 70 6C 69 73  |bplist00bybiplis|
00000010  74 31 2E 30 A1 01 5F 10 26 7A 6F 74 65 72 6F 3A  |t1.0.._.&zotero:|
00000020  2F 2F 73 65 6C 65 63 74 2F 6C 69 62 72 61 72 79  |//select/library|
00000030  2F 69 74 65 6D 73 2F 4E 4A 56 45 52 56 32 32 14  |/items/NJVERV22.|
00000040  16 00 00 00 00 00 00 01 01 00 00 00 00 00 00 00  |................|
00000050  02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
00000060  3F                                               |?|
00000061

But in DEVONthink:

image
image556×698 55.6 KB

This file does not have a Finder comment. The wherefroms attribute is the only attribute it has. (Thus, it is not mixed up with the behavior we were discussing upthread.)

Is there something that needs to be done to make DEVONthink “notice” the change to the attribute value?

(And to answer preemptively the likely question “can’t the attribute be added before the file is indexed?”: not easily. DEVONthink is indexing a folder to which items are added by Zotero, and there’s no available way to make Zotero itself add the wherefrom attribute. This means an extra step or program must be inserted somewhere. The current approach involves triggering a smart rule in DEVONthink to run an external program that sets the wherefrom attribute value, but the triggering event is DEVONthink indexing the file. I suppose we could come up with some more elaborate scheme in which DEVONthink indexes a different folder, and something sync’s the Zotero database folders with the location indexed by DEVONthink after having done the wherefroms attribute manipulations, but that’s bound to be more fragile and error-prone.)

36

If changing the xattrs don’t generate a filesystem event, DEVONthink shouldn’t update anything. I’d say this appears to be the case.

37

Hmm. That’s probably the case. You gave me an idea, though: there’s probably some way to force a filesystem event by some other means. I will look into this.

1 Like
38

The URL is not updated by File > Update Indexed Items, the file’s contents are indexed, tags & comments updated and secondary metadata (e.g. size & dates) too. Basically only data that can be changed in the Finder or other apps.

39

I’m trying to understand this. Are you saying that, for a file that has been indexed, if another app changes the metadata attribute com.apple.metadata:kMDItemWhereFroms on that file, then there is still nothing that will make DEVONthink notice the change?

40

Exactly. The only app doing this that I’m aware of is actually Zowie :slight_smile: