Oh, I get it now. OK, then that’s less disturbing. However, IMHO still unexpected (and apparently undocumented, unless I missed it). So, my two points, about documentation and having a preference, still stand.
I wonder whether the problem would be fixed if Zowie used either com.apple.metadata:kMDItemWhereFroms or the Finder comment but not both.
I wonder whether the problem would be fixed if Zowie used either com.apple.metadata:kMDItemWhereFroms or the Finder comment but not both.
It already has that ability (it’s one of the available methods), but there is a different issue that comes from writing into the wherefroms metadata attribute. I hope to have a detailed explanation of that shortly.
But again, I want to point out that the behavior we’ve been discussing in this thread is independent of Zowie. I think the issue merits discussion on its own. At the very least, if it’s not documented anywhere, it should be. Approximately a zillion times more people use DEVONthink than Zowie; the latter is largely inconsequential, but DEVONthink’s behavior here may catch a lot of other people unaware, even if they don’t report it.
So far I’m actually not aware of any similar reports which are not related to Zowie.
1 Like
OK, I tested it with DEVONthink 3.8 on a macOS 10.14 system and confirm that there’s a problem, at least in my environment.
If I write a com.apple.metadata:kMDItemWhereFroms attribute value with a URL to a PDF file, and then import that file into DEVONthink (by dragging it into a database from a Finder window), DEVONthink shows the value in the URL metadata field.
If I have an indexed folder of PDF files, pick a file in that folder, and write a com.apple.metadata:kMDItemWhereFroms attribute value to a file that is already indexed, DEVONthink does not show the URL in the URL metadata field. This remains true even if I run Update Indexed Items on the parent indexed folder, or on the individual file in question for that matter. I also tried restarting DEVONthink. I can verify that the attribute does exist on the file by going to a shell and doing this:
> xattr -l "/Users/mhucka/databases/zotero-bibliography/storage/H8WLLTPK/Library Futures Foundation 2021 — Report — Controlled Digital Lending - Unlocking the Library’s Full Potential.pdf"
com.apple.metadata:kMDItemWhereFroms:
00000000 62 70 6C 69 73 74 30 30 62 79 62 69 70 6C 69 73 |bplist00bybiplis|
00000010 74 31 2E 30 A1 01 5F 10 26 7A 6F 74 65 72 6F 3A |t1.0.._.&zotero:|
00000020 2F 2F 73 65 6C 65 63 74 2F 6C 69 62 72 61 72 79 |//select/library|
00000030 2F 69 74 65 6D 73 2F 4E 4A 56 45 52 56 32 32 14 |/items/NJVERV22.|
00000040 16 00 00 00 00 00 00 01 01 00 00 00 00 00 00 00 |................|
00000050 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000060 3F |?|
00000061
But in DEVONthink:
This file does not have a Finder comment. The wherefroms attribute is the only attribute it has. (Thus, it is not mixed up with the behavior we were discussing upthread.)
Is there something that needs to be done to make DEVONthink “notice” the change to the attribute value?
(And to answer preemptively the likely question “can’t the attribute be added before the file is indexed?”: not easily. DEVONthink is indexing a folder to which items are added by Zotero, and there’s no available way to make Zotero itself add the wherefrom attribute. This means an extra step or program must be inserted somewhere. The current approach involves triggering a smart rule in DEVONthink to run an external program that sets the wherefrom attribute value, but the triggering event is DEVONthink indexing the file. I suppose we could come up with some more elaborate scheme in which DEVONthink indexes a different folder, and something sync’s the Zotero database folders with the location indexed by DEVONthink after having done the wherefroms attribute manipulations, but that’s bound to be more fragile and error-prone.)
If changing the xattrs don’t generate a filesystem event, DEVONthink shouldn’t update anything. I’d say this appears to be the case.
Hmm. That’s probably the case. You gave me an idea, though: there’s probably some way to force a filesystem event by some other means. I will look into this.
1 Like
The URL is not updated by File > Update Indexed Items, the file’s contents are indexed, tags & comments updated and secondary metadata (e.g. size & dates) too. Basically only data that can be changed in the Finder or other apps.
I’m trying to understand this. Are you saying that, for a file that has been indexed, if another app changes the metadata attribute com.apple.metadata:kMDItemWhereFroms on that file, then there is still nothing that will make DEVONthink notice the change?
Exactly. The only app doing this that I’m aware of is actually Zowie 
Well, now we have an even better answer to this earlier question:
Because we don’t have a choice: nothing will make DEVONthink detect changes made to the file attribute underlying the URL field, and thus when using an external application, the only way to add a URL to a file indexed by DEVONthink is to do it indirectly (in this case, by first writing the URL as a Finder comment and then duplicating it to the URL field).
That’s right and that’s why the Finder comment should actually be sufficient for the current workflow. Changes of the comment are detected and according to A new tool for Zotero users a smart rule copies the comment to the URL field. But probably I’m missing something important
I may be getting confused about something. When asking about why the redundant info is needed, are you talking about the duplication we’re doing in DEVONthink (i.e., the part about copying from the Finder comment to the URL field)? Or are you wondering about why one would use an external tool to write in both the Finder comment and the URL field simultaneously?
About this. According to your description I guess that the Finder comment plus a smart rule should be sufficient.
Ah, okay. I’m sorry if there was confusion about this – this is not what I’m doing (and I don’t think others are doing) and not needed for the current scheme. It is possible to write both simultaneously with Zowie (instead of one or the other), but as you noted, there’s no need.
The reason I went down the rabbit hole of dealing with the wherefroms field is that if DEVONthink detected changes to the attribute value and used it for the URL, then there would be no need to the roundabout approach of first writing the Finder comment and then using a smart rule to update the URL. People could write Zotero select links directly to the wherefroms metadata field, and that alone would be enough – no smart rules or other copying needed. (Avoiding the need for a smart rule would be beneficial, because of timing issues involving interaction with Zotero.)
It would not be necessary to write to both the Finder comment and the wherefroms in either scenario.
1 Like
An update for anyone coming to this thread: I’ve released a new version of Zowie that has a new option (-s or --space on the command line) which makes it write the Finder comments with a trailing space in the text. This is a hack, I know, but it has the effect of making the Finder comment and the URL in DEVONthink sufficiently different from each other that DEVONthink will no longer ignore the Finder comment, when the comment is copied to the URL field by the typical smart rule approach. I find the result less confusing, and others may too, but the option is off by default and has to be turned on deliberately.
As an aside, I’ve also developed some better ways of producing runnable self-contained single-file applications from Python programs. The new release of Zowie now comes with binaries for all versions of macOS.
3 Likes
Downloaded the new version, ran it with the -s and -o flags on my entire library and now it’s all nice and clean. Thanks for figuring that out!
Glad to hear it worked! Thanks for letting me know.
1 Like
@cgrunenberg One last question related to this topic. I was reviewing my smart rule for copying the URL from the Finder comment to the URL field in DEVONthink, and at some point in the past I used “every minute” as the “perform the action” triggering event. I now wonder if there is a more efficient way to trigger the rule. Will any other event be generated if an external application changes the Finder comment on a file indexed by DEVONthink?
No, that doesn’t cause any smart rule events and therefore a scheduled smart rule is still necessary.
1 Like