# Checksums ... ;-)

**URL:** <https://discourse.devontechnologies.com/t/checksums/73483>\
**Category:** DEVONthink\
**Created:** [November 12, 2022, 10:59pm UTC](https://discourse.devontechnologies.com/t/checksums/73483 "2022-11-12T22:59:00Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 21, 2022, 10:40pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/41 "2022-11-21T22:40:05Z")

</div>

> [@Ventura, DT3, Arq, Wasabi: Immutable backups failing silently](https://discourse.devontechnologies.com/t/ventura-dt3-arq-wasabi-immutable-backups-failing-silently/73361/3):
>
> [Michael Tsai - Blog - Ventura Bug Disables Security Software](https://mjtsai.com/blog/2022/11/02/ventura-bug-disables-security-software/)

Thanks, going to read this!

Meanwhile, I opened a support ticket (over the link from @BLUEFROG above)

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 21, 2022, 10:47pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/42 "2022-11-21T22:47:15Z")

</div>

I did as recommended in this article and removed DT from Full Disk Access, then started it again - and I got asked to give DT access to Network Volumes, which I gave.

Also, I added it again to “Full Disk Access” …

Sadly, the problem remains.

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 21, 2022, 11:04pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/43 "2022-11-21T23:04:08Z")

</div>

And then I tested again:

Created a new text file from DTTG in the above …/ARCHIVE group:

 ![image](https://devontech-discourse.s3.dualstack.us-east-1.amazonaws.com/uploads/original/3X/a/9/a9e9fd2f8997a2e43295f620384305749e0eb84a.jpeg)

It got uploaded to WebDAV and … bammmm… it also showed up fine in DT:

![Screenshot 2022-11-22 at 00.04.27](https://devontech-discourse.s3.dualstack.us-east-1.amazonaws.com/uploads/original/3X/2/c/2c1812170c0d215f1b6812a07bab903c3a155316.png)

So, the problem only exists for certain file types, but not for others.  
Very strange.

But DT can indeed download new files from WebDAV and write them to the indexed folders - just not for every content.

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [November 22, 2022, 8:16am UTC](https://discourse.devontechnologies.com/t/checksums/73483/44 "2022-11-22T08:16:38Z")

</div>

> [@tja](#):
>
> But DT can indeed download new files from WebDAV and write them to the indexed folders - just not for every content.

Did you check the ownership & permissions of the folders? Sometimes a permission issue is just a permission issue. You could also enable the option to ignore the ownership on the network volume.

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 22, 2022, 9:41am UTC](https://discourse.devontechnologies.com/t/checksums/73483/45 "2022-11-22T09:41:06Z")

</div>

I think I proved that this is not the problem:

As the logged in user, both a terminal shell and DT can write to the folder!  
The permissions are also visibly OK and there are no ACLs that could be a problem.

DT can write itself to the target folder, when creating a new file from within DT!

And DT can write new files that it get’s over WebDAV when such a new file is created on an iPad (and uploaded to WebDAV).

But it does not work for all type of files, as it seems - of course the technical explanation may be different.

All of this was shown in my above text-snippets and screenshots.

DT strongly says, that there is no permission to access the folder “ARCHIVE” while it can write to it without problems for new textfiles - regardless of how it get’s added (by creating from DT or by creating over WebDAV from an iPad)

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 22, 2022, 9:42am UTC](https://discourse.devontechnologies.com/t/checksums/73483/46 "2022-11-22T09:42:18Z")

</div>

> [@cgrunenberg](#):
>
> You could also enable the option to ignore the ownership on the network volume.

What excactly do you mean by this?

Anyways, DT can write to it!

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [November 22, 2022, 11:56am UTC](https://discourse.devontechnologies.com/t/checksums/73483/47 "2022-11-22T11:56:25Z")

</div>

> [@tja](#):
>
> I think I proved that this is not the problem:

Well, that’s at least the error returned by macOS. Therefore it’s either a wrong error message or maybe a temporary issue of the WebDAV server, e.g. maybe due to concurrent access by multiple devices.

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 22, 2022, 1:32pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/48 "2022-11-22T13:32:18Z")

</div>

I don’t know how to repeat this in a way that is more clear…

DT can add lots and lots of files coming from WebDAV without any problem at all.  
There is no problem with WebDAV!

DT can also see all of the WebDAV files (as the names and sizes are shown) and it can download the WebDAV files - but only to some cache folder and then cannot move them to the destination folder:

At least, the log files says that the file got download to “NSSourceFilePathErrorKey=/Users/tja/Library/Caches/TemporaryItems/DEVONcloudy 12795/Main Thread/360/IMG\_0001.png”

Also, DT clearly has the rights required to do this - as tested with textfiles:

Created on an iPad, the files get uploaded to WebDAV and DT on the Mac perfectly fine downloads them to the indexed folders!

This problem is strange but seems to be a local DT problem!  
Or, of course something from macOS.

Meanwhile, I also rebooted the Mac just to be sure … no change

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 22, 2022, 1:40pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/49 "2022-11-22T13:40:20Z")

</div>

To prove that the download from WebDAV works, I let this run:

```auto
(base) tja@mini:~$ while ! ls /Users/tja/Library/Caches/TemporaryItems/DEVONcloudy\ 12795/Main\ Thread/* 2>/dev/null ; do : ; done
total 400
-rw-r--r--@ 1 tja staff 201443 Nov 22 14:38 IMG_0001.png
(base) tja@mini:~$

```

As you can see, the loop exited when the PNG file was correctly downloaded from WebDAV to the cache folder!

But DT cannot move it to the destination folder, as it says in the logs.

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [November 22, 2022, 1:43pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/50 "2022-11-22T13:43:25Z")

</div>

> [@tja](#):
>
> DT can add lots and lots of files coming from WebDAV without any problem at all.

A network volume mounted in the Finder is actually not the same as directly accessing a WebDAV server (like the synchronization does). Did you also check the permissions of `/Users/tja/Library/Caches/TemporaryItems/`?

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 22, 2022, 1:44pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/51 "2022-11-22T13:44:30Z")

</div>

Also, to show this again, I added 2 more files to DTTG:

 ![image](https://devontech-discourse.s3.dualstack.us-east-1.amazonaws.com/uploads/original/3X/b/d/bd132b88f12e7ed57522a85efbb70066a3914274.jpeg)

 ![image](https://devontech-discourse.s3.dualstack.us-east-1.amazonaws.com/uploads/original/3X/1/a/1ad747769a0ab30cac00a5fc8b4cbd1428f9bd00.jpeg)

So, they got added on iPad…

And then, looking on DT:

 ![Screenshot 2022-11-22 at 14.42.59](https://devontech-discourse.s3.dualstack.us-east-1.amazonaws.com/uploads/original/3X/2/e/2eea06ce943504e759e0f3952bf2f7dbad831d60.png)

They appeared without problem!

But NOT the PNG files…

Per terminal:

```auto
(base) tja@mini:/Volumes/DT_MAIN/ARCHIVES$ ls
total 48
drwxr-xr-x 1 tja staff 192 Jul 30 23:17 archive/
drwxr-xr-x 1 tja staff 416 Jul 30 23:17 DO_NO_DELETE/
drwxr-xr-x 1 tja staff 160 Jul 30 23:17 BACKUPS/
-rw-r--r-- 1 tja staff 5 Nov 21 23:58 createdByDDTG.txt
-rw-r--r-- 1 tja staff 1087 Nov 22 14:38 daily_dates.log
-rw-r--r-- 1 tja staff 1681 Nov 22 14:38 rsync_XTRMQ-T7_Touch2TB.sh

```

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 22, 2022, 1:49pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/52 "2022-11-22T13:49:37Z")

</div>

Yes, please see above - the PNG get’s correctly downloaded to the cache folder!

But it cannot be moved - while other downloaded files can get moved!

Very strange

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 24, 2022, 8:04pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/53 "2022-11-24T20:04:03Z")

</div>

@BLUEFROG  
@cgrunenberg

I found the problem!

First, how I did this:

I let a loop run to wait for DT to download a file from WebDAV and copied it to ~/TMP

`(base) tja@mini:~$ while ! cp -p /Users/tja/Library/Caches/TemporaryItems/DEVONcloudy\ 12795/Main\ Thread/*/* ~/TMP/ 2>/dev/null ; do : ; done`

This produced the following file:

```auto
(base) tja@mini:~$ ls ~/TMP/IMG_0001.png
-rw-r--r--@ 1 tja staff 201443 Nov 12 20:35 /Users/tja/TMP/IMG_0001.png

```

Now, I just tried to simulate what DT would be doing:

```auto
(base) tja@mini:~$ mv ~/TMP/IMG_0001.png /Volumes/DT_MAIN/ARCHIVES/
mv: /Volumes/DT_MAIN/ARCHIVES/IMG_0001.png: unable to move extended attributes and ACL from /Users/tja/TMP/IMG_0001.png: Operation not permitted

```

This gives a harmless error message, because the target file system cannot handle those unimportant ACLs or extended attributes …

The “mv” did work, of course:

```auto
(base) tja@mini:~$ ls ~/TMP/
total 0

(base) tja@mini:~$ ls /Volumes/DT_MAIN/ARCHIVES/
total 448
drwxr-xr-x 1 tja staff 192 Jul 30 23:17 archive/
drwxr-xr-x 1 tja staff 416 Jul 30 23:17 DO_NO_DELETE/
drwxr-xr-x 1 tja staff 160 Jul 30 23:17 BACKUPS/
-rw-r--r-- 1 tja staff 201443 Nov 12 20:35 IMG_0001.png
-rw-r--r-- 1 tja staff 5 Nov 21 23:58 createdByDDTG.txt
-rw-r--r-- 1 tja staff 1087 Nov 22 14:38 daily_dates.log
-rw-r--r-- 1 tja staff 1681 Nov 22 14:38 rsync_XTRMQ-T7_Touch2TB.sh

```

So, the file is where it should be.

The only explanation for this is, that DT “interpretes” the apparent error message from the move, because of the missing ACL / extended attribute support und simply removes this file again!

This is very frustrating, as I see no way to fix this by myself!  
Files from anywhere may have added ACLs / extended attributes… and I have no way of removing them.

Only an option to DT that allows to ignore ACLs / extended attributes or ignores the error messages from “mv” could help.

I cannot change the fact that some files come with some ACL / extended setting - but I have no interest in them!

Any chance for such an option to DT?  
Any other idea how to fix this?

Many thanks

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 24, 2022, 8:11pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/54 "2022-11-24T20:11:05Z")

</div>

As it seems, some files from iPhone or iPad just come with ACLs / extended attributes set and WebDAV seems to be able to handle them too.

But not all file systems can do this!

This is normally not a problem at all …  
You just get a message that the ACLs / extended attributes could not be copied.

But DT handle this in a way that such files can never ever be downloaded!

This is a problem.

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 24, 2022, 8:19pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/55 "2022-11-24T20:19:31Z")

</div>

I also checked, what exactly is so important althought it should be unimportant:

```auto
(base) tja@mini:~/TMP$ ls -le IMG_0003.png
-rw-r--r--@ 1 tja staff 151794 Nov 12 20:35 IMG_0003.png

```

So it’s not ACLs!

```auto
(base) tja@mini:~/TMP$ ls -l@ IMG_0003.png
-rw-r--r--@ 1 tja staff 151794 Nov 12 20:35 IMG_0003.png
	com.apple.assetsd.UUID 16
	com.apple.assetsd.addedDate 50
	com.apple.assetsd.assetType 2
	com.apple.assetsd.avalanche.type 2
	com.apple.assetsd.creatorBundleID 21
	com.apple.assetsd.customCreationDate 50
	com.apple.assetsd.dbRebuildUuid 36
	com.apple.assetsd.deferredProcessing 2
	com.apple.assetsd.favorite 2
	com.apple.assetsd.grouping.state 8
	com.apple.assetsd.hidden 2
	com.apple.assetsd.importedBy 2
	com.apple.assetsd.originalFilename 12
	com.apple.assetsd.publicGlobalUUID 36
	com.apple.assetsd.timeZoneName 13
	com.apple.assetsd.timeZoneOffset 4
	com.apple.assetsd.trashed 2
	com.apple.assetsd.videoComplementVisibility 2

```

It’s Apple crap in extended attributes!  
😑

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 24, 2022, 8:34pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/56 "2022-11-24T20:34:22Z")

</div>

And finally, this is really a problem of DT:

Both “cp” and “mv” work flawlessly and do NOT return with an exit code that is not 0:

```auto
(base) tja@mini:~/TMP$ cp -p ~/TMP/IMG_0003.png /Volumes/DT_MAIN/ARCHIVES/
cp: /Users/tja/TMP/IMG_0003.png: could not copy extended attributes to /Volumes/DT_MAIN/ARCHIVES/IMG_0003.png: Operation not permitted
(base) tja@mini:~/TMP$ echo $?
0

```

```auto
(base) tja@mini:~/TMP$ mv ~/TMP/IMG_0003.png /Volumes/DT_MAIN/ARCHIVES/
mv: /Volumes/DT_MAIN/ARCHIVES/IMG_0003.png: unable to move extended attributes and ACL from /Users/tja/TMP/IMG_0003.png: Operation not permitted
(base) tja@mini:~/TMP$ echo $?
0

```

This proves that the commands work totally find - the error is just a message that should be ignored, it is NOT an actuall error that would exit != 0

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 24, 2022, 8:57pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/57 "2022-11-24T20:57:59Z")

</div>

And finally, this may well explain my constant problems with the number of items being different between iDevices and DT on Mac …

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [November 25, 2022, 7:48am UTC](https://discourse.devontechnologies.com/t/checksums/73483/58 "2022-11-25T07:48:32Z")

</div>

> [@tja](#):
>
> This proves that the commands work totally find - the error is just a message that should be ignored, it is NOT an actuall error that would exit != 0

Actually DEVONthink doesn’t use shell scripts for copying/moving but Cocoa’s filemanager instead which returns a useless error presented by DEVONthink unfortunately.

What kind of filesystem does the server actually use? I’ve never experienced this on my own despite using various WebDAV servers. And how did you exactly add/create these files in DEVONthink To Go?

Anyway, thanks for the useful debugging - highly appreciated.

---

<div class="post-metadata">

**Author:** ![tja](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/tja/32/7004_2.png) [@tja](https://discourse.devontechnologies.com/u/tja)\
**Post date:** [November 25, 2022, 1:06pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/59 "2022-11-25T13:06:42Z")

</div>

Ah, I replied to this by email.

Did you get this?

---

<div class="post-metadata">

**Author:** ![cgrunenberg](https://discourse.devontechnologies.com/user_avatar/discourse.devontechnologies.com/cgrunenberg/32/7172_2.png) [@cgrunenberg](https://discourse.devontechnologies.com/u/cgrunenberg)\
**Post date:** [November 25, 2022, 1:08pm UTC](https://discourse.devontechnologies.com/t/checksums/73483/60 "2022-11-25T13:08:00Z")

</div>

I didn’t receive any email actually. Or do you refer to your ticket?

[Previous page](https://discourse.devontechnologies.com/t/checksums/73483.md?page=2)

[Next page](https://discourse.devontechnologies.com/t/checksums/73483.md?page=4)
