Some exported Files and Folders can't be read by backup programmes

mikebore · April 28, 2020, 11:32am

I have a really strange problem on one of my two Macs, both running latest DT3 on latest Catalina. The other Mac does not have the problem at all.

On the problem Mac I exported all the Files and Folders in a small database to a folder in my user directory. When I try to back up this folder both Carbon Copy Cloner and Chrososync give a load of errors about some of the files, not all. The problem is when reading the exported files (not writing). Chronosync mentions permissions problem.

The permissions on the folder exported from DT3 are:
michaelb: read and write
Staff: read only
Everyone: read only

I have changed these with Get Info to all three being read and write, and applied to all enclosed, but the problem persists. I verified that the permissions had changed on some of the problem files.

I have run repairHomePermissions from Terminal while in Recovery as this Apple KB without effect.

I am currently running full installer again as recommended in the Apple KB.

The same happens with all my six databases, I have just been focussing on this small one for diagnostics.

Another snippet is that if I export the Files and Folders to an external on the good Mac, then move that external to the bad Mac and copy them into the same user folder there, they backup fine. So the problem is some obscure property of some exported files on the bad Mac. I have not seen any other strange behaviour on the bad Mac.

Clearly this is a problem with this one Mac since the other is OK, but I am running out of ideas!

Unfortunately Bad Mac is my main 27" iMac (2017), and Good Mac is a 12" MacBook (2017).

Thanks

cgrunenberg · April 28, 2020, 11:40am

These permissions should actually be sufficient for you. Is the Finder able to copy this folder?

mikebore · April 28, 2020, 12:49pm

Thanks. Yes. Finder can copy it.

Have completed the OS reinstall and that has not solved anything.

cgrunenberg · April 28, 2020, 12:51pm

If the Finder is able to copy them then any app should be able to do this actually. Does this also happen using a new database and some dummy files?

BLUEFROG · April 28, 2020, 1:29pm

What are the names of a few files?

mikebore · April 28, 2020, 2:21pm

I have been focussing on a particular folder for diagnostics work. It has 17 files in and three of these will not back up by CCC and Chronosync. I exported again to the Desktop and now see that iCloud will not upload these three either. See screenshot, which gives you some names.

I tried creating a new database and added these 17 files to it. DT3 would not accept these three files.

mikebore · April 28, 2020, 2:32pm

I have also tried running xattr in Terminal to see if any attributes could explain what is going on. In the output below, the first three files examined are the same bad ones as my previous post. The last two are good file in the same folder of 17. This is beyond my expertise, but nothing is shouting at me!

Last login: Tue Apr 28 14:49:26 on ttys000

FIle 1

michaelb@iMac11 ~ % xattr /Users/michaelb/Desktop/BRITISH\ LAND\ 2005\ iMac/Groups\ 2005\ final.doc

com.apple.fileutil.PlaceholderData

com.apple.macl

com.apple.metadata:kMDItemOMUserTagTime

com.apple.metadata:kMDItemOMUserTags

com.dropbox.attributes

com.dropbox.attrs

michaelb@iMac11 ~ %

File 2

michaelb@iMac11 ~ % xattr /Users/michaelb/Desktop/BRITISH\ LAND\ 2005\ iMac/Matrix2005_rnd6.5.xls

com.apple.fileutil.PlaceholderData

com.apple.macl

com.apple.metadata:kMDItemOMUserTagTime

com.apple.metadata:kMDItemOMUserTags

com.dropbox.attributes

com.dropbox.attrs

michaelb@iMac11 ~ %

File 3

michaelb@iMac11 ~ % xattr /Users/michaelb/Desktop/BRITISH\ LAND\ 2005\ iMac/Qualifiers\ letter\ final.doc

com.apple.fileutil.PlaceholderData

com.apple.macl

com.apple.metadata:kMDItemOMUserTagTime

com.apple.metadata:kMDItemOMUserTags

com.dropbox.attributes

com.dropbox.attrs

michaelb@iMac11 ~ %

File 4

michaelb@iMac11 ~ % xattr /Users/michaelb/Desktop/BRITISH\ LAND\ 2005\ iMac/Matrix2005_rnd1.xls

com.apple.fileutil.PlaceholderData

com.apple.macl

com.apple.metadata:kMDItemOMUserTagTime

com.apple.metadata:kMDItemOMUserTags

com.dropbox.attributes

com.dropbox.attrs

michaelb@iMac11 ~ %

File 5

michaelb@iMac11 ~ % xattr /Users/michaelb/Desktop/BRITISH\ LAND\ 2005\ iMac/UK\ land\ prizes\ 2005.doc

com.apple.fileutil.PlaceholderData

com.apple.macl

com.apple.metadata:kMDItemOMUserTagTime

com.apple.metadata:kMDItemOMUserTags

com.dropbox.attributes

com.dropbox.attrs

michaelb@iMac11 ~ %

mikebore · April 28, 2020, 2:40pm

I thought it was just worth re-expressing the problem in the context of this folder of 17, which is typical of my Files and Folders across six databases.

If I export this folder of 17 on good Mac, all 17 will backup by CCC and Chronoisync and upload to iCloud.

If I export this folder of 17 on bad Mac the three identified files cause errors in Chronosync and CCC, and will not load to iCloud. The three also cannot be imported to a fresh DT3 database.

All six databases pass verification on both good and bad Macs.

BLUEFROG · April 28, 2020, 3:07pm

Does DEVONthink report an error when trying to import them?

mikebore · April 28, 2020, 3:24pm

Interesting. I said I had exported the 17 files to a folder on my Desktop, so I assumed these are now 17 standalone files. I just looked at the log when I try and add the three bad files to a new TEST database and see the attached. No mention of the TEST database but DT3 seems to recognise them as being in the CHESS database which is where the folder of 17 originally resided.

This shows my repeated attempts to add the 3 bad files to new TEST databases.

Does this make any sense? or help understand the original problem?
Thanks

BLUEFROG · April 28, 2020, 3:41pm

Hold the Option key and choose Help > Report bug to start a support ticket.
Please ZIP and attach the three files in question. Thanks!

mikebore · April 28, 2020, 4:05pm

I have submitted a BR and linked to this thread.
It was impossible to send the three sample bad files either in a zip or unzipped. Zipping failed with an “Unknown error”. When I tried to attach them unzipped, the email would not send as it “had attachments which were still being loaded”.

BLUEFROG · April 28, 2020, 4:39pm

See my response on your ticket.

mikebore · April 28, 2020, 4:40pm

I have replied. I will try exporting the sample test folder to a different location in case that makes them mailable.

mikebore · April 29, 2020, 2:33pm

To summarise where this has got to:

The initial problem was that hundreds of files exported from two my databases on my main Mac (iMac) were unreadable by any programme, for no discoverable reason (attributes, permissions etc). It was not a one off event, but was repeatable.

The same files exported from the same two databases on my other Mac (MacBook) were all readable and no problems.

I discovered the problem because, for the first time since moving to DT last Dec, I did a full export from all my databases as an extra backup and test of exportability. The unreadability errors were revealed when my backup programmes, CCC and Chronosync, threw up hundreds or errors when attempting to back up the exports. (I do normal backup of the databases themselves including verified archives. This was additional).

So the problem was something local to my iMac, not per se a DEVONthink problem.

Even though not apparently a DT bug the experience has dented my confidence in DEVONthink a bit because the problem was undetected. The affected databases passed verification and there was no indication that anything was wrong, either with DT or the iMac generally. But all the time the affected files were effectively locked into the DT databases, fully usable while in the database, but unexportable. (found later that if exported individually the files were readable, but not if part of a bulk export).

Jim Neumann has been very responsive and supportive on the support ticket but the root cause has not so far been found.

To move forward I have done the following things:-

Deleted the two affected databases, and created new ones by importing the files exported from my MacBook.
I have done every maintenance activity I can think of on the iMac, short of erasing and reinstalling from scratch. The iMac has been well behaved in all other use, no sign of anything wrong except the DT export.

After doing the above I did a test full export from all databases on the iMac, and everything is fully readable. Both CCC and Chronosync can read every file. So for now the problem has been removed.

I will now do weekly exports to see if the problem recurs.

zit · December 2, 2020, 9:31pm

I have had many similar problems with files having the attribute
“com.dropbox.attrs 26”
Problems with changing ownership, copying/moving etc.
Nothing to do with devontechnologies

BLUEFROG · December 2, 2020, 11:18pm

Welcome @zit
Were the problems with those file operations in the Finder?

zit · January 21, 2021, 1:38am

It’s been a while. Didn’t have Notifications turned on.
Don’t recall exact details.
I solved the problems by copying the files to a FAT volume, and then back again.

BLUEFROG · January 21, 2021, 6:13am

Interesting. Thanks for the follow-up.