CloudKit: Missing Manifest & Broken Database Structure

I’ve been syncing my primary database and Global Inbox between my iMac, MacBook Pro, iPad, and iPhone using CloudKit with good results, until recently when I’ve been getting iCloud (CloudKit) manifest missing and database structure issues.

My setup:
The iMac is running DEVONthink 3 Pro version 3.8.5 and macOS Monterey 12.5.
The MacBook Pro is running DEVONthink 3 Pro 3.8.5 and macOS Catalina 10.15.7.

The iPad and iPhone are running DEVONthink To Go 3 version 3.5.2 (17229) on iPadOS / iOS 15.6

I am not using Bonjour syncing. On both Macs, “Enable incoming connections” is off and Bonjour syncing is “off:” on both iOS devices.

Both Macs and the iPad and iPhone are set to sync the Global Inbox and my primary database, and the iPad and iPhone have CloudKit set to download files “Always”. So effectively the two databases are mirrored on all devices.

What I’ve done
A wile back I started getting “manifest missing” log messages on all my devices. Initially I believe many, many files were being listed in in the log, but after doing a “Clean” on the iMac the logs on each device list a single file name as having the missing manifest, and it’s the same on all devices.

I researched this in DEVONthink Help and in this community and tried various steps.Not sure I can remember everything I tried, but I did “Verify Location Quickly” on the iMac, and eventually I resorted to “Clean Location”.

I still get the manifest missing messages, but more ominous is that now, when I “Verify Location Quickly” I receive the following message in the log:

Broken or not yet complete data structure of database “my database name”.

I get this same message on both Macs, which appears to be related to the CloudKit sync store, not the locally stored database(s).

I’ve been troubleshooting this unsuccessfully on and off for weeks. Perhaps this warrants a trouble ticket / bug report but I thought I wad ask here first.

Regards,

Jim

Apple’s Cloudkit misbehaving again. Search this forum for “missing manifest”.

If you need sync, then setup and try another sync service. I use Bonjour, Dropbox, and supplemented with WebDav with a local Synology NAS.

Thanks, rmschne.

Yes, before I posted I read probably a dozen threads concerning “missing manifest” and “broken database structure”, and I followed a number of suggestions from those threads.

I have use Dropbox in the past as a sync store but was under the impression that CloudKit is more reliable these days. Unless I’m mistaken, and Dropbox is the preferred alternative, I’d prefer to stick witih CloudKit.

Regards,

Jim

That’s your choice. I prefer to focus on my DEVONthink content instead of debugging Apple’s stuff.

Haven’t done the statistics, but based on numerous reports here, Apple’s CloudKit is the one third party service reported most often as being unreliable for some. Sometimes clears on its own – sometimes not.

At minumum check you are not using Apple’s “optimise disk” with your iCloud service. That will break sync as DEVONthink expects the files to be in place on the local disk.

1 Like

We don’t have a cloud service we advocate. However, CloudKit has been very unreliable in the past several months.
And if you don’t need to use a remote sync option, we suggest you use a sync on your local network.

Thanks, BLUEFROG, that’s pretty definitive. I’m willing to transition off CloudKit in favor of reliability!

I’ll search the community of course, but can you say anything about Dropbox reliability “lately”?

Last, I need to learn more about local sync options, so I’ll read the documentation. If anyone has pointers to useful discussions, I’m interested!

rmschne, I see you use a combination of dropbox “supplemented by” a local sync option. I’m interested to hear more! Are you syncing the same databases to both Dropbox and local sync stores?

Regards,

Jim

Absence of reports does not prove absence of reliability issues. We see so many reports about Apple’s services failing yet I cannot remember any about Dropbox that dealt with unreliability vs. user error in setting up.

Dropbox runs on computers, and is surely not perfect, of course.

Using Dropbox to sync to phone and iPad, I have only my “Works in Progress” database that holds stuff I want to read and stuff that I’m working on (writing, editing, etc.), or stuff that I might need for travelling (bookings, certificates, etc.).

I “forget about it”. For me Dropbox “just works.” I do not attempt to sync everything that exists on the real computers for “just in case wanted”.

These few databases also set to sync with Bonjour which only kicks-in when the device is inside the local network.

You’ll find I’ve written here so many other times on what and how I sync. Search here.

I don’t use Local Sync Store.

As mentioned in my reply above, I use Bonjour and WebDav on the local network. Both sync everything between an iMac and a Macbook.

See page 60 of the current and outstanding “DEVONthink Handbook” for more info on synching.

Remember -

  • sync only if you need to
  • sync only what you need to.
1 Like

Dropbox had a crash with their API last release but that appears to be addressed now. Otherwise, there haven’t been many recent reports of issues with it.

Note: You do not have to use the Dropbox application on your machine to use Dropbox as a sync location. You don’t even need the Dropbox app installed! Our sync engine deals directly with Dropbox’s servers.