CloudKit sync "CKErrorDomain 15"

DEVONthink
21

This deserves several upvotes, I really really appreciate this.

Sure, I also was at the blink of leaving the forum forever, but first wrote privately to the developers which helped a lot, but not perfectly, as we see now. Still, yes, we all could do better. And I wonder where the women are – except for Steffi from Steffi’s Cloud who has some really knowledgable posts.

2 Likes
22

Coming back to this since I’ve noticed increasing errors for CloudKit for a while.

I first switched to the legacy iCloud for some months, but noticed data loss. The local iCloud sync store cache also balooned up to 70 GB, which can’t be influenced manually.

So, I switched back to CloudKit and its server errors. I thought 1 or 2 successful syncs every hour would be good enough, but this also seems to cause data loss when further using files between unsuccessful syncs.

I’ve seen heavily highlighted PDFs reverting to some earlier version while working with them, loosing 70%+ of my highlights, depending on when the last successfull sync happened.

That’s why I don’t feel like the server errors are an acceptable annoyance any more, but actually make the sync unusable (when using iCloud). Since some mentioned this being a cosmetic issue in the log, I thought I’d share.

Since Bonjour and WebDav isn’t an option due to limitations (iOS shallow sync), it seems like paying for Dropbox is the only way. No data loss there, but I hate to pay so much for this without any other need for it.

I think it would be nice if an S3 compatible sync option could actually be considered, so I could pay $2 for sync instead $10.

23

Are you sure the data loss is related to the sync? I’ve been using CloudKit sync for two years and I’ve never had data loss. If the sync doesn’t work immediately, it usually works a few mins later, and I’ve never had an issue with DTTG losing amends I’d made. (Not saying you haven’t experienced that, but it’s worth checking what’s actually happening here, and also checking how your devices are handling conflicts.)

2 Likes
24

Unfortunately, yes. 100%. Some details below.

  • I’ve been using CloudKit since it first became available. The errors were infrequent, and I didn’t notice any problems either for some time.
  • Months ago (July 2023), it wouldn’t sync for weeks at a time.
  • I couldn’t even delete the data from DT because the errors prevented cleaning the sync location.
  • I deleted everything from iCloud settings (after emailing with Jim) and set everything up again later.
  • At this point, I had already deleted the iOS app to isolate the issue.
  • At some point, the errors subsided and I was able to upload everything again (still not using any other device with the sync).
  • For a while, CloudKit sync at least worked once or twice every few hours, so I accepted it, like others mentioned here.
  • At some point, I noticed the highlights data loss while working with PDFs (and still not using any other device with the sync).
  • The errors weren’t actually the problem here. Only after a while of errors, followed by a successful sync, did this happen. The open PDFs not only lost most of their highlights, but they even went back in page position after the sync.
  • I have seen this happen 30+ times after first noticing it. It only occurs when sync didn’t happen for a while due to errors, followed by a successful sync, which I isolated by keeping the log window open at all times and practically waiting for it to happen (costing me hours overall).
  • I also set up smart groups showing the amount of PDF annotations in the column view and made screenshots of the numbers before working with files. Sure enough, those numbers frequently changed back to an earlier sync state.
  • These were files I used in their own small database, which I set up just for testing further. It was the only database that was synced with CloudKit at this point.

There are obviously a few reasons that make me think it’s only related to iCloud sync:

  • It’s only happening with iCloud, never with Dropbox :laughing:
  • It’s only happening at times of many errors, followed by a successful sync, as if reverting to an earlier sync state.

Also, it’s easy to miss, since it’s only showing up when using highlights in PDFs. The modification date metadata does not catch it. If I hadn’t seen it “live” once while working with a PDF, I wouldn’t even know. Lots of screenshots and isolated database testing, all without using any other device with the sync store, proved it to such a certain degree that I concluded that this way of syncing was utterly unusable, at least on my Mac (I only use DT on 1 Mac).

I’ve repeated the process multiple times and still enable the iCloud sync in settings every few days for testing. The last successful sync was days ago. I can’t even get the test database to sync due to the errors. Once, it was 2 weeks between a successful sync. So, even without the highlights issue, this wouldn’t be a viable option most of the time.

I went through all of this effort because I actually want to use iCloud. It worked well for some time. I have no use for Dropbox otherwise and I’m super annoyed about paying $10 for some sync data.

25

It’s a sad fact that the issue is sporadic and some people aren’t affected… at the moment. Whether it will continue to behave, we can’t say. Fingers crossed for those people. However, while we don’t advocate for any cloud-service and we always recommend a local sync unless a remote one is needed, it seems like $10 is a small price to pay to reduce annoyance and return to productivity.
:slight_smile:

PS: We continue doing what we can with each release, but we don’t control all the variable in the equation. If we could fix it, we undoubtedly would.

2 Likes
26

Yeah, it’s so weird, I’ve used iCloud for some time without issues myself.

I mostly wanted to mention the PDF highlights loss issue, because it’s easy to miss. Who knows how many of my PDFs reverted their highlights before I started making screenshots of the number in column view to actually track this.

I can’t use Bonjour because my iOS device doesn’t have enough space and shallow sync only (edited:) “works” (makes sense) with remote sync.

Yeah, I’m probably more annoyed that I invested so much time over the last few months, lost data, and still have to pay for Dropbox in the end :joy:

Still hoping that an S3-compatible sync could be considered at some point.

27

Really?

28

and shallow sync only works with remote sync.

@rmschne’s short comment is accurate. Shallow sync doesn’t require a remote sync. Bonjour supports it as long as a Bonjour sync is feasible in the current environment.

1 Like
29

What I should have said was that shallow sync doesn’t give access to the files on iOS, unless I’m in the same location/network with both devices, making it unfeasible in this scenario.

It’s not like I have a mansion that would require me to “travel” for 10 minutes to my DT Bonjour server at the other side of the estate :sweat_smile:.

I guess the above would be a use case for Bonjour with shallow sync, but unless I’m missing something, Bonjour with shallow sync is pointless on the go.

1 Like
30

It depends on where the Bonjour server device is. I used Bonjour on the road a ton, using sync-by-wire between my Mac and mobile devices.

31

Hardly “pointless”. If expected that everything would be available on a small iOS device on demand, then yes, it doesn’t do that.

But if you make changes to things that you do have on iOS, when you get back in range of Bonjour sync it will sync.

That’s why I have a database “Works in Progress” and if I work while “out and about”, changes will sync back.

I recommend, given your situation as described, you buy a new iOS device with more storage or change to a more reliable sync service than Apple can apparently provide.

32

Got it, but this doesn’t apply to me use case. I have no reason to use DTTG on iPhone if I have my Mac with me. But thanks for mentioning this, I never thought about the wired connection. It doesn’t apply to me, but it’s good to know.

33

I suppose I should have said “pointless” for my use case, or for anyone who wants access to all of their DT data on the go without carrying the server device at all times or having enough free space to sync everything.

I mean, of course I want everything available on demand. It’s not just about working with a smaller set of files in progress, but having access to my archive as well.

So, Bonjour never was a solution to begin with. It’s nice that this is a use case for some people, and I should have spoken in less general terms before, but it’s certainly no replacement for remote sync, given its limitations.

That’s what I already did.

34

Hello everyone

I like many others am encountering these errors sporadically - but today, I did a bit more experiments.
See, I have other apps syncing through CloudKit, same protocol as DTTG. Whereas the latter stubbornly refuses to sync and only mumbles “CKErrorDomain 15” errors, the other apps do just fine and happily sync changes from Mac to iPhone and the reverse.
Is everything to blame on CloudKit? I might be wrong, and probably am. Still, what does DTTG do with CloudKit to perform its syncing duties, that somehow causes these errors - which the other apps don’t?

Thanks

35

There are a lot of prior posts explaining that what DEVONthink does to sync is fundamentally different than other apps, especially those for which syncing is simple. Best to read those past posts to get the “why”. There are also other sites on internet that explains complexity and recent changes Apple made to their sync services, some not quite successful according to those reports.

The new info today from @cgrunenberg about Apple using three third party server vendors (if not more) is interesting and is beginning to help us understand why different people have different experiences. Probably on different sync services with probably no guarantee your computer will connect to same all the time.

If you want sync to work, use a different method. Bonjour and Dropbox (if you need internet method) just works and avoids all this problem re-reporting here.

36

Such as…?

37

Amazon (AWS) and Google Cloud are certainly being used for iCloud storage, with Apple being one of, if not the, largest customer. Microsoft Azure was in use, but it appears to have been for different purposes.

So yes, we can add data centers as yet another variable to include in the scenario.

1 Like
38

Such as Bear or Ulysses

39

Hello rmschne

Thank you for your explanations, I’ll certainly look into the simple vs complex sync approaches in the forum in order to better understand why some other apps appear more resilient to the underlying infrastructure hiccups.
I’m unsure whether Dropbox is an equivalent alternative here as it would be a supplementary, costly, service - so maybe Bonjour over LAN.
For information, there have been reports of Apple Services outages recently (see here: The App Store was down, along with Apple TV, Apple Podcasts, and Apple Music - The Verge), and iCloud is part of the impacted ones. That might explain a surge in the syncing incidents as well.

Thanks
Fritz

1 Like