Using Bonjour sync and cloud sync (iCloudKit) simultaneously. Useful?

To all sync experts

I recently upgraded to DTTG 3x and iCloudkit.

Since DTTG 3x did not do the conversion as promised in the manual, I had to “feed” my iOS Devices (2 iPhones and 3 iPads) from scratch.

I have 5 databases with total 82GB and about 60.000 objects. Therefore, I decided to use bonjour sync to get the 82GB on my iOS devices. Since, it is the sync mode with the highest speed.

For normal operation I prefer cloud sync – syncing independently everywhere and simultaneously on multiple devices. I experience that bonjour can only handle one sync at a time.
In the past, I was using or better suffering :wink: iCloud (legacy) sync to keep the remote devices up to date.
So, I am very happy that I can us now iCloudKit which is much faster.

After successfully transferring the database onto the iOS device with bonjour, I activated the iCloud CloudKit.

And here I got confused:
I expected that the iCloudKit sync would take now only a short time – the database was already transferred and up to date – but I took hours and hours. It seems that it did a full sync again.

@BLUEFROG, I saw that you normally recommend using bonjour and also using it in combination with cloud sync settings.
From my experience I can’t follow that recommendation for two reasons:
• If the job bonjour sync does not “satisfy” the iCloud sync, why bother with two systems?
• Bonjour sync is only able to serve one device at time, so it creates a bottleneck.

Do I overlook something or is there a mistake in my setup?

Best regards
Herb

I’ll give you my experience.

I run Bonjour with an iMac (the machine that accepts incoming), Macbook, iPad, and iPhone. The two macs i sync everything. I’m selective for the IOS devices for only those things I really need.

I also run iMac to Dropbox (to support me when, if ever, I can return to my Road Warrior days and have an external sync method), and WebDav local on Synology NAS server. Same strategy as above for what gets syncd.

I really pay not much attention to synching as when I do I notice that the files I want are always there when I want them. When I bother to look at the sync status (last sync time stamp), all syncs are quick and up to date. As I can only use one machine at a time I really don’t mind that Bonjour is only one at a time. If Bonjour waiting on a device, then the other sync method usually covers it. I do not notice any bottlenecks since, as I say, the files are there when I want them. Yes, there is a queue, but for me it is not important.

Of course your first synce with CloudKit will take a long time with 82 GB!! and subsequent syncs depend on what changed.

If your criteria for not doing as @BLUEFROG recommends (and I do too), then no, these recommendations won’t work for you. But for me, I care little about monitoring sync process, more about having the files in place where I want them when I want them and two sync methods seem to do the trick for me.

Just my two bits.

1 Like

Hi @rmschne
thanks for sharing your workflow.

My MacMini is the only device staying at home.
The other mobile devices (5x iOS and 2 MacBooks) gain from the location independent cloud sync.
My wife and I have decided to run our apple devices under one iCloud Apple Id for several reasons:
• Sharing our photos in one iCloud photolibary
• Sharing a lot of things (e.g. DT) over iCloud.

With many devices requiring sync, it often happens that bonjour ist busy or the Mac Mini has powered down :frowning:
And the fact, that DTTG3 will be switched off by the iOS powermanagenent system makes things even more difficult.

I really appreciate bonjour for speed reasons.
But I am irritated that a complete bonjour sync does not prevent iCloud CloudKit sync to do it all over again. I hope @BLUEFROG will figure out that this is actually a mistake in my setup.

Furthermore, since iOS devices with 128GB or even 256GB are affordable, I decided to sync all databases completely, since I cannot predict or don’t want to guess what I am going to need on the road.

Like you, I don’t want to worry about sync :slight_smile:

I am confident that iCloud CloudKit will address my sync needs in the future.
On the devices where the databases were moved, the daily changes are synced quickly and without errors by iCloud CloudKit.

There is no mistake.
Sync methods don’t share data so if you’re doing a fresh upload to CloudKit, where there is no sync data, then it needs to do a full upload to Apple’s servers.

Imagine you tell a story to a friend.
Later you meet up with another friend. You’d need to tell that friend the entire story as the knowledge of the first friend has no bearing on what the second friend knows.
Sync operates in the same way, with each location being a different friend for DEVONthink to talk to.

4 Likes

I can’t follow you.
DT 3.6.2 running on my mac had already done a complete iCloud CK sync. So the data was already on Apple Servers. So, no need for any upload when another device starts syncing.

Furthermore, on the iOS devices the complete and up-to-date databases were already in place by using the quicker bonjour sync. So, no need to download data.

I assumed that comparing the metadata (similar to shallow sync) would do the trick and switching to iCloud CK sync should be done in a breeze.

Why isn’t that case?

What is then advantage of having a cloud and bonjour system in parallel?

There’s still going to be transactional data that is synced between the device and Apple’s servers. There was no data in the CloudKit info for DEVONthink To Go so it has to do its own accounting.

Also bear in mind we are investigating some errors from Apple’s servers which is slowing down an initial sync.

What is then advantage of having a cloud and bonjour system in parallel?

It has nothing to do with making one sync method faster due to the other. A Bonjour sync is for fast and private syncing on your network. A remote sync can be used for syncing off your network. For people who need to sync off their network, this is the advantage.

1 Like

I know that Apple isn’t the first address for cloud computing – Steve had to apolozige at least one :wink:
And I am full of respect what the DT Teams makes possible for us despite all the challenges created by apple – not only the cloud issue, remember die PDF engine disaster.

Of course, there has to be lot of transaction to check if the local database data is in line with what is present on the apple server.
And then to up and down load only the real data which is missing.

But to me it seems that my full 80 GB of database data is transferred again, and for each of the devices hooked up to the iCloud CloudKit sync.

At least, I know for now, that I can skip using bonjour for my setting.

Thanks for your prompt response as always.

You’re welcome.

I have the same requirement wanting to use both Bonjour sync and iCloud sync. I need a few clarification.

I use MacBook as the Bonjour server. I use iPad to sync with MacBook via Bonjour.

Assuming I have done a complete sync via Bonjour. If I have 10,000 documents in my MacBook, I now have 10,000 identical documents on my iPad.

I then setup iCloud (CloudKit) sync store in MacBook. I don’t do anything yet on my iPad/iPhone. I wait for it to complete the sync to iCloud. So, it now has the same 10,000 documents n iCloud.

If I turn on the iCloud sync now on my iPad, I will expect it to try to initiate a sync but then realise that there is nothing to sync as it already has the same identical 10,000 documents.

Is this expectation correct? Or will it resync again since the iCloud is a different sync store?

If it is the later, then creating two sync store actually slows down the sync process as for every single document/file, it needs to sync twice to separate sync stores.

That actually depends on the sync settings on the iPad, afaik. You can set them to “download always” or “download manually”.

I’d expect it to check the 10000 documents to see if some of them where perhaps changed after the last sync. But maybe that is what you mean by “resync”.

So yes, it will slow down the sync process. But is that really relevant except for maybe the initial sync? Or are you expecting to change 10000 documents regularly, triggering repeated uploads and downloads?

1 Like