Synchronising causing duplicates

I’ve posted about this previously, and thought it was solved; however, it reappeared, and my last post noting this appears to have gone unnoticed (apologies therefore for reposting the issue).

For context…

On my iMac, I index some folders from various iCloud locations. On my MB Air, I synchronise the same database, and do the same with DTTG on my iPhone, iPad Pro and iPad mini.

All well so far.

However, when I add folders or files (ether via DT, directly in iCloud or other apps that save to the iCloud folders), the MB Air creates duplicates of those folders and files. Nothing else does this (the iMac or other devices). However, as the indexed folder is iCloud, it then syncs back those duplicates to the other devices.

To ‘solve’ this, I tried cleaning everything (basically started from scratch) and then on the MB Air, unticked “Synchronise Contents of Indexed items” on the info icon of the sync preferences. Seemed to work for a while, but then the issue returned.

A permanent solution I am currently using is to not synchronise the database on the MB Air - I just imported the database and am not syncing it. It updates perfectly via the indexing of the iCloud folders. This seems to indicate that the MB Air is first syncing the database change, then ‘seeing’ the same changes in the indexed folders and adding those too, thus duplicating everything new. Incidentally, a duplicated file or folder is not duplicated in the iCloud folders.

The issue with this solution is that changes to the database - not the files/folders - (e.g. add smart groups, rules, tags etc.) obviously don’t sync.

Any ideas of how I can keep the benefits of this solution (i.e. no duplicates) but sync the database?

Are you saying that the duplicates start and are created first on the iCloud shared folder? If yes, then that’s a mystery probably unrelated to DEVONthink, I would guess.

Just to check, also, iCloud on the MacBook Air (or any Mac, actually) is not set to “optimise disk storage”.

Hi,

No, as I said later in the post, no duplicates are created in the iCloud folders, only in the MB Air database (and then synced to other databases).

On the MB Air, yes, optimise storage is currently on…

Bad idea for indexed folders: Apple may decide at any moment to remove the file from local storage to „optimise“ storage.

2 Likes

As DEVONthink expects indexed files (and DEVONthink files for that matter) to be “local” and not deleted from the MacBook Air as you instructed iCloud to “optimise disk storage”, no telling what problems that can cause.

Recommend turning that off.

Hi,

My apologies - just physically checked, and the MB Air has the optimise storage option unticked :innocent:

Is the duplicate option in case of sync conflicts (see Preferences > Sync) en- or disabled on all devices?

Yes, both on the iMac and MB Air, and on iPhone and 2 x iPads… all set to ‘latest’…

And all Macs use the same version of macOS, meaning that the path of the iCloud folders is the same?

Absolutely. Latest current OS, no public betas etc…

Do the duplicates in the MB Air database have the same path and are located in the same group?

Yes, path is exactly the same, and in the same group - ‘pure’ duplicates. Both ‘reveal’ to the same non-duplicated item (either file or folder) in iCloud.

If both reference the same path, then moving one to the trash and emptying the trash should fix the duplicate but not remove the item in iCloud Drive.

Hi. Yes, that is the ‘resolution’ that fixes the issue immediately - I delete the ‘second’ version of the item, empty Trash, and that fixes it. The actual item in iCloud is, as you say, untouched.

However, obviously I’d like to be able to not have to do this :upside_down_face:

Are you able to reproduce this issue, ideally using a test database and an indexed iCloud test folder so that we might be able to reproduce this too?

Okay - I’ll set something up, and see if that reproduces it. I’ll detail what I did and the results shortly :slight_smile:

1 Like

Okay…
Set up a new database on iMac (Test database). Indexed a new folder in iCloud (Test for DEVONThink). Added a document to the iCloud folder, appeared in DT fine. Added a new folder (Test folder 1), appeared fine in DT.

Synced it (CloudKit). Imported into MB Air, and synced fine. Added a new document on iMac from an app directly into the iCloud folder - synced fine to iMac DT and MB Air DT - no duplicates (by this stage, the problem database would have duplicated that).

Added the test database on iPhone - synced, and duplicates :frowning: See screenshot. Option for latest is ticked.

And sorry - just noticed that the iCloud version only contains 1 note, not 2 (it has 'Take Notes", but not another Word doc that is in the iMac and MB Air versions). Also, the duplicates in iPhone are not synced/reproduced in the iMac and MB Air databases.

Thank you for the instructions, we’ll check this. By the way, which version of DEVONthink To Go do you use?

Version 3.5.4 (17234)

Hmm… no luck so far. :confused: