DT3 creates lots of duplicates when syncing an indexed iCloud folder

Hi there,

let’s start by describing my setup:

  • Macbook running DT 3.7.2 / OS X 11.4 (20F71) / temporarily on
  • Mac Mini Server (Mid 2010) running DT 3.7.2 / OS X 10.13.6 (17G14042) / always on for processing OCR on incoming scans
  • Synology NAS running a WebDAV server
  • Both Macs share the database “dokumente” via WebDAV through a sync store residing on the NAS
  • Both Macs have iCloud Drive enabled using the same Apple ID

My goal is to:

  • have all our household’s paperworks digitally stored in one iCloud Drive folder (with lots of subfolders)
  • use DT3 to index that folder
  • share access with my wife via iCloud shared folders
  • be able to re-organize the structure of my “document system” using iCloud Drive on my iPhone while I’m on my commute - this way I could work around the lacking ability of DTTG to create folders on the fly while moving a file to a different location
  • use DTTG on the run if I need to find something - since iCloud Drive is lacking a full-text search

When I ran my proof of concept I quickly ended up with a folder convoluted with duplicates of documents and folders:

I think iCloud must be the problem maker here since when I was indexing a “regular” folder outside of iCloud everything worked just fine. I’d just like to know:
Is it just not possible at the moment to get things to properly sync the way I intended?

Now, I saw other people reporting similar issues, so I made sure it’s set to “keep the last document” - here’s my sync config:


Same goes for the Mac Mini:

Thanks and cheers,
Lorenz

Steps how to reproduce this would be great, thanks in advance. E.g. are you able to reproduce this using a new database and few files/folders?

Hi @cgrunenberg, will I be able to share a screen recording here?

Depending on the size this should be possible. Ideal would be steps from scratch (new database, new indexed folders) so that it should be possible to reproduce this.

Hi @cgrunenberg,

I recorded a screencast. Even though I took some time to prepare this, I’m still pretty noob when it comes to DT. So the screencast is a bit lengthy (approximately 9 min). The interesting stuff starts at around 3:45 where I index the iCloud folder to the newly created DB that’s synced through the sync store in my NAS’ WebDaV.

I would really appreciate you taking the time to watch me struggling :wink:

Thanks & best regards,
Lorenz

Thank you for the screen recording! According to your screenshots you’re synchronizing your documents & desktop folder to iCloud. Is this correct and enabled on both computers? Does the issue also happen in folders directly created in iCloud Drive at the top level (especially not in Documents or Desktop) or using folders not located in iCloud Drive at all?

Yes.

Yes.

No. I tried that already and everything works just fine. But that’s not a solution for me since I cannot edit the files and folders on the go.

I also tried indexing a folder in OneDrive - same result :slightly_frowning_face:

Why not?

If you are using DEVONthink and DEVONthink To Go, you would have access to imported files if the database is synced between the two.

See my comment here please Add a new group while sorting documents - #4 by VeryDisco

To be honest, I’m not really surprised or disappointed that my idea didn’t work out. I also think that the issue is not DT here but more the fact of two concurrent sync services (DT sync store & iCloud) running on two different machines syncing the same data source.

Still, it would have been nice :hugs:

I think I’ll go with an indexed folder on my MacBook that doesn’t reside in any cloud service and use DTTG like you suggested for accessing the data when I’m away from my computer.

Is it only you working the files in icloud?

Yes.

But ultimately I would have liked the ability to share (read-only) links to some files or folders.

I didn’t even realize until now, but if this would have worked I could have shared a sub-folder with other people and give write access to them - and they could edit stuff (indirectly) in my DT database. Powerful stuff :wink:

Hi,

when I tried this some versions of DT ago (maybe the Version 2 - DTPO) I had the same result. But when now un-selecting the sync of index file contents (screenshot) in the sync store settings before adding the database for syncing everything works as expected, at least for me on an iCloud sync store.