DT3B7 Sync problems with indexed files

The setup: DT3 synced between two computers via bonjour.

The problem: Indexed files keep getting re-indexed (they receive a new uuid) thus showing up twice or thrice in the same database.

My first hypothesis as to the culprit: Dropbox’s new sync mechanism. (Now you can select individually for each file if it will be available locally or stay only in the cloud). → Could be part of the problem when this happens with dropbox.

What I tried: Moved my pdfs over to iCloud.

What happened: Not much. The problem persists.

Any thoughts? This is somewhat maddening. I have to declutter my database over and over again.

Are only indexed items located in cloud folders (Dropbox, iCloud etc.) affected? When exactly does the duplication happen and finally, should the sync create duplicates in case of conflicts (see Preferences > Sync)?

Dropbox’s new sync mechanism.

Are you referring to Dropbox’s Smart Sync ?

If so, and if you are allowing the files to be pushed only to the cloud, it would be inadvisable to do this.You should only be indexing local resources.

I cannot answer this since I don’t have any indexed files that is not stored in a cloud folder.

Usually when I switch computers.

Yes, but the file is not being duplicated. It is just being indexed again. For instance, many pdfs that I have on my Bookends folder on iCloud just showed up at my database’s root directory after I finished working on the macbook and started DT3 up on the iMac.

Yes.

All indexed files where local at all times.

I experienced a similar issue yesterday by moving an indexed file in the Finder into a subfolder (also indexed) which contains already an indexed file with the same name. Afterwards the indexed subgroup contained two instances in DEVONthink (different UUID, same path). At least this issue will be fixed by the next release, not sure if that’s what’s happening on your machine too.

1 Like

Chris, this is happening to me a lot (duplicates and unnecessary re-indexing over Macs). It happened in iCloud Drive, then I moved files to my NAS (via SMB share), still happened but with less frequency, and after that I moved the files into Dropbox, and still happens.

I have Smart Sync enabled but not for the indexed files. I mean, the folders that are indexed in DT, have the local stuff set.

It does not happens always, but sometimes once I touch (even only read) a file in DTTG or my second MAC, duplication starts, no matter where the file is located. And it is a chain of duplicates: Mac B duplicates, then, as Mac A is closed, I remove the duplicate in Mac B, but when I go to Mac A and syncs, I have a new duplicate of the duplicate… And sometimes even the duplicate file remains in the sync database and I can download it)…

It happens even in local folders (both indexed in two Mac with exact same files), but with less frequency.

It is very strange, because this used to happen to me with DT2x and DTTG, but with very low frequency, but the combination of DT3 and DTTG makes this happens almost all times. So frequently I’ve stopped using DTTG and my second Mac for indexed items until this will be resolved (I expect on RTM version).

I think it is not a Dropbox/NFS/iCloud Drive problem but a DTx/DTTG one, because if all people say the emperor is dressed, well, he is dressed (oh, sometimes not). But you know, Occam Razor is here. Apart of my own sync problems in my very complex scenario, I’m reading a lot of complains here in relation to Sync…

BTW, I’m retaking DTTG after last update to see if at least the Sync problems went minimized. I had a lot of problems in DTTG, like “phantom” files: files that aren’t in DT3 nor in file system, but DTTG still have them and can download from Sync Database. Or empty Trash bin in DT3, sync, go to DTTG, sync, and see the trashed files in DT3 one more time. Repeat until you go to DTTG and empty trashes from DTTG… Things that never happened with DT2/DTTG2…

In case of indexed files located in cloud folders (e.g. Dropbox) and edited on iOS this is unfortunately a known issue, the cloud app (e.g. Dropbox.app) creates the duplicates after synchronizing such changes back to the Mac. But we’ll check if it’s possible to work around this.

Viewing a PDF document could already be sufficient too up to DEVONthink To Go 2.7.1 as PDF documents were sometimes automatically fixed and saved.

In case that this will happen again - what’s the filename of the original and the duplicate? And which one has the newer modification date?

I did some experiments. I have a specific Dropbox folder called “Scraps” where I store some files to be reviewed later before to be sent to DT. I indexed that folder from MacBook Pro at job (Mac 1). When I arrived home, that folder were indexed as well in my main iMac (Mac 2), that is my “DT Server”.

With DT in Mac 2 open, Removed all sub-folders and put the files inside each folder in “Scraps” one. I did it in Finder, but that folder is in Dropbox. I didn’t looked inside DT, as I went to dinner.

After dinner, I went to my DTTG and did some reading. I annotated “A robótica por la Religión.PDF” in DTTG. This file originally was in “Así habló Cicerón”, was moved into “Scraps” and “Así habló Cicierón” folder deleted from Finder. After I annotated it, I waited DTTG synchronize and then went to my iMac (Mac 2) and found this mess:

“Magically” some deleted folders were recreated in Scrap folder, basically all files that were “touched” (read or annotated) in DTTG. Now DT (and DTTG as it re-synchronized from DT) has some duplicate files. Read/annotated ones in the sub-folders. New capture:

From left to right: Annotated file in sub-folder, that was in “Scraps” folder when annotated, duplicated file in its right folder but before annotate it. Another read file moved into its original folder (this one is not duplicated), and same for “¿Prostituyó…”. First one is the annotated (“magically” moved int original folder and next to it, the duplicated one.

Both DT and DTTG are synchronized, and synchronization database was regenerated this weekend.

One more capture:

I’m trying to resolve this mess. I’ve deleted duplicated files in Finder, move the annotated ones into the right folder, and the result is the capture. Still a “fake” “A la robótica por la Religión” that is not in file system.

Ok. Now run an “update indexed folders”:

Oh, no, now all is completely messed. Still some “phantom files”… Now I go to my DTTG and synchronize. “Magically” a new folder inside “Scraps” has been created after iPad synchronization in file system:

Ok, let´s forget this by moment.

If you don’t see some DOC files in DT but you see them in Finder, those files are in Trash. We empty “trash”, but doc files aren’t deleted from file system, as they have been deleted form DTTG (I don’t know if this is a bug or a “feature”, but is not important here).

However, let’s try to resolve all this mess. Deleting some files from Scraps folder inside DT and file system. Delete not wanted files in DT, deleting a new phantom created folder, DT updates itself on deleting the folder from Finder and…

… one more time, DT duplicated a file and restored inside it the DOC files that aren’t in file system!!! A new manual “Update indexed files” and all seems to be correct now.

Let’s switch on iPad and synchronize it. After some go an back, this is the result:

iPad lacks some files… and they aren’t syncing anymore.

And now is when I switch off all my aThings and iThings, take my Note 10+ and Tab S6 and continue with other stuff.

Where is the Dropbox location on each machine?

Exactly same absolute folder, even the same profile folder: /Users/rfog/Dropbox

However, the main problem is not the syncing between both Mac, but between DT and DTTG, because when I described all of the above, Mac 1 was switched off at office (well, not really off but in a Faraday housing, as it is in a safe with other company I work for laptops/critical stuff).

Just for the record: my sync issues happened using just two Macs. I do have DTTG, but I have not been using it.

Sorry for having colonized your thread. I will stop posting here.

That is not what I meant!
It is just that you said that

so I just wanted to state for the record that I had problems while syncing between two Macs.

This is by no means my thread :wink:
Please, carry on.

Ok, let’s say it is our thread. :blush:

I’ve found something interesting that is valid for your problem and my problem when the files are in a Dropbox folder: The new Dropbox File Manager that can be optionally installed in a Mac.

Do you have it running? Because I’ve done some testing at work in same shared folder, but now with that tool completely closed (via Cmd-Q), and now there haven’t had any issue…

However, this could had worked by a chance and I need to do more testing.

I mean, this program:

Yeap, I have it running on both Macs. Indeed I thought this was the cause of the problem, but then it happened again using iCloud.

I just updated to the latest version, I mean, the official release. Let us see how it will behave!

I have been doing some testing with the final version and hasn’t failed yet.

Same thing here. So far so good :upside_down_face:

Great. Please let us know if this should happen again, thanks!

1 Like