DT3B7 Sync problems with indexed files

@rfog and @Bernardo_V, et al:

While it is unclear what impact the smart sync and the Dropbox File Manager application have in this instance (though I suspect they are problematic), I think it underscores the caution that should be exercised when using indexed resources in a cloud-synced folder with DEVONthink and new features of cloud services like those.

And while opinions are just that, here’s an interesting bit on Dropbox’s behavior regarding the app:

I knew that, and I voluntarily subscribed to it. It is more usable than the very obsolete Finder.

However, currently I had more issues with that program closed. Same creation/deletion of folders in Finder, that DT didn’t wanted and once an “Update Indexed Items” is run, not only re-creates externally deleted folders and files, but even remove existent files.

No sync with DTTG and no other Mac involved. Only a Dropbox indexed folder (marked as local always) DT and Finder.

For me, this is completely broken even in final version, and is a DT problem because it happens with iCloud Drive and SMB shares as well.

After some more testing, the problem arises at the moment you delete a folder inside an indexed one from outside of DT does not matter if the new Dropbox File Manager is running or not. At that moment, DT becomes completely crazy. I haven’t tested doing the folder movements inside DT.

Steps to repro:

  • Go to an indexed folder shared in Dropbox.
  • From Finder, create two folders and put one file inside, lets say Folder1, Folder2 and File1.pdf and File2.pdf inside of respective folders.
  • Wait until DT acknowledges the changes.
  • Always from Finder, copy File1.pdf and File2.pdf into base folder. Then delete Folder1 and Folder2.
  • At this moment, DT becomes crazy, not recognizing all of the file movements.
  • Put DT in base folder of Folder1 and Folder2 folders, and execute a “File -> Update Indexed Items”. Normally, one of both deleted folders are recreated and things start behaving wrong, like creation of deleted folders, false files (even with changed extensions html by pdf)…

I don’t know how to reproduce it, but it happened again. :confused:
This time on a much smaller scale and with seemingly random files which I didn’t even touched on either computer.

I couldn’t reproduce this, was Dropbox.app running concurrently? Does this still happen without Dropbox and/or without automatic sync (set Preferences > Sync > Schedule to “On Demand”)?

But in case of cloud folders usage of File > Update Indexed Items might indeed be necessary as filesystem events aren’t sufficient unfortunately. E.g. after moving a file into the database on computer A and Dropbox.app synchronizing this change to computer B, the received filesystem events are identical to deletions in the Finder. But DEVONthink can’t delete the item as it’s a cloud folder and a move into database might have happened somewhere but wasn’t synchronized yet.

Currently I cannot test without On Demand because then I will get a completely full disk, however, the indexed part is set as Local, that I think could be the same as having disabled On Demand for that folder, and Dropbox is behaving perfect.

Said that, and considering your second paragraph, and some guess, I had the iPhone/iPad off, but not “switched off”, and then perhaps the issue could be in the push notifications, that wakes up the iThings and then forced this “undefined” sync status.

Then, I assume that once all iThings synchronized, the mess will disappear, as one device regulates other once a round of “synchronizations” go around?

(I know this is a very difficult thing to do, like multicore synchronization, deadlocks and semaphores waiting forever, with the added of being completely distributed and de-located and not inside one chip…)

And my configuration is complex: one iPhone, two iPad (11 and 13) and 3 Mac (yes, I need to purchase an extra seat, I’m using now the “grace period” - or throw mi 3rd Mac into the channel -I live in The Netherlands, sigh).

The suggestion was only for the Mac and just changes the schedule, nothing else. There’s no “on demand” download on the Mac like on iOS (yet).

I think I’m missing you… You mean Dropbox “On Demand” or DTTG On Demand?

I’m sorry, used the wrong description. On the Mac in Preferences > Sync it’s actually the schedule “Manually”.

Ok, then I set it Manually on Mac, and when finish messing with files, then do a manual sync… Sounds good.

Thanks.

That’s a possible workaround. But are you still able to reproduce your “Steps to repro” (see above) after setting sync to manually?

This is getting me crazy.

First, sorry for the delay, this week has been a “release one” (I think you know about that :wink:) )

Second, I’ve tried some times with sync set in manually, automatic, with a Steve Jobs photo at my side, and another of Bill Gates as well, and hasn’t failed. No-one-fail-in-at-least-ten tries.

Thank you for the info! I’d suggest to test this again using 3.0.1 which will improve the sync of indexed items located in cloud folders.

Never failed since then, except so and then I get a duplicate file with “copia” added at the end of the file name (I suppose “copy” in English) in indexed folder I use to communicate between Android/Windows and DT, and I think it happens at the moment I “touch” a file in DTTG, but so and then.

BTW, still waiting for 3.0.1. No hurry, as it is working fine now. But you told me test in 3.0.1 and I expected a soon release.

DEVONthink 3 does definitely not include a localized string “copia”, therefore the duplication seems to happen somewhere else.

Yes, but could be the bug you told us is resolved in 3.0.1 that makes finder generate duplicates.

Said that, I’m happy with the current status of “failure”: so and then (good scientific measurement :joy:) between 3 macOS, 2 iPad and one iPhone.

For example, since I wrote previous message, it hasn’t happened in a full weekend of use.

Whenever I create a new group in a indexed dropbox folder and move records into it (that are all indexed as well) on another Mac, I end up with this:

The folder appears faded and with the cloud symbol next to the finder icon.
I can fix this by moving the folder into the database and then moving it back again to the indexed folder.

It seems to me that this new dropbox system makes it incredibly difficult for DT3 to deal with indexed files. Or is this not the case?

P.s. I am using the latest version, of course.

Are you using Dropbox’s smart sync option?

Yeap. You asked this before :point_up_2:
All the files are local at all times. :wink: