Unreliable Remote Sync with Dropbox

DEVONthink
,
1

My remote syncing (via Dropbox) has become much less reliable in recent months.

I regularly find databases have stopped syncing - even though there are no errors logged at either side. The sync progress indicator completes, and all looks good. However, the files have not been synced. There is no particular database/s always affected, or pattern that is discernible. It happens at least every 2-3 weeks and sometimes as many as 3 or 4 databases have stopped syncing. The problem is it may be a couple of weeks before anyone realises they are out of sync.

Shut down reboot, verify & repair, (on both side) no errors, try again, still no success. Verify remote syncstore “Quickly" or "Thoroughly” also does not help.

In the end only a “Clean” of the syncstore fixes the issue. When the databases are 10-15GB in size, cleaning is a major data usage problem!

It happens so often I have had to put a text file with a colour label in the inbox of every database, and a smart group shows all the colour labels. I added a second Mac in my office, and all databases are on both machines. Every week I change the label colour and wait to see if the change propagates through. This obviously only proves the syncstore is ok, but it has helped.

We have 13 DEVONthink users around the world, and each share between 4-8 databases via Dropbox. The files that change in the various databases would probably average between 100k to 1MB in any one day, and maybe up to 100MB if a lot changed.

Up until few months back the sync was absolutely bullet proof for a very long time. If any sync did stop (very rare), a verify and repair would fix the issue and sync would restart itself.

It feels like something in a more recent DT3 build has made the sync process more fragile, but with no error shown, there is no way to know.

My location has has always had problems with the internet connection occasionally being dropped mid sync, but it always picks up again and continues when the connection comes back.

Likewise Dropbox has thrown glitches and error messages on odd occasions, but again, the sync always picked back up again.

One last oddball thought. One database did stop syncing more noticeably than others. It had RSS feeds which no longer were live and always errored in the log. I deleted those feeds and that database syncs much the same as the others. That was the only database with RSS feeds.

Any suggestion what to look for next time the sync fails?

1 Like
2

I’m not using databases on the scale you are but, for what its worth, I’ve had to abandon Dropbox for sync stores because of the same problems you mention.

I had used Dropbox with DTPro for many years without any particular problems. I continued to use Dropbox after updating to DTP 3 again without any particular problems.

However, some time in mid-2020 I started to experience the same problems you have encountered. There are many similar posts on the forum complaining about this and various mitigations suggested. Nothing seemed to improve reliability.

Since Dropbox changes its software every two or three weeks, and Devonthink say they haven’t changed anything on their side, I concluded it was probably an issue with Dropbox.

I ended up moving my sync stores to a WebDAV server which has been working perfectly well so far; its been about 5 months since I changed over. I’m using NextCloud 20 on a virtual private server appropriately secured; the sync stores are also encrypted.

3 Likes
3

That is good to know - but not sure if we have the capability to set-up a WebDAV server ourselves. The standard sync options are very to setup and run.

Any suggestions for alternatives to Dropbox that have a reputation for being more stable?

4

Peter,

There are commercial WebDAV services available that you could subscribe to. I don’t have any experience with them (runing my “play” WebDAV server myself on a Synology NAS), but I know some suggestions have been mentioned in this forum before. Maybe do a quick search.

5

There’s a lengthy description in the tips subforum here for a Synology NAS.

1 Like
6

I hope they’re working on fixing the sync issue. I see people discussing Devonthink syncing issues in forums, blogs, and podcasts. It’s the most frustrating aspects of using Devonthink. I’m going to stick around for a while longer, but I just wish they would prioritize syncing issues rather than working on a new DT to go app. This is a key feature that should just work.

7

fyi I find it does “just work" (and has for years) with Bonjour and WebDAV on the local network and DropBox to support my road warrior life (when allowed someday, I guess but I kept it turned on with hope).

1 Like
8

From the feedback on here I’ve learned that it is almost certainly Dropbox’s problem with constant updates, and NOT DT, so let’s not direct any complaints in the wrong direction!

I will change to a WebDAV service, or possibly try CloudMe.

2 Likes
9

Works like a charm for me :+1:t2:

2 Likes
10

CloudMe also works like a charm for me (switched from Dropbox after having the same sync problems others are having).

2 Likes
11

If you have a local NAS storage (i.e. Synology) you do not need an additional Dropbox. Just use a VPN and connect to your home network.

12

I of course know that. I would rather have the professionals at Dropbox manage security (on my encrypted files kept there) than me messing with a VPN. This strategy as a result of years of experience with computing/networking security.

That being said, as I’m not doing any “road warrior” work in the “current situation”, so I keep Dropbox running simply to prove in this forum that it “just works” for me.

13

“Security” and “Dropbox” do not go together. It does not matter which cloud - files sometimes disappear. They are good for temporary files and sharing.

14

I knew you would say that! :wink:

15

OK - something new. :stuck_out_tongue:

Setting up the VPN is more easy and quicker than setting up a Dropbox and the connection.

16

However, the performance over VPN isn’t as good.

17

That can’t be made as a blanket statement, it depends on what device is terminating the VPN on your network and what your Internet service is. I didn’t want the complexity of VPN and trust HTTPS WebDAV, so I stopped using VPN…but I see similar performance between the two. My connection back to my Synology is exponentially faster than Dropbox was, but then I am on a gigabit service at home. The “remote” device gets connection rate transfer for my NAS, that was never remotely true for Dropbox :wink:

As an example, my work laptop sits on my home network but has a full tunnel to my employer network…so even to access my local NAS, the traffic goes over the VPN over the Internet to my corporate network, then through their web proxy/filter service, back out to the Internet and back to my house. I am able to get 100mpbs+ transfer rates to my NAS, I was lucky to get 10% of that to Dropbox consistently. In fact my WebDAV experience is so much better that I never even know when a sync occurs, however I would have frequent issues with DT when trying to “quit” due to the slow sync on quit when using Dropbox. On my WebDAV it is instantaneous :wink:

On the primary subject here, the problem you are going to face with geo-distribution is that of the service behind Dropbox likely doesn’t provide a guaranteed instant consistency across geo-regions. So it is possible that user across two continents actually see different data when their sync occurs. This falls within the CAP theorem explanation of the tradeoffs made when deploying services like this.

18

Thanks for sharing your experience.

19

I may have spoken too soon, as my WebDAV sync started having significant issues.with *.manifest missing of database <datbase name> when trying to import the WebDAV DB to other devices. I will fork this into its own thread if I can’t find an existing one that is relevant.

20

Unfortunately these same Dropbox sync issues started plaguing my setup about the same time as you. Switching database to sync via iCloud did not resolve the issue. Sync has become so unreliable that it can no longer be trusted and now all databases are access through a single computer. This has become especially annoying since, previously, DevonThink was a great way to keep information in a centralized location across iOS and Mac OS X. Hopefully it gets fixed soon.