Speed up sync to Dropbox?

I’m going on about 24 hours for a 1.5GB database. Any ideas to speed this up?

So many variables in a network transfer, especially an upload, and 1.5GB of data. Not really. Sorry.

My (limited) experience has been that uploading even fairly small amounts of data to the Dropbox “cloud” can vary from glacially slow to relatively quick. The congestion variance usually seems more with their service than general upload bandwidth.

Ok, I’m going to abandon Dropbox and go with a large USB flash drive. Not optimal, but I think I can make it work to keep my computers in sync.

Funny, I was just now thinking of how you could use a local syncStore on a USB thumbdrive for this purpose.

If they’re all LAN connected then maybe direct connection synching is an option?

However, it seems that’ll only work between two systems. Under Synchronizing in the documentation it says:

• Direct connection between two Macs

I’m unclear on part of this:

• Direct connection: A direct link between two instances of DEVONthink, usually on two Macs; synchronizing through direct connections does not require sync stores but requires both devices to be switched on and running DEVONthink.

usually on two Macs” implies something else is possible, but what? Two instances of Dt running on a single Mac under different accounts (with Fast User Switching)?

I believe the “usually” piece means most people would be doing bilateral sync – between a desktop and laptop, say. But, less “usually”, one could sync multiple machines: A <-> B <-> C, or A <-> B and A <-> C, and so on.

On Dropbox slow uploads – there’s a setting in Dropbox Preferences to adjust up/down transfer rates. I’ve not tested this scientifically, but it’s worth a tweak:

Ok so I set up a local sync store on a USB flash drive, performed my sync in about an hour (pretty good compared to Dropbox at 40+ and still going), took the USB drive home and my import to my home machine does not complete successfully. This is with a new database that I created with a mail archive import of about 16000 messages, tried to sync with Dropbox and then cancelled, and next tried the local sync store. Again, the sync to copy to the flash drive works, its when I try to import to another Mac. Frustrating is an understatement. I get tons of errors about mail messages containing no text or two messages when there should be one.

I will try again. Is there another alternative?

Is this database in a Dropbox folder?

If you are still having problems, please try the Sync operation again AFTER following these instructions…

To enable logging, type the following in /Applications/Utilities/Terminal.app and restart DEVONthink:

defaults write com.devon-technologies.thinkpro2 DTSyncDebug -bool YES
defaults write com.devon-technologies.thinkpro2 DTSyncDebugHigh -bool YES

This will log more detail to your system log, which can be viewed in /Applications/Utilities/Console.app.

After you’ve attempted the Sync, open Console, type “DEVONthink” in the filter field in the upper right-hand corner, and copy and paste ALL of the window’s contents into a new text document. Start a Support Ticket at our DEVONtechnologies Support Ticket System and attach the document to your reply. Thank you.

BTW, I just validated this process with an HFS+ formatted 8GB thumb drive I happened to have in my pocket.

I did do a Rebuild on my database since I had some errors but it worked nicely. Do note that I cannot verify whether this success can be attributed (in full or in part) from it being HFS+ formatted instead of FAT32. I will test this also when I get a moment (and another drive).

Would that mean A, B, C are three instances of DEVONthink? That’s a way “usually on two Macs” after “A direct link between two instances of DEVONthink, …” makes more sense to me without being more explicit.

Couldn’t the results of A ↔ B ↔ C synching be the same as A ↔ B and A ↔ C, maybe dependent on when items are updated in each instance and how any synching conflicts are handled?

I think Don’t limit is a relatively new Upload rate option; it’s not mentioned on the help page that the circled ‘?’ opens. Maybe it gives more upload bandwidth to Dropbox than Limit automatically (the default), especially when there’s other competing upload traffic?

Sure … it’s just an example. If everyone in an office were syncing instances of the same database, then they’d be advised to establish a set procedure and sequence and stick to it. This is an @arnow-type of question – maybe he’ll chime in with an opinion. Probably should be a new topic – since we’re getting far OT from the Dropbox speed question.

Dropbox says:

and

So, in most cases, it appears the Dropbox default is to favor user browsing experience over data transmission.

With the same procedure, db/app availability requirements fulfilled, presumably results wouldn’t differ using a direct connection sync vs. a sync store?

I originally chimed in wondering if abowen500 might be able to use a direct connection sync instead of a local sync store as the Dropbox alternative, since narrowing focus to better understand which of the former two would be personally preferable. I don’t mind if that subtopic dies here since abowen500 has already chosen to use a local sync store, while I’m currently leaning more towards direct sync.

Thanks; I hadn’t looked far enough.

Thanks for all the assistance. I’m not sure which way I’m headed. I’ve had limited success with some of my databases with a local syncstore on a usb drive. Direct connection isn’t an option for me (work vs. home).

Gotcha. Thanks for clarifying that.

I tried to sync a 10 Gb database to dropbox and gave up after 4 days and many resumes. The only result is that my dropbox load increased with 10% and I don’t know how to remove it. (By the way: Bluefrog, can you please have a look at the thread in which I asked how to remove the sync remainders from dropbox and answer my question about the transactions folder? :wink: )

Like you, I switched to using a 16 GB USB stick and for me that works very well. Speed depends on the type of USB you use (I use USB 3) and the format (HFS+ is much faster than FAT).

I didn’t import the database from the sync store though.

Instead, I copied the database from the first computer to the USB and moved it from there to the second computer. At the second computer I created a sync store on the USB stick (‘add new local store’ in the ‘… with these locations pane’ in DEVONThink’s Sync preferences) and synced the database on that computer with that store (before the sync I did a ‘very and repair’ followed by a ‘backup and optimize’, just to be sure). After the sync finished (45 minutes) I took the stick to the first computer, added its store in the sync preferences of DEVONThink on that computer (‘add existing local sync store’ in the ‘… with these locations pane’) and synced the database on that computer with the store (again after a ‘very and repair’ and ‘backup and optimize’). It finished in 3 minutes and everything works without problems now.

Perhaps, your problem can be solved in the same way (that is by copying the database to the second location rather than by importing it from the store). :slight_smile:

:smiley:

At the moment I have only two computers (an no job :frowning: ) so I can’t experiment with this setup, but I think it should work, provided you avoid loops. I am pretty sure that A ↔ B ↔ C ↔ A will give problems. So, indeed, it is very important to have a procedure and sequence and stick to it and the sequence should be such that there are no loops.

I guess that if you want to sync more than two copies of a database, using a sync store is much easier than direct syncing and, hence, safer because in that case the sequence is irrelevant.

Hmmm, I wonder if a sync store is always “safer” in the case of multiple synchronizations. Consider the case where A,B, and C are machines with local instances of database Z – each of which have changed some thing on their instance. Then, say A ↔ syncstore-Z and B ↔ syncstore-Z and C ↔ syncstore-Z occurs concurrently. At the end of this sync event, does syncstore-Z contain all changes made on A, B, and C, and does A contain the B+C changes, B the A+C, C the A+B changes? I think the answer is “sometimes yes, and sometimes no”. Only by manual inspection and/or homemade logging tools would users A, B and C know for sure.

I haven’t tested this, and I cannot test it because of lack of computers, but I have assumed that concurrent syncs are impossible. That is I have assumed that if B tries to initiate a sync with Z when A is syncing with Z, B will have to wait until A is ready. I could be mistaken – DevonTechnologies’ information is too scanty to be of use for this kind of scenario’s :frowning:

Did you experiment with this? If so I would be very interested to hear the results!

I guess that whether or not concurrent syncs are possible in this scenario Z will have all the changes, but A, B, and/or C might lack some of the changes.

However, this doesn’t make the sync unsafe - there are no changes lost and no verification errors created. The only problem is that not all the instances are up to date.

Loops in a direct sequence, on the other hand, can easily result in loss of changes and/or verification errors.