Looking for some thoughts. Sync works OK but I am thinking about some changes and wanted input to think about. I have a Mac mini, MacBook Pro 15inch, new Mac book air, 2015 iPad Pro 12.9 and a new iPad Air. I work from home and have a wifi / internet connection that test at down load speeds of 150-225 MB/s on Speedtest over wifi, so a decent connection. Upload is usually 10-20.
My basic conundrum is a database with 6,000 notes; around 5 million words, and is 7.1 GB in size. It is slow to sync among the various computers I use. For comparison I have another database with 1300 notes, 1 million words, 2.5 GB which syncs basically ok, and a third one with 350 notes, 17.5 million words [yes, odd , but has many patents and highly technical dispcriptions] and is 2.5 GB and syncs ok. All other databases are less 500 notes, a million words and under 500 MB in size and sync quickly among the computers and iPads.
My question: I can logically split the large database into 3 smaller databases without impacting usage or function. Does this make sense to do?
I have read in the forums and in the database description of Bill DeVille’s databases in the manual that databases larger than the ones I use should sync fine. But with the different computers / iPads I use they are not always caught up so I am looking for a way to improve this so they all have the latest database changes.
Do to the way I work and live inn multiple places, I cannot easily reduce the number of tools I use.
I have a very similar situation with a large database and multiple devices to Sync to.
I had major challenges syncing as one large database with both DT2 and DT3. With DT3 I divided it up into several databases and the improvement in both syncing as well as general speed/responsiveness has been very notable. I wish I had done that long ago.
You can split the databases, yes, and perhaps see a reduction in sync time for an individual database, but if you’re syncing 3 databases that contain the same data footprint as the formerly single database, I think you’ll net out to the close to the same amount of total time in sync.
Do you need to synchronize data across all those devices? All the time? I assume you might – but is it essential? That’s a lot of overhead == the time you spent handling all the updates.
I sync between a MacBook Pro, two iPad Pros, and my iPhone. But I don’t sync all the same databases on all those devices. The MacBook has all my databases, but the other devices only have the one, two, or three that I need when I use that device. So, in effect, I have a different strategy for each device.
Good points. In actuality , if I split the big database into 3 parts, a majority of the changes would be in only one of the 3 databases, say daily. The other 2 new databases might only be changed 1-2 times per week. So the total footprint might be similar to the original larger database but the need for syncing is now asymmetrical.
Correct on your last 2 paragraphs. I have a full set of databases on the 2 MacBooks which tend to be in 2 different parts of the USA. Only selected databases are synced to the 2 iPads.
And yes it is overhead heavy and I want to reduce the time spent handling updates.
As a side note, I have been testing DTPO 3 with smaller databases [non-critical] and it does seem to work better, ie faster and with less handling. Not sure if this is a function of the smaller database, improvements in DTPO 3 or both.
In reality, I ran into a lot of issues where either iCloud or Dropbox would just hang up and not sync.
So a large database which would take several days to get an initial sync to work would instead sync in just a few hours after I broke it up into separate databases.
I actually stopped using DT2 for a while because of this problem - even though my use case is spot-on perfect for Devonthink and I love the app otherwise. I am very glad that I gave DT3 a try and in the process stumbled into realizing how I can better handle the large database sync issue.
Plus the interface of DT3 makes it extremely easy to work with data in multiple databases simultaneously.
Plus the sync time is notably slower on iOs; it turns out that at least for me DTTG works much better if I restrict it to syncing only the database(s) that I truly use on iOs and not try to store 100% of everything on my phone.
Worth mentioning, in case it is not obvious, that “splitting” a database means creating a new database, or more than one, and moving data from the original database to the new database(s). (Export from old then import to new is a fast way to do this.)
I prefer to stop all syncing of the original database on all devices before I undertake the split, remove the databases from all iOS devices, and do the split on only one device.
I then sync the new database(s) to a new sync store. And then, for laptops, restore the databases to the other laptop(s) from that new sync store. Then I sync with the iOS devices.
The point is to stop using the original database everywhere. Make sure there’s only one instance of it left anywhere – which I zip then backup to multiple clouds. Then carefully start syncing in a stepwise manner with the other devices. I prefer using DEVONthink sync to propagate database instances to laptops than other means such as external drives. Just my own idiosyncratic shoe-tying to make sure I don’t fall down.
Never duplicate a database and then do surgery on all the duplicates. It’s tempting to do that if one has a lot of replicants and smart groups. But it results in no end of troubles, in my experience.
Good points. What I had in mind was creating 3 new clean databases, moving the relevant data into them form the original big database, syncing where I needed them, and then keep the original on the two Macs, taking it off all other devices.
In details, doing what Korm delineates except use 3 new databases and not use the original one.