Question to the fellowship: how can a 4GB database possibly require an 18GB local sync store!? I am stunned! Can I expect the size of this store to shrink after it’s been imported into the second machine? Thank you.
Are you indexing files in your database?
Yes, quite a few.
Indexed files are always Synced - regardless of the Sync method. This cannot be turned off. This is almost certainly where the extra data is coming from.
Yes, Jim, that makes sense.
On the face of it (hardly a position from which to talk about such complicated matters, but thanks for your patience 
), I would prefer a different scheme, in which the syncing of all indexed content itself would be the responsibility of the user (e.g. via Dropbox). This would, I imagine, enable DevonThink to sync only the index as well as the links/paths to the indexed content, not the actual content itself. The result would be a much smaller sync store, at a time in history when syncing of large files is not yet very fast or even 100% dependable.
That of course would make syncing with the iOS version highly problematical, given the sandbox model that iOS utilizes.
By the way (or should I open another thread for this), is there a way to verify the consistency of a sync store? I am having a few nagging problems syncing via my local sync store (and BitTorrent sync). The problems are so far restricted to items I delete—they sometimes reappear, either in the main database or in the trash (depending where they last were). Curiously, this even happened with an indexed file that I had physically deleted in the Finder and in DT from both machines—which means that the sync mechanism found the file in the sync store and took the liberty to restore it!
Thanks.
Syncing with iOS (DEVONthink to Go) is a different case, I think. With DTTG we explicitly configure it to sync these files, and/or these groups, and there is no concept of indexing on iOS anyway.
I can see your point with DEVONthink database sync – the transfer of indexed files from one machine to another via DEVONthink sync is unexpected. DEVONthink doesn’t tell us “I am going going to transfer external files to the sync store – do you want to proceeed?” It just does it. On the other hand, if the indexed files were not copied into the sync store and from there into the 2nd (or more) instances of the database on other machine(s), then DEVONthink would have to track “where” the files are to avoid problems like Verify & Repair failing on the remote machine due files “missing” because they were not synced.
Yes, that was precisely my point. And I was thinking forward, thinking of the future versions of DTTG, realizing that the only way to sync to iOS files that are indexed on OS X would be by including the full content of these files in the sync store. This is in fact the best reason I can surmise for this design decision, which otherwise results in humungous sync stores.
Either track the indexed files, as you suggest or, as I imagine it, actually relegate the responsibility for any broken links to the user. DevonThink is actually quite brutal in providing no mechanism for ensuring the integrity of links with indexed files: move an indexed file from its expected location, or rename it, and you get a broken link. (This doesn’t have to be that way, incidentally. BibDesk has solved the same problem by relying internally and transparently on aliases. The BibDesk user can freely move/rename indexed attachments without any problems.) I think we’re all used to handling our indexed files with care. So, if DevonThink relegates responsibility for the integrity of these links to the user at the local (single machine) level, what is the point of this extreme measure (humungous sync stores) for the sake of integrity at the 2-machine level? I find this a dubious benefit with a disproportionate cost. (I am talking about a literal financial cost here: I had to purchase a 35USD/yr BitTorrent Sync subscription to accommodate my 18GB sync store for a 4GB database).