Sync beta: what should I expect?

I have DevonThink Pro Office running on two machines, my desktop at home (which is also the home server) and my work laptop. I have multiple databases on each of these machines. Once upon a time, these databases were perfectly in sync. Over the last couple of years, I have allowed the databases to diverge. Consequently, the content on each machine is perhaps 90% identical. That said, each machine’s databases contain some documents that are lacking on the other. Further, each machine’s databases have had a few deletions that were never made on the other machine.

When I tried the sync beta, I expected to wind up with the union of the contents of the databases, with no duplicate files or groups. Thus, I expected to see files or groups that were deleted only on one machine re-appear in the synched databases. I tested, of course, on copies of my original databases, just in case.

What I got was rather different.

  • Some groups synched up correctly. Most of these were groups whose contents were identical on both machines. A few were groups containing some files that were identical on both machines along with some files that were unique on one or the other.
  • Most groups were duplicated. That is, I wound up with a database that contained two groups named “Economics & Policy” (for example), each group holding the files from one machine only. Note that most files within these groups were true bit-wise duplicates of each other – the Unix cmp utility found them to be identical.
  • Some of the duplicate files were detected by DT as duplicates, and were appropriately indicated as such in the UI.
  • DT failed to detect most of the duplicate files as duplicates.

Was my expectation reasonable? Am I seeing syncing failures, or is this expected behavior? Is the sync solution intended to cope with synching up partially diverged databases, or does it only handle propagating changes between databases that were identical on first sync?

Other related questions:

  • What should happen when I have moved (or replicated) a file to different groups on different machines (without editing the file)? Imagine a .rtf file originally in SomeGroup on both machines. Move it to OtherGroup on machine A, and to YetAnotherGroup on machine B. Now sync the machines. What is the expected result? Does the answer change if the move happened before syncing for the first time vs. after syncing has started?
  • What happens when I change the group hierarchy? For this case, imagine that I shuffle groups around without editing any files. Assuming that syncing is already in place, I would expect changes on one machine to be reflected on the other after a sync operation on each machine, right? But what should I expect when the changes to the hierarchy were made before the first sync?
  • What should happen if I edit a file on Machine A, and then sync both machines? Assuming that the machines were in sync before the edit, I’d expect to see the edit propagate to the other machine. But what should happen if the edit occurred before the first sync ever? Should this look like two different files? Should it look like a conflict? What do you do with conflicts, anyway?

Speaking as a geeky software type myself (Ph.D. in Software Engineering), I’d love to see the document that lays out the various cases and the expected result of each. Given that the sync plugin has to be built to handle all possible cases (or, at least, all interesting cases) and that you need unit tests for each supported case, I’d expect that you already have some sort of document or chart explaining the expected behavior. Certainly I’d have to write it down to be sure I hadn’t missed an important case (more likely, many important cases). YMMV, of course.

I also had trouble with doing a round-trip sync. See the response I posted today to someone else’s topic for more details.

System information: Mac Pro early 2008 and Macbook Pro mid 2010, both running 10.8.2 with all updates. Synching over wired LAN to a sync store hosted on a disk attached to the Mac Pro that is shared via AFP. Results described are identical regardless of which machine creates the sync store.

Xenophon, thanks to have invested so much time six months ago to help to improve the DevonThink sync.

It seems everything you described then is more or less still a question today.

See my posts:

and

I am very surprised you had no answer. And very surprised also it seems you were not too much listened to…

Many times, forum posts quickly turn into offline discussions (though I am not claiming this is the case here - I honestly don’t know offhand). But a lack of response on the forums is not an indiciator that no is has been listened to.

In any case, it’s certainly not an indicator it has been listened to! :wink:

What I notice is only this:

  • I worked more than ten hours this week-end trying to solve my little problems with DevonThink, with limited success.
  • I have observed more or less everything that was evocated by Xenophon happening right there, in front of me, on my iMacs and MacBooks, with a DevonThink version that is now not any more a beta version.

I would have prefered to observe quite the contrary on both topics…

I am fond of DevonThink (until 2.4.3) so what I say is said only to help in the future.

There must be someone at DEVONtechnologies who knows the answer to all of my questions – if only because these are the kinds of questions whose answers define “correct operation” for the synch capability. It would be great to know what is supposed to happen during synchronization. There are lots of possible correct outcomes depending on the choices made by the developers. Reporting bugs would be much easier if we know which outcomes are expected and which are unintentional.

Even an informal blog post would help. Does this message constitute a sufficient request, or should I contact customer service directly?

Bluefrog? Beuller? Anyone?

The first question I have is… are you running the publicly released Sync plugin now?

Sync mirrors the databases.

If there are copies of the same document, in different states due to individual editing, a conflict resolution window will appear for you to determine which is the copy to Sync.

@ Xenophon Thanks for the help!
@ Bluefrog I propose some examples (non exhaustiv list):

database D1

doc A
last modified 13-03-15 12:00 by X

doc B
last modified 10-03-15 10:00 by X

doc C
deleted 11-03-15 08:00 by X

database D2

doc A
last modified 13-05-15 11:50 by Y

doc B
deleted 13-05-15 11:00 by Y

doc C
last modified 10-05-15 09:00 by Y

Some questions:

Do you think we have enough datas provided by the software to take the right decision in the first case (A)? I am not so sure, the software is only going to ask the only person syncing (X, Y or …Z?) “do you prefer the X version or the Y version?”

In the second case (B), will the doc B be deleted on both Macs? Will I be informed if I am Y that the doc has been modified by X? Will I be informed if I am X that Y decided to delete the doc? Will I be informed of the history of this doc if I am Z?

More or less the same questions in the third case ©.

Without a common server to “time” the events how do you know what are the intents of the team members?

Wouldn’t it be better to log all the events and to keep successive versions of all the documents to be able to restore the right version if lost by error? Wouldn’t be more metadatas about the documents concerned and an opportunity to visualize them serious helps for the users?

@Franck:

I have not tested each of your scenarios presented but your example leaves a larger (and in my mind, more important) question hanging: The question of protocol and administration.

If this is a single User scenario, which is many times the case, then it’s easier to determine what version is appropriate to Sync. In a multi-User environment, it is imperative that protocols are put into place to alleviate these situations from occurring. Especially concerning file management (deletions, database restructuring, etc.), this is so important for oversight to determine what is and isn’t appropriate.

As far as keeping successive versions, extra metadata, etc., this is beyond the current scope of Sync as it’s implemented. It also complicates the transactional processes by requiring more preferences that may not be desirable in all Sync situations leading to more complexity in the Sync Prefs. You could file an official feature request through our DEVONtechnologies Support Ticket System.

Thank you for this answer.

I have already submitted a support ticket. With no answer until now.

As I am a blog author and a customer trainer, I must be registered as a journalist in the DevonThink database. Perhaps it’s the explanation for the lack of answer?

Regarding sync, as far as I remember, and I have worked managing information systems more or less directly since 1983…, it has never been nor easy, nor simple. To try to keep it simple is not at all so simple, lots of situations can happen where you need lots of metadatas and complete versionings. If you don’t do that, the prices to pay are lots of “mistakes” (= bad choices) by the software or by the human beings using it.

We are a small team and we already have the problem. We are now using DevonThink only on one iMac running the legacy 2.3.5 version. On the other iMacs, it’s now not allowed any more to use DevonThink. We don’t use anymore DevonThink 2.5.1 and we don’t try any more to use sync as it is now. Does it seem the administration and the protocol the DevonThink team imagined their users to adopt? Is it good practice?

We are a small team but we can have from time to time some influence in large accounts. For instance, in recent past, we coached a customer in the choice of a CRM for around 2.000 users. We think there could be an opportunity for DevonThink to grow not only in the SME Market, but also in bigger companies. But even in the SME market, as soon as you reach 2 users, you need a bullet proof sync.

And our experience seems to prove this first version has still to be improved quite a little. If I had known, I would have invested more time to try to work with the DevonThink team during the beta test… Sorry! :wink: