Sync Issues

Any assitance welcome.
Setup sync via dropbox, but “verification of database failed” is the error message. This occurs on first sync attempt.
If i sync just global inbox it works ok.

Any suggestions??

Run Tools > Verify & Repair on the failing database.

I’m having similar issues. I’m using Dropbox Sync. Until recently all was working. I got the error message also and ran the Tools -> Verify…

There was an error found, and corrected. Running the Tools -> Verify… reports no errors but the database are not synchronized. It doesn’t appear that I’ve lost any information in the original database, but I’m afraid of taking any additional steps for fear of data loss.

Bryan

Tools > Rebuild Database…

Having some issues with the local server database sync process.

Conflict Resolver.
There is no indication of how many records need to be resolved. Sometimes it may be just a few, but sometimes it can be hundreds. Knowing this, with a live count . Eg. Initial Total Unresolved Records & Records Still Remaining Unresolved (& %). Would be very useful. Would gives the user an idea of the scale of the solve (and time needed to process). Also, I know there is a slider, but this has no scale, range or numbers.

Box: “Resolve All Conflicts The Same Way”.
What does this mean? For example. It could be:
Use the records that have the most recent Modification Date-Time.
Use the records that have the most recent Content Modified value.
Use all the records from the Local Database (or Server).
Always use the record that smells most sweetly of new-mown grass.

Apparently identical Records (in Conflict).
Sometimes the two “conflicted” records seem to be identical. And all the modification dates/times are also the same. So, why the difference error? Perhaps some helpful clue could be provided. Eg. “Same content, different folder” or “Tag Name Change” etc. Sometimes, there are conflicts listed for records that have not been touched for several weeks. Why might this occur?

Partial.
Let’s say the user manually resolves a bunch of record conflicts, but there are still more to go (and there is no count). The user’s available time has run out. But there is no way to proceed with the Sync with the resolved records so far, leaving any remaining conflicts for a later date. So the user has to start again at a later date. Quite annoying!

Rebuild Database.
After a database rebuild there seem to be hundreds of new conflicts. Why is this? Should the rebuild not just restore to the most recent condition, leaving the same number of conflicts, if any?

Thanks.

A simple verify and repair did not work. I am still having issues, but I’m having trouble pinpointing what is happening. Here’s what I know so far.

I have two computers, and iMac desktop and a MacBook air laptop. I mostly use the desktop at work with DevonThink Pro as my daily project management tool.

I sync with Dropbox realtime / daily at work. Although dropbox syncs almost daily on my laptop, it may be many days between usage of DevonThink Pro on that computer.

Things will fine for about a week. Then, I will get a sync error on the work machine, usually due to a “file not found” type of error. I will run Tools -> Verify and Repair, which reports that the problem has been found. I select repair and the program reports that the problem has been fixed. But when I do the next sync, the sync log reports the same problem again. Run Tools->Verify and Repair again, and the same problem is found and reported as repaired. I can do this over and over until I manually delete the offending link, remove it from the trash, verify and repair and then re-sync.

At present, when this happens, it is only a file or two, and typically seems to be related to an email as the originating document.

At this point, I’ve always been able to recreate the entry after the fix, but I’m concerned about data loss.

Any thoughts?

P.S. This is a fairly large file. I am very careful not to start DevonThink until I am sure that Dropbox has fully synced with the computer. I do automatic syncing in DevonThink upon database opening and closing.

Is your database stored in a Dropbox folder? If so it MUST be removed from this location.

Also, did you try the Tools > Rebuild Database…?

No, it is not stored in Dropox. It is stored locally on my computer and synced with the other computer via the dropbox sync procedure.

Tools verify and repair or rebuilding the database will not help at all in this case. Dropbox will not sync the database if it detects improper files. I don’t know how you use your database but in my case I use it extensively with DTTG a dreadful product. It will generate false notes. That is, if you manually scan through the records in your database, there will be a record that has a header and nothing in the data field. This record will not sync to Dropbox. Dropbox will stop sync’g at this point. You need to be able to get to the “commit” point in the sync for a true sync to happen.

Fortunately, if you have the activity monitor on while your sync’g everytime Dropbox fails it will identify the record it has failed. Record the name correctly and do a search and you’ll find this record. Look at. It will be corrupted. Delete. Dropbox will now continue on. However, if there is more than one record that is corrupted, Dropbox will stop at each one and you will need to delete each one. Eventually, the whole database will sync.

I spent a whole afternoon once deleting multiple records and then finally my database was fully sync’d. Once sync’d, even if a false/corrupted record gets back into your database, it Dropbox will keep sync’g. However, should you need to do a full sync from scratch or Dropbox goes into a full sync from scratch mode that can happen, if there are any false records in your database, you will have to go through the above process.

There is absolutely no way around the above process. It is very manual and extremely labour intensive. Sometimes if you do a column view by file size, any file that is a size of zero is a corrupted/false record. Just delete them all as there’s nothing in them even though when they were created there was.

As I said, I’ve found the majority of this comes from DTTG and is persistent and replicable. There is no way to know either when DTTG or why DTTG will do this. There is nothing special about any of these records that sync into your DEVONthink database incorrectly. However, have fun with the mess. I sure have and i can assure it is a mess.

I was ready to throw my iMac out the window that afternoon that I spent all that time cleaning up the db. Fortunately, I haven’t had a problem since. I don’t know why. But what you are experiencing is real and so don’t feel like you’re imagining something as you’re not.

One thing that I do occasionally now and this is for orphaned and phantom files generated by DTTG is just to de-install and re-install DTTG and then re-sync it from scratch. This ensures that it is clean and that the data in it is solid as generated from DEVONthink. It might be one reason I’ve never run into this problem since. Don’t hesitate to do this either. It really doesn’t take long. The re-install of DTTG is fast and simple and my database re-sync’s to my two IOS devices very quickly.

It’s too bad the answer to this isn’t more forthcoming as there is an answer to this.

Anyway, that’s what you’ll have to do and with proper maintenance procedures you should be able to avoid a future recurrence of the issue. Be proactive.

Try a Sync to a local syncStore on your Desktop.

kerryd,

Thank you for your explanation. Based on your description, I think I’m going to resort to trying a USB sync. I can’t do a direct sync because my Mac at work is on a secure vLAN. I can’t link the two machines because the wireless does not have access to that vLAN so there is no easy way to do a direct link, or a link to a sync store on the work machine.

Do you know, is there anything special I need to do to “remove” the links to the Dropbox sync store? Can I just turn off that synchronization and start a new one on a USB stick?

Thank you for your help.

Bryan

All you really need to do is go into preferences/sync and disable sync with Dropbox and delete your dropbox account.

You might then want to go into your Dropbox folder itself, apps/DEVONthink and delete the DEVONthink folder.

By deleting that folder, if you do decide you want to sync via Dropbox you’ll be starting fresh and clean. You’ll likely find it works for you.

If there are specifics I can help you with around this I’d be glad to. I know it extremely well. I think we might have gotten married somewhere along the way :slight_smile:

Kerry

Oh and I forgot to mention that if your database does have bad records and I’m pretty certain this is the case be aware they’re there. They won’t cause you any problem except if you sync with Dropbox. However, Dropbox will identify the record it’s stalling at in the activity log and it’s then easy to find that record and delete it.

So you’re database will work fine except if you go to do a Dropbox sync if those bad records exist.

I never did think to ask do you use DTTG. Just curious?

Thank you for the additional information. At this point, I don’t believe I have any bad records. Everything seems to be working well at the moment.

I am using DTTG, but “use” is a relative term. It is on my iTouch, but I’ve only synced it a couple of times. The majority of my use is on my work iMac and my MacBook Air. I bought DTTG in spite of all the bad reviews, but never really got into using it that much.

In doing some further investigation, I’m coming to find that (and this is from memory only, so my observations may be incorrect) most of the records that end up failing are emails that I dragged and dropped from MS Office Outlook for Mac. It seems like, again only from memory, that when I do that, the record establishes a “link” and doesn’t really import the information unless I click on the record to display the link. Could this be tied to the Dropbox empty record issue that you mentioned? Just a thought.

I’ve not yet moved to syncing to a local store. I’ve still been playing with the dropbox sync and have some new information. I’ll try to do this as succinctly as possible. I’m losing data in the dropbox sync process. Here’s what happened:

I checked, verified and rebuilt all databases and I wasn’t getting any errors or warnings on either machine (desktop or laptop). I’ve concentrated on monitoring a single database, and I believe, to the best of my ability, that the database was in sync and contained the same data on both machines.

I edited a rich text file contained in (not linked) DevonThink Pro Office. That “record” was on the screen when I closed DevonThink. The program is set to automatically sync with dropbox upon start and exit. I confirmed that dropbox was completely synced before shutting down that (desktop) computer.

I went to my laptop and allowed dropbox to fully sync. I opened DevonThink Pro Office on my laptop, which is set to sync on program start and exit. The text that was added to the rft record on the desktop machine did not appear in the rtf record on the laptop. I did a verify and repair and a rebuild and the text did not re-appear. I received no errors with ether of these processes. I closed DevonThink and let dropbox full sync.

This morning, when I returned to the desktop and opened DevonThink, the record that had been edited appeared on the screen upon startup, showing the text that had been added during the previous day’s edit. As DevonThink performed its sync upon startup, the text disappeared and is no longer in the database once the sync activity bar showed the sync was complete. The text literally disappeared as I was watching the screen. The record and the “original” text I added was still there, but not the “edit”. I did a verify and a rebuild. No errors. No text.

It’s almost like some edits are not getting transferred to dropbox via the sync.

I guess I will be moving to syncing with a local store on a flash drive. I really don’t want to do that. That forces me into a manual process, which has its own perils instead of an automatic one. But there seems to be issues with the dropbox sync process.

Any thoughts?

Did you delete the database on the second machine and reimport it afresh?

Such as…?

I had deleted the database and reimported a couple of days prior (as part of the previous issues and process), but not right before this process.

First chance I get, I’m going to go back and try to replicate the issue. I’ll try to replicate before deleting the database and re-importing, and then again after deleting the database and re-importing, to see if I get a different result.

The other perils include:

  1. Forgetting to grab the USB stick and take it with me
  2. Losing the USB stick

Both result in not having the latest data with me.

  1. Speed, the USB stick is quite a bit slower than dropbox for the sync.

I would get a new stick then. A local syncStore should be much faster than a Dropbox (or other cloud service’s) Sync.

There’s a ton less transactional data that needs to go on and there’s no network congestion / protocols to contend with. Even on my old 2009 Macbook, it’s away faster than a Dropbox Sync. (On my 2012 Macbook it feels orders of magnitude faster!)

Still, I have to legally say, YMMV. 8)

I’ve run some more tests.

Yes, I agree. The stick is much faster. I was pleasantly surprised. It seems that the dropbox sync does quite a bit of “file by file” work while the stick sync does the whole “store” at once. It is quite a bit faster.

That’s the good news. The bad news is that the problem persists. I get errors. I do a verify and repair. The verify and repair reports that records were repaired and that all is well, yet when I try to sync, the sync reports that it failed and won’t sync. I run verify and repair and the exact same errors are reported again.

I agree with the post above regarding Outlook / emails. In all cases, I’m coming to find that the errors relate to records from Outlook for Mac 2011. Some of the emails coming in from Outlook are showing a “File Missing” error, even though the emails are still in outlook.

At this point, I’m not sure why this is happening. When I delete these records, everything is fine from that point on, until I bring in more emails. (I am using DevonThink as a place to collect emails in sort of a ToDo list.) I’m also a little confused as to whether DevonThink is storing a “copy” or a “link”, or if it’s trying to do both. Maybe it’s something I’m doing, but I’m seeing some inconsistency here.

Now that I’ve narrowed down to an interaction between outlook and DevonThink, I’ll try to pinpoint this further.

A quick update.

The USB sync is going well, with two exceptions. I’ll get to that in a moment.

The problems I was having with Outlook emails resolved itself after a while. I believe there is a problem with the interaction of Outlook emails and using the Dropbox sync. Something about the Outlook record in DevonThink does not allow for reliable syncing via Dropbox. Once I had been using the USB sync for a period of time, I was able to manually correct those issues and they have not cropped up since.

I do have two new issues with the USB sync:

  1. Sometimes DevonThink gives me an error message that says “You do not have permissions to sync on this drive” or something like that. The USB is set to full permissions for everyone. The only recourse is to shut the computer down and restart. Once I do that, the error goes away and DevonThink will sync. I am very careful to properly eject and remove the USB in all cases. Ejecting and re-inserting the USB does not clear this problem The only recourse I have found is to restart the computer.

  2. I have had data disappear from a record. To the best of my ability, here’s what I believe is happening. On computer one, If you have a .rtf record, edit that record in the document window of the three pane view, then sync and exit, the most recent edits are not always saved. If you then sync on another computer (computer two), not only are the changes not there, but when you go back to computer one, DevonThink sees the “unchanged” record from computer two as the most recent, and deletes the edited information from computer one. On computer one, I actually see the information come up on the screen, then disappear upon syncing. This situation is consistent and repeatable. Since there is no manual way to force a save, I’m not sure what to do about this situation.