I am syncing through a dtsyncstore on a Webdav server and after reducing the number of connections to 6 I am happy to report that the new sync seems to be working very well indeed.
Lately the log is collecting “manifest file missing”, after one of the two Macs froze completely due a problem and had to be restarted. I am assuming that the problem had nothing to do with DTPO, yet I am unsure if the message means that the dtsyncstore on the remote server is damaged and needs to be recreated or if the local DTPO database are having a problem.
Only the iMac that had to be restarted shows these messages in the log, not the MacBook at home and both appear to be syncing fine, despite the iMac showing these messages.
Should I recreate the local database or the remote dtsyncstore?
Okay, perhaps the easier question: Where does the manifest file reside and the state of which database (local db or remote dtsyncstore) does it reflect?
Verify & Repair reports no issues at all.
I do not use Magenta Cloud, the Webdav server uses Seafile and has been rock solid before the iMac had to be revived by literally pulling the plug. It is this computer that reports the “manifest file missing” in the log, not the Macbook that syncs to the same dtsyncstore, which makes me suspect that the syncstore may be okay and only the database on the iMac broken.
Is it possible that it is something akin to an undeleted lock file if the iMac froze during a sync (that had to be terminated abnormally)?
I have created files on either computer and they were synchronized without issues.
Ah, there is a new feedback:
Broken or not yet complete manifest chain of database XYZ
plus the 5 more explicit manifest file warnings that I had been seeing before
Before I do that and re-upload the database to the Webdav server
Is there a way to find out if both databases (DTPO) are in sync? They appear to be and test documents were synced back and forth but it would be nice to know before I make the local database on my Macbook the sole source for the dtsyncstore and later the iMac, especially considering that the sync itself may be compromised.
The dtsyncstore serves as a sync hub for several DTPO databases. Is there a way to find out which one is affected? Or should I do the same for every database that syncs to the dtsyncstore?
I cleaned the dtsyncstore and re-uploaded the databases one by one. The third one threw an error of the kind:
18:52:00: 24dc71305e6022c3f4ae957eb8061e1…/receipts/20e33d0f135a6dcd6e70a0330fecfb389cbd79e72c6f1f8fb…receipt/001/5e6054b8e8d4e6bc9db011…4770349320a8da94afe07.5 Verification failed.
(the “…” were included by me and replace similar strings)
Verify and Repair for the local DTPO database still runs through without an error.
Five minutes later (I had just manually issued “Verify location quickly” by right clicking in the Sync Preferences) the activity bar pops up again, uploads another 200 items or so and logs “successfully verified”.
For the following two databases I reduced the number of connections from 4 to the smallest value possible (2).
I received similar error messages again for two other databases and like before they disappeared after DTPO was forced to resume syncing.
All subsequent sync operations after the initial upload went without a hitch. There seems to be a minor bug perhaps but as far as I can see one without major consequences.
Even using only 2 simultaneous connections the whole process went fast, really fast. Although the server is physically in the same house, I was connected by Wifi only and with 5 databases about 10 GB in total the whole repair took maybe 45 minutes if that long.
Thanks a lot, Devonthink with the new sync is a dream come true for me. For the first time ever I am leaving my portable hard drives and USB sticks at home, which is really liberating.
Prion
PS: To the best of my knowledge the problem solved above was not caused by DTPO and besides the error messages in the log the actual sync appeared to have worked well regardless.
It all went well for a few weeks but as of today the dreaded “manifest file missing” error message is back…again.
I can narrow it down to one particular database by unchecking the databases one by one in the Sync preferences, the error message also gives details about the particular file name missing - is there anything that can be done short of cleaning and uploading to the syncstore yet again?
On the Mac. I did intermittently sync with an iOS device but the problem was reported on the MacOS side. The verification of the database goes through without feedback, i.e. seems to be okay.
This time I am not entirely sure that the server is at fault here. Before uploading the database after the first problem (see above) It transferred the content to a new and empty database at which time DTPO complained several times about UUIDs (I am not sure I remember the technical term here) already being in use. I also admit that I forget what I did about it. I’ll try to repeat it now.
The box regarding verifying uploaded items is checked BTW.
Question: Using the cryptic ID of the manifest file in the error message, is there a way for me to identify the offending file in the database so that I could carry out a more targeted removal?
The re-upload of the many gigabytes seems more like a sledge hammer-inspired cure.
On the computer that is experiencing the manifest issue? Then you could try this but please make a backup first:
Delete the database via DEVONthink Pro’s File menu, this removes the sync status of this database too
The database is now located in the Finder’s trash, move it to its former location
Open the database again
Merge the database with the remote database, this should be a lot faster than uploading the database again
No, especially as the databases might not be in sync afterwards. It’s intentional that sync stores use chains & checksums to verify the contents and to detect server and other issues.
In the meantime I checked the content of the database in question and noticed that a folder with relatively large video files was really causing the issues. Since this folder really should not have been in this database in the first place I removed it from that database and re-uploaded to the cleaned syncstore, which thankfully went a lot faster and without issues.
But thank you very much for the detailed description of what to do should the problem occur again, duly noted for future reference.
Okay, understood plus I now have the workaround above, thanks.
I ran into the same problem with the “missing manifest-file” twice. Right now is the second time it happened.
The first time I ended up reuploading the databases that had missing manifest-files.
Now I am really p****ed that it happened again, since the problematic databases have 5,6 GB and 14,3 GB. That takes REALLY long to upload (and I have to work and now, again, I cannot work anywhere). On ALL connected Macs the folders/groups in the main window show the same amount of files. The only difference is shown in database-information, where the database on one of the Macs has one PDF-file more than the others (which is strange, since the file-counters show the same amount of files).
Why is it not possible to tell DEVONthink just download/upload the missing file? It just sucks.
Sorry, I really love DEVONthink. So much, that I totally rely on it. But this is the second, unnecessary upload of 20GB (which takes about 2-3 days).
The verification of the sync-Store tells me there are several files “missing” or “not available yet”.
Verification of the databases show me no problems.
Syncing tells me about the missing manifest-files.
Is there anything I can do without uploading the databases again? How can I find out which files are “missing”?
Thanks in advance (and sorry for my harsh words but I am really upset),
Which cloud service do you use? In addition, please verify all databases in the sync store on their own (e.g. use the contextual menu of the database in Preferences > Sync) to figure out which database is broken.
After uploading the two broken databases (for 2 days), everything was fine again. Until the day before yesterday. Now three databases are “broken”, due to “missing manifest-files”.
The sync with this particular webdav-service was working flawlessly. It wouldn#t be so bad if there was a CLEAR how-to, for how to get rid of that error. But it just says “missing manifest-file” with some cryptic filename and that’s it. No hint to a solution whatsoever.
Worst thing is, that I ave to find out myself, wich files are out of sync. With databases containing thousands of files (99,99% is PDFs) in hundreds of groups is not so much fun.
I always click "sync everything_ before closing the app. I always wait for the activity-window to show no more action.
It just s***ks.
I just don’t know what to do anymore. As I said, I totally rely on DEVONthink for both my work and my private stuff. But I need a 100% working sync.
Can I hope for a “perfect” sync in v3.0?
Thanks in advance,
lxc
I
I use WEBDAV (secure), hosted by Strato.de. It’s called “HiDrive”.
Of course, as I mentioned,above, I have checked which Database is broken.