Hi, DevonThink is my current go to solution for organizing research and administrative information. I have steadily accumulated a large collection of materials and use iCloud for synchronization between two desktops and two iOS devices with DevonThink to Go installed. Local databases are stored on an external drive. This works really well; however, as the size of databases grow so too the cache in ~/Library/MobileDocuments … /DevonThink (268.8 GB) and other hidden files such as Sync.log in Cloudy (10 GB??). Unfortunately, Apple’s design decision to make the local iCloud sync store non-relocatable to a different storage device means that a certain point one iCloud sync just become impossible to use further. I have 5GB of room left on the system drive. Part of the issue seems to be a set of database duplicates that cropped up over time in iCloud. So, I have two questions.
-
How can duplicate iCloud sync stores on local databases be safely identified and removed to eliminate needless duplication? The local databases only amount to 61 GB, a small fraction of the size of the storage sync for these databases on iCloud.
-
Why is the Sync.log file in Cloudy so large. 10GB seems enormously excessive. Is there a safe way to trim that file down?
Cheers
I’m having similar problems. I have asked in another post if we could get more information about how sync works so we can understand these issues.
I’d really like to see more status information for debugging. It’s all “black box” and difficult to debug.
I find the iCloud solution to be very unreliable. I understand the fault lies with Apple but I have very low confidence that things are synched and rely on check files created just to detect sync failures.
I love DTG with a passion and want to be able to trust it. I’m very worried that I’ll silently lose files.
Why not use one of the non-iCloud alternative means of sync? Or two of them?
Good question. I pay for Apple iCloud disk space but don’t want to pay for DB or one of the others. I’m retired and money is an issue.
Maybe I’m missing something, I’d love to use direct sync to my own sever over the internet, that would be ideal. But they don’t let us specify the host by IP address or DNS name, just bonjour.
So I limp along with iCloud.
Of course, although switching from iCloud to another sync solution imposes its own costs. (Good I am stuck at home and have a bit of extra time!), nor does switching address the issue of the multiple duplication of synced databases within iCloud (61GB of databases somehow ballooning to 268.8 GB) and the seeming difficulty of cleaning all this up. Apple’s insistence on the shadow mirroring of iCloud files is an issue only Apple can rethink. Hopefully, the duplication of databases by DevonThink is just a rare quirk.
Maybe you can set up your own free WebDav server
1 Like
Sync.log is usually a quite small file (few KB up to several MB). If it’s really that huge then you’re experiencing lots of sync issues. This could also explain the used disk space in ~/Library/Mobile Documents as macOS & iOS should evict the local files after up-/downloading.
You could try to verify the sync store thoroughly (see contextual menu in Preferences > Sync after pressing the Alt modifier key), this should also remove the local copies. And you can of course delete the log file, a new one will be automatically created.
Thank you for the clarification. I moved sync to Dropbox for the time being and—eventually—cleaned up the iCloud sync store with a trip to the office computer and turning off iCloud for DevonThink to Go on my devices. So far, so good. With luck whatever issue that led to the duplication of databases on iCloud will not occur again.
I think I have similar issues.
When I started using DevonThink I used CloudKit for synchronisation. Initially it was ok, but later started giving errors, having long lags between devices, and incomplete synchronisation. I recently switched to Dropbox and since then synchronisation seems to be reliable again.
However, my DT3 sync file in Dropbox is huge (100Gb) and has taken up all the remaining hard drive space on my old MacBook. My newer iMac is not challenged.
I add notes that I create, and those copied from the internet, direct to DT3. I also index pre-existing files stored in other Dropbox folders. As most of my self-created notes are tiny Markdown files, I am beginning to wonder if the indexed files (mainly pdfs) have been duplicated to the DT3 sync file in Dropbox?
Any suggestions would be appreciated. I have generally found DT3 intuitive and I am able to find and use files rapidly, as well as making use of smart groups and rules, and many other features. However, I have not previously needed to look in detail into synchronisation and it remains something of a mystery to me.
Good move to move away from Apple cloud services. A few starter suggestions. Others can chip in.
: version of DEVONthink and DEVONthink To Go?
: how big are the databases that you sync? See in Menu: File → Database Properties.
: imported or index files or mixed?
: Perhaps “Verify Database Quickly” on the sync location?
: Perhaps “Verify and Repair” databases?
: Did you “clean” the iCloud sync location when abandoning it? (Not sure that affects Dropbox Sync locations, but mentioning none-the-less)
: Maybe a brute force fix would be to start again with Dropbox sync. Clean it then sync again. Only sync those databases that you need.
: Consider Bonjour (also) as it has no need for sync locations on cloud servers. Usually faster and more reliable.
: read and take advice from the Troubleshooting chapter of “DEVONthink Manual”. Sync covered on page 188 3.9.1 version
Thank you for your rapid and helpful response. In answer to your questions:
DT3 3.9.1 and DtG 3.7.
Six databases, 9Gb, 700Mb, 2.3Gb, 48Mb, 2.6Gb. 41.8Gb.
Mix of imported and index files. Smaller databases mainly imported Markdown files. Larger databases contain many indexed pdf and other larger files.
Do not (yet) know how to quick verify sync location.
All databases show successful Verify and Repair, but largest database has 6 checksum errors.
I did not “clean” the CloudKit sync. I was waiting to see if Dropbox worked before abandoning CloudKit completely, but I think I can now safely delete any files in CloudKit sync.
Will consider a brute force fix if I cannot make progress in other ways first.
Bonjour would make a lot of sense, but I need to check how to set it up.
I have just downloaded the latest manual and will ever to those pages.
Sync is in progress at the moment and is taking a long time for my largest database. I will leave that running in the background and return to troubleshooting later today or tomorrow before I add any new data.
I appreciate your advice.
Frankly, all the questions you posed are answered by reading the outstanding “DEVONthink Manual”.
Sync will take the time it takes.
Good luck.
You should not have any Dropbox sync data on your Mac. See In & Out > Sync > Dropbox in the built-in Help and manual.
Thank you. I will re-read the manual (latest version) and see what I have done wrong when I next get free time from 7th June. Devonthink is synchronising, but perhaps not as it should. I appreciate your suggestions and encouragement.
