Big data Usage on "mobile documents" on new Mac as well as iOS

Dear Forum,

after upgrading I am currently in the process of setting up a new MacBook. After setting up iCloud and even before setting up DT3 (Pro) I found the Folder under user/library/mobile documents/devonththinkprooffice to contain 36Gb where I know for a fact my databases contain at most 12-14Gb. I have seen similar behavior when doing a complete sync and download on iOS where at one point 50Gb where used up by those smaller databases. (There was a way to solve this but I forgot how and now I am stuck with these data hogs and I am too scared to delete stuff.).

Please advise on how to fix this and maybe even how to prevent it.

All the best,

Johannes

If you’re going to use the Legacy sync, it - meaning iCloud - is going to store data on your local machine. iCloud is not under our control so there’s no guarantee of prevention.

Regarding the space used, I would disable a DEVONthink / DEVONthink To Go Legacy sync on any device.
Then on the Mac, go into System Preferences > Apple ID > iCloud and click the Manage button. Select DEVONthink To Go then click the Delete documents and data button.
This preference doesn’t give a realtime view of the deletion process, so close and return to the Manage section occasionally to check the process. Once iCloud has removed the data from Apple’s server, the DEVONthink To Go entry will disappear.

After it’s done, you could use a CloudKit sync, which syncs directly to Apple’s servers. However, in this case you should only upload one database at a time, letting it finish, then uploading the next, etc. to avoid Apple’s servers from throttling you.

1 Like

Jim’s response is, of course, assuming that you actually have copies of your databases, which you are going to put on your new MacBook - and haven’t, like this unfortunate fellow, forgotten to back up the database files and were hoping to restore from sync (which, nobody who knows will ever tire of pointing out, is not backup).

3 Likes

Dear All,

Thanks for the swift replies. I do have a local copy on my main mac and, occasionally back it up manually. (Would definitely do so before trying your suggestion). I still do use legacy sync as I had peoblems after trying the new protocol. Will have to revisit.

How does sync work anyways on multiple devices if not all files are stored on iCloud as well as on a local machine?

If syncing is not backing up, how can I make sure that the backup I am commencing on my main machine it is indeed complete if I had been adding quite a few files on iOS and my secondary mac?

Cheers for helping out!

J.

How does sync work anyways on multiple devices if not all files are stored on iCloud as well as on a local machine?

What files aren’t stored on iCloud?

If syncing is not backing up, how can I make sure that the backup I am commencing on my main machine it is indeed complete if I had been adding quite a few files on iOS and my secondary mac?

Your backup application should use a snapshot-style backup, e.g., Time Machine. If you continue to use your machine while its backing up, it will never technically be “complete” unless you’re not using it while it’s backing up. But even then once you make changes, it won’t be backed up until the next time backup kicks off. However, in the grand scheme of things, the interval is usually sufficiently short so as to not worry about it.

1 Like

Maybe I wasn‘t quite clear. Everyone here says icloud sync is not a backup. However, for files to be available on all synced devices all files would necessarily be stored on icloud, right?

While it would seem that way, the technicalities are different.
Your files are not stored in a sync location (except Bonjour, which is a direct connection). It is raw, chunked, and optionally encrypted sync data about the files, not the files themselves.

1 Like

As such you are right: you can retrieve your files from a sync store to DEVONthink (using DEVONthink & to go) - so if you set up a new device, you can sync all your files to it from a sync store. That is assuming all your files have been put there, and the store is intact. But: if you suffer data loss on a device (accidental deletion, malware, corruption), that loss will be propagated to your sync store and any attached devices. There is no versioning - so sync fails even the most basic test for a backup.

Sync is great - it is not backup. If you are interested in backup, some users - myself included - posted their ideas on strategies here.

1 Like

Cool. That is the first time I am an clear on that.

Thus,

  1. how can I verify that all files of the databases are synced and locally available on a given device?
  2. if that has been achieved, Should I use the backup dialogue or something else to save a complete backup of the databases to hide away somewhere?
  3. can I the somehow purge all icloud data created by devonthink?
  4. lastly, I would then like to redistribute the database backup files to all Macs and be set from there plus simply re enabling / newly enabling icloud sync and wait until all is uploded?

Hope this is clearly expressed and makes sense. Thanks for taking the time.

any takers on my last post? cheers.

1 how can I verify that all files of the databases are synced and locally available on a given device?

You could look at the file counts in the Info popup of a database in DEVONthink To Go and File > Database Properties for the same database in DEVONthink.

2 if that has been achieved, Should I use the backup dialogue or something else to save a complete backup of the databases to hide away somewhere?

What backup dialogue ?

If you have proper primary backups running, e.g., Time Machine, the databases (and your other data) should be backed up.
Please refer to the Help > Documenation > Gettings Started > A Word About Backups for more information.

3 can I the somehow purge all icloud data created by devonthink?

Regarding the removal of iCloud Legacy sync data, I responded to this a week ago:

Here are the instructions again (and some additions)

  1. On the Mac, go into System Preferences > Apple ID > iCloud and click the Manage button. Select DEVONthink To Go then click the Delete documents and data button.
    This preference doesn’t give a realtime view of the deletion process, so close and return to the Manage section occasionally to check the process. Once iCloud has removed the data from Apple’s server, the DEVONthink To Go entry will disappear.
  2. Then hold the Option key, control-click the iCloud sync location and choose Verify Location Thoroughly. The results will be shown in Window > Log.
    1. In DEVONthink’s Preferences > Sync, right-click the iCloud sync location and choose Clean Location. This will clear out any sync data.
  3. After the clean is reported successfully in Log window or popover, disable the iCloud (Legacy) sync location.

4 lastly, I would then like to redistribute the database backup files to all Macs and be set from there plus simply re enabling / newly enabling icloud sync and wait until all is uploded?

Do you need to use a remote sync option like iCloud?
See this forum post:

2 Likes

cheers for taking the time!!! brilliant!

You’re welcome. :slight_smile:

Hi bluefrog, hi forum,

the new sync works seamlessly as of now and all documents load on demand. however, the creep factor has set in and I see again 70plus GB of storage used up on icloud even now that Inuse cloudkit. how can that be? any ideas,

cheers,

j.

Are you indexing data in DEVONthink?
If so, is that why you’re expecting there to be less sync data in iCloud?

yes, I do index some folders. but after rebuilding as per your instructions usage was normal/ as to be expected from size of local DBs.

secondly, I think I jinxed it. Bought a new iPad and step by step set up cloudkit one DB at a time. There many items again that I see as preview but cannot download. (these are not indexed files (would it matter if they were?) but present items in the DB.

thanks again