Mobile Documents Folder, iCloud Sync and Drive Management

Just now setting up DEVONthink with iCloud with the hopes of avoiding my internal Macintosh HD completely. Apparently, I am not thanks to the Mobile Documents folder.

  1. I have all of my Databases on a 5TB external drive, happy as can be
  2. I have 2TB of iCloud Storage that I am not using and so I wanted to use iCloud for DEVONthink storage and go directly from my external databases to iCloud, avoiding my Macintosh HD altogether.

Not so fast.

I have noticed that in this syncing process all of a sudden my Macintosh HD is filling up like crazy. Come to find out that it is my Library -> Mobile Documents -> DEVONthink Pro Office -> DEVONthink folder that has 115GB of data in there!

So now what?

I am in the process of moving the Mobile Documents folder to an external drive and putting a Symbolic Link in its place. Not sure if this going to work, but we will see.

Any other suggestions? If not it’s back to Dropbox, I guess?

iCloud syncing always happens locally, then iCloud does its own syncing. There is no way around this.

As far as you linking the Mobile Docs folder, it’s not officially supported nor advocated, so bear that in mind.

So as it sends it to the cloud does the data remove itself from the Mobile Documents folder?

I am just setting up my first DEVONthink databases. I have 3 databases syncing to icloud with DTTG on my iphone. 2 of the databses together total about 12 gb and the third does not have anything in it. yet.

I have a macbook pro with a 250 gb hard drive.

In the process of setting up my databases, I have been trying to clean up space on my macbook pro by moving larger files (e.g. video files) to my 2 tb external hard drive. I then was just indexing these files into my DEVONthink database so it woudn’t take up space on my macbook pro hard drive.

Somehow during this process of trying to free up space on my macbook pro, I noticed that the same mobile documents folder that the OP referenced above has reached a size of 100+ gb which has filled up my hard drive to maximum capacity. My actual DEVONthink databases are only about 12 gb.

My computer is barely working now as the hard drive is so full.

What can I do to fix this?

Help, please!

Update - After letting things sit for a couple hours, my Mobile Documents folder is now down to 30 GB.

Not sure why, but I am glad to see it freed up a bunch of space on its own.

Did you clean the location as I suggested in your support ticket?

I did not. I just let it sit for a couple hours and the mobile documents folder size went from 100 + GB down to 30 GB. Also, the free space on my hard drive on my laptop went from 0 to about 70 GB.

Does that size for the mobile documents folder make sense? I am still a bit confused by this. I have a few different pictures in my head of how this is working but I am getting results back that do not match up perfectly with these pictures.

Hopefully, I can better describe my situation and formulate my question clearly to help with my confusion.

Goals
I am trying to index 100 + GB of video so I can store the files on an external hard drive and not take up all that space on my laptop. I would like the file’s metadata to come up in a DEVONthink search if it is pertinent to my topic I am searching for. I would then expect I would need to connect my external hard drive if I want to view/download an indexed video.

I would like to be able to have the same indexed file’s metadata come up in a search on my phone that is synced via iCloud. I didn’t anticipate being able to simply download these files on demand. I figured I may be able to if my external hard drive was connected to my laptop, though, I wasn’t sure if it works this way with an external hard drive. If there is a way to download these on demand without having to store the 100 + GB of files on my laptop, great! If not, it is not a deal breaker to only be able to view the metadata.

Point of Confusion
I am getting very confused right now by the fact that I can only view/download the indexed videos on my laptop if the external hard drive is connected (this part makes sense) but I am somehow able to download one of these videos on my phone without the external hard drive connected to my computer. On my laptop, the indexed video file says “Volume not mounted” and then shows the path to the video. On my phone, when I clicked download file, it downloaded it somehow. Not all of the files that I indexed will download on my phone, though. Just a few of them have so far.

Here are the ways that I have understood what may be happening, but I am not sure which (if any) is correct.

Scenario #1
Indexed file contents are stored in full in the mobile documents folder.

From your responses to my support email I took it to be that the mobile documents folder would contain the actual file contents of the files that you have indexed so that you can download them on demand from other synced devices (if you set it up this way as opposed to unchecking the Synchronize contents of indexed items). If this is this the case, should my mobile documents folder be at least as big as the files I indexed (100 + GB)?

If this is the way that sync works with indexed files, is it possible that iCloud or my laptop realized I didn’t have enough room to store all the data in my mobile documents folder and it just stopped uploading all the file contents? I can see the indexed folders and metadata on my phone, though.

Also, why are these files available to download on my phone on demand but not on my computer?

Scenario #2
The same metadata only (not full file content) is stored in the mobile documents folder for indexed files that are stored locally in the database file.

This is how I anticipated indexing and syncing to work. I figured just references/links to the location of the files would be in the database and sync locations with the ability to download them on demand. In my particular case, since my files are on an external hard drive, I also expected they will not be available to download on my phone unless that external hard drive is connected to my laptop.

If this scenario is correct, given my indexed file size total of 100+ GB, would a 30 GB mobile documents folder size make sense? Also, why can I download only some of the videos on demand on my phone?

Again, I just can’t make sense of what is hapenning given the results I am seeing above. Any clarification would be greatly appreciated!

I also wondered where my disk spaces goes. My 20 GByte database added additional 40 GByte to the Mobile Documents folder. So I cleaned up the sync store and after setting up sync again Mobile Documents now holds only 30 GByte.

Can you explain why the sync folder is bigger than the plain data. Why is this data duplication happening and is there a way to avoid it (directly storing in iCloud Drive?)?

I’m using DT3 Beta.

What’s the size of the database according to File > Database Properties?

This is an older topic but instead of making a new one I figured I’d just ask here.

I have the same issue, my home server is a mac mini with 128gb of internal storage. I don’t need more because all my data is on external storage RAIDs

But with the Mobile Documents folder, it’s still eating up most of my internal storage just for syncing even though the actual DT database is already on my drive.

Is there no way around this? I want to leave my Users home directory on the internal drive if possible

There is no way for us to fully control this folder as it’s under iCloud’s jurisdiction. It can download data as it sees fit to make it available on the devices as part of Apple’s “your data… everywhere” mantra.

Hmm that’s what I thought.

Since DEVONthink is the only app I’m using that’s facing these problems (big sync folder, sometimes inconsistent files across devices, etc) I wonder a bit why that is? (Though i’m guessing because of heaps of files and big databases).
Is there a different way to use iCloud drive besides "put it in the folder and wait for iCloud to upload? Like programtically call their APIs?
(I have little experience with iOS/Mac development so I’m not sure what iCloud offers)

Having the database twice on drive, once in the Mobile Documents folder and once inside the actual location is a bit odd…

(I guess there is CloudKit but that looks like it’s only “the developer pays for this database”. Maybe worth considering this as a paid option at some point?)

Is there a different way to use iCloud drive besides "put it in the folder and wait for iCloud to upload? Like programtically call their APIs?

We are working within boundaries set by Apple. They don’t allow direct access to their servers like Dropbox does.

Having the database twice on drive, once in the Mobile Documents folder and once inside the actual location is a bit odd…

It’s not “the database twice”. From Help > Documentation > In & Out > Sync > Q & A