Importing very large mbox file?

Hello all.

I’m very new to Devonthink, but recently purchased Devonthink Pro 3. One of the things I’d hoped to use it for is to index all my old data, including email. I have an mbox file created by Google Takeout which is 49GB. Upon importing into Devonthink, the interface freezes and I get the “rainbow wheel”. I’ve left it for around 4 hours, but eventually had to force quit. I’ve tried it twice, but same behavior.

Is this amount of data too much to expect of DT3? This is on a Apple Macbook Pro 16", 2.4 GHz 8-Core Intel Core i9, 64 GB 2667 MHz DDR4.

I appreciate any help you can provide.

Depending on the number of emails and the size/kind of the attachments (which are indexed too) this might require a lot of time and memory. Do you remember how much memory DEVONthink used before force quitting? See e.g. Memory tab of Activity Monitor.app. Or do you know how many and what kind of emails the mbox contains?

I’m not certain on the amount of memory. The types of emails include just over 10 years of various business emails from my account. Standard sort of stuff.

I’m working on a script to break these up by sender, so one sender per file. A few questions:

  1. Assuming I’m able to import in smaller batches, can I assume that Devonthink can handle this much data once imported?
  2. Is there a way to script the import of mbox files from the command line? I’m a programmer of over 20 years, so willing and able if the capabilities exist.

Thanks,
d1rewolf

This is hard to tell without knowing the number of emails and the kind/size of the attachments.

This isn’t scriptable.

Are there known limits to the amount of emails DT can handle? Are there known limits to the kind and size of attachments?

Per database not more than 300000 items and 300 million indexed words are usually the recommended limits. How many of such databases can be opened concurrently depends on the amount of RAM, without sufficient RAM the performance might be a lot slower.