You slightly misunderstood the emphasis in my question. There are lots of reasons to store important emails locally. I’m just questioning why you want to store your entire email archive locally (or anywhere else).
If it were me, my first step would be a large scale purge. Whatever import method you choose will be a lot easier for 500 emails than for 50,000.
My Workflow: Managing 800,000+ Emails in DEVONthink via MailMate & Hazel
I currently manage around 800,000 emails inside DEVONthink (DT). However, my strategy differs significantly from the official documentation. Rather than relying on DT’s built-in mailbox archiving, I use an automated pipeline that feeds raw .eml files directly into the DT Inbox.
This setup has proven to be incredibly reliable, with messages appearing in DT within seconds of hitting my mail server. Here is how I put it together.
The Automation Pipeline
Ingestion (MailMate): I use MailMate to connect to my various email accounts. MailMate is ideal for this because it reliably downloads every single message as an individual .eml file in its local filesystem cache.
Monitoring & Import (Hazel): I have Hazel monitor MailMate’s cache folders. When a new .eml file appears, Hazel copies it into the DEVONthink Inbox. I use Hazel’s Finder tags to track what has already been processed and prevent duplicate imports.
Organization (DEVONthink Smart Rules): Once the .eml files land in DT, a few Smart Rules take over. They classify the emails, rename them based on the message subject, and move them to the appropriate database. Because they are imported as raw files, DT perfectly extracts and indexes all the native email metadata.
Database Management & Archiving
To keep performance snappy, I split my archive into two tiers:
Active Database: Contains only the most recent email (roughly the last 1–2 years). This database is lightweight and always open.
Archive Database: Holds the bulk of the 800,000 emails. It sits closed most of the time and is only loaded on demand when I need to run a deep search.
Key Caveats & Edge Cases
If you are looking to replicate this, keep the following in with mind:
App Dependencies: This requires MailMate to be running constantly for real-time indexing (though if it’s closed, Hazel will immediately catch up upon launch).
Email Client Limitations: This workflow hinges on MailMate’s transparent filesystem cache. Other clients (Apple Mail, Outlook, MailMaven, etc.) handle caching in proprietary ways that I couldn’t reliably automate. A lightweight IMAP connector might replace MailMate, but it likely wouldn’t play nice with my M365 accounts.
Resource Usage: It requires a decent amount of RAM and CPU, but on modern hardware, it’s a non-issue.
Deletions & Moves: Messages moved within the mail client (e.g., from Inbox to Archive) do not cause duplicates because of Hazel’s tracking tags. Messages deleted from the mail client stay in DT permanently.
Spam Handling: I don’t import my spam folder. However, if a false positive is moved from Spam back to the Inbox, Hazel catches it and pushes it to DT normally.
There are likely minor edge cases where this isn’t 100% airtight, but I keep a separate raw email archive “on ice” just in case.
Curious to hear if anyone else is taking a similar filesystem-first approach to email archiving!
Disclosure: AI used to tighten this post up and make more readable, but the content is 100% human created.
If I understand you correctly @kewms , you’re question is why capture absolutely everything?
In my own case, I’m working on the assumption that I’ve already deleted anything that I might not want to refer to again.
I operate as close to an Inbox-zero scenario as I can. So if I’ve read an email and won’t need to refer to it again then I’ll delete it as soon as it lands it. I periodically do a bit of a cleanup too where I delete things like dispatch notifications for postal/courier deliveries that have arrived.
I’d expect to retain some stuff for my lifetime, in the spirit of a personal archive.
I use Apple Mail (Mac and iPad)
with an applescript to import emails as .eml files
Only selected emails (those I move to a mail folder labeled Devonthink)
I just keep my email archives on an IMAP server running on my Synology box. With Maildir+ format the system can easily handle a single folder of 30,000+ messages, and Apple Mail can index and search it. The on-disk format for Maildir is basically just the raw text, like .eml files, so it’s easy to turn it into something else if I ever feel the need. I guess it’d be nice if DEVONthink was an IMAP client though…
MailMaven 1.2.0 added AppleScript support, so I put together a quick solution to handle my email archiving. The script enumerates my accounts and mailboxes, exports native .eml files straight into my DEVONthink Inbox with specific Finder tags, and lets DEVONthink’s smart rules handle the rest of the enrichment and filing.
Instead of relying on Smart Mailboxes, which gave me some trouble, the script uses MailMaven’s native keywords and a simple look-back window (measured in days) to capture new messages. To keep spam out of my archives, I added a 5-minute delay rule so my regular spam filters have time to clear out the junk first.
It runs seamlessly on demand or scheduled via Keyboard Maestro or cron. Relying on AppleScript feels much more robust than directly monitoring the local filesystem where an email client stores its raw downloads. Next, I’m experimenting with exporting messages directly to Markdown with a local LLM-generated summary at the top to boost future searchability.