Automatically importing emails using an always-on Mac (mini)

I found many topics on this forum where people try to find a reliable way of importing all emails into DEVONthink automatically to handle them in the inbox there using its advanced categorization features. But I couldn’t find any solution that seemed to work reliably and was easy to maintain at the same time. After trying out a lot of different approaches and seeing most of them fail, I finally found a solution that works for me which I want to share.

Prerequisites

My solution has two caveats though:

  1. You need an always-on Mac running as a “server”, such as a Mac mini. Otherwise, Apple Mail rules won’t work reliably.
  2. And you need a DEVONthink seat in your license (thankfully, all licenses come with 2 seats = 2 devices – so if you’re using DT on only one Mac, your existing license should do – otherwise you have to buy one for currently $99).

If none of these hold you back (like in my case, because I already use a Mac mini as a media server + for backups), then read on to learn about how my solution works. It’s based on Apple Mail rules and a single line of code adjustment in DEVONthinks’ pre-shipped AppleScript files.

Step 1: Setting up DEVONthink on the always-on Mac

  1. Download & install DEVONthink Pro on the always-on Mac (adjust import preferences if needed)
  2. Install the additional tools you need (like OCR) and make sure to include the Apple Mail plugin & scripts
  3. Setup the Sync for the database you want to import the emails to (out of scope for this guide)

Step 2: Setting up Apple Mail

  1. Set up one or multiple email accounts you want to forward emails from to DEVONthink in Apple Mail
  2. Create a folder (via + button when hovering over account name) in one of your email accounts and name it exactly DEVONthink (that’s where we’ll move emails as a backup)
  3. Create two subfolders inside DEVONthink (via right-click on the folder > “New mailbox…”) and name them Incoming and Outgoing (we’ll also import sent emails, so let’s keep them separate)
  4. Turn on the Apple Mail plugin (in Apple Mail > Settings > General tab > Manage plugins)
  5. Change the email accounts “Description” field in Settings > Accounts you created the DEVONthink folder in to Mail Hub (or keep description in mind, you’ll need it later)

Step 3: Adjusting DEVONthinks’ AppleScript files

  1. Open Apple Mail settings, go to the Rules tab, and press “Add Rule”
  2. In the “Perform the following actions” section, choose “Run AppleScript”
  3. In the values dropdown for “Run AppleScript”, select “Open in Finder” (you can cancel the rule setup dialog in Mail for now)
  4. Duplicate the script “Mail Rule - Add messages to DEVONthink” to import the email as an .eml file (including attachments) – or if you prefer to save import the attachments as separate documents, duplicate “Mail Rule - File messages & attachments” instead
  5. Rename the duplicated script to “Send to DEVONthink + move to DT-Incoming folder”
  6. Double-click the renamed script to open it in the “Script Editor” app
  7. Add the following line right before the last end repeat towards the end of the file and save the file (via Cmd + S):
set mailbox of theMessage to mailbox "DEVONthink/Incoming" of account "Mail Hub"

(If you didn’t rename your email accounts description to Mail Hub before, adjust it here.)

The result should look like this:

  1. Now go back to the Finder window with the scripts (or repeat until step 3), this time duplicate the new script we just created
  2. Rename the duplicated script to “Send to DEVONthink + move to DT-Outgoing folder” (note the Sent at the end)
  3. Double-click the renamed script and in the line we added, change DEVONthink/Incoming to DEVONthink/Outgoing and save the file (Cmd + S)

Step 4: Setting up the Apple Mail rules

  1. Open the settings in the Mail app, navigate to the “Rules” tab, and press “Add Rule” (if it was open already, you can skip this step)
  2. Change the description to Incoming to DEVONthink
  3. Select any above the conditions section
  4. Set the first condition to To, is equal to and your main email address as the value
  5. Add another condition for each alternative email address you want to forward (if any)
  6. In the “Perform” section, set the action to Run AppleScript and select the Send to DEVONthink + move to DT-Incoming folder (note the ‘Incoming’!)

The end result should look like this:
Screen Shot 2022-09-10 at 23.46.58

  1. Press “OK” and select “Do not apply” for now so we can finish the setup without delays
  2. Repeat steps 1 to 7, but 3 changes: a) name it “Outgoing to DEVONthink”, b) select From for the condition type, and c) select the Send to DEVONthikn + move to DT-Outgoing folder (note the ‘Outgoing’!)
  3. Press “OK” and select “Do not apply” again for a more controlled migration

Permissions, Validation & Initial Migration

  1. To test things out, and also to get any Apple permission dialogs, right-click on any email you received in your main account and press “Apply Rules” at the bottom
  2. On first usage, this should ask you for permissions to access DEVONthink – give all permissions asked for
  3. The rule might not have worked on first try (due to lacking permissions), so revert any changes (if anything happened) and repeat step 10
  4. Validate that the email was moved to the DEVONthink/Incoming folder and that it was sent to DEVONthink, too (if not, something went wrong, check the rule & script)
  5. Now right-click an outgoing email you’ve sent yourself and choose “Apply Rules”
  6. Validate that the email was moved to the DEVONthink/Outgoing folder and that it was sent to DEVONthink, too (if not, something went wrong, check the rule & script)
  7. Now, to test if this works with new emails, ask someone to send you a test email (or use an email address you haven’t configured) – this email should land in DEVONthink/Incoming and DEOVNthink app automatically (wait a few seconds until Apple Mail gets it)
  8. Next, test if this works with sent emails, by sending a test email from another device to a friend – this one should land in DEVONthink/Outgoing and DEVONthink app automatically
  9. When these things worked as expected, you can now select all mails in all your folders (Cmd + A), right-click and press “Apply Rules” to apply the rules to all your emails, or start with just a few to further validate things work as expected

Now just keep your Apple Mail app and DEVONthink open on this Mac and things should work.

Please note that I’m taking no responsibility if anything breaks in the process and you lose any data. I’ve tried to come up with a safe approach, keeping the emails on your Mail server as a backup. Also, only one line adjustment in the scripts to not mess around.

While I tested this setup extensively, I did not use it long term yet, so I will report if I run into any issues, but I wanted to share before I forget the details. If you run into any issues or have suggestions for improving the workflow, feel free to share so we can learn all together.

2 Likes

My desktop computer is a Mac Mini
It is always on, however not necessarily a requirement

My mobile device is an iPad, where I usually review email
I prefer a manual selection approach during the review;
trashing email or moving to the Devonthink folder for archiving

Devonthink folder …(that’s where we’ll move emails as a backup)

Devonthink is my tool for archiving/backup
I don’t store email in the mail app
I use the Devonthink folder as a trigger within the Send to DEVONthink script

That’s also a way to use it and it appears that your expected use cases isn’t the same as mine. Maybe I wasn’t clear enough, so let me clarify: My approach is supposed to solve the case for people who don’t want to look into both the inbox in DEVONthink for their scanned mail and the inbox in their Email client for electronic mail, but want to have a single place to review new stuff, one inbox. You seem to be fine with reviewing emails in the email client, so this solution isn’t for you.

1 Like

Thanks for the thorough write up. And indeed, let us know if you see any issues crop up! :slight_smile:

1 Like

Experience report after 5 days of usage

The approach works very well overall!

All emails get forwarded as expected, except for one type of e-mails that the above rules don’t cover: Emails you receive when mentioned in the BCC field, because then you’re not the “receiver” of the email, but some other email address is. To cover this case as well, I adjusted my settings from “Step 4” by changing from “To is equal to your@email.address” to “From is not equal to your@email.address” and selecting “If ALL of the following conditions are met”.

Another issue I’ve had was that sometimes the Mail app would hang during applying of the rules. But this is likely a bug in the macOS 13 “Ventura” beta I have currently installed on my Mac mini. And in any case, I found a workaround by running a small shell script that will restart the Mail app every 5 minutes which I wrote. While I’m sure there are many other ways to do this, here’s the script in case anyone wants to reuse it:

#!/bin/bash
set -euo pipefail

echo $(date) –– 'Attempting to quit Mail normally ...'
timeout 2 osascript -e 'quit app "Mail"' || true

echo $(date) –– 'Force-quitting Mail (if not quit already because it hangs) ...'
killall "Mail" || true
sleep 2

echo $(date) –– 'Starting Mail again ...'
open -j -a "Mail"  # the `-j` option ensures it's started as hidden, remove to foreground-start it
sleep 2

Just copy the contents to a file named restart_mail.sh and call chmod +x path/to/restart_mail.sh in the Terminal app to make it an executable. Then setup a daemon (or something else) that runs it every 5 minutes (I used the paid LaunchControl app to do that).

Let’s see how things will evolve long-term. I’ll try to keep you updated.

2 Likes

Confirmed; initial email review in the mail client instead of DT
I’m interested in the opinion of other DT users on this

I’m using the same components for my implementation,
but lack the automatic mail import-to-dt rule
As such, I had to set up automatic scheduling for the import script

1 Like