Noob question…
How long should it take to archive 50 thousand email messages?
- Macbook air M3 24 GB Memory 256 GB “disk”
- Devonthink 4 last version
Guesstimates?
It’s impossible to say. It’s akin to saying, “How long will it take to get to California?” Are you in the US? New York or Nevada? Are you flying, driving, biking, or walking? You know the weather and traffic, etc. all come into play, etc., etc. There are many variables involved.
If these are stored in multiple submailboxes, process them individually. Also, they should ideally be local to the machine. See the Troubleshooting > Email Import Issues section of the built-in Help and manual.
1 Like
OK.
Thanks
You’re welcome 
@gdigesu , What was the answer?
Hi -
I tried it with ~ 5 thousand messages.
Performance on my Mac - M4 Mac Mini 16 GB memory 256 GB disk - was not satisfactory.
I abandoned the idea of regularly doing it.
Did you try doing smaller chunks? Smaller chunks consistent with prior advice from DEVONthink Support on this forum. 5,000 “regularly” seems a bit excessive. Dozens a week or so maybe a better target?
1 Like
Are you on macOS Tahoe?
Quite right.
My expectation was the tool would “quickly” scan the “mail database”, identify the messages that required backup, and do only those, like Arq, or iDrive do when doing “general” backup.
Maybe it is just me that is doing it wrong.
26.5
How would it do that? How do you determine which messages “require backup” if you throw 50000 of them at a program?
2 Likes
It is not rocket science, sir.
Incremental backup …
Incremental backup implies that there’s an initial backup and that the increment would simply consist of the changes after the last backup.
All of that is very different from “archiving 50000 emails”.
And DT is not a backup app. It will check every email you throw at it and compare it with those it already has. If it has any. In the best case, that’s an O(n) process.
3 Likes
I ask because I was considering doing something like this myself. I frequently use DEVONthink for storing emails related to active projects. But I have been considering building an archive of historical emails so that I can just take them down after their cloud servers. This wouldn’t be an incremental process. I’ll be looking to process a couple of mailboxes but with many thousands of emails.
I could use something other than Devonthink for this (for instance a local–only mail client) but it seems like DT will be able to do this, but maybe it’s too ambitious?
I have read the tutorial, the manual and searched the forum but I am still a bit confused about what a good approach here looks like? They are two different tasks, I guess: (1) importing a big bundle of historical emails and (2) incrementally adding the most recent emails. If I had started step 2 back when I started sending email (1997?), I’d never need to do step 1, but I suspect a lot of people come to this point with a big back-catalogue.
And I know, DT is not just a dumping ground for everything, but out of curiosity, does anybody keep all their emails in a Database?
I guess my question would be why do you want to import all of your email? And what structure already exists?
For instance, I have annual folders of financial emails that I keep for tax reasons. I also have completely trivial email threads – probably the majority by volume – that I don’t need to keep at all.
1 Like
FWIW I import my email archive in Mail to an “Email Archive” database periodically and then move it from my Mac onto a NAS every 12 to 18 months or so and start a fresh DT database for the next 12 to 18 months. It works well and I’ve never had a problem, but there would only be 1,500 or so emails, not 50,000. There are a lot of attachments, though. I also don’t archive every email I receive in Mail, which also helps cut down on the number of emails.
@stephenjw What process do you use to bring over those emails: importing a Unix Mailbox or The Import Sidebar?
It’s a fair question. Two reasons:
(1) I’d like to have it all in the one place. That is all the mail across all my different addresses stored in one place, which is easily searchable. Ideally I’d be able to put some general (as in coarse, not necessarily perfect) level of organisation on it. For instance, to separate out (using tags or smart folders) the trivial stuff (newsletters, etc) from the meaningful correspondence.
(2) I’d like to take the emails of the third-party servers. Lots of reasons for this, all of them personal and they wouldn’t necessarily be a concern for everyone. In my case, I pay for the storage on some of these email accounts. It’s not a fortune, but it seems unnecessary. Also, I think there’s a compelling environmental case to be made to not unnecessarily store stuff on the cloud. I appreciate that this is contrary to the current dominant doctrine. Ultimately, I think I just like the idea of storing stuff locally.
When I read online about solutions to this, I am directed to MailSteward or EagleFiler. I have played around with MailSteward and it’s really fast, but the UI takes a lot of getting used to. Some people online have described using EagleFiler in a manner that resembles Devonthink, which brings me back to here, because I already use Devonthink as my main workbench.
The import sidebar. It remembers what’s already been imported and adds the emails that have been added. It can take a while if there’s a lot of emails.
That is not the sidebar but DT itself: It re-purposes the e-mail’s UUID as its own UUID and doesn’t re-import an e-mail by checking that.