Some days ago I added an indexed local folder to an existing database. This indexed folder is on my mac (no google drive or icloud or similar).
Soon after I started getting some verification problems.
Today opening this database makes DVT “hang”. Opening DVT without his 1 database –> everything fine. Opening only this one database –> verification and apparantly around 7xx files “missing” and DVT “hangs” and is marked as such.
The indexed folder is the IMAP folder of MailMate: lots of eml files (> 100000). Is that the reason?
Anyway, since I think this IMAP indexed folder is the reason: I would like to be able to remove this folder from the database? Any suggestions?
The only other thing that changed today is: I installed the ScanSnap ix500 scanner and I changed the chat AI model to claude (was Ollama).
Well, it seems not be hanging anymore. I opened all databases except the “faulty” one and let them synchronise for a while until I saw no more traffic. Then I opened the faulty one, let if verify and confirmed “repair” when asked. It showed 7xx missing files but the database opened en DVT is stable at this point.
Any suggestions about indexing the IMAP folder of mailmate?
Don’t. But that’s probably not what you want to hear. Indexing a file over whose content you don’t have control ist never a good idea. Even less so b with 100000 items and probably growing.
Mailmate is an IMAP mail client. So, it should not have an “IMAP folder”. Perhaps it downloads your IMAP server folder(s) onto your Mac. Which, in my mind, goes against the idea of an e-mail server. It serves, so the data should mostly remain on the server, except for some caching.
Now let’s assume your “IMAP folder” is a mirror image of what is stored on your IMAP server(s). What happens if you delete an e-mail on your IMAP server, perhaps using another client? Will Mailmate then also remove the file from its folder? What happens if you move a file in your indexed database into another group (say from “Spam” to “Holidays” or whatever). Will Mailmate still be able to find that file in its database?
One of the reasons I use Mailmate is precisely because it lets met work offline with IMAP based servers and it keeps up with changes elsewhere. Together with DVT it’s one of the program’s I have been using since very long. It has smartfolders and a lot of programmability and it has a very good search function. (I recently added Obsidian but I think that may be another rabbithole for an ADD brain).
Anyway, as (I believe) for so many of us, email has also become a database and archive for past projects and I had the idea of trying to bring DVT and mailmate archive together. It would help double tagging, using AI, reading newsletters, … There is a plugin but it isn’t very efficiënt so I tried to circumvent it …
Any suggestions of email and DVT are definitely welcome
Sorry, that’s not my area of expertise. If I were confronted with 100k e-mails, I’d start weeding them out because I’d think that there are many that I will never need again.
Some personal background: When I was still working, it was as an editor of a monthly IT magazine. So, there were articles in different states of editing, e-mails, PDFs, what not. At the beginning, I kept everything. Only to notice that I never looked at older articles (aka “projects”) again. Then I started keeping only the last 10 or so articles and related documents. And even with those, I rarely looked back.
You’re situation may be totally different – if you’re an architect, you might need to keep everything for 30 years. But e-mails… well, in my experience, 80% of them are just white noise some days after they were received.
Do you mean “ADHD brain”? If not, what does ADD stand for?
ADD is similar to ADHD but without the physical hyper-activity. Typical people with procrastination, tool-focus, dopamine-searching, lot’s of projects at the same time, lot’s of tabs open, many documents reading at the same time, information hoarding, … :).
Instead of weeding out +100K (it’s actually around 700K emails) I was trying to get to some form of “see also” so that I do not need the manual labour of “weeding out” the emails.
But thanks again, and suggestions are always welcome (and I don’t care if the answer does not match my wish of idea - on the contrary).
Then there’s another one: Use Mailmate’s search capabilities instead of indexing everything into DT.
There was, btw, a similar thread here some time ago, if I remember correctly. Ah, there it is: How to index appr. 120.000 emails in an iCloud database?
Perhaps that gives you some ideas.