Email Setup with isync, notmuch, afew, msmtp and Emacs
I was asked recently about how I have my email client setup. As I naturally do, I replied with something along the lines of the following.
I use isync, notmuch, afew and msmtp with emacs as an interface, let me get you a link on how I did my setup from my blog.
To my surprise, I never wrote about the topic. I guess this is as better time as any to do so.
Let’s dig in.
Bird’s-eye View
Looking at the big list of tools mentioned in the title, I could understand how one could get intimidated but I assure you these are very basic, yet very powerful, tools.
First task is to divide and conquer, as usual. We start by the first piece of the puzzle, understand email.
In a very simplified way of thinking of email is that each email is simply a file. This file has all the information needed as to who sent it to whom, from which server, etc… The bottom line is that it’s simply a file in a folder somewhere on a server. Even though this might not be the case on the server, in this setup it will most certainly be the case locally on your filesystem. Thinking about it in terms of files in directories also makes sense because it will most likely be synchronized back with the server that way as well.
Now you might ask, what tool would offer us such a way to synchronize emails and my answer would be… Very many, of course… come on this is Linux and Open Source ! Don’t ask silly questions… But to what’s relevant to my setup it’s isync.
Now that I have the emails locally on my filesystem, I need a way to interact with them. Some prefer to work with directories, I prefer to work with tags instead. That’s where notmuch comes in. You can think of it as an email tagging and querying system. To make my life simpler, I utilize afew to handle a few basic email tasks to save me from writing a lot of notmuch rules.
I already make use of emacs extensively in my day to day life and having a notmuch interface in emacs is great. I can use emacs to view, tag, search and send email.
Oh wait, right… I wouldn’t be able to send email without msmtp.
isync
isync is defined as
a command line application which synchronizes mailboxes.
While isync currently supports Maildir and IMAP4 mailboxes, it has the very logical command of mbsync. Of course !
Now, isync is very well documented in the man pages.
man mbsync
Everything you need is there, have fun reading.
While you read the man pages to figure out what you want, I already did that and here’s what I want in my ~/.mbsyncrc.
##########################
# Personal Configuration #
##########################
# Name Account
IMAPAccount Personal
Host email.hostname.com
User [email protected]
Pass "yourPassword"
# One can use a command which returns the password
# Such as a password manager or a bash script
#PassCmd sh script/path
SSLType IMAPS
CertificateFile /etc/ssl/certs/ca-certificates.crt
IMAPStore personal-remote
Account Personal
MaildirStore personal-local
Subfolders Verbatim
Path ~/.mail/
Inbox ~/.mail/Inbox
Channel sync-personal-inbox
Master :personal-remote:"Inbox"
Slave :personal-local:Inbox
Create Slave
SyncState *
CopyArrivalDate yes
Channel sync-personal-archive
Master :personal-remote:"Archive"
Slave :personal-local:Archive
Create Slave
SyncState *
CopyArrivalDate yes
Channel sync-personal-sent
Master :personal-remote:"Sent"
Slave :personal-local:Sent
Create Slave
SyncState *
CopyArrivalDate yes
Channel sync-personal-trash
Master :personal-remote:"Junk"
Slave :personal-local:Trash
Create Slave
SyncState *
CopyArrivalDate yes
# Get all the channels together into a group.
Group Personal
Channel sync-personal-inbox
Channel sync-personal-archive
Channel sync-personal-sent
Channel sync-personal-trash
The following will synchronize both ways the following folders:
- Remote “Inbox” with local “Inbox”
- Remote “Archive” with local “Archive”
- Remote “Sent” with local “Sent”
- Remote “Junk” with local “Trash”
Those are the only directories I care about.
With the configuration in place, we can try to sync the emails.
mbsync -C -a -V
notmuch
You can read more about notmuch on their webpage. Their explanation is interesting to say the least.
What notmuch does, is create a database where it saves all the tags and relevant information for all the emails. This makes it extremely fast to query and do different operations on large numbers of emails.
I use notmuch mostly indirectly through emacs, so my configuration is very simple. All I want from notmuch is to tag all new emails with the new tag.
# .notmuch-config - Configuration file for the notmuch mail system
#
# For more information about notmuch, see https://notmuchmail.org
# Database configuration
#
# The only value supported here is 'path' which should be the top-level
# directory where your mail currently exists and to where mail will be
# delivered in the future. Files should be individual email messages.
# Notmuch will store its database within a sub-directory of the path
# configured here named ".notmuch".
#
[database]
path=/home/user/.mail/
# User configuration
#
# Here is where you can let notmuch know how you would like to be
# addressed. Valid settings are
#
# name Your full name.
# primary_email Your primary email address.
# other_email A list (separated by ';') of other email addresses
# at which you receive email.
#
# Notmuch will use the various email addresses configured here when
# formatting replies. It will avoid including your own addresses in the
# recipient list of replies, and will set the From address based on the
# address to which the original email was addressed.
#
[user]
name=My Name
[email protected]
# [email protected];[email protected];
# Configuration for "notmuch new"
#
# The following options are supported here:
#
# tags A list (separated by ';') of the tags that will be
# added to all messages incorporated by "notmuch new".
#
# ignore A list (separated by ';') of file and directory names
# that will not be searched for messages by "notmuch new".
#
# NOTE: *Every* file/directory that goes by one of those
# names will be ignored, independent of its depth/location
# in the mail store.
#
[new]
tags=new;
#tags=unread;inbox;
ignore=
# Search configuration
#
# The following option is supported here:
#
# exclude_tags
# A ;-separated list of tags that will be excluded from
# search results by default. Using an excluded tag in a
# query will override that exclusion.
#
[search]
exclude_tags=deleted;spam;
# Maildir compatibility configuration
#
# The following option is supported here:
#
# synchronize_flags Valid values are true and false.
#
# If true, then the following maildir flags (in message filenames)
# will be synchronized with the corresponding notmuch tags:
#
# Flag Tag
# ---- -------
# D draft
# F flagged
# P passed
# R replied
# S unread (added when 'S' flag is not present)
#
# The "notmuch new" command will notice flag changes in filenames
# and update tags, while the "notmuch tag" and "notmuch restore"
# commands will notice tag changes and update flags in filenames
#
[maildir]
synchronize_flags=true
Now that notmuch is configured the way I want it to, I use it as follows.
notmuch new
Yup, that simple.
This will tag all new emails with the new tag.
afew
Once all the new emails have been properly tagged with the new tag by notmuch, afew comes in.
afew is defined as an initial tagging script for notmuch. The reason of using it will become evident very soon but let me quote some of what their Github page says.
It can do basic thing such as adding tags based on email headers or maildir folders, handling killed threads and spam.
In move mode, afew will move mails between maildir folders according to configurable rules that can contain arbitrary notmuch queries to match against any searchable attributes.
This is where the bulk of the configuration is, in all honesty. At this stage, I had to make a decision of how would I like to manage my emails ?
I think it should be simple if I save them as folders on the server as it doesn’t support tags. I can derive the basic tags from the folders and keep a backup of my database for all the rest of the tags.
My configuration looks similar to the following.
# ~/.config/afew/config
[global]
[SpamFilter]
[KillThreadsFilter]
[ListMailsFilter]
[SentMailsFilter]
[ArchiveSentMailsFilter]
sent_tag = sent
[DMARCReportInspectionFilter]
[Filter.0]
message = Tagging Personal Emails
query = 'folder:.mail/'
tags = +personal
[FolderNameFilter.0]
folder_explicit_list = .mail/Inbox .mail/Archive .mail/Drafts .mail/Sent .mail/Trash
folder_transforms = .mail/Inbox:personal .mail/Archive:personal .mail/Drafts:personal .mail/Sent:personal .mail/Trash:personal
folder_lowercases = true
[FolderNameFilter.1]
folder_explicit_list = .mail/Archive
folder_transforms = .mail/Archive:archive
folder_lowercases = true
[FolderNameFilter.2]
folder_explicit_list = .mail/Sent
folder_transforms = .mail/Sent:sent
folder_lowercases = true
[FolderNameFilter.3]
folder_explicit_list = .mail/Trash
folder_transforms = .mail/Trash:deleted
folder_lowercases = true
[Filter.1]
message = Untagged 'inbox' from 'archive'
query = 'tag:archive AND tag:inbox'
tags = -inbox
[MailMover]
folders = .mail/Inbox
rename = True
max_age = 7
.mail/Inbox = 'tag:deleted':.mail/Trash 'tag:archive':.mail/Archive
# what's still new goes into the inbox
[InboxFilter]
Basically, I make sure that all the emails, in their folders, are tagged properly. I make sure the emails which need to be moved are moved to their designated folders. The rest is simply the inbox.
Note
The read / unread tag is automatically handled between notmuch and isync. It’s seemlessly synchronized between the tools.
With the configuration in place, I run afew.
afew -v -t --new
For moving the emails, I use afew as well but I apply it on all emails and not just the ones tagged with new.
afew -v -m --all
msmtp
msmtp is an SMTP client. It sends email.
The configuration is very simple.
# Set default values for all following accounts.
defaults
auth on
tls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile ~/.msmtp.log
# Mail
account personal
host email.hostname.com
port 587
from [email protected]
user [email protected]
password yourPassword
# One can use a command which returns the password
# Such as a password manager or a bash script
# passwordeval sh script/path
# Set a default account
account default : personal
Emacs
I use Doom as a configuration framework for Emacs. notmuch comes as a modules which I enabled, but you might want to check the notmuch’s Emacs Documentation page for help with installation and configuration.
I wanted to configure the notmuch interface a bit to show me what I’m usually interested in.
(setq +notmuch-sync-backend 'mbsync)
(setq notmuch-saved-searches '((:name "Unread"
:query "tag:inbox and tag:unread"
:count-query "tag:inbox and tag:unread"
:sort-order newest-first)
(:name "Inbox"
:query "tag:inbox"
:count-query "tag:inbox"
:sort-order newest-first)
(:name "Archive"
:query "tag:archive"
:count-query "tag:archive"
:sort-order newest-first)
(:name "Sent"
:query "tag:sent or tag:replied"
:count-query "tag:sent or tag:replied"
:sort-order newest-first)
(:name "Trash"
:query "tag:deleted"
:count-query "tag:deleted"
:sort-order newest-first))
)
Now, all I have to do is simply open the notmuch interface in Emacs.
Conclusion
To put everything together, I wrote a bash script with the commands provided above in series. This script can be called by a cron or even manually to synchronize emails.
From the Emacs interface I can do pretty much everything I need to do.
Future improvements I have to think about is the best way to do email notifications. There are a lot of different ways I can approach this. I can use notmuch to query for what I want. I could maybe even try querying the information out of the Xapian database. But that’s food for thought.
I want email to be simple and this makes it simple for me. How are you making email simple for you ?