Replacing the mail system¶

One afternoon I decided to replace my local mail system. No real reason, just time to try some new things out.

The current system is postfix, procmail, fetchmail and SpamAssassin. I only really push mail through either the SMTP server at UKFSN and work from home, I only really use procmail to deal with the SpamAssassin stuff, and fetchmail is just there because it makes life a whole lot easier to work with external tools than just trying to use sylpheed.

Note

UKFSN now provides SpamAssassin as a free service to its users, so the requirement for that is gone. I can just set up the scoring rates and skip the whole SpamAssassin thing at home.

So first things first I’ve installed GNU mailutils, because I know it is going be useful purely because of the MH utilities(MH is the only mail format worth using ;). I’ve also decided to use it as a replacement for local and remote delivery of mails(the mail.local and mail.remote commands). Because of the cool way you can just specify remote mailboxes to the MH utilities I also decided quickly it was time to drop gbiff2, it is much easier(and prettier) just to use something like:

#! /bin/bash
while true
do
    new_messages=$( messages -q pop://<login>:<password>@<server> )
    echo "${new\_messages} new mail(s)" \
        | osd_cat \
        -f '-monotype-arial black-medium-r-normal-*-*-240-*-*-p-*-iso8859-1' \
        -c red -p bottom -A center -s 5
    sleep 10m
done

Note

I already had this feeling the change over would include a lot of shell scripts as replacements for commands. Or as I like to call it the TLW, and there is a prize in it for whoever figures out what TLW is first.

Note

It would actually make a lot more sense to start this in Ruby, especially now I have the XOSD bindings working. But that isn’t the way it works for me, it is just so much easier to make a bunch of shell scripts as issues arise.

Next up I decided to remove fetchmail, I either might as well just use MH utilities or write another shell script(probably using bash‘s net redirection).

After toying with the bash script idea for some time, then remembering that I have access to a few systems with net redirections disabled(standard Debian package) I decided to not go the cool script for cool scripts sake. So it looks a little something like this:

#! /bin/bash
new_messages=$( inc --truncate --file pop://<login>:<password>@<server> | wc -l )
echo "${new\_messages} new mail(s) received" \
    | osd_cat \
    -f '-monotype-arial black-medium-r-normal-*-*-240-*-*-p-*-iso8859-1' \
    -c red -p bottom -A center -s 5

Note

Obviously there is absolutely no error checking happening with either of these scripts, but it isn’t difficult to add. I’ve just decided to show a cut down version in here to get the point across. If you want the full versions drop me a mail.

Uhmm, so what about SpamAssassin? Yeah, this was the difficult one. If only there was a --in-place option, because nowadays I tend to just “mark as read” spam lately I’ve seen too many false positives to push it all in to another folder.

For now I’ve decided to use a heavyweight sylpheed filtering file, if spam starts to get through it I will probably write my own inc command using the mailutils libraries. The easy way out ;)

Anything left?

Postfix
- Replaced with the mailutils mail.local and mail.remote
procmail
- No longer needed, but guimb and sieve are available from mailutils to replace it.
fetchmail
- Replaced with the mailutils inc
SpamAssassin
- Currently not used, I’m hoping the the static filters are enough for now. Again this should be easy to replace to some degree with the filtering available from mailutils.

Changes¶

Since I wrote the original body of this text I’ve changed a lot of things. Firstly the whole system is written in C, mainly out of the need for increased functionality and speed. It now supports dealing with my IMAP4 account at work with no trouble at all. It includes a nice GTK GUI for configuring the tools, and actually sits nicely as a plug-in to sylpheed-claws.

These changes have allowed me to address my concerns about mail filtering too. I now link to PCRE, which provides the capabilities to do all the content based sorting I used to do with procmail. I’ve built with support, although I don’t use it, for the SpamAssassin library. As a side effect of some of the functions available in the mailutils libraries support a simple built-in vacation (1) type of functionality.

The simple truth here is the whole thing is less than a few thousand lines clean private code built on top of the mailutils infrastructure, and once compiled it is only 600 kB(or 1.3 MB if you include the GTK interface and sylpheed plug-in). Yeah I know that sounds like a lot, and I suppose in some ways it is but it is incredibly fast. From server connect until the mail is filtered there are zero external processes started and even heavy RegExp processing and MH folder sorting is three times as fast on a 100 mail in 1 MB import compared to the old procmail/fetchmail system.

Am I going to continue using this at home? Yes, it has actually worked out fantastically especially for a bored afternoon’s original work. Not only does the system work faster and more efficiently than the old method, but it also looks really cool with the XOSD based status reports. In fact I find it quite ugly to watch the --no-xsod console output now.

Changes in March¶

Since the last update, I’ve almost entirely rewritten the client from scratch to support some previously missing features like HTML rendering(but still no authoring).

Since the 24th of March gmh(the final MUA name) has replaced sylpheed-claws on all the systems at work and I’m yet to receive any negative feedback. If this continues until the middle of April then it will be replacing sylpheed too.

The only real missing feature for the MUA now is documentation, but because it borrows heavily from the standard interfacing techniques in almost every GUI MUA it doesn’t really need it (but it is slowly getting written).

Note

I suppose in a funny way the most unexpected thing has happened, I’ve started using sylpheed-claws at home again. I’m sure it is a boredom thing, so I expect it to go away anytime soon.

The most significant change to happen to the MUA since its inception is support for a mini-command bar at the bottom of the interface, which allows you to manually enter MH commands and operate on the mailboxes. A lot of people seem to like the idea, but some people have complained about it not supporting a full terminal mode so I guess it has its uses(and there is possible room for improvement).

As for the server situation at home, it is just a cleaned up version of the system described originally. There has been a lot of work done on error checking and code clean up. I’ve started building the system against a live CVS checkout of GNU Mailutils, in the hope that I can eventually start to push changes upstream.

Changes in April: The final chapter¶

The whole suite is now available for checkout via deltaforce.

sylpheed{,-claws{,_gtk2},-gtk2} have all been replaced right across AST, and we will no longer be shipping them in updates or new products.

The calendar module is now fully implemented and supports a very cool dep system which allows for automated generation of “distributed appointment ordering” notices(see the paper in the source tree).

This is the final update here, as development has now moved to the AST site.

JNRowe

Navigation