TFM:Fun with Procmail

From ProgSoc Wiki

Jump to: navigation, search

Contents

Fun with Procmail

Anand Kumria, Telford Tendys

Introduction

Procmail is a program designed to make your email life easier. It works like a robot secretary who opens your new mail, reads it and then decides which is important, what can be thrown away and where to file the messages that you are too busy to read right now but probably will want to look at later.

Things to read

This chapter provides you with enough to keep your progsoc mail in order but does not attempt to replace the real procmail documentation which you can read by using the following:

man 1 procmail
man 1 formail
man 1 egrep
man 5 procmailrc
man 5 procmailex
man 5 procmailsc

And point your favorite web browser to:

Disclaimer

The examples here may not work on every system with every mail reader, the presumption is that you are using one of the ProgSoc user machines, you may need to tweak paths and other details.

Setting things up

Follow these steps but try to think about what you are doing as well.

The .forward file

Edit the file $HOME/.forward and put just this single line into it (quotes and all):

"|exec /usr/bin/procmail"

On some \UNIX\ systems (e.g. *.it.uts.edu.au) you will need to use:

/var/spool/forward/~username/.forward

(you'll have to make the directory in your name under forward). Putting the above line into your $HOME/.forward file will redirect all your inbound email to procmail and let it take care of things.

The .procmailrc file

Edit the file $HOME/.procmailrc and put these lines into it:

# Tiny procmailrc file for testing and getting started
LOGFILE=$HOME/log/procmail
:0 c
oldmail

Note that  :0 is a colon followed by a zero, not a letter O. All procmailrc files contain these little surprised smilies, they mark the beginning of a rule. Each rule in procmail is part of a big "OR" expression so the first rule that turns out to be true is the rule that is entrusted with delivery (not counting copying flags, non-delivering recipes and other complicated stuff that you can worry about later).

In order to make procmailrc files sound friendly and to distinguish them from other files full of rules, each rule is refered to as a "recipe". When you write a stunningly useful computer program, then you too will be able to invent peculiar and slightly offbeat names for configuration components. Under the load of sufficiently large numbers of people exerting such creative forces, the world of computing becomes rather interesting.

Extra directories and permissions

These shell commands will create some extra directories and put the correct permissions on everything:

mkdir $HOME/oldmail
mkdir $HOME/log
chmod og-rwx $HOME/oldmail $HOME/log
chmod a+x $HOME
chmod a+r $HOME/.forward
chmod og-rw $HOME/.procmailrc
chmod og-w $HOME $HOME/.forward

Just testing

Now try sending yourself a small mail message and you should see a copy of it turn up in the oldmail directory and you should see the $HOME/log/procmail file now contains an entry. Note that you get a copy or your messages in oldmail and you should still see them in your regular mailbox file like normal. This makes a handy backup in case you delete the wrong thing.

Backing up and cleaning up

To stop your oldmail directory getting too big, the following can be added to the end of your initial .procmailrc file:

:0 ic
| cd oldmail && rm -f dummy `ls -t msg.* | sed -e 1,16d`

To explain a little about what is going on here, those letters after the smiley are flags.

The c flag tells Procmail to make a copy of the email and perform the action on the copy, and then continue processing the .procmailrc file.

The i flag indicates to Procmail that it should ignore errors while writing. The action line which begins with a bar ( | ), simply enters the oldmail directory, and removes the oldest 16 files (messages).

The rest of the flags are explained in the man page.

An alternative way to clean up is from the command line by using the following (which deletes backup mail older than 10 days):

find $HOME/oldmail -mtime +10 -exec rm -i "{}" ";"

Let's make a cake!

Baking with Procmail

Procmail has the notion of recipes. Recipes are instructions for putting food together. In the Procmail sense, they are instructions regarding what should be done with incoming email. There are two types of recipes, delivering and (oddly enough) non-delivering; they are defined as:

  • Delivering recipes are those that cause header and/or body of the mail to be written into a file, absorbed by a program or forwarded to a mail address.
  • Non-delivering recipes are those that cause the output of a program or filter to be captured back by procmail or those that start a nesting block.

Example and Explanation

The Example

LOGFILE=$HOME/log/procmail
:0 c
oldmail

:0 ic
| cd oldmail && rm -f dummy `ls -t msg.* | sed -e 1,16d`

# Throw away duplicate messages
# (i.e. messages with an ID already seen before)
:0 Wh: formail.lock
| /usr/bin/formail -D 5000 $HOME/Mail/msgid

# Grab mail from important people and put it into my mailbox
:0:
* ^From.*anton
$DEFAULT
:0:
* ^From.*nokia
$DEFAULT

# Add more important people here ----

# Ignore people who are not worth listening to
:0:
* ^From.*(annoying@progsoc.uts.edu.au|annoying@it.uts.edu.au)
/dev/null

# Add more annoying people here ----

# redirect the ProgSoc@ mailing list into Mail/progsoc
:0:
* ^Sender:.*progsoc@progsoc
Mail/progsoc
:0:
* ^CC:.*progsoc@progsoc
Mail/progsoc

# redirect the R developers list into Mail/r-devel
:0:
* ^Sender:.*r-devel@stat.math
Mail/r-devel

# Add more lists here ---- 

# Any other list mail that I get gets redirected too
:0:
* ^Precedence:.*(list|bulk)
Mail/other-lists

# Grab any mail explicitly sent to me (or CC'd to me)
# put these messages into my regular mailbox
:0
* ^(TO|CC):.*telford
$DEFAULT

# Filter any mail autoclassified as spam into its own box.
:0
* ^X-Spam-Flag: YES
Mail/spam

# Everything else is something I do not understand,
# I may want it but cannot be sure at this stage
:0:
Mail/questionable

Making use of this

For normal mail reading I would just type:

mutt

This will give me all the important messages and the ones that are explicitly sent to my address. Most spam mail either goes to a list or gets bulk delivered with no regard to the name of the target.

If I feel like seeing what has been happening on the progsoc list:

mutt -f $HOME/Mail/progsoc

The Explanation

The first two recipes are to backup the most recent 16 emails that I received.

The formail recipe will "deliver" mail into the formail if the mail is a duplicate of earlier mail. Formail recognises duplicates by storing the message ID numbers in a file. Any message "delivered" to formail is thrown away so only non-duplicate messages get past this rule.

When the mail came from an address that I know is someone I like, I make sure it goes to the defualt mailbox (my normal box).

When the mail came from an address that I want to filter out then I send it to /dev/null to save me the trouble of deleting it manually. Note that I have specified the full address of the person I want to ignore but only a partial address of the people that I regard as important. We call this, "to err on the side of caution" but you may consider it better to do things the other way round.

Progsoc runs a spam-detecting program called SpamAssassin on its servers. All mail that you recieve through your Progsoc account include a special "X-Spam-Flag" header (among others), and you can use this header to auto-filter your mail. SpamAssassin tends to err on the side of caution, which means you're far more likely to see mail which should have been classified as spam than you are not to see mail which shouldn't have been.

Mailing Lists

Since most progsoc members are on progsoc@progsoc.uts.edu.au it should be easy to appreciate the benefit of separating list mail from regular mail. You want to take a look at the list mail but you don't want to carefully sift through every item. I tend to give the list mail a once over now and then if I have time after checking my personal mail.

I have also included a rule for the R developers list to make it easy to see how the patterns work, you may subscribe to various weird and wonderful lists so you can add more of your own.

Most polite list robots will put a "bulk" flag onto their mails so you can separate out that sort of mail as well.

But Wait...

The tail end consists of two slightly tricky recipes, I notice that spammers usually do not bother to write my name on the messages that they send. Since I don't like spam and I do like polite people who address their messages to me correctly, I only grab hold of messages that really have my name on them.

The very final recipe just provides a place to throw anything that didn't fit into any other category. This is not my default mailbox so I only bother looking at this stuff on occasion.

Accessing other mailboxes in Pine

So, you now have 30 mail boxes, how can you read them all? Using Pine, you need to add these as folders. Folders are just file which contain mail messages (accessed via the L key).

Edit .pinerc in your home directory, and look for the section below:

# List of incoming msg folders besides INBOX, e.g. ={host2}inbox, {host3}inbox
# Syntax: optnl-label {optnl-imap-host-name}folder-path
incoming-folders=

Make some modifications to it as per the instructions, and it will probably come out looking something like this.

# List of incoming msg folders besides INBOX, e.g. ={host2}inbox, {host3}inbox
# Syntax: optnl-label {optnl-imap-host-name}folder-path
incoming-folders=OzEmail {mail.ozemail.com.au}inbox,
       FIT {lucy.it.uts.edu.au}inbox,
       POSTMASTER Mail/postmaster,
       PROGSOC Mail/progsoc,
       SYSTEM Mail/system,
       WWW Mail/www

The first two entries allow access to mail stored on either OZEmail or Faculty of IT computers remotely. The last four access the mail boxes created by procmail.

Personal tools