Get and install mail programs

Switching from exmh to Thunderbird

In October 2005, I switched away from fetchmail/exmh/nmh/procmail to Thunderbird with IMAP. I tried Apple's Mail.app first, but it hung a few times and smashed SCD's mail server, so I switched to Thunderbird. Reasons to change from exmh to Thunderbird were:
  1. Exploit fast indexed searching. The old way was find/grep, which was pathetic, or glimpse, which couldn't search headers well. Now I have spotlight-style indexed searching, which is much better.
  2. Better integration with other applications.
  3. A better GUI. Exmh is old. Good pull-down menus to do configuration, and a look that's compatible with other applications on the Mac.
  4. Get away from the "all mail is on the laptop" mode. I want to be able to read mail from other machines, like in an Internet cafe. I could do that before, but I didn't have a model for thinking about where the mail is. Now, I have folders that are local and folders that are on the IMAP server. I can move mail that I want to save into a local folder, and leave crummy mail on the IMAP server, or something. I need to find out the best scheme for this: under Tools->Account Settings->Offlin & Disk Space, there are interesting possibilities.
  5. Integration with Microsoft Office tools.
To make the move, I had to figure out how to get my old mail folders "into" Thunderbird. I wrote a Perl script named mh2mbox.pl that used the nmh "packf" program to make mbox-format files out of my old MH folders, then copied the mbox files to the place that Thunderbird stores local folders, which I found under Thunderbird's Tools->Account Settings->Local Folders. I copied the mbox files and Thunderbird automatically created index files named .msf (Mail Summary File), making Thunderbird's search capability work.

To get LDAP working with Thunderbird, I followde Lynda McGinley's excellent instructions, paraphrased here:

converting MH folders to mbox format

To move from exmh to Thunderbird, I had to convert my existing mh mail to mbox format. The secret is packf, an nmh program.

How I send/receive mail on Linux systems

I use fetchmail to get my mail from mail.ucar.edu via IMAP and store it in ibex, my desktop. I use procmail to sort the mail into folders (directories), exmh as a mail reader, spamassassin to filter spam, XEmacs to compose mail. I run a postfix daemon to receive messages from fetchmail and to handle messages composed in exmh/xemacs.

When I'm on a foreign network, like a conference wireless network or a hotel room network, I use the VPN client and ssh to ibex to read mail. If I'm on a foreign machine like at an Internet cafe, I use the web interface to the CommuniGate software at https://mail.ucar.edu. I don't use http://webmail.ucar.edu, which is another web-bsed email interface written by Leonard Sitongia.

How I receive mail

  1. I run fetchmail
  2. fetchmail gets mail messages from the IMAP server on mail.ucar.edu
  3. fetchmail feeds each message to the postfix daemon, listening on port 25
  4. By default, the postfix daemon would append the message to /var/spool/mail/siemsen, but my $HOME/.forward file causes it to run procmail instead. postfix then writes a log message to /var/log/maillog file.
  5. procmail reads $HOME/.procmailrc and decides what to do with the message, (usually runs rcvstore), and logs a message to $HOME/Procmail/log. If you don't have a $HOME/.procmailrc file for some reason, procmail will quietly do nothing and your email will end up in /var/spool/mail/siemsen.
  6. rcvstore writes the message into a folder under $HOME/Mail

How I send mail

  1. In exmh, I click "Comp", or reply to a message, or forward it
  2. exmh runs "comp" or "repl" or "forw" to create a new file in the ~/Mail/drafts directory, call it dfile
  3. comp or repl or forw inserts headers from the ~/Mail/components or ~/Mail/replcomps or ~/Mail/forwcomps files into the dfile. See 'man mh-format' for details about the format of these template files.
  4. exmh runs "exmh-async gnuclient dfile" (see man 'gnuclient') where dfile is the name of the new file in ~/Mail/drafts. This behavior was specified in exmh->Preferences->Editor Support.
  5. A running XEmacs, which has had gnuserv-start executed as part of initialization, wakes up.
  6. XEmacs opens the dfile and, since ~/.xemacs/init.el said to, displays a new frame with the buffer in mh-letter-mode. mh-letter-mode is part of mh-e, which is a standard XEmacs package. See MH-E home page.
  7. I write the letter
  8. I hit f12 or click "Done"
  9. XEmacs invokes petes-save-and-stop
  10. petes-save-and-stop invokes gnuserv-edit
  11. gnuserv-edit marks the current gnuserv editing buffer as "done", and the buffer's client (exmh) is notified
  12. exmh displays the "What now?" window
  13. I click "Send"
  14. exmh runs nmh "send" (see 'man send')
  15. "send" reads ~/.mh_profile (see 'man mh_profile') and then runs nmh "post" (see 'man post')
  16. "post" adds a "Date:" line. I have 'masquerade: draft_from' turned on in /usr/local/nmh/mts.conf. I also have "From:" lines defined in the template files named ~/Mail/components, ~/Mail/replcomps and ~/Mail/forwcomps. As described in 'man mh-tailor' and 'man post', this combination causes "post" to not add a "Sender:" line, and use the "From:" line from the template file for both the envelope "From" line and the internal "From:" line in the message.
  17. "post" runs /usr/lib/sendmail, which is a link to /usr/sbin/sendmail, which is a special version that was installed when postfix was installed. (from the postfix docs) The Postfix sendmail program invokes the privileged postdrop program which deposits the message into the maildrop directory. Actually, what appears to happen at this point according to the /var/log/maillog file is that something starts the smtpd daemon, which logs a message to the /var/log/maillog file. Perhaps /usr/lib/sendmail sends a packet to the SMTP port, and inetd starts an smtpd daemon?
  18. (from the postfix docs) The pickup daemon reads the file from the maildrop directory and feeds it to the cleanup daemon, which logs a message to the /var/log/maillog file. According to the man page, the cleanup daemon adds From: and other message headers and then inserts the result as a single queue file into the incoming queue, and notifies the qmgr daemon of the arrival of new mail.
  19. Somewhere in the postfix processing described in the last 2 steps, postfix uses the myorigin value defined in the /etc/postfix/main.cf file to set the outbound "From" line. If myorigin is set to the default value of "$myhostname", the envelope "From" line in outgoing messages will contain the incorrect value "siemsen@gazelle.ucar.edu". When myorigin is correctly set to "$mydomain" in the file, the envelope "From" line in outgoing messages is supposed to contain the correct value "siemsen@ucar.edu".
  20. The qmgr daemon wakes up, logs a message to the /var/log/maillog file, and starts another instance of the smtpd daemon.
  21. The smtpd daemon actually ships the mail to mail.ucar.edu, and terminates.
  22. mail.ucar.edu forwards the message to its final destination.

How I send/receive mail on Mac systems

When I started the Mac Mail program the first time, it asked me some questions. I said my incoming server was IMAP, the server was mail.ucar.edu, the outgoing server was mail.ucar.edu, and I didn't check the "Authentication" box on the "outgoing" screen.

I downloaded and installed letterbox.app, which makes Mail.app display the content next to the list of emails.

MIME

Reading MIME attachments
Exmh handles MIME in received mail messages using metamail. You should also consider the mhstore program. Also see stripmime or ripmime.

Images in exmh - see Images in exmh

Composing a message containing a MIME attachment
As of 2003-07-10, XEmacs on gazelle can't create MIME attachments like it used to. Now, I use the Web interface to the Communigate software running on mail.ucar.edu. To create sugh a message:

You can insert a file as a MIME attachment when composing messages in XEmacs mh-letter mode. Use C-c C-m tab to insert a MIME placeholder. When finished composing the message, just before exiting, use C-c C-e to expand all the placeholders and insert all the MIME headers. Some MIME types you can use are

Word - application/msword
Excel - application/vnd.ms-excel
PowerPoint - application/vnd.ms-powerpoint
Forwarding mail that contains MIME attachments
Subject: Re: Forwarding MIME mails without nested MIME
From: Neil W Rickert
Date: Wed, 15 Jun 2005 15:47:38 -0500 (14:47 MDT)
To: Discussion list for EXMH users

Stephan Skrodzki wrote on Jun 15, 2005:

Maybe it's a FAQ, but I haven't found it while googling around for some time:

How do I forward a Mail with MIME Attachments so, that most MUAs understand that?

I only found two forward possibilities:

a) plain forward: this generates messages without any MIME parts -> useless

It isn't completely useless. An mh user can use "burst" to extract the forwarded message(s) as individual messages.

b) mh mime forward (forw -mime): this generates nested MIME, which many MUAs (Apple Mail, evolution etc.) do not understand...
Any mailer that doesn't understand this format has broken mime support.
Is there something I haven't seen yet?
Send the person two messages:

(a) a short note indicating that you are resending a message seperately,

(b) the original message, resent using "dist" (in mh/nmh) or "redidstribute" from inside exmh.

Even mailers with broken mime support should be able to handle that.

-NWR

metamail

Metamail decodes MIME messages, and is needed by exmh. I found RPMs for it at a Red Hat Fedora "legacy" site, and then did
rpm -i metamail-2.7-29.i386.rpm sharutils-4.2.1-8.7.x.i386.rpm

junkfilter

Junkfilter filters out junk mail. It requires procmail.

procmail

To filter (sort) mail into mh folders, you can use slocal or procmail. The slocal way is easy, but it doesn't work well. It gets confused by "From " lines inside mail messages. The procmail way is better, but is harder to set up (surprise). To use procmail, first you have to get and install procmail. See the procmail home page for details. Procmail is installed as part of the Red Hat 7.2 distribution, so I didn't need to get/install it, but I did need to allow sendmail to execute it via sendmail's restricted shell, by doing this:

(as root)
ln -s /usr/bin/procmail /etc/smrsh

Here are notes about how to configure and debug procmail filtering. Note: when experimenting with sending yourself mail, check the ~/Procmail/log file for procmail's log messages.

Create a .procmailrc file...

cd
cat >.procmailrc
#
# For an explanation of all this, see
# http://www.ii.com/internet/robots/procmail/qs
#
# where I store my mail
MAILDIR=$HOME/Mail

# so procmail can find rcvstore
PATH=/usr/lib/nmh:/usr/bin:/usr/local/bin

# what to do with mail that doesn't match any recipes
DEFAULT=$MAILDIR/inbox

# Set to yes when debugging
VERBOSE=no

# Remove ## when debugging; set to no if you want minimal logging
## LOGABSTRACT=all

# Directory for storing procmail-related files
PMDIR=$HOME/Procmail

# Put ## before LOGFILE if you want no logging (not recommended)
LOGFILE=$PMDIR/log

# recipe files
#INCLUDERC=$PMDIR/testing.rc
INCLUDERC=$PMDIR/main.rc
^D
chmod 644.procmailrc
Create the main filter file. Each of the rules here has to refer to an exmh folder that already exists. Create the folder using the New button in exmh.

cd
mkdir Procmail
cd Procmail
cat >main.rc

:0
* ^TO_ABILENEV6-L
| rcvstore +abilenev6

:0
* ^TO_ABILENE-OPS-L
| rcvstore +abilene-ops

:0
* ^TO_abilene-techs
| rcvstore +abilene-techs

:0
* ^From:.ARSystem
| rcvstore +arsystem
etc. ^D
Create a .forward file...

cd
cat >.forward
"|exec /usr/bin/procmail"
^D
chmod 644 .forward

Get and install nmh

Nmh didn't come as part of Red Hat Fedora Core 2, so I had to get it. I got the latest version, nmh-1.1rc3, which is beta. Livin' large.

cd /usr/src/nmh-1.1-RC3
./configure
Then I edited the config.h file and commented-out DBMPWD. Probably doesn't matter.
make
make install
This stores nmh binaries in /usr/local/nmh/bin and man pages in /usr/local/nmh/man, so you'll have to update the .bashrc file. Nmh 1.0.4-9 came installed on gazelle. Most of the nmh binaries (inc, show, etc) are installed in /usr/bin. Some binaries are in /usr/lib/nmh.

Update the .bashrc file to add

if [[ ($AMROOT = "no") && (${HOST} = 'proteus' || ${HOST} = 'gazelle' ) ]]; then
    MHLIB=/usr/lib/nmh
    if [[ -d ${MHLIB} ]]; then
        if [ $DEBUGGING ]; then
            echo ".bashrc: setting up nmh..."
        fi
	export MHLIB
	pathadd ${MHLIB}
#        /usr/bin is already in the path
#        manadd /usr/local/nmh/man
    else
	unset MHLIB
    fi
fi
      

Configure nmh

First, configure nmh so that it knows the name of the local host, local domain, masquerading, and the SMTP server to send mail to. If you don't do this, then outbound mail may contain "siemsen@localhost.localdomain" in the internal "From" field instead of "siemsen@ucar.edu", and mail bounce in some mail servers. See 'man mh-tailor' and 'man mts.conf' and the above description of how I send mail. Edit /usr/local/nmh/mts.conf and set
localname: gazelle
localdomain: ucar.edu
masquerade: draft_from
servers: mail.ucar.edu

When nmh configures an outbound message, it formats the headers by following the templates contained in four files:

~/Mail/components
controls formatting when I compose a new message
~/Mail/replcomps
controls formatting when I reply to only the sender of a message
~/Mail/replgroupcomps
controls formatting when I reply to the sender and all the CCs of a message
~/Mail/forwcomps
controls formatting when I forward a message

Note that the versions of these files in /usr/local/nmh/etc are not the ones that count, so don't bother editing them.

If you don't have the proper templates in these files, you can have some very irritating effects. For a few years, these files were slightly wrong, and when I forwarded or replied to a message, the envelope From line in my outbound messages was "siemsen@proteus.ucar.edu" instead of "siemsen@ucar.edu". This caused replies to those messages to bounce. As described above, you need "From:" lines in these template files. As of 2004-03-04, here are the "correct" contents of those files:

components

To:
      From: Pete Siemsen <siemsen@ucar.edu>
Subject: Fcc: outbox --------

replcomps

      From: Pete Siemsen <siemsen@ucar.edu>\n\
%(lit)%(formataddr %<{mail-reply-to}%?{reply-to}%?{from}%?{sender}%?{return-path}%>)%<(nonnull)%(void(width))%(putaddr To: )\n%>\ %<{subject}Subject: Re: %{subject}\n%>\ %<{date}In-Reply-To: Your message of "%<(nodate{date})%{date}%|%(pretty{date})%>."%<{message-id} %{message-id}%>\n%>\ Fcc: +outbox\n\ --------

replgroupcomps

      From: Pete Siemsen <siemsen@ucar.edu>\n\
%(lit)%(formataddr %<{mail-reply-to}%?{reply-to}%?{from}%?{sender}%?{return-path}%>)%<(nonnull)%(void(width))%(putaddr To: )\n%>\ %<{cc}Cc: %{cc}\n%>\ %<{subject}Subject: Re: %{subject}\n%>\ %<{date}In-Reply-To: Your message of "%<(nodate{date})%{date}%|%(pretty{date})%>."%<{message-id} %{message-id}%>\n%>\ Fcc: +outbox\n\ --------

forwcomps

To:
      From: Pete Siemsen <siemsen@ucar.edu>
Subject: cc: Fcc: +outbox --------

Get and install exmh

Web to the EXMH home page and click on exmh-2.7.2.tar.gz. Save it in /usr/src.

cd /usr/src
gunzip exmh-2.5.tar.gz
tar xf exmh-2.5.tar
rm exmh-2.5.tar
cd exmh-2.5/
wish exmh.install

This starts an Exmh Install window with buttons and a Install EXMH window containing help text. In the Exmh Install window, change the paths to the following:

MH slocal program: /usr/local/nmh/lib/slocal
Root dir of faces store: <blank it out since I don't use faces>
Your default domain: ucar.edu
Audio play command: /usr/bin/play

Click on the I have read the instructions button, then the Install button, then the Really Install button. Wait until it's done, then click on the Quit button.

Ok, now you can run it. The first time, it'll ask if you want to create a drafts directory. Click Yes.

When I ran exmh as siemsen, I got this message: BgRegister X server insecure (must use xauth-style authorization); command ignored It isn't very clear what the problem is. I posted a whine to exmh-users mailing list. Responses were to run xdm, which will take care of this. It does, but I don't want to run xdm. After some pain, I finally found that doing this:

xauth generate :0.0 .
or
xauth add :0 . `mcookie`
to create a $HOME/.Xauthority file and then starting X with the "-auth $HOME/.Xauthority" option worked. The trouble is, now doing a "su" and then trying to run an X application gives this error:
Xlib: connection to ":0.0" refused by server
Xlib: Invalid MIT-MAGIC-COOKIE-1 key
xterm: can't open display :0.0
To fix this, as root, copy siemsen's .Xauthority file to /root

Customize exmh

Link Exmh to Netscape

handle text/html MIME attachments
When an email message contains a MIME attachment that is text/html, we want exmh to silently extract the html into a temporary file and then warp Mozilla to the file. To configure exmh to do this, click on Preferences, the WWW, then set "how to display text/html" to "external". Set the Mosaic program name to Mozilla. Set the Netscape command to "/usr/bin/mozilla -remote openURL($xuri)". Click Dismiss, then click Save, then restart exmh.
don't just warp Mozilla every time
(THIS DOESN'T WORK, I DUNNO WHY)
After setting up things as in the last paragraph, you may find it to be annoying, because the Netscape window unexpectedly leaps up as you go from message to message. To make exmh *not* automatically pop up, put the following in your ~/.exmh/exmh-defaults file. When a message contains HTML, exmh will display a window that says There are alternative views of the following. You can then right-click on the window to pop up a menu that offers Show alternative: text/html, which will cause the message to be displayed in Netscape.
*mime_alternative_prefs: text/plain text/richtext text/html

You'll probably have to restart exmh to see the effects of this.

Make URLs in email messages clickable
You can make exmh highlight URLs that appear in mail messages, and if you click on the URL, exmh will cause a running netscape to warp to the URL. Click Preferences, then WWW, then turn on "Scan for URLs in messages", then set "Netscape command" to "/opt/netscape/netscape -remote openURL($xuri)". For this to work, you have to have Netscape running.

Link Exmh to XEmacs

You can link exmh to a running XEmacs process. When you compose or reply to a message in exmh, XEmacs is used as the editor. This can be set up so that instead of starting a fresh copy of xemacs for every message you compose, you communicate with a single XEmacs process. This is described in Ben Escoto's Exmh/XEmacs Integration page. Following Ben's instructions, click on Preferences, then Editor Support. In the resulting window, in the Editor command field, type
exmh-async gnuclient
You'll probably have to restart exmh to see the effects of this. This causes exmh to run gnuclient whenever you compose a message. Exmh does so in an asynchronous process, so exmh isn't paused while the gnuclient runs. Gnuclient is a way to tell a running XEmacs to do things. The XEmacs has to have been told to run gnuserv to listen for gnuclient "calls" (see below). Gnuclient communicates with gnuserv using a Unix domain socket in ${TMPDIR}. For this to work, you'll need to add this to your .bashrc file:
export EXMHTMPDIR=~/tmp
So: gnuclient asks XEmacs to open a frame containing the compose template. When you're done using XEmacs to compose the message, hit C-c Y and confirm it, and XEmacs will hand the message back to exmh, which will display a "What Now?" window. Click "Send" to send the message.

Then copy Ben's elisp into your .xemacs/init.el file. This sets things up so that when you compose a message in exmh, exmh causes XEmacs to open a frame and put it into mh-letter-mode. When you're finished composing the message, hit Control-C T. See the .emacs file for details.

One of the features of Ben's ELisp code in the .emacs file is support for "Supercite", an Emacs package that helps you cite someone else's email message in your own email message. This is great for replying to mail. See the info page for Supercite for details. The long and the short of it is, when you put Ben's ELisp fragment in your .emacs file, and then reply to a message from exmh, you can use the C-c C-y command to insert the message you're replying to into the reply.

Link Exmh to acroread

When a mail message contains a MIME attachment that's a PDF file, exmh displays the file name highlighted. If yau click on the highlighted name, exmh looks in the ~/.mailcap file to see what program to run for the given MIME type.

Move scrollbars to the left

Click on Preferences, then Windows and Scrolling. In the resulting window, change the Vertical Scrollbar Side to "left".

Get rid of the Inc button

Once you've created the .forward and .procmailrc files, start exmh, click Preferences, then Incorporate Mail, then set the Ways to Inc button to none. Click Dismiss, then IMPORTANT!!! click Save, not Dismiss. This tells exmh not to sort incoming mail into folders, and to remove the Inc button from the exmh user interface. This is cool, because incoming mail will be sorted into folder as it arrives, when fetchmail executes procmail, which reads the .procmailrc file, which sorts each mail message into a folder using rcvstore.

Address book feature

The "address database" feature of exmh can be used if you follow the directions on Ben Escoto's Exmh/XEmacs Integration page. When turned, on, exmh will save all the email addresses in messages that you send, and store them in ~/.exmh/exmh-addrs. Then, when you compose messages, you can use C-TAB to complete email addresses.

I dunno how good this is. To turn it off to prevent exmh from saving up addresses, go to Preferences->Address Database and unclick the top button.

Mouse wheel support

Turn on Preferences->Windows & Scrolling->Wheel Mouse.

Glimpse

I got Glimpse from http://webglimpse.net/ into /usr/src, and built it.
cd /usr/src
gunzip glimpse-latest.tar.gz
tar xf glimpse-latest.tar
rm glimpse-latest.tar
chown -R siemsen:datacomm glimpse-4.18.1p
exit (back to "siemsen")
cd /usr/src/glimpse-4.18.1p
(per README.install)
sh configure
make
su
cd /usr/src/glimpse-4.18.1p
make install
This install some binaries in /usr/local/bin, including glimpse and glimpseindex, and man pages for them.

To index my mail folders, do

glimpseindex -o ~/Mail
This writes index files named .glimpse* in my home directory. The glimpse program will look at those files, like
glimpse abcdef
Now try, from within exmh, Search->All  messages in all folders (glimpse) to bring up the exmh Glimpse window. In that window, click index to index all the Mail folders. See http://localhost/exmh-2.7.2/Search.html.

image display in EXMH

Exmh will use the entries in /etc/mailcap to decide what to do with MIME attachments. By default, my Red Hat system was using gthumb to try to display images, and gthumb wasn't working. So I installed kview:
(as root)
yum install kdegraphics
and then changed the entry in /etc/mailcap to make it say
image/*; /usr/bin/kview %s

postfix

On 2003-06-02, I stopped using sendmail and started using postfix, which is faster and simpler. I went to the postfix home page and got postfix-2.0.10.tar and did
as root
cd /usr/src
tar xf postfix-2.0.19.tar
rm postfix-2.0.19.tar
cd postfix-2.0.19
make
groupadd -g 501 -f postdrop
groupadd -g 502 -f postfixgrp
useradd -u 98 -g 502 -c "Pete Siemsen" postfix
make install
hit RETURN to all prompts
This finsihed with
Warning: you still need to edit myorigin/mydestination/mynetworks
parameter settings in /etc/postfix/main.cf.

See also http://www.postfix.org/faq.html for information about dialup
sites or about sites inside a firewalled network.

BTW: Check your /etc/aliases file and be sure to set up aliases
that send mail for root and postmaster to a real person, then run
/usr/bin/newaliases.
I modified /etc/postfix/main.cf as follows:
default_privs = nobody
myhostname = ibex.scd.ucar.edu
myorigin = ucar.edu
inet_interfaces = localhost
mydestination = $myhostname, localhost.$mydomain
mynetworks_style = host
mailbox_command = /usr/bin/procmail
smtpd_banner = $myhostname ESMTP $mail_name
Then I created a postfix startup file in /etc/init.d containing
#!/bin/bash
#
# postfix        Starts postfixd/klogd.
#
#
# chkconfig: 2345 98 20
# description: postfix, a replacement for sendmail
### BEGIN INIT INFO
# Provides: $postfix
### END INIT INFO

# Source function library.tcl8.4.7/unix
. /etc/init.d/functions

RETVAL=0

start() {
 	echo -n $"Starting postfix: "
	postfix start
	RETVAL=$?
	return $RETVAL
}
stop() {
	echo -n $"Stopping postfix: "
	postfix stop
	RETVAL=$?
	return $RETVALtcl8.4.7/unix
}
reload() {
	postfix reload
}

case "$1" in
  start)
  	start
	;;
  stop)
  	stop
	;;
  restart|reload)
  	reload
	;;
  *)
	echo $"Usage: $0 {start|stop|reload|restart}"
	exit 1
esac

exit $?
    
Then I did
as root
cd /etc/init.d
./sendmail stop
chmod +x postfix
chkconfig --add postfix
chkconfig --del sendmail
newaliases
./postfix start
Look in /var/log/maillog for log messages about postfix starting and stopping.

fetchmail

Fetchmail uses IMAP to get my mail from mail.ucar.edu. Web to fetchmail home page for details.

cd /usr/src
gunzip fetchmail-5.9.5.tar.gz
tar xf fetchmail-5.9.5.tar
rm fetchmail-5.9.5.tar
cd fetchmail-5.9.5
./configure (ignore makedepend error messages)
make
make install
ln -s /usr/sbin/sendmail /usr/lib/sendmail
Configuring fetchmail consists of creating a .fetchmailrc file in your home directory. A GUI program called fetchmailconf is provided to help you do this. I couldn't get fetchmailconf to write a good .fetchmailrc file. My best efforts produced a file containing only a timestamp comment. So I had to actually read the man page and learn the syntax used in the file, which wasn't trivial. Sigh.

While developing a .fetchmailrc file, it can be helpful to run fetchmail --version, which will tell you the version and other interesting information about what fetchmail would do, without doing anything.

Fetchmail fetches mail from the server. Fetchmail delivers the mail to the local postfix daemon, which finds my ~/.forward file and invokes procmail to sort the mail into folders.

If fetchmail gives you messages like

skipping message siemsen@mail.ucar.edu:1 not flushed
...it means that you've used another IMAP client to read your mail, and the other client marked the messages you read as "seen" on the server. Fetchmail won't fetch "seen" messages, so you'll continue to get the "skipping" messages until you delete the "seen" messages from the IMAP server.

Port forwarding IMAP

On 6/12/2001, at a conference, I connected to a local network, both cabled and wireless. When I executed fetchmail, I got the following error:

131 messages for siemsen at proteus.
reading message 1 of 131 (740 header octets) fetchmail: SMTP error: 250 2.0.0 Reset state not flushed
reading message 2 of 131 (2406 header octets) ..fetchmail: SMTP error: 250 2.0.0 Reset state not flushed
reading message 3 of 131 (2014 header octets) .

At each of the above failures, it displayed "octets)..", then waits exactly 80 seconds (80, not 60), then displays the message from fetchmail and immediately moved on to the next message.

In an attempt to "fix" it, I Greg Woods suggested that I use ssh port forwarding. I did so, as follows. Note that it didn't fix the problem. Then, later, it started working, for no reason that I can fathom. The next day, it failed again, the same way, and I didn't get it working.

To set up port forwarding, you first start an ssh session to mail.ucar.edu and include port forwarding of the IMAP protocol. Then you run fetchmail in a way that does IMAP to localhost instead of to mail.ucar.edu.

(as root)
ssh -l siemsen -L 143:mail.ucar.edu:143 mail.ucar.edu

(as siemsen)
fetchmail -f ~/.fetchmailrc-wireless

Configure aliases

Postfix runs chrooted, and uses the /etc/postfix/aliases file. To reduce confusion, delete the /etc/aliases file, which was used by sendmail. Edit the /etc/postfix/aliases file. Set root to "siemsen@ucar.edu". This will cause all messages to "root" or "postmaster" to go to "siemsen@ucar.edu".

Set up the ~/.mh_aliases file, so I can use shortcuts when I compose new mail messages. It should contain, like,

delynn: delynnmarie@yahoo.com
fmc: fmc@ucar.edu
colburn: Scot Colburn
scot: Scot Colburn

Create houseclean-Petes-home.pl

Copy this Perl script from elsewhere, and install a cron job to run it periodically. It deletes old deleted mail from my ~/Mail directory.

SPAM

When an inbound email message for siemsen@ucar.edu is received by UCAR, it first goes through the "blacklist filter". The source address is compared to a list of known spam sources. If it matches, it is dropped. To turn on blacklist filtering for siemsen@ucar.edu, go to WEQC by going to http://www.people.ucar.edu/ and putting in "siemsen" and then clicking on the resulting "Pete Siemsen" link. Set the "Spam Block" level to "2", which is tho highest. That will turn on blacklist filtering and greylist filtering.

If an inbound email message for siemsen@ucar.edu passes the blacklist filtering, it enters the MAIA system, where it first goes through the virus scanner. IF it contains a virus or a Windows .exe MIME attachment, it is dropped. You can't control virus scanning.

Next, the message goes through SpamAssassin, where it is assigned a score. To control the score threshold past which the email is considered spam, follow the directions at UCAR Spam page. Set the level to "High", which equals a SpamAssassin score of 3.

Finally, the email message is delivered, and your mailer does it's own filtering. Mac Mail uses a Bayesian filter that I train by assigning mail to "junk" or not.

mailing lists

To find out who is on a mailing list, go to niwot (it has the ldapsearch command installed) and use my lds alias, like so:
niwot$ lds nets-outage
cn=nets-outage, ou=aliases, dc=ucar, dc=edu
sn=nets-outage
fwd=nets, cpg, hmail@netserver
description=NETS Outages (David Mitchell)
niwot$
This tells you that nets-outage forwards to nets and cpg and hmail@netserver. See (ldapsearch) for details.

If it's a mailman list, the "forward" line will be an address like xyz@mailman. Such lists can be "advertised" or "unadvertised". If it's advertised, you'll find the list at http://mailman.ucar.edu/mailman/admin. If it's unadvertised, you won't find it there, and you have to go directly to the link for the individual list. For instance, the link for frgp-outage list is http://mailman.ucar.edu/mailman/admin/frgp-outage

using IMAP mailers

I use mh-style mail system, where mail is fetched from an IMAP server and stored locally. An alternative is to use what I'll call IMAP mailers like Netscape/Mozilla or Evolution, which store all the mail on the SCD mail server named mail.ucar.edu.

PROs of IMAP mailers

PROs of fetchmail, exmh, etc.

Conflicts when using MH and Evolution or Mozilla and the MH system

I use the exmh/fetchmail system as my primary mail system. When I use Evolution, it displays the unfetched mail that is sitting on mail.ucar.edu. If I read a message, it gets moved to somewhere, and won't be fetched the next time I do fetchmail. So if I read mail with Evolution, the mail is "lost" to the MH system.

using Evolution to read/write mail

I experimented with using Ximian Evolution to read/write mail under Linux. This might be better than exmh, as it'll let me store email and folders on mail.ucar.edu instead of a local directory. The first time you start Evolution, it'll ask configuration questions. Set the incoming part to use IMAP to mail.ucar.edu. Set the outgoing part to mscan.ucar.edu. Set the Automatically check for new mail every checkbox to off

using Mozilla to read/write mail

Using Mozilla to read/write email allows me to save email and folders on the SCD mail server instead of in an mh-style local directory.

Maia Mailguard

On 2005-03-17, I had gotten a few emails from Maia about mail to nets-www or npad, so I read the UCAR Mailguard System page and configured Maia to filter email addressed to Pete Siemsen. I followed the instructions an that page, copied below so that I have them even if someone moves the page:
Before Mailguard will perform content-based filtering on your mail, you must configure a minimum of two things. To do so, log in to Mailguard system, then:
  1. Link your e-mail addresses to your account:
    • Go to the Settings page (click the "cog" wheel icon).
    • Enter your main UCAR address in the Username field, e.g. joe@ucar.edu. Enter your UCAR Central Authentication Services (UCAS) password, then click the "Add E-Mail Address" button (you must click the button, hitting return will not work as expected).
    • Return to the Settings page. Enter your divisional address (e.g. joe@division.ucar.edu) in the same way.
    • Return to the Settings page. Enter your ncar.ucar.edu address (e.g. joe@ncar.ucar.edu) in the same way.
    • It does not matter (in UCAR's environment) which address is Primary, as long as all of the addresses at which you receive mail are linked to your account.
  2. Enable Spam filtering:
    • Go to the Settings page.
    • Click on your Primary address, e.g. joe.
    • Click the Enabled radio button under Spam Filtering.
    • Click the "Update ALL Addresses' Settings" button (so this change will apply to all the addresses you have linked to your account).

Address comments or questions about this Web page to the Network Engineering & Technology Section at nets-www@ncar.ucar.edu. The NETS is part of the Scientific Computing Division of the National Center for Atmospheric Research, which is part of the University Corporation for Atmospheric Research.
Last modified: Mon Oct 16 16:10:10 MDT 2006