"Linux Gazette...making Linux just a little more fun!"

The Answer Guy

By James T. Dennis, tag@lists.linuxgazette.net
Starshine Technical Services, http://www.starshine.org/

'sendmail' Log Jams and Capacity Problems

running extra 'sendmail -q' processes

From Robert Cotran on 28 Apr 1998

Hi there. You seem to know your stuff about Linux. I've been running it for two years. I have a little server at home. I was at first running Slackware, I'm not running RedHat 5.0. Here's my problem. After a while of running, my sendmail jams up. It doesn't let in any mail and it doesn't send out any queued mail. It's the strangest thing. And I had the same problem running Slackware. I figured that RedHat (having more updated programs) would be OK, but it's not. After a while, any queued messages just stay stuck in the /var/spool/mqueue dir, and incoming connections on port 25 are either REAL slow, or not possible. After rebooting however, all the queued mail zips out, and incoming connections are fine again. Sorry to bug you, but I just figured you might have an idea. Thanks for any help you can give me!

Rob

You don't say what sort of volume your trying to bump through there or what sort of connection you have to the net. So I'll have to assume the worst.

When I was postmaster/sysadmin of a very high volume site I found that I needed to occasionally run extra copies of the command 'sendmail -q' concurrently to help reduce the backlog in the queue. This command simply makes 'sendmail' to one sweep through its queue (mostly outgoing) and look for items that are ready for a retry.

'sendmail' has locking mechanisms (those x* and l* files) to prevent concurrency issues. So this is a safe procedure.

Typically I find that it's the DNS requests that bog down the processing, so running a caching nameserver on the localhost and making it the primary entry in your /etc/resolv.conf can be a big win.

A trick I've used when a system gets temporarily really overloaded is to tar/gzip up a bunch of qf* and df* file pairs (queue control and data) --- ftp that to another system and extract them into a new directory. Then I have that system run several queue run processes in parallel to the other. This is particularly handy if the other system has a separate pipe (connection to the Internet).

In Robert Harker's 'sendmail' Seminar (which I took a few months ago) he describe a "requeue" mailer. This is an advanced technique that he would use to requeue some mail into a separate, low priority, queue directory. The idea is to prevent a relatively small number of messages to sites that are dead or down, and messages to bogus addresses (from your users' typos, etc) from clogging things up for the other 70 or 80 percent that could be expedited.

Another thing to seriously consider and investigate is whether you're site is being used as a spammer's relay. If your SMTP processing load has suddenly and drastically increased then you might have had some of your systems "hijacked" by spammers.

To understand this a little bit of explanation is in order. The Internet was founded in spirit of co-operation so 'sendmail' was designed and configured (for the last 20 years) to accept mail from anyone, to anyone. It would gladly relay mail at the request of any MTA (mail transport agent).

This was practical and reasonable --- since mail that needs multiple hops to get somewhere (say from department to hub/gateway, to other site's hub, to recipient's department and thence to recipient's home host) shouldn't need to carry "authentication" information for every leg of its journey.

However 'spammers' use this openness to send a few copies of a message with hundreds or thousands of recipients listed for each. This allows someone with a 28.8 modem to chew up thousands of gigabit hours of bandwidth over the entire 'net.

In any event you can find some instructions on how to use the recent configuration options in 'sendmail' to prevent some of this abuse. Look at http://www.sendmail.com/ for info on the new 8.9 release which supports a number of anti-spam features for details. (As you might expect this site also has lots of other sendmail information as does the older and now slightly out-of-date http://www.sendmail.org/). Sendmail 8.9 includes support for a FEATURE (m4 macro set) that links you into the "Real-Time Black Hole List" (RBL) that is maintained by Paul Vixie (author of DNS BIND/named and of the Vixie 'cron' among other things).

Like all sendmail FEATURE's this is optional.

For information information about the RBL look at http://maps.vix.com/rbl/

For more information about the fight against spam look at: http://www.cauce.org/

For the best free support on the net for sendmail related issues subscribe to the comp.mail.sendmail newsgroup.

Copyright © 1998, James T. Dennis
Published in Linux Gazette Issue 29 June 1998

	versions	lilo	virtdom	kernel	winmodem	basicmail	betterbak
	shadow	dell	dumbterm	whylinux	redhat	netcard	macrovir
	newlook	tacacs	sendmail	dialdppp	ppp233	msmail	procmail