Listwork How-To
About this Document
This document is meant as a
technical reference for members of the listwork working group.
Are looking for
help with administration, moderation or use of single mailing lists? Please refer to the
Listwork FAQ.
Are you looking for
information about the mail server itself?
SaraiInfo has it.
To keep track of changes to this and the other documents ('topics', i.e.
ListWorkHowto) within this TWiki web (i.e. Sysadmin), please subscribe to the notifications at
WebNotify. Some statistics on changes of this and other pages are available at
WebStatistics.
Mailing Lists
Creation, Renaming, Suspension, Removal, Resetting Passwords
Creating Mailing Lists
The easiest way to create new lists is through Mailman's
list creation web interface.
However, if you prefer console, use this instead:
$ sudo /usr/sbin/newlist
Aliases will be created automagically whether you use web interface or console.
NOTE: It seems like in many or even all cases, the automatically generated and sent passwords do never arrive at the list admins. For this reason it is recommended to manually set a password and to manually send it to the lists' admins, preferrably through secure communication (GPG or IRC with SSL encryption on both sides).
After the list has been created, go to the
list administration page, look for your newly created list and click it. Log in using the newly created password (if you happen to know it) or using the global admin password. Then enter in the list description and related list categories. Also check the other basic settings.
When you are done, make sure the list shows up well at
http://lists.indymedia.org.
Renaming Mailing Lists
Let's say the old name is "oldname" and the new name is "newname".
- cd to the directory where you've installed Mailman. Let's say it's /mailman, like in newsarai:
$ cd /mailman
and cd to the 'lists' subdirectory:
$ cd lists
You should now see the directory 'oldname'. Move this to 'newname':
$ mv oldname newname
- Now cd to the private archives directory:
$ cd ../archives/private
You will need to move the oldname's .mbox directory, and the .mbox file within that directory. Don't worry about the public archives; the next few steps will take care of them without requiring you to fiddle around in the file system:
$ mv oldname.mbox newname.mbox
$ mv newname.mbox/oldname.mbox newname.mbox/newname.mbox
- You should now regenerate the public archives:
$ sudo /usr/sbin/mmarch newname
- You'll likely need to change some of your list's configuration options, especially if you want to accept postings addressed to the old list on the new list. Visit the admin interface for your new list:
- Go to the General options
- Change the "real_name" option to reflect the new list's name, e.g. "Newname"
- Change the subject prefix to reflect the new list's name, e.g. "[Newname] " (yes, that's a trailing space character).
- Optionally, update other configuration fields like info, description, or welcome_msg. YMMV.
- Go to the Privacy options and to the Recipient filters section
- Add the old list's address to acceptable_aliases. E.g. "[13]oldname@dom.ain". This way, (after the /mailman/data/aliases changes described below) messages posted to the old list will not be held by the new list for "implicit destination" approval.
- Now you want to update the /mailman/data/aliases file to include the aliases for the new list, and forwards for the old list to the new list. Note that these instructions are for Sendmail style alias files, adjust to the specifics of how your MTA is set up.
$ sudo vi /mailman/data/aliases
- Find the lines defining the aliases for your old list's name
- Copy and paste them just below the originals.
- Change all the references of "oldname" to "newname" in the pasted stanza.
- Now change the targets of the original aliases to forward to the new aliases. When you're done, you will end up with /mailman/data/aliases entries like the following (YMMV):
# Forward the oldname list to the newname list
oldname: newname
oldname-request: newname-request
oldname-admin: newname-admin
oldname-owner: newname-owner
newname: "|/usr/local/mailman/mail/wrapper post newname"
newname-admin: "|/usr/local/mailman/mail/wrapper mailowner newname"
newname-request: "|/usr/local/mailman/mail/wrapper mailcmd newname"
newname-owner: newname-admin
$ sudo postalias /mailman/data/aliases
Removing Mailing Lists
We do not delete mailing lists, we usually temporarily remove, suspend (or 'graveyard') them, so they may be reactivated later on and their archives remain intact.
Temporary Removal of Mailing Lists
To graveyard a mailing list, run
$ sudo sh /usr/lib/mailman/bin/graveyard $MAILINGLIST
You need to replace $MAILINGLIST by the name of the list you wish to graveyard, of course. For example:
$ sudo sh /usr/lib/mailman/bin/graveyard imc-test-editorial
Permanent Removal of Mailing Lists
Follow the instructions given above at
Temporary Removal of Mailing Lists. You will be given a hint on how to permanently remove the list archives.
Restoring Mailing Lists
To bring a list back from the dead, use the following recipie:
Copy the old data from the saved lists directory to the live lists data directory:
$ sudo cp -pr /var/lib/mailman/saved-lists/thelist /var/lib/mailman/lists/thelist
Update the aliases:
$ sudo /usr/lib/mailman/bin/genaliases
Reload the aliases:
$ sudo postalias /var/lib/mailman/data/aliases
Resetting lost List Administration Passwords
Login to this lists'
administration interface using the global list admin password. You can then change the list admin password using the form in the passwords section. When you email the list admin with the password, be sure to use encrypted communication and or urge the list admin to change the password once they receive it.
from sarai : go to mailman dir (/var/lib/mailman) and issue :
$ bin/change_pw --listname=$name_of_list
new password should be automatically emailed to list-owners.
Archives
Removing a Post from the Archives
CURRENT archives
The list moderators have the option of hiding posts in the administrative interface. In case this is not sufficient (read: we don't want this post in our drives at all), one needs to edit the mailbox of the mailling lists at hand:
You either need sudo access on sarai or be in the 'list' group on sarai. You
need to have two terminals opens: one is used to lock the mailling list, the other to edit.
$ cd /var/lib/mailman
$ sudo bin/withlist $listname
Loading list $listname (unlocked)
Unknown list: lists
The variable `m' is the lists MailList instance
>>> m.Lock()
When the prompt is back after you've typed "m.Lock()", the list is locked, you can work on the second terminal.
- In the second terminal
- make a backup of /var/lib/mailman/archives/private/$listname.mbox/$listname.mbox
- edit it and change whatever is needed, save it.
- Back to the the first terminal:
>>> m.Unlock()
>>> ^D
- The list is now unlocked, just regenerate the archives:
$ sudo bin/arch --wipe $listname
OLD archives
This is when a post is somewhere at
http://archives.lists.indymedia.org only.
Just go to /mailman/archives.lists.indymedia.org and edit whatever is needed, there's no need to lock anything whatsoever.
Corrupted Archives
Every now an then, mailing list archives get corrupted. Some posts may be missing or split up into two posts or the archives simply stop at some point but it is known that there was more list traffic after this point and the archiving setting has remained activated for this list.
To fix this for
${MAILINGLIST}
and SOS ticket number
${SOSTICKET}
, you should do the following:
Login via ssh, then
# Create a new temporary directory and move the current archives there, so that you may revert this in case something goes wrong
mkdir /mailman/temp/ticket-${SOSTICKET}
mv /var/lib/mailman/archives/public/${MAILINGLIST}/database /mailman/temp/ticket-${SOSTICKET}/
# Now regenerate the archives, this will take a bit.
/usr/lib/mailman/bin/arch ${MAILINGLIST} /var/lib/mailman/archives/private/${MAILINGLIST}.mbox/${MAILINGLIST}.mbox
Now check whether the public list archives for said mailing list exist and look good:
http://lists.indymedia.org/pipermail/${MAILINGLIST}/ to make sure it worked out fine. If it didn't, you can
rm -rf /var/lib/mailman/archives/public/${MAILINGLIST}/database
and
mv /mailman/temp/ticket-${SOSTICKET}/database /var/lib/mailman/archives/public/${MAILINGLIST}/
to go back to where you started from. Please notify the other listworkers in this case.
If you're sure it worked out fine, you can safely remove the backups:
rm -rf /mailman/temp/ticket-${SOSTICKET}/
Subscribers
Determine List Subscriptions of a single user
Use the following shell command:
sudo /usr/lib/mailman/bin/find_member EMAIL_ADDRESS_AS_REGEX
You will need to replace EMAIL_ADDRESS_AS_REGEX by a (Python style) regular expression which will match the email address you are looking for. In case of
joe@foe.org you would use
sudo /usr/lib/mailman/bin/find_member joe@foo\.org
Running
sudo /usr/lib/mailman/bin/find_member --help
will give additional options for reducing and extending your search.
Removing subscribers from single, multiple or all lists
Use the following shell command:
sudo /usr/lib/mailman/bin/remove_members LISTNAME EMAIL_ADDRESS
You will need to replace EMAIL_ADDRESS by the email address you would like to remove from list defined by LISTNAME. In case of
joe@foe.org you would use
sudo /usr/lib/mailman/bin/remove_members tests joe@foo.org
To remove a subscriber from
all mailing lists hosted at lists.indymedia.org, use this shell command:
sudo /usr/lib/mailman/bin/remove_members --fromall EMAIL_ADDRESS
Again, replace EMAIL_ADDRESS by the email address of the subscriber you want to remove. Processing this command can take a couple of seconds, so stand by.
Running
sudo /usr/lib/mailman/bin/remove_members --help
will give additional options for mass removals and omitting acknowledgements.
Changing subscribers subscription addresses
Remove the member from all lists (without telling the user or admin):
sudo /usr/lib/mailman/bin/remove_members -N -n --fromall EMAIL_ADDRESS
Note down the lists that they were removed from and subscribe them to those:
echo EMAIL_ADDRESS > sub
for LIST in imc-foo imc-bar ; do
sudo /usr/lib/mailman/bin/add_members -w n -a n -r sub $LIST
done
Maintenance
Main List Info Page (%LISTWORK_URL_LISTPAGE%)
Adding Offsite Lists
To add a list to the
http://lists.indymedia.org page that is actually hosted on a different machine, simply run sudo /usr/lib/mailman/bin/addoff and answer the questions. This script will add an entry into the /etc/mailman/off-site-lists.xml file and then refresh the list of lists display.
Removing Offsite Lists
To remove an offsite List from the
http://lists.indymedia.org page, make a backup from /etc/mailman/off-site-lists.xml first. After this edit /etc/mailman/off-site-lists.xml and delete the rerfering part. Make sure you delete the whole entry between
<list name='$LISTNAME'>
and
</list>
Now be patient for a maximum of one hour and the
http://lists.indymedia.org page should be updated.
Jeez, the mail server sure has been acting unreliably lately. I want to email all the list admins and tell them so they don't worry too much. You can make a list of all the email addresses of all the listwork administrators by going into /var/lib/mailman and typing this:
sudo grep owner aliases | awk '{print $1}' | cut -d: -f 1
Moderation Queues / Pending Posts
Clearing all Messages out of a Lists' Moderation Queue
sarai$ rm ~mailman/data/heldmesg-listname-*
or, if need be:
sarai$ sudo -u mailman rm ~mailman/data/heldmesg-listname-*
After that, you'll need to load the list's admin page in a web browser in order to refresh the database.
Determining Lists which have the highest Number of held Messages
jb wrote a couple of scripts to help with this. All of them return a listing of mailing lists with pending moderation requests.
(The email addresses, server and account names were changed.)
ADMIN_STATUS sudo /usr/lib/mailman/bin/admin_status
Scope:
- UNKNOWN, but somehow limited (270 lines of output)
Returns:
- the number of pending moderation requests (for each list)
Ordered by:
- # of pending posts (descending)
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/admin_status | grep "imc-ireland-newswire"
84 imc-ireland-newswire
WARN_DORMANT sudo /usr/lib/mailman/bin/warn_dormant
Scope:
- UNKNOWN, but somehow limited (348 lines of output)
Returns:
- the number of pending moderation requests (for each list)
- the time span (in days) since the first unhandled moderation request was generated (for each list)
- the email address(es) of the list admin(s), or 'NO-OWNER' if there is none (for each list)
- in the end, it gives the total number of lists matching the scope
Ordered by:
- mailing list names (ascending)
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/warn_dormant | grep "imc-ireland-newswire"
imc-ireland-newswire 84 pending 235 days admin1@example.com,admin2@example.com
WARN_NOADMIN sudo /usr/lib/mailman/bin/warn_noadmin
Scope:
- ASSUMABLY all mailing lists (918 lines of output)
Returns:
- the number of pending moderation requests (for each list)
- in the end, it gives the total number of lists matching the scope
Ordered by:
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/warn_noadmin | grep "imc-ireland-newswire"
('imc-ireland-newswire', 84)
WARN_PENDING sudo /usr/lib/mailman/bin/warn_pending
Scope:
- UNKNOWN, but somehow limited (82 lines of output)
Returns:
- the number of pending moderation requests (for each list)
- the email address(es) of the list admin(s), or 'NO OWNER' if there is none (for each list)
- in the end, it gives the total number of lists matching the scope
Ordered by:
- mailing list names (ascending)
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/warn_pending | grep "imc-ireland-newswire"
imc-ireland-newswire: 84 pending, owners: admin1@example.com, admin2@example.com
List pending posts and warn admins
There's a new script ( bin/warn_pending ) that print who's not too well administering her/his list and can send messages to admins to tell them they should work a bit...
$ cd /var/lib/mailman
$ bin/warn_pending --help
Print stats about pending e-mails or warn admins.
Usage:
warn_pending [options]
Where:
--listname=listname
-l listname
Include only the named list in the search.
--exclude=listname
-x listname
Exclude the named list from the search.
--number N
-n N
Minimum number a pensing messages to have a warning.
Default is 50.
--send
-s
Actually send a warning to list owners. Default mode
is to print.
--from=address
-f address
Set the from header and the enveloppe address to something
different than mailman-bounces@. This is no-op without the
previous options.
...
So, the standard usage is:
# to print what list have more than 50 pending messages
$ sudo -u list bin/warn_pending
# to send a mail to these admins from me@indymedia.org (bounces
# and errors will go to me@
$ sudo -u list bin/warn_pending -s <b>-f me@indymedia.org</b>
-- replace me@ by listwork@ or something.
Clearing Messages stuck in the Mailman Queue
Another administrative task I sometimes do and sometimes forget to do.
The queue directory for mailman is located at
/var/lib/mailman/mailman/qfiles
This directory should usually only have messages 2 minutes old or less, and each one should have a corresponding .db file.
So it should look something like this:
/var/lib/mailman/qfiles] deanna@sarai$ ls -lt
total 300
drwxrwsr-x 2 root mailman 249856 May 1 17:53 ./
-rw-rw-r-- 1 mailman mailman 125 May 1 17:53 1901d219fddaeb68f8792f80d9a7811441cd4121.db
-rw-rw-r-- 1 mailman mailman 764 May 1 17:53 1901d219fddaeb68f8792f80d9a7811441cd4121.msg
-rw-rw-r-- 1 mailman mailman 122 May 1 17:53 c62490476131e380a35b6fca11dbbea050d62956.db
-rw-rw-r-- 1 mailman mailman 5644 May 1 17:53 c62490476131e380a35b6fca11dbbea050d62956.msg
-rw-rw-r-- 1 mailman mailman 125 May 1 17:53 c0c191aebac1f5b6c18996ce488ffc7db72fae0f.db
-rw-rw-r-- 1 mailman mailman 784 May 1 17:53 c0c191aebac1f5b6c18996ce488ffc7db72fae0f.msg
-rw-rw-r-- 1 mailman mailman 122 May 1 17:53 033585956ccfb90317c56e095c7a8ef4511d791c.db
-rw-rw-r-- 1 mailman mailman 5667 May 1 17:53 033585956ccfb90317c56e095c7a8ef4511d791c.msg
-rw-rw-r-- 1 mailman mailman 125 May 1 17:53 af189dec68a4bfac9d5f4eefba09863b6001cfce.db
-rw-rw-r-- 1 mailman mailman 1831 May 1 17:53 af189dec68a4bfac9d5f4eefba09863b6001cfce.msg
drwxrwsr-x 22 mailman mailman 4096 Apr 29 16:44 ../
But for reasons unknown to me, sometimes there are orphaned *.msg files in there, or ones with corrupt or 0-byte .db files.
So periodically the directory should be checked for these. These are message that have not been delivered. Take a look at the file with cat or grep or whatever and find out who it was meant for, or just do this:
$ grep ^Delivered-To af189dec68a4bfac9d5f4eefba09863b6001cfce.msg
Which should give you:
Delivered-To: listwork@indymedia.org
or something similar.
So now you must re-send the message, like so:
$ cat af189dec68a4bfac9d5f4eefba09863b6001cfce.msg | /usr/lib/sendmail listwork@lists.indymedia.org
Important
If the Delivered-To header has @lists.indymedia.org, you will have to redeliver to @indymedia.org, and vice versa. If you don't pay attention to this the mail will bounce and be lost. This is to get around postfix's attempt to prevent mail loops.
You should then be able to safely delete the message, as it is now being re-processed.
Hope this makes some sense.
Trouble Shooting
Stuck Traffic on Lists
I'm an impatient sort of fellow lately and I noticed some mail I wrote wasn't getting through to a list. And since what I have to say is important I went and had a look and it seems there were stale lock files in ~mailman/locks which was preventing mail from being delivered for about 11 hours today. I presume this correlates with sarai being rebooted. There were about 1000 messages in the qfiles directory.
Anyway, if list traffic isn't flowing take a look in the locks directory. If there are lock files present, look for the pid they indicate. If the PID doesn't exist, it is a stale lock. In that case, feel free to remove it.
Switched Lists
There's some strange thing at work: every now and then lists get switched: imc-latina becomes imc-mineapolis radio, maritimes becomes chiapas, etc. There's probably something wrong with a hash method used by mailman (or maybe our listwork kiddies do drink-typing B) ), well: a method to get things back to normal is to:
* see when did the list work fine; * get the config.db from the associated backup: (on kropotkin)
tar zxvf ~mailman/mailman.[date].tar.gz mailman/lists/[list name]/config.db
copy this config.db to sarai:~mailman/lists/[list name]/config.db.
Unfortunately, some changes may be lost when one's doing that (subscriptions, options/passwd changes, etc).
NotAMemberError / Sticky users
If you get a message from cron like the following or a NotAMemberError from the upgrade scripts, this means you have a sticky member (a bogus email address in only parts of the membership list).
Date: Fri, 2 Apr 2004 09:02:22 -0800 (PST)
From: Cron Daemon <root@sarai.indymedia.org>
To: list@sarai.indymedia.org
Subject: Cron <list@sarai> /usr/lib/mailman/cron/disabled
Traceback (most recent call last):
File "/usr/lib/mailman/cron/disabled", line 219, in ?
main()
File "/usr/lib/mailman/cron/disabled", line 162, in main
if mlist.getDeliveryStatus(member) <> MemberAdaptor.ENABLED:
File "/var/lib/mailman/Mailman/OldStyleMemberships.py", line 139, in getDeliveryStatus
self.__assertIsMember(member)
File "/var/lib/mailman/Mailman/OldStyleMemberships.py", line 113, in __assertIsMember
raise Errors.NotAMemberError, member
Mailman.Errors.NotAMemberError: someuser@somedomain.sometld
Bogus e-mail addresses can get in Mailman, but it is hard to get them out. For long, we had Mailman sending root@indy warnings because joe@foo@.ca was messing things. No way to remove joe@ with standard tools. Joe@ was removed with bin/withlist on imc-maritimes-news:
$ cd /var/lib/mailman
$ sudo -u list bin/withlist imc-maritimes-news
Loading list imc-maritimes-news (unlocked)
The variable `m' is the imc-maritimes-news MailList instance
>>> m.Lock()
>>> del m.members['joe@foo@.ca']
>>> del m.user_options['joe@foo@.ca']
>>> del m.bounce_info['joe@foo@.ca']
>>> del m.delivery_status['joe@foo@.ca']
>>> del m.passwords['joe@foo@.ca']
>>> del m.usernames['joe@foo@.ca']
>>> m.Save()
>>> m.Unlock()
>>> ^D # hit Control-D here.
and joe@ is gone.
You can also remove joe@foo@.ca using the following:
$ cd /var/lib/mailman
$ sudo -u list bin/rmsticky imc-maritimes-news 'joe@foo@.ca'
If this does not work it means there are some weird characters. Run this:
$ sudo /usr/lib/mailman/bin/dumpdb somelist > pickle.txt
$ less pickle.txt
At this point, search for the offending characters, copy them out and run this:
$ sudo /usr/lib/mailman/bin/rmsticky -i somelist 'hammam2@rpi.edu \t\xa0'
Good to know
A bunch of little items that are helpful to know.
Listinfo Pages are cached
Pages are served statically, the following file is cron'd to run every 30 minutes but can also be run manually.
/usr/local/bin/listinfo_cache.pl
Patching Mailman
Patches
Some patches we use at Indymedia are available at
http://lists.indymedia.org/patches/ and
http://lists.indymedia.org/patches.tar.gz
The first url is the patches we have currently installed (minus some fixes). The second url contains the patches that we intend to install when we next upgrade mailman.
These locations may change. You will probbaly run into questions and should send an email to the listwork mailing list if you intend to use any of these patches. Some of the patches depend on other ones to do their work.
See also
MailmanContr - Indymedia contributions to the Mailman software.
Many more patches (not by Indymedia) are at
http://sourceforge.net/tracker/?atid=300103&group_id=103
Applying patches
Copy the .dpatch files from above into debian/patches/ in your unpacked debian mailman source directory and edit debian/patches/00list, adding the names of the patches you will use. Then build the package as normal. Depending on the patches you use, you may need to edit patches like so (for example) to change what other patches one depends on, or to correct conflicts:
$ dpatch-edit-patch imc-00-indymedia-changelog
Email Forwarders
Adding, Editing, Deleting
You need sudo access on sarai.indy to do this. We have setup a script which will make it easy for you to edit the file which contains the mappings defining the email forwarding:
$ sudo /usr/local/sbin/editalias
You are now editing a Postfix config file, namely
/etc/postfix/maps/virtual_indymedia
.
* To
add a new forwarding address, page down to the bottom of the file and press
i
to put an entry like this:
# Requested by Che from IMC Chile, added by Jane 1 April 2006
# OR:
# In response to SoS ticket #2006032900034
foo@indymedia.org: fred@nothotmail.com
This means that email sent to foo@indymedia.org will be forwarded to fred@nothotmail.com. Make sure you add a note as given in the above example (lines starting with # are notes) on what was changed so we have a bit of documentation.
* To
edit or to
delete an entry, use
/
(the slash) to enter search mode, type the email address you want to change, press enter. If the first result is not what you were looking for, press
n
until you end up at the entry you need to change or remove. Then press
i
and edit or remove the entry in question.
If you're changing an entry, make sure you add a note as given in the above example (lines starting with # are notes) on what was changed so we have a bit of documentation. However, if you're deleting it, you should also delete all of its backlog and not leave a comment.
When you finished editing (by pressing escape, then typing ":x" and pressing enter), the script will automatically run two more commands ("postmap /etc/postfix/maps/virtual_indymedia" and "/etc/init.d/postfix force-reload") in the background. That's it.
Listing
To get a plain alphabetically sorted list of all the currently setup email forwarders, use this command:
cat /etc/postfix/maps/virtual_indymedia | grep "." | grep -v "^#.*" | sort | less
Counting
To find out how many email forwarders are currently setup, use
cat /etc/postfix/maps/virtual_indymedia | grep "." | grep -v "^#.*" | wc -l
Spam
Checking the Record Spam Score
zgrep -i 'discarded: SpamAssassin score was ' /var/log/mailman/vette* \
| sed -e 's/^.*discarded: SpamAssassin score was *\([0-9.]*\).*/\1/g' \
| sort -n -r -u |less
Checking spamd bayes db
how to see which list has the most spams in its bayes db (top 10):
mysql> select id,username,spam_count,ham_count,token_count from bayes_vars order by spam_count desc limit 10;
+-----+--------------------+------------+-----------+-------------+
| id | username | spam_count | ham_count | token_count |
+-----+--------------------+------------+-----------+-------------+
| ... | ...
change
order by
and
limit
values for other results.
Adding spam or ham to a list's bayes db
what we want to do is pipe the full message into
sa-learn
(this will add the message to the mysql backend that spamd uses)
if we want to use an email from the list's archive, one good way to do this is to open the mbox archive with mutt like this:
mutt -R -f /var/lib/mailman/archive/private/$listname/$listname.mbox
(the -R is a good idea because we want to read the mbox read-only so we don't make silly changes to the archive)
then, select the message and use the
|
(pipe) command to pipe the message into sa-learn. you'll want to give it the following arguments:
sa-learn -u $listname --[spam|ham]
(use the actual listname, and decide whether the message is spam or ham. if the message scored outside of the autolearn threshhold then it will have already been learned, and sa-learn will have no effect)
Blacklisting Email Addresses
There are two blacklist mechanisms before email hits mailman or is processed otherwise (email aliases etc). The first one is Postfix' access checks. The second one is SpamAssassin. It is better to blacklist these at the MTA level, i.e. using Postfix' access checks, because SpamAssassin is slower and this way less email needs to be processed on its way from the MTA to SpamAssassin.
Postfix Access Checks
However, Postfix does not offer much to test against. It is still enough in some cases, though. Postfix allows you to reject email based on the MAIL FROM address as well as parts thereof. The domain part in the MAIL FROM line is resolved to IP addresses before the checks are applied, so you can also put IP addresses.
For example,
69.22.169.128 REJECT
would reject all email sent by any mail server where a MAIL FROM line contains either
69.22.169.128
or a FQDN resolving to this IP address.
To add, change, or remove an entry to the blacklist, edit
/etc/postfix/checks/access
, then run
sudo postmap /etc/postfix/checks/access
and
sudo /etc/init.d/postfix reload
.
To test your rules, you can do:
postmap -q 69.22.169.128 hash:/etc/postfix/checks/access
which should echo
REJECT
if your rule matches (and you didn't forget to update the access.db file).
You can also whitelist email addresses (or domains) appearing in the MAIL FROM line by appending
OK
the email address. For the exact format, just have a look at the other lines in this file.
SpamAssassin Rules
In those cases where Postfix' simple and dirty blacklisting doesn't fit, SpamAssassin is the way to go. In addition to its fine-grained blacklist, SpamAssassin also offers a whitelist. A really long whitelist is compiled regularly by a cron job (initially setup by Deanna) which is supposed to encompass all the people subscribed to all indymedia lists (this is pretty old info, may be outdated).
This is what the SpamAssassin blacklist looked like in 2002:
blacklist_from *@the-financial-news.com (this is pretty old info, may be outdated)
# this is the "Unlimited Free Calling SPAM" - christopher 4 april 2002
blacklist_from gintare@netzero.net
The list is stored in
/etc/mail/spamassassin/local.cf
. After editing it, run
sudo /etc/init.d/spamassassin restart
If you have questions, see
man Mail::SpamAssassin::Conf
. Free (as in hallicugenic herbal potions) SpamAssassin rulesets wizardry courses are also available at
the SpamAssassin wiki on Apache.org and
the Exit0 wiki. Readymade and regularly updated magic spells are available at
The Rules Emporium as well as on
the SpamAssassin wiki.
Stop Spam in Progress by Emergency Procmail Rules
Here's something I do from time to time if a spammer has hijacked our system. Since we have process limits set and spamassassin only does one message at a time, one spammer can tie us up for long periods of time. If you're on the shell and want to do something about it:
Add a rule to /etc/procmailrc-spamtrap and /etc/procmailrc-spamd with their address in it, like this:
:0
* ^From.Marketingwap@millenniumdata.com
/dev/null
Then kill all the running procmail processes, which will cause procmail to requeue the mail and subject it to the new rules.
$ for proc in `ps auxw | grep [p]rocmail | awk '{print $2}'`; \
do sudo kill $proc ; echo $proc ; done
Now watch them all get sent to the trash in /var/procmail/procmail.log.
In /etc/postfix/checks/access put the user@address or domain on a line
followed by 'OK' then run
sudo postmap /etc/postfix/checks/access
. You can take a look at other entries there (they are located at the bottom of the file) for clues to the format, or
man access
on sarai.
You might also want to add them to the spamassassin whitelist in
/etc/mail/spamassassin/*.cf
in case the SA checks on the same blocklists cause them to get tagged as spam. We should probably zero out all those checks that we're doing in postfix in spamassassin so that they're not run twice.
Quick Menu:
--
ChristopherMitchell - 01 Jun 2002
--
GaBa - 01 Jun 2002 - Traducciones
--
LuiS - 09 Jun 2003 - Updated the command paths to be used in newsarai
--
LuiS - 12 Oct 2003 - Moved the FAQ section to
ListWorkFAQ
--
ChrisC - 28 Oct 2003 -- added stuff about new script for editing mbox files
--
PaulWise - 13 Jan 2004 --
--
JiBe - 02 Feb 2004 -- Added stuff about joe@foo@.ca and
delpost
--
AlsteR - 10 Dec 2004 -- Updated list creation hints
--
AlsteR - 03 Jan 2005 -- Added find_member and remove_members
--
AlsteR - 04 Jan 2005 -- Reorganized the topic
--
PaulWise - 05 Jan 2005 -- Fix some typos and stuff
--
AlsteR - 05 Sep 2005 -- Reordered this page so you don't find what you're looking for
--
AlsteR - 02 May 2006 -- Updated email forwarding addresses ('aliases') how-to
--
AnA - 19 Nov 2006 -- updated the file where old archives are - edit archives section
--
AlsteR - 14 Dec 2006 -- moved and rewrote instructions for fixing broken archives
--
AnthonyG - 22 Dec 2006 -- removed reference to sending a message to the listwork discussion list after a new list has been created (the
SoS system takes care of that now)
--
CaD - 12 Jun 2008 added "Removing offsite List"