Listwork How-To

About this Document

This document is meant as a technical reference for members of the listwork working group.
Are looking for help with administration, moderation or use of single mailing lists? Please refer to the Listwork FAQ.
Are you looking for information about the mail server itself? SaraiInfo has it.

To keep track of changes to this and the other documents ('topics', i.e. ListWorkHowto) within this TWiki web (i.e. Sysadmin), please subscribe to the notifications at WebNotify. Some statistics on changes of this and other pages are available at WebStatistics.


Mailing Lists

Creation, Renaming, Suspension, Removal, Resetting Passwords

Creating Mailing Lists

The easiest way to create new lists is through Mailman's list creation web interface.

However, if you prefer console, use this instead: $ sudo /usr/sbin/newlist

Aliases will be created automagically whether you use web interface or console.

ALERT! NOTE: It seems like in many or even all cases, the automatically generated and sent passwords do never arrive at the list admins. For this reason it is recommended to manually set a password and to manually send it to the lists' admins, preferrably through secure communication (GPG or IRC with SSL encryption on both sides).

After the list has been created, go to the list administration page, look for your newly created list and click it. Log in using the newly created password (if you happen to know it) or using the global admin password. Then enter in the list description and related list categories. Also check the other basic settings.

When you are done, make sure the list shows up well at http://lists.indymedia.org.

Renaming Mailing Lists

Let's say the old name is "oldname" and the new name is "newname".

- cd to the directory where you've installed Mailman. Let's say it's /mailman, like in newsarai:

$ cd /mailman

and cd to the 'lists' subdirectory:

$ cd lists

You should now see the directory 'oldname'. Move this to 'newname':

$ mv oldname newname

- Now cd to the private archives directory:

$ cd ../archives/private

You will need to move the oldname's .mbox directory, and the .mbox file within that directory. Don't worry about the public archives; the next few steps will take care of them without requiring you to fiddle around in the file system:

$ mv oldname.mbox newname.mbox

$ mv newname.mbox/oldname.mbox newname.mbox/newname.mbox

- You should now regenerate the public archives:

$ sudo /usr/sbin/mmarch newname

- You'll likely need to change some of your list's configuration options, especially if you want to accept postings addressed to the old list on the new list. Visit the admin interface for your new list:

  • Go to the General options

  • Change the "real_name" option to reflect the new list's name, e.g. "Newname"

  • Change the subject prefix to reflect the new list's name, e.g. "[Newname] " (yes, that's a trailing space character).

  • Optionally, update other configuration fields like info, description, or welcome_msg. YMMV.

  • Save your changes

  • Go to the Privacy options and to the Recipient filters section

  • Add the old list's address to acceptable_aliases. E.g. "[13]oldname@dom.ain". This way, (after the /mailman/data/aliases changes described below) messages posted to the old list will not be held by the new list for "implicit destination" approval.

  • Save your changes
- Now you want to update the /mailman/data/aliases file to include the aliases for the new list, and forwards for the old list to the new list. Note that these instructions are for Sendmail style alias files, adjust to the specifics of how your MTA is set up.

  • Run
$ sudo vi /mailman/data/aliases

  • Find the lines defining the aliases for your old list's name
  • Copy and paste them just below the originals.
  • Change all the references of "oldname" to "newname" in the pasted stanza.
  • Now change the targets of the original aliases to forward to the new aliases. When you're done, you will end up with /mailman/data/aliases entries like the following (YMMV):

      # Forward the oldname list to the newname list
      oldname:         newname
      oldname-request: newname-request
      oldname-admin:   newname-admin
      oldname-owner:   newname-owner

      newname:         "|/usr/local/mailman/mail/wrapper post newname"
      newname-admin:   "|/usr/local/mailman/mail/wrapper mailowner newname"
      newname-request: "|/usr/local/mailman/mail/wrapper mailcmd newname"
      newname-owner:   newname-admin

  • Finally, run
$ sudo postalias /mailman/data/aliases

Removing Mailing Lists

We do not delete mailing lists, we usually temporarily remove, suspend (or 'graveyard') them, so they may be reactivated later on and their archives remain intact.

Temporary Removal of Mailing Lists

To graveyard a mailing list, run

$ sudo sh /usr/lib/mailman/bin/graveyard $MAILINGLIST

You need to replace $MAILINGLIST by the name of the list you wish to graveyard, of course. For example:

$ sudo sh /usr/lib/mailman/bin/graveyard imc-test-editorial

Permanent Removal of Mailing Lists

Follow the instructions given above at Temporary Removal of Mailing Lists. You will be given a hint on how to permanently remove the list archives.

Restoring Mailing Lists

To bring a list back from the dead, use the following recipie:

Copy the old data from the saved lists directory to the live lists data directory:

$ sudo cp -pr /var/lib/mailman/saved-lists/thelist /var/lib/mailman/lists/thelist

Update the aliases:

$ sudo /usr/lib/mailman/bin/genaliases

Reload the aliases:

$ sudo postalias /var/lib/mailman/data/aliases

Resetting lost List Administration Passwords

Login to this lists' administration interface using the global list admin password. You can then change the list admin password using the form in the passwords section. When you email the list admin with the password, be sure to use encrypted communication and or urge the list admin to change the password once they receive it.

from sarai : go to mailman dir (/var/lib/mailman) and issue :

$ bin/change_pw --listname=$name_of_list

new password should be automatically emailed to list-owners.

Archives

Removing a Post from the Archives

CURRENT archives

The list moderators have the option of hiding posts in the administrative interface. In case this is not sufficient (read: we don't want this post in our drives at all), one needs to edit the mailbox of the mailling lists at hand:

You either need sudo access on sarai or be in the 'list' group on sarai. You need to have two terminals opens: one is used to lock the mailling list, the other to edit.

  • In the first terminal:

      $ cd /var/lib/mailman
      $ sudo bin/withlist $listname
      Loading list $listname (unlocked)
      Unknown list: lists
      The variable `m' is the lists MailList instance
      >>> m.Lock()

When the prompt is back after you've typed "m.Lock()", the list is locked, you can work on the second terminal.

  • In the second terminal
    • make a backup of /var/lib/mailman/archives/private/$listname.mbox/$listname.mbox
    • edit it and change whatever is needed, save it.

  • Back to the the first terminal:

      >>> m.Unlock()
      >>> ^D

  • The list is now unlocked, just regenerate the archives:
      $ sudo bin/arch --wipe $listname

OLD archives

This is when a post is somewhere at http://archives.lists.indymedia.org only.

Just go to /mailman/archives.lists.indymedia.org and edit whatever is needed, there's no need to lock anything whatsoever.

Corrupted Archives

Every now an then, mailing list archives get corrupted. Some posts may be missing or split up into two posts or the archives simply stop at some point but it is known that there was more list traffic after this point and the archiving setting has remained activated for this list.

To fix this for ${MAILINGLIST} and SOS ticket number ${SOSTICKET}, you should do the following:

Login via ssh, then
# Create a new temporary directory and move the current archives there, so that you may revert this in case something goes wrong
mkdir /mailman/temp/ticket-${SOSTICKET}
mv /var/lib/mailman/archives/public/${MAILINGLIST}/database /mailman/temp/ticket-${SOSTICKET}/
# Now regenerate the archives, this will take a bit.
/usr/lib/mailman/bin/arch ${MAILINGLIST} /var/lib/mailman/archives/private/${MAILINGLIST}.mbox/${MAILINGLIST}.mbox

Now check whether the public list archives for said mailing list exist and look good: http://lists.indymedia.org/pipermail/${MAILINGLIST}/ to make sure it worked out fine. If it didn't, you can rm -rf /var/lib/mailman/archives/public/${MAILINGLIST}/database and mv /mailman/temp/ticket-${SOSTICKET}/database /var/lib/mailman/archives/public/${MAILINGLIST}/ to go back to where you started from. Please notify the other listworkers in this case.

If you're sure it worked out fine, you can safely remove the backups:
rm -rf /mailman/temp/ticket-${SOSTICKET}/

Subscribers

Determine List Subscriptions of a single user

Use the following shell command: sudo /usr/lib/mailman/bin/find_member EMAIL_ADDRESS_AS_REGEX

You will need to replace EMAIL_ADDRESS_AS_REGEX by a (Python style) regular expression which will match the email address you are looking for. In case of joe@foe.org you would use sudo /usr/lib/mailman/bin/find_member joe@foo\.org

Running sudo /usr/lib/mailman/bin/find_member --help

will give additional options for reducing and extending your search.

Removing subscribers from single, multiple or all lists

Use the following shell command:

sudo /usr/lib/mailman/bin/remove_members LISTNAME EMAIL_ADDRESS

You will need to replace EMAIL_ADDRESS by the email address you would like to remove from list defined by LISTNAME. In case of joe@foe.org you would use

sudo /usr/lib/mailman/bin/remove_members tests joe@foo.org

To remove a subscriber from all mailing lists hosted at lists.indymedia.org, use this shell command:

sudo /usr/lib/mailman/bin/remove_members --fromall EMAIL_ADDRESS

Again, replace EMAIL_ADDRESS by the email address of the subscriber you want to remove. Processing this command can take a couple of seconds, so stand by.

Running

sudo /usr/lib/mailman/bin/remove_members --help

will give additional options for mass removals and omitting acknowledgements.

Changing subscribers subscription addresses

Remove the member from all lists (without telling the user or admin):

sudo /usr/lib/mailman/bin/remove_members -N -n --fromall EMAIL_ADDRESS

Note down the lists that they were removed from and subscribe them to those:

echo EMAIL_ADDRESS > sub
for LIST in imc-foo imc-bar ; do
   sudo /usr/lib/mailman/bin/add_members -w n -a n -r sub $LIST
done

Maintenance

Main List Info Page (%LISTWORK_URL_LISTPAGE%)

Adding Offsite Lists

To add a list to the http://lists.indymedia.org page that is actually hosted on a different machine, simply run sudo /usr/lib/mailman/bin/addoff and answer the questions. This script will add an entry into the /etc/mailman/off-site-lists.xml file and then refresh the list of lists display.

Removing Offsite Lists

To remove an offsite List from the http://lists.indymedia.org page, make a backup from /etc/mailman/off-site-lists.xml first. After this edit /etc/mailman/off-site-lists.xml and delete the rerfering part. Make sure you delete the whole entry between
<list name='$LISTNAME'>
and
</list>
Now be patient for a maximum of one hour and the http://lists.indymedia.org page should be updated.

Contacting all List Admins

Jeez, the mail server sure has been acting unreliably lately. I want to email all the list admins and tell them so they don't worry too much. You can make a list of all the email addresses of all the listwork administrators by going into /var/lib/mailman and typing this:

 sudo grep owner aliases | awk '{print $1}' | cut -d: -f 1 

Moderation Queues / Pending Posts

Clearing all Messages out of a Lists' Moderation Queue

sarai$ rm ~mailman/data/heldmesg-listname-*

or, if need be:

sarai$ sudo -u mailman rm ~mailman/data/heldmesg-listname-*

After that, you'll need to load the list's admin page in a web browser in order to refresh the database.

Determining Lists which have the highest Number of held Messages

jb wrote a couple of scripts to help with this. All of them return a listing of mailing lists with pending moderation requests.

(The email addresses, server and account names were changed.)

ADMIN_STATUS
sudo /usr/lib/mailman/bin/admin_status
Scope:
  • UNKNOWN, but somehow limited (270 lines of output)
Returns:
  • the number of pending moderation requests (for each list)
Ordered by:
  • # of pending posts (descending)
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/admin_status | grep "imc-ireland-newswire"
        84      imc-ireland-newswire

WARN_DORMANT
sudo /usr/lib/mailman/bin/warn_dormant
Scope:
  • UNKNOWN, but somehow limited (348 lines of output)
Returns:
  • the number of pending moderation requests (for each list)
  • the time span (in days) since the first unhandled moderation request was generated (for each list)
  • the email address(es) of the list admin(s), or 'NO-OWNER' if there is none (for each list)
  • in the end, it gives the total number of lists matching the scope
Ordered by:
  • mailing list names (ascending)
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/warn_dormant | grep "imc-ireland-newswire"
imc-ireland-newswire 84 pending 235 days admin1@example.com,admin2@example.com

WARN_NOADMIN
sudo /usr/lib/mailman/bin/warn_noadmin
Scope:
  • ASSUMABLY all mailing lists (918 lines of output)
Returns:
  • the number of pending moderation requests (for each list)
  • in the end, it gives the total number of lists matching the scope
Ordered by:
  • no particular order
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/warn_noadmin | grep "imc-ireland-newswire"
('imc-ireland-newswire', 84)

WARN_PENDING
sudo /usr/lib/mailman/bin/warn_pending
Scope:
  • UNKNOWN, but somehow limited (82 lines of output)
Returns:
  • the number of pending moderation requests (for each list)
  • the email address(es) of the list admin(s), or 'NO OWNER' if there is none (for each list)
  • in the end, it gives the total number of lists matching the scope
Ordered by:
  • mailing list names (ascending)
Example output:
me@mailmanhost:~$ sudo /usr/lib/mailman/bin/warn_pending | grep "imc-ireland-newswire"
imc-ireland-newswire: 84 pending, owners: admin1@example.com, admin2@example.com

List pending posts and warn admins

There's a new script ( bin/warn_pending ) that print who's not too well administering her/his list and can send messages to admins to tell them they should work a bit...

$ cd /var/lib/mailman
$ bin/warn_pending --help
Print stats about pending e-mails or warn admins.

Usage:
    warn_pending [options]

Where:
    --listname=listname
    -l listname
        Include only the named list in the search.

    --exclude=listname
    -x listname
        Exclude the named list from the search.

    --number N
    -n N
        Minimum number a pensing messages to have a warning.
        Default is 50.

    --send
    -s
        Actually send a warning to list owners.  Default mode
        is to print.

    --from=address
    -f address
        Set the from header and the enveloppe address to something
        different than mailman-bounces@.  This is no-op without the
        previous options.
 ...

So, the standard usage is:

# to print what list have more than 50 pending messages
$ sudo -u list bin/warn_pending    

# to send a mail to these admins from me@indymedia.org (bounces
# and errors will go to me@
$ sudo -u list bin/warn_pending -s <b>-f me@indymedia.org</b>
   -- replace me@ by listwork@ or something.

Clearing Messages stuck in the Mailman Queue

Another administrative task I sometimes do and sometimes forget to do.

The queue directory for mailman is located at

/var/lib/mailman/mailman/qfiles

This directory should usually only have messages 2 minutes old or less, and each one should have a corresponding .db file.

So it should look something like this:

 /var/lib/mailman/qfiles] deanna@sarai$ ls -lt
 total 300
 drwxrwsr-x    2 root     mailman    249856 May  1 17:53 ./
 -rw-rw-r--    1 mailman  mailman       125 May  1 17:53 1901d219fddaeb68f8792f80d9a7811441cd4121.db
 -rw-rw-r--    1 mailman  mailman       764 May  1 17:53 1901d219fddaeb68f8792f80d9a7811441cd4121.msg
 -rw-rw-r--    1 mailman  mailman       122 May  1 17:53 c62490476131e380a35b6fca11dbbea050d62956.db
 -rw-rw-r--    1 mailman  mailman      5644 May  1 17:53 c62490476131e380a35b6fca11dbbea050d62956.msg
 -rw-rw-r--    1 mailman  mailman       125 May  1 17:53 c0c191aebac1f5b6c18996ce488ffc7db72fae0f.db
 -rw-rw-r--    1 mailman  mailman       784 May  1 17:53 c0c191aebac1f5b6c18996ce488ffc7db72fae0f.msg
 -rw-rw-r--    1 mailman  mailman       122 May  1 17:53 033585956ccfb90317c56e095c7a8ef4511d791c.db
 -rw-rw-r--    1 mailman  mailman      5667 May  1 17:53 033585956ccfb90317c56e095c7a8ef4511d791c.msg
 -rw-rw-r--    1 mailman  mailman       125 May  1 17:53 af189dec68a4bfac9d5f4eefba09863b6001cfce.db
 -rw-rw-r--    1 mailman  mailman      1831 May  1 17:53 af189dec68a4bfac9d5f4eefba09863b6001cfce.msg
 drwxrwsr-x   22 mailman  mailman      4096 Apr 29 16:44 ../

But for reasons unknown to me, sometimes there are orphaned *.msg files in there, or ones with corrupt or 0-byte .db files.

So periodically the directory should be checked for these. These are message that have not been delivered. Take a look at the file with cat or grep or whatever and find out who it was meant for, or just do this:

 
 $ grep ^Delivered-To af189dec68a4bfac9d5f4eefba09863b6001cfce.msg

Which should give you:

 Delivered-To: listwork@indymedia.org

or something similar.

So now you must re-send the message, like so:

 
 $ cat af189dec68a4bfac9d5f4eefba09863b6001cfce.msg | /usr/lib/sendmail listwork@lists.indymedia.org

Important

If the Delivered-To header has @lists.indymedia.org, you will have to redeliver to @indymedia.org, and vice versa. If you don't pay attention to this the mail will bounce and be lost. This is to get around postfix's attempt to prevent mail loops.

You should then be able to safely delete the message, as it is now being re-processed.

Hope this makes some sense. smile

Trouble Shooting

Stuck Traffic on Lists

I'm an impatient sort of fellow lately and I noticed some mail I wrote wasn't getting through to a list. And since what I have to say is important I went and had a look and it seems there were stale lock files in ~mailman/locks which was preventing mail from being delivered for about 11 hours today. I presume this correlates with sarai being rebooted. There were about 1000 messages in the qfiles directory.

Anyway, if list traffic isn't flowing take a look in the locks directory. If there are lock files present, look for the pid they indicate. If the PID doesn't exist, it is a stale lock. In that case, feel free to remove it.

Switched Lists

There's some strange thing at work: every now and then lists get switched: imc-latina becomes imc-mineapolis radio, maritimes becomes chiapas, etc. There's probably something wrong with a hash method used by mailman (or maybe our listwork kiddies do drink-typing B) ), well: a method to get things back to normal is to:

* see when did the list work fine; * get the config.db from the associated backup: (on kropotkin)

 tar zxvf ~mailman/mailman.[date].tar.gz mailman/lists/[list name]/config.db 

copy this config.db to sarai:~mailman/lists/[list name]/config.db.

Unfortunately, some changes may be lost when one's doing that (subscriptions, options/passwd changes, etc).

NotAMemberError / Sticky users

If you get a message from cron like the following or a NotAMemberError from the upgrade scripts, this means you have a sticky member (a bogus email address in only parts of the membership list).

Date: Fri,  2 Apr 2004 09:02:22 -0800 (PST)
From: Cron Daemon <root@sarai.indymedia.org>
To: list@sarai.indymedia.org
Subject: Cron <list@sarai> /usr/lib/mailman/cron/disabled

Traceback (most recent call last):
  File "/usr/lib/mailman/cron/disabled", line 219, in ?
    main()
  File "/usr/lib/mailman/cron/disabled", line 162, in main
    if mlist.getDeliveryStatus(member) <> MemberAdaptor.ENABLED:
  File "/var/lib/mailman/Mailman/OldStyleMemberships.py", line 139, in getDeliveryStatus
    self.__assertIsMember(member)
  File "/var/lib/mailman/Mailman/OldStyleMemberships.py", line 113, in __assertIsMember
    raise Errors.NotAMemberError, member
Mailman.Errors.NotAMemberError: someuser@somedomain.sometld

Bogus e-mail addresses can get in Mailman, but it is hard to get them out. For long, we had Mailman sending root@indy warnings because joe@foo@.ca was messing things. No way to remove joe@ with standard tools. Joe@ was removed with bin/withlist on imc-maritimes-news:

$ cd /var/lib/mailman
$ sudo -u list bin/withlist imc-maritimes-news
Loading list imc-maritimes-news (unlocked)
The variable `m' is the imc-maritimes-news MailList instance
>>> m.Lock()
>>> del m.members['joe@foo@.ca']
>>> del m.user_options['joe@foo@.ca']
>>> del m.bounce_info['joe@foo@.ca']
>>> del m.delivery_status['joe@foo@.ca']
>>> del m.passwords['joe@foo@.ca']
>>> del m.usernames['joe@foo@.ca']
>>> m.Save()
>>> m.Unlock()
>>> ^D            # hit Control-D here.

and joe@ is gone.

You can also remove joe@foo@.ca using the following:

$ cd /var/lib/mailman
$ sudo -u list bin/rmsticky imc-maritimes-news 'joe@foo@.ca'

If this does not work it means there are some weird characters. Run this:

$ sudo /usr/lib/mailman/bin/dumpdb somelist > pickle.txt
$ less pickle.txt

At this point, search for the offending characters, copy them out and run this:

$ sudo /usr/lib/mailman/bin/rmsticky -i somelist 'hammam2@rpi.edu \t\xa0'

Good to know

A bunch of little items that are helpful to know.

Listinfo Pages are cached

Pages are served statically, the following file is cron'd to run every 30 minutes but can also be run manually.

/usr/local/bin/listinfo_cache.pl

Patching Mailman

Patches

Some patches we use at Indymedia are available at http://lists.indymedia.org/patches/ and http://lists.indymedia.org/patches.tar.gz

The first url is the patches we have currently installed (minus some fixes). The second url contains the patches that we intend to install when we next upgrade mailman.

These locations may change. You will probbaly run into questions and should send an email to the listwork mailing list if you intend to use any of these patches. Some of the patches depend on other ones to do their work.

See also MailmanContr - Indymedia contributions to the Mailman software.

Many more patches (not by Indymedia) are at http://sourceforge.net/tracker/?atid=300103&group_id=103

Applying patches

Copy the .dpatch files from above into debian/patches/ in your unpacked debian mailman source directory and edit debian/patches/00list, adding the names of the patches you will use. Then build the package as normal. Depending on the patches you use, you may need to edit patches like so (for example) to change what other patches one depends on, or to correct conflicts:

$ dpatch-edit-patch imc-00-indymedia-changelog


Email Forwarders

Adding, Editing, Deleting

You need sudo access on sarai.indy to do this. We have setup a script which will make it easy for you to edit the file which contains the mappings defining the email forwarding:

$ sudo /usr/local/sbin/editalias

You are now editing a Postfix config file, namely /etc/postfix/maps/virtual_indymedia.

* To add a new forwarding address, page down to the bottom of the file and press i to put an entry like this:
 # Requested by Che from IMC Chile, added by Jane 1 April 2006
 # OR: 
 # In response to SoS ticket #2006032900034
 foo@indymedia.org: fred@nothotmail.com 

This means that email sent to foo@indymedia.org will be forwarded to fred@nothotmail.com. Make sure you add a note as given in the above example (lines starting with # are notes) on what was changed so we have a bit of documentation.

* To edit or to delete an entry, use / (the slash) to enter search mode, type the email address you want to change, press enter. If the first result is not what you were looking for, press n until you end up at the entry you need to change or remove. Then press i and edit or remove the entry in question.

If you're changing an entry, make sure you add a note as given in the above example (lines starting with # are notes) on what was changed so we have a bit of documentation. However, if you're deleting it, you should also delete all of its backlog and not leave a comment.

When you finished editing (by pressing escape, then typing ":x" and pressing enter), the script will automatically run two more commands ("postmap /etc/postfix/maps/virtual_indymedia" and "/etc/init.d/postfix force-reload") in the background. That's it.

Listing

To get a plain alphabetically sorted list of all the currently setup email forwarders, use this command: cat /etc/postfix/maps/virtual_indymedia | grep "." | grep -v "^#.*" | sort | less

Counting

To find out how many email forwarders are currently setup, use cat /etc/postfix/maps/virtual_indymedia | grep "." | grep -v "^#.*" | wc -l


Spam

Checking the Record Spam Score

     zgrep -i 'discarded: SpamAssassin score was ' /var/log/mailman/vette* \ 
     | sed -e 's/^.*discarded: SpamAssassin score was *\([0-9.]*\).*/\1/g' \ 
     | sort -n -r -u |less

Checking spamd bayes db

how to see which list has the most spams in its bayes db (top 10):

mysql> select id,username,spam_count,ham_count,token_count from bayes_vars order by spam_count desc limit 10; 
+-----+--------------------+------------+-----------+-------------+
| id  | username           | spam_count | ham_count | token_count |
+-----+--------------------+------------+-----------+-------------+
| ... | ...

change order by and limit values for other results.

Adding spam or ham to a list's bayes db

what we want to do is pipe the full message into sa-learn (this will add the message to the mysql backend that spamd uses)

if we want to use an email from the list's archive, one good way to do this is to open the mbox archive with mutt like this:

mutt -R -f /var/lib/mailman/archive/private/$listname/$listname.mbox

(the -R is a good idea because we want to read the mbox read-only so we don't make silly changes to the archive)

then, select the message and use the | (pipe) command to pipe the message into sa-learn. you'll want to give it the following arguments:

sa-learn -u $listname --[spam|ham]

(use the actual listname, and decide whether the message is spam or ham. if the message scored outside of the autolearn threshhold then it will have already been learned, and sa-learn will have no effect)

Blacklisting Email Addresses

There are two blacklist mechanisms before email hits mailman or is processed otherwise (email aliases etc). The first one is Postfix' access checks. The second one is SpamAssassin. It is better to blacklist these at the MTA level, i.e. using Postfix' access checks, because SpamAssassin is slower and this way less email needs to be processed on its way from the MTA to SpamAssassin.

Postfix Access Checks

However, Postfix does not offer much to test against. It is still enough in some cases, though. Postfix allows you to reject email based on the MAIL FROM address as well as parts thereof. The domain part in the MAIL FROM line is resolved to IP addresses before the checks are applied, so you can also put IP addresses.

For example,

69.22.169.128 REJECT

would reject all email sent by any mail server where a MAIL FROM line contains either 69.22.169.128 or a FQDN resolving to this IP address.

To add, change, or remove an entry to the blacklist, edit /etc/postfix/checks/access, then run sudo postmap /etc/postfix/checks/access and sudo /etc/init.d/postfix reload.

To test your rules, you can do:

postmap -q 69.22.169.128 hash:/etc/postfix/checks/access

which should echo REJECT if your rule matches (and you didn't forget to update the access.db file).

You can also whitelist email addresses (or domains) appearing in the MAIL FROM line by appending OK the email address. For the exact format, just have a look at the other lines in this file.

SpamAssassin Rules

In those cases where Postfix' simple and dirty blacklisting doesn't fit, SpamAssassin is the way to go. In addition to its fine-grained blacklist, SpamAssassin also offers a whitelist. A really long whitelist is compiled regularly by a cron job (initially setup by Deanna) which is supposed to encompass all the people subscribed to all indymedia lists (this is pretty old info, may be outdated).

This is what the SpamAssassin blacklist looked like in 2002:

 blacklist_from *@the-financial-news.com (this is pretty old info, may be outdated)
 # this is the "Unlimited Free Calling SPAM" - christopher 4 april 2002 
 blacklist_from gintare@netzero.net 

The list is stored in /etc/mail/spamassassin/local.cf. After editing it, run sudo /etc/init.d/spamassassin restart

If you have questions, see man Mail::SpamAssassin::Conf. Free (as in hallicugenic herbal potions) SpamAssassin rulesets wizardry courses are also available at the SpamAssassin wiki on Apache.org and the Exit0 wiki. Readymade and regularly updated magic spells are available at The Rules Emporium as well as on the SpamAssassin wiki.

Stop Spam in Progress by Emergency Procmail Rules

Here's something I do from time to time if a spammer has hijacked our system. Since we have process limits set and spamassassin only does one message at a time, one spammer can tie us up for long periods of time. If you're on the shell and want to do something about it:

Add a rule to /etc/procmailrc-spamtrap and /etc/procmailrc-spamd with their address in it, like this:

   :0
   * ^From.Marketingwap@millenniumdata.com
   /dev/null

Then kill all the running procmail processes, which will cause procmail to requeue the mail and subject it to the new rules.

$ for proc in `ps auxw | grep [p]rocmail | awk '{print $2}'`; \
do sudo kill $proc ; echo $proc ; done

Now watch them all get sent to the trash in /var/procmail/procmail.log.

joe@foo cannot send Emails to Indymedia

In /etc/postfix/checks/access put the user@address or domain on a line

followed by 'OK' then run sudo postmap /etc/postfix/checks/access. You can take a look at other entries there (they are located at the bottom of the file) for clues to the format, or man access on sarai.

You might also want to add them to the spamassassin whitelist in /etc/mail/spamassassin/*.cf in case the SA checks on the same blocklists cause them to get tagged as spam. We should probably zero out all those checks that we're doing in postfix in spamassassin so that they're not run twice.


Quick Menu:


-- ChristopherMitchell - 01 Jun 2002

-- GaBa - 01 Jun 2002 - Traducciones

-- LuiS - 09 Jun 2003 - Updated the command paths to be used in newsarai

-- LuiS - 12 Oct 2003 - Moved the FAQ section to ListWorkFAQ

-- ChrisC - 28 Oct 2003 -- added stuff about new script for editing mbox files

-- PaulWise - 13 Jan 2004 --

-- JiBe - 02 Feb 2004 -- Added stuff about joe@foo@.ca and delpost

-- AlsteR - 10 Dec 2004 -- Updated list creation hints

-- AlsteR - 03 Jan 2005 -- Added find_member and remove_members

-- AlsteR - 04 Jan 2005 -- Reorganized the topic

-- PaulWise - 05 Jan 2005 -- Fix some typos and stuff

-- AlsteR - 05 Sep 2005 -- Reordered this page so you don't find what you're looking for

-- AlsteR - 02 May 2006 -- Updated email forwarding addresses ('aliases') how-to

-- AnA - 19 Nov 2006 -- updated the file where old archives are - edit archives section

-- AlsteR - 14 Dec 2006 -- moved and rewrote instructions for fixing broken archives

-- AnthonyG - 22 Dec 2006 -- removed reference to sending a message to the listwork discussion list after a new list has been created (the SoS system takes care of that now)

-- CaD - 12 Jun 2008 added "Removing offsite List"
Topic revision: r72 - 31 May 2012, DimitrisT
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback