Geoffroy Carrier wrote:
On Tue, Jan 20, 2009 at 09:10, RedShift <redsh...@pandora.be> wrote:
I've been working on my own mailing list manager, it's completely written in
PHP and uses SQL as a backend, but at this point a bit specific to our
setup. Because our company is all about open source I intend on releasing it
as such. It's in need of an overhaul anyway, I'll get hacking at it again
ASAP.

I'd love to be an early adopter :)

Plus I'd be happy to contribute...




Ok so here's what I've got so far:


When I was designing my anti-SPAM filter, I noticed you're just running a series of code 
after each other. To keep everything nice and tidy I split up the various routines into 
small pieces of code designed to do one thing and then pass on to the next filter. My 
project has the codename "redsky". Yes, it's cliché but I couldn't think of 
anything else.

By itself, redsky is pretty useless. It's only capable of running a series of 
filters after each other. The real power is in the filters itself. The 3 
filters that come shipped with redsky are designed to interface with another 
piece of software called proxsmtpd (but can be used in other ways as well, they 
just use STDIN and STDOUT), but there's no reasons you could write your own 
input/output filters and make redsky a mini-SMTP server.


Here are the packages that are of interest:

* phpemailtoolbox (http://aur.archlinux.org/packages.php?ID=22237)

This is a small set of email related functions and classes. It is required by 
redsky.

* redsky (http://aur.archlinux.org/packages.php?ID=22261)

This is the main redsky code. It contains three filters:
* redsky_read: this reads the email in question from STDIN. It is designed for 
interfacing with proxsmtpd, but can be used in a generic manner as well.
* redsky_discard: when the discard flag is set on an email, this filter will 
log this event.
* redsky_out: this outputs the email to STDOUT, again designed to interface 
with proxsmtpd but is suitable for general purpose as well. In the mailinglist

* redsky-filters (http://aur.archlinux.org/packages.php?ID=24393)

This is a set of complementary filters for redsky, it has for example SMTP 
blacklist checking, whitelisting, autoreplies, etc... They are mainly designed 
to use in a postfix content scanner context. Only the redsky_sqlconn filter is 
required for mailinglist manager.

* mlm-filters (http://aur.archlinux.org/packages.php?ID=24399)

This is the most interesting set of filters. Mlm stands for Mailing List 
Manager. There's a queueing part and a sending part.

For the queueing part:
* mlm_getinfo: Fetch information for a certain list using SQL
* mlm_loopdetect: Check if a message is looping or not
* mlm_bouncehandler: If the message received was sent to a bounce address, 
store it in an SQL table
* mlm_getsubscribers: Get the subscribers from SQL for the current list
* mlm_access: Determine if a sender has access to the list. It currently 
supports 4 methods: use an ACL, only allow subscribers, require a passphrase or 
no limitations
* mlm_subjecttag: Tag the subject if the mailinglist is configured for that
* mlm_addfooter: Add a footer to the emails if the mailinglist is configured 
for that
* mlm_queuer: Write a spoolfile and put all the recipients into an SQL table

For the sending part:
* mlm_singleinstance: Only allow a single instance of redsky to be run
* mlm_queuebuilder: Build a queue of messages to be sent and their respective 
recipients
* mlm_queuerunner: Process the earlier built queue using sendmail to send 
messages


redsky is written to support multiple configurations. In the MLM case, you 
would have two configuration files (examples are shipped with mlm-filters), one 
for the queuer and one for the sender. The queuer would be called by the MTA 
while to sender could be executing with a cronjob. Use -c to supply an 
alternate config, for example:

redsky -c /etc/redsky.queuerunner.config.php



You will notice that the redsky-filters and mlm-filters come shipped with 
example configuration files. You will find them in /usr/share/php/redsky/. Some 
filters depend on each other, so it is important in which order they are 
executed. For example, the mlm_access cannot determine if the sender is a 
subscriber when mlm_getsubscribers hasn't run first, because it uses the 
information from mlm_getsubscribers for this.


Now maybe the hardest part, integration with an email server. I'll only handle 
postfix. Our entire infrastructure is based on MySQL, however all redsky code 
is written to use PDO, so it should be compatible with any databaseserver that 
PDO supports.

You can get our database schema from http://projects.webmind.be/redsky/mail.sql

In postfix terms, you would add entries to your alias table to send certain addresses to 
the "mlm". The examples below are made to work with the database schema above, 
but all queries can be modified in the configuration files.

virtual_alias_maps = mysql:/etc/postfix/mysql/lists.cf

lists.cf:

user = USER
password = PASSWORD
hosts = HOST
dbname = MAILDB

query = SELECT
                        'mlm' AS goto
                FROM
                        lists
      LEFT JOIN
        domains
      ON
        domains.domain_id = lists.domain_id
      WHERE
                (lists.list_post_address = '%u' OR
                lists.list_bounce_address = '%u')       AND
                domains.domain_name = '%d'              AND
                domains.domain_type = 'virtual'         AND
                domains.domain_enable_email     = 'yes'

The above will send all mail sent to a mailing list address to 'mlm'.
Now postfix knows it needs to send any list emails to mlm. Now we need to tell 
postfix that all email sent to the local 'mlm' address to be piped to our mlm 
program. For this we create a transport table, that has 
'm...@localhost.localdomain' transported to the 'mlm' destination.

Now we have to define what the mlm destination is. Open up master.cf and add:

mlm     unix    -       n       n       -       -       pipe
        user=nobody     flags=O argv=/usr/bin/redsky -c 
/etc/redsky.queuer.config.php

Note the O flag, this will add an X-Original-To header, which is required to 
identify wich original address the message was sent to.


To allow the messages to be distributed, just call /usr/bin/redsky -c 
/etc/redsky.runner.config.php

Ok, all this information was probably hard to swallow. I will try to answer any 
questions you have regarding the implementation. If you want to ask me a direct 
question you can find me on IRC, freenode with the nick RedShift. I am in 
#archlinux.

It's also easier if you start by testing it out directly from the shell instead 
of immediatly integrating it into your MTA.


Best regards,

Glenn

Reply via email to