Geoffroy Carrier wrote:
On Tue, Jan 20, 2009 at 09:10, RedShift <redsh...@pandora.be> wrote:
I've been working on my own mailing list manager, it's completely written in
PHP and uses SQL as a backend, but at this point a bit specific to our
setup. Because our company is all about open source I intend on releasing it
as such. It's in need of an overhaul anyway, I'll get hacking at it again
ASAP.
I'd love to be an early adopter :)
Plus I'd be happy to contribute...
Ok so here's what I've got so far:
When I was designing my anti-SPAM filter, I noticed you're just running a series of code
after each other. To keep everything nice and tidy I split up the various routines into
small pieces of code designed to do one thing and then pass on to the next filter. My
project has the codename "redsky". Yes, it's cliché but I couldn't think of
anything else.
By itself, redsky is pretty useless. It's only capable of running a series of
filters after each other. The real power is in the filters itself. The 3
filters that come shipped with redsky are designed to interface with another
piece of software called proxsmtpd (but can be used in other ways as well, they
just use STDIN and STDOUT), but there's no reasons you could write your own
input/output filters and make redsky a mini-SMTP server.
Here are the packages that are of interest:
* phpemailtoolbox (http://aur.archlinux.org/packages.php?ID=22237)
This is a small set of email related functions and classes. It is required by
redsky.
* redsky (http://aur.archlinux.org/packages.php?ID=22261)
This is the main redsky code. It contains three filters:
* redsky_read: this reads the email in question from STDIN. It is designed for
interfacing with proxsmtpd, but can be used in a generic manner as well.
* redsky_discard: when the discard flag is set on an email, this filter will
log this event.
* redsky_out: this outputs the email to STDOUT, again designed to interface
with proxsmtpd but is suitable for general purpose as well. In the mailinglist
* redsky-filters (http://aur.archlinux.org/packages.php?ID=24393)
This is a set of complementary filters for redsky, it has for example SMTP
blacklist checking, whitelisting, autoreplies, etc... They are mainly designed
to use in a postfix content scanner context. Only the redsky_sqlconn filter is
required for mailinglist manager.
* mlm-filters (http://aur.archlinux.org/packages.php?ID=24399)
This is the most interesting set of filters. Mlm stands for Mailing List
Manager. There's a queueing part and a sending part.
For the queueing part:
* mlm_getinfo: Fetch information for a certain list using SQL
* mlm_loopdetect: Check if a message is looping or not
* mlm_bouncehandler: If the message received was sent to a bounce address,
store it in an SQL table
* mlm_getsubscribers: Get the subscribers from SQL for the current list
* mlm_access: Determine if a sender has access to the list. It currently
supports 4 methods: use an ACL, only allow subscribers, require a passphrase or
no limitations
* mlm_subjecttag: Tag the subject if the mailinglist is configured for that
* mlm_addfooter: Add a footer to the emails if the mailinglist is configured
for that
* mlm_queuer: Write a spoolfile and put all the recipients into an SQL table
For the sending part:
* mlm_singleinstance: Only allow a single instance of redsky to be run
* mlm_queuebuilder: Build a queue of messages to be sent and their respective
recipients
* mlm_queuerunner: Process the earlier built queue using sendmail to send
messages
redsky is written to support multiple configurations. In the MLM case, you
would have two configuration files (examples are shipped with mlm-filters), one
for the queuer and one for the sender. The queuer would be called by the MTA
while to sender could be executing with a cronjob. Use -c to supply an
alternate config, for example:
redsky -c /etc/redsky.queuerunner.config.php
You will notice that the redsky-filters and mlm-filters come shipped with
example configuration files. You will find them in /usr/share/php/redsky/. Some
filters depend on each other, so it is important in which order they are
executed. For example, the mlm_access cannot determine if the sender is a
subscriber when mlm_getsubscribers hasn't run first, because it uses the
information from mlm_getsubscribers for this.
Now maybe the hardest part, integration with an email server. I'll only handle
postfix. Our entire infrastructure is based on MySQL, however all redsky code
is written to use PDO, so it should be compatible with any databaseserver that
PDO supports.
You can get our database schema from http://projects.webmind.be/redsky/mail.sql
In postfix terms, you would add entries to your alias table to send certain addresses to
the "mlm". The examples below are made to work with the database schema above,
but all queries can be modified in the configuration files.
virtual_alias_maps = mysql:/etc/postfix/mysql/lists.cf
lists.cf:
user = USER
password = PASSWORD
hosts = HOST
dbname = MAILDB
query = SELECT
'mlm' AS goto
FROM
lists
LEFT JOIN
domains
ON
domains.domain_id = lists.domain_id
WHERE
(lists.list_post_address = '%u' OR
lists.list_bounce_address = '%u') AND
domains.domain_name = '%d' AND
domains.domain_type = 'virtual' AND
domains.domain_enable_email = 'yes'
The above will send all mail sent to a mailing list address to 'mlm'.
Now postfix knows it needs to send any list emails to mlm. Now we need to tell
postfix that all email sent to the local 'mlm' address to be piped to our mlm
program. For this we create a transport table, that has
'm...@localhost.localdomain' transported to the 'mlm' destination.
Now we have to define what the mlm destination is. Open up master.cf and add:
mlm unix - n n - - pipe
user=nobody flags=O argv=/usr/bin/redsky -c
/etc/redsky.queuer.config.php
Note the O flag, this will add an X-Original-To header, which is required to
identify wich original address the message was sent to.
To allow the messages to be distributed, just call /usr/bin/redsky -c
/etc/redsky.runner.config.php
Ok, all this information was probably hard to swallow. I will try to answer any
questions you have regarding the implementation. If you want to ask me a direct
question you can find me on IRC, freenode with the nick RedShift. I am in
#archlinux.
It's also easier if you start by testing it out directly from the shell instead
of immediatly integrating it into your MTA.
Best regards,
Glenn
