If you're doing that much reworking things, I bet this would be an ideal time to impliment shared folders and related issues recently discussed (multiple recipients receiving the same message - rework the db tables just a bit so you only store a single copy of a message, and keep track of what mailboxes it belongs too).
If you want to get really ambitious, and end up doing schema changes anyways, you could stick in the injector side of header caching, and the pop3/imap daemons could but optimized to use that later. :) ---- Original Message ---- From: Aaron Stone <[email protected]> To: [email protected] Subject: [Dbmail-dev] The wonderful flaws of injector.c, and a plan to move ahead Sent: Sat, 19 Apr 2003 12:44:18 -0700 (PDT) I finished reading through main/pipe, injector and mini-injector. Now I recall the old thread about the gazillion simultaneous pipes chocking someone's system. injector.c sets up an open sendmail process for each and every forward, then begins reading in the message from stdin and after each chunk of data, loops the pipes and sends on them, then loops the local user id numbers and inserts a block into the database. Repeat until stdin has finished. This is an interesting but unworkable architecture because a large set of forwarding addresses will cause the system to die. A throttled system would be much more effective. Was this discussed at the time? As for main/pipe, I was not able to find if there was code to use the temporary tables. This would be the way out for injector.c, of course. Where I'm at now is wanting to merge main/pipe with injector. Not sure about bounce.c and forward.c, although I'd lean toward keeping them. The code path I see is... - Start up "new-injector" - Process command line arguments - Read the header from stdin (but not hard coded in each function!) - Set up the lists of local user ids, forwards, bounces (no such user) - Create a temporary message - Read in the entire message, storing to the temporary table - Calculate the message size, and set up the list of quota bounces - Get the 'SENDMAIL_THROTTLE' config value (whatever we call it) - Allocate ( fwd.total < THROTTLE ? fwd.total : THROTTLE ) pipes while( db_get_me_some_data_from_the_temporary_table() ) { - Get the first/next message block from the database - For each pipe, send out the data } - Repeat this procedure for the local users (Except that it should be abstracted to the database driver layer. Some versions of the supported db's can simply use a subselect and not require reading/buffering/writing back to the database. Much better!) - Send the no such user bounces and the over quota bounces, again using the throttle mechanism. Naturally, the order of a few things can be changed around, but this is the basic idea. Questions, comments? Eelco and Roel, anything important that I left out? Aaron -- End Original Message -- -- Jesse Norell jesse (at) kci.net
