'lo, I got some emails asking me what teergrubing was after this post which I managed to delete before replying to. Hopefully the people who were wondering about it will catch this message.
On Thu, Aug 19, 2004 at 03:40:34AM +1000, Pascal Hakim wrote: > Hi, > <snip> > Delays were caused on this list by the high number of gmail > subscribers, which caused some gmail servers to start teergrubing > lists.debian.org. > <snip> Firstly, that sentence above is incorrect. It should have be: "Delays on this list were caused by the high number of gmail subscribers on this list, combined with the fairly high volume of mails that comes out of it". Basically, what's currently happening is that mails getting sent to gmail.com are taking a _long_ time to get processed. While we get an answer almost straight away on HELO, RCPT TO:, or Mail From, we have to wait until we get a reply at the end of the DATA section of the email. I've done some tests, and it seems to be taking at least 5 minutes to give us an OK code after the message has been sent. While this is not a problem if you are only sending a couple of emails, when you get to the levels that lists.debian.org has to deal with, it starts being a problem. We've setup our MTA so that it can be trying to deliver up 270 messages at any point in time. By holding a connection open for a long period of time, processes that would normaly be delivering mail just sit there waiting for a successfull delivery message. Since there weren't enough processes to deliver messages, the queues were getting longer and longer causing some people to have extreme delays in getting emails from lists. The listmaster team stuck a band-aid on the problem by having some processes set aside to be used uniquely by gmail. That way the problem now only affects people with an address at gmail and no others. If we look at some quick numbers, we realise quite quickly that we're still having quite a problem. In the last 90 minutes, 350 messages were accepted by gmail. Since there are 50 processes dedicated to delivering mail there, it's taking on average 12 minutes for a message to an individual gmail subscriber to go through. We still don't know what's causing the delays when we send mail to gmail. There's two main possibilities I can think of. (a) is that gmail is doing some content filtering after mail reception but before accepting a message. This way it can bounce messages it believes to be spam. (b) is that google is holding connections open to what it thinks are propbably spammers. This is what teergrubing refers to. The idea behind it is that most spammers do not use real SMTP clients, but rather, something that mostly works. By holding a connection open for a long time, a spammer is prevented from sending more spam out, as it has one less process and one less port it can send spam from. As we can see from what's happening to us, this is a very effective technique in stopping someone from delivering mail. There's a fairly interesting article about teergrubing at: http://www.iks-jena.de/mitarb/lutz/usenet/teergrube.en.html At the moment, I'm guessing that (b) happened. I'm still trying to get more information, so it's hard for me to make more than a guess at this point however. Cheers, Pasc -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
