working on getting a dual core dual cpu 64b 2MB cache
xeon 2.8GHz w/12GB RAM and dual copper em(4) put
in place in front of our MX vip for a greylisting spamd(8).
i've got a similar machine with faster CPU ( 3.0 GHz / 4MB )
but it "only" has 4GB of RAM with 4.0 installed on it now
that was planning on using but i'm wondering if i'm going
to definately need that extra 8GB of RAM.
reason being we want to keep as much crap from talking to the MXs
as we can (and thus to the spamassassin and all that stuff)
- we also have some RBLs we use to flat out 5xx some
incoming clients with - would like to use those for the
<spamd> table with spamd-setup(8) - don't want them to go through
greylisting trials and do want to be able to specify the "blacklist"
message they see based on what RBL they're in. i'm not the biggest
fan of RBLs, but we use some of them, so if spamd(8) can help us
use them *better*, hooray.
i did a few tests with pfctl(8) and spamd-setup(8) and see
a concerning difference in the amount of time it takes to complete
the operation of populating the addresses ( mostly in part because
of the size of these RBLs, i believe ).
here's a comparison of loading the 'CBL' list (i sorted it first,
ascending) into pf via pfctl and then via spamd/spamd-setup.
# wc -l /root/rbldns/cbl/cbl.zone.plain.sorted
3899399 /root/rbldns/cbl/cbl.zone.plain.sorted
# pfctl -FT ; echo "table <cbl> persist file
\"/root/rbldns/cbl/cbl.zone.plain.sorted\"" | time pfctl -vvf- -Tl
1 tables deleted.
table <cbl> persist file "/root/rbldns/cbl/cbl.zone.plain.sorted"
0m12.52s real 0m4.48s user 0m7.97s system
# cat /etc/spamd.conf
all:cbl:
cbl:black:msg="CBL":method=file:file=/root/rbldns/cbl/cbl.zone.plain.sorted
# pfctl -FT; time /usr/libexec/spamd-setup
2 tables deleted.
20m19.53s real 5m38.13s user 7m23.78s system
watching top(1), the greater part of 12m are spent in spamd-setup
at ~100% cpu, then at about 13m spamd looks like it starts getting
infos from spamd-setup, spamd itself starts having 90%+ CPU
until the 20m mark when spamd-setup disappears.
spamd-setup held onto its memory allocation until right near the end
(~20m), here's a line from top -b a second or two before spamd-setup
was out of the processlist:
15386 _spamd 64 0 127M 76M onproc/2 - 6:51 100.05% spamd
31029 root 2 0 951M 950M sleep/0 netio 12:58 0.00% spamd-setup
spamd itself is now sitting at
15386 _spamd 2 0 209M 210M sleep/2 select 7:56 0.00% spamd
so wrt the spamd-setup taking 20m there, i'm guessing that the only
thing i can do to make that quicker would be to opportunistically
supernet the source data into a file and then have spamd-setup use
that one instead, but it is likely to be the case that the source
data might not be very supernet-able too; i was looking at using
more than just the CBL list, so instead of "only" about 3.8M entries,
having something more like 12 million.
the ruleset for pf i'm planning on keeping as plain as i can, probably
not using altq for anything, but i'd like to use the source tracking
options to keep the amount of connections from any specific non-whitelisted
hosts down to a minimum. i'm anticipating i'll need to significantly
jack up the default state/srctrack limits, which is ok, but primarily
i'm trying to make certain that it's reasonable to put a spamd/pf box
in front of the MXs and have it weather the storm and do what i
hope it will benefit us with.
dmesg and some details about the amount of email we get are in:
http://marc.theaimsgroup.com/?l=openbsd-pf&m=116482979827642&w=2
with the exception that instead of the 1950s it's now going to probably
be a 2850 ( the guy with 12GB RAM ) as the primary and failover to a 1950
who'll just pass stuff without greylisting.
--
jared
