Hi Diane and Sylvestre,
Sorry to drop in, but I was also looking into this and am actually not
sure about Diane's patch. I found some completely different reasons for
why some log lines get missed.
I am also using fail2ban 0.11.2 with exim 4.94.2. Yes, I also see it
misses some lines, but I also still see a lot of lines being matched.
None of my log lines contains a `pid`, so I doubt that `%(pid)s` is
the reason for missing some lines.
Also, the `%(pid)s` part is actually defined as optional in
/etc/fail2ban/filter.d/exim-common.conf:
pid = (?: \[\d+\])?
Also, what happens if someone does log `pid`, because with the patch
the filter will only expect whitespace, not a `pid` anymore.
Perhaps Diane is using a different log format or watching a different
log file than I do? Maybe affected by something like this issue where
the user had two timestamps in the log and was also able to solve it
by using a custom `pid` prefix:
- https://github.com/fail2ban/fail2ban/issues/3060
(exim logs from journal do not match)
Furthermore, Diane's patch does two more things in addition to removing
the `%(pid)s` that I have doubts about:
First, the line about `SMTP call`: It seems the patch actually reverts
some improvements made between fail2ban 0.10.2 and 0.11.2, such as the
additional matching of `syntax or protocol errors` and
`last command was`. I do not see why this is necessary.
Second, Diane's patch adds a new line to the filter to also match
lines containing `LOGIN authentication mechanism not supported`. I do
not know if this is a good addition, I do not need it myself, but
either way, as far as I can tell this is simply not in the upstream
filter. I wonder if this is not a bit too big of a Debian-specific
deviation from upstream. Perhaps it should better be forwarded or
directly submitted upstream.
Actually, I found some other reasons for why fail2ban misses some log
lines:
Example 1: `SMTP call` in exim 4.94.2 has a new field at the end.
exim 4.92:
2021-08-28 08:40:29 SMTP call from census9.shodan.io [71.6.167.142]
dropped: too many syntax or protocol errors (last command was "?")
exim 4.94.2:
2021-10-14 17:20:55 SMTP call from census6.shodan.io [66.240.236.119]
dropped: too many syntax or protocol errors (last command was "?", NULL)
fail2ban 0.10.2 misses both, because `syntax or protocol errors` was
only added in 0.11.2.
fail2ban 0.11.2 matches the exim 4.92 line, but not the 4.94.2 line,
because it does not expect the last field (`, NULL`).
Example 2: No match on encrypted connections.
Already with the previous fail2ban 0.10.2 + exim 4.92 and now still
with fail2ban 0.11.2 + exim 4.94.2.
exim 4.92 and 4.94.2, connection without TLS:
2021-10-01 15:27:31 H=(win2012r2RDP) [77.247.110.246]
F=<spam...@tiscali.it> rejected RCPT <spam...@tiscali.it>: relay not permitted
2021-10-20 14:33:00 H=(win2012r2RDP) [77.247.110.115]
F=<spam...@tiscali.it> rejected RCPT <spam...@tiscali.it>: relay not permitted
Both match with fail2ban 0.10.2 and 0.11.2.
exim 4.92 and 4.94.2, connection with TLS:
2021-10-01 15:08:40 H=(xhU9K1I7) [119.91.134.193]
X=TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no F=<w...@edlp.nl> rejected RCPT
<726357...@qq.com>: relay not permitted
2021-10-17 08:52:37 H=(LH5pvsWxa) [36.25.57.206]
X=TLS1.2:ECDHE_SECP256R1__RSA_SHA512__AES_256_GCM:256 CV=no F=<w...@edlp.nl>
rejected RCPT <897855...@qq.com>: relay not permitted
Both missed by fail2ban 0.10.2 and 0.11.2.
I did not finish checking all missed lines, but I suspect there are
more specific causes that need to be addressed and doubt that removing
`%(pid)s` and reverting some parts to 0.10.2 is the right solution.
Best regards,
Peter
