...or not. I didn't realize the wiki was closed to contributors. If any of you watching this thread is a contributor, please make this change.
On 07/08/2016 06:25 PM, Alan Liddell wrote: > > Thanks Zurd for taking the time to answer this. Turns out it was a > really simple mistake. After reading this closed bug report > <https://github.com/fail2ban/fail2ban/issues/1453> (specifically the > comment here > <https://github.com/fail2ban/fail2ban/issues/1453#issuecomment-222469723>) > I realized that whitespace after the beginning-of-line anchor (^) was > actually necessary. > > The entry in the wiki > <http://www.fail2ban.org/wiki/index.php/MANUAL_0_8#Filters> has this > to say, though: > >> If the failregex is anchored with a leading |^|, then the anchor >> refers to the start of the remainder of the line, /after/ the >> timestamp and intervening whitespace. > > So if I'm reading this correctly, this is actually inaccurate, since > according to the bug report and that specific comment the whitespace > after the timestamp needs to be explicitly included. Unless someone > gets back to me telling me why I'm wrong, I'm going to change the > entry in the wiki. Thanks for your time everyone. >> This was the logical end of a sequence of more general regexes to make >> sure I wasn't losing my mind. Here's the one I started out with: >> >> ^\[error\] \d+#\d+: \*\d+ \S+\(\) \"\S+\" (failed|is not found) \(2\: No >> such file or directory\), client\: <HOST>\, server\: \S*\, request\: >> \"(GET|POST|HEAD) \/\S+ \S+\"\, .*?$ >> >> When you expand <HOST> out to |(?:::f{4,6}:)?(?P<host>\S+) |as the wiki >> suggests (http://www.fail2ban.org/wiki/index.php/MANUAL_0_8#Filters), >> you get >> >> ^\[error\] \d+#\d+: \*\d+ \S+\(\) \"\S+\" (failed|is not found) \(2\: No >> such file or directory\), client\: (?:::f{4,6}:)?(?P<host>\S+)\, >> server\: \S*\, request\: \"(GET|POST|HEAD) \/\S+ \S+\"\, .*?$ >> >> (I tried this one as well), which debuggex.com says should match. >> Compiling this last regex in both Python 2 and Python 3, I get a match >> when I run it against this line. It's not the over-general regex. The >> timestamp, from fail2ban-regex output in my first, should also match. So >> what gives? >> >> > Date: Wed, 6 Jul 2016 21:43:33 -0400 >> > From: Zurd <zurd33@...> >> > Subject: Re: [Fail2ban-users] fail2ban refuses to match even most >> > basic acceptable regex >> > To: fail2ban-users@... >> > Message-ID: >> > <CAFPUJG7_4_LjJCH0HWZJEbKngjaOU_hPizLuuVo_K-LSY2X9wQ@...> >> > Content-Type: text/plain; charset="utf-8" >> > >> > I am by no means expert but I can see that having a filter so simple and so >> > small doesn't work. I don't think it's really a bug though as fail2ban >> > could be compromised about 1 or 2 years ago and the regex had to be remade. >> > I think having as little as possible of .* helps. Maybe an expert can >> > comment on this? Isn't there a wiki page on how to create regex on the >> > website of fail2ban? I can't seem to find anything. >> > >> > As for your regex, I can make it work like so below by just adding a few >> > characters/words: >> > >> > $line= >> > '2016/07/05 23:10:26 [error] 2359#0: *21 open() >> > "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657" >> > failed (2: No such file or directory), client: 198.143.46.17, server: _, >> > request: "GET /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657 >> > HTTP/1.1", host: "www.appleipadwallpapers.com"' >> > >> > $regex= >> > '^.*, client: <HOST>, server: _, request:.*$' >> > >> > Or just copy and paste this: >> > fail2ban-regex '2016/07/05 23:10:26 [error] 2359#0: *21 open() >> > "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657" >> > failed (2: No such file or directory), client: 198.143.46.17, server: _, >> > request: "GET /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657 >> > HTTP/1.1", host: "www.appleipadwallpapers.com"' '^.*, client: <HOST>, >> > server: _, request:.*$' >> > >> > >> > >> > >> > On Wed, Jul 6, 2016 at 4:15 PM, Alan Liddell <alan.c.liddell@...> >> > wrote: >> > >> >> Hi all, >> >> >> >> I checked the GitHub and asked on IRC (nobody around at the time) and >> >> couldn't find anything like this. I'm running fail2ban 0.9.3 on Fedora >> >> 24, Python 2.7.11/3.5.1, trying to check Nginx error logs for bots. >> >> Here's the line: >> >> >> >> $ line='2016/07/05 23:10:26 [error] 2359#0: *21 open() >> >> >> >> "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657" >> >> failed (2: No such file or directory), client: 198.143.46.17, server: _, >> >> request: "GET >> >> /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657 HTTP/1.1", >> >> host: "www.appleipadwallpapers.com"' >> >> >> >> Here's the regex: >> >> >> >> $ regex='^.*<HOST>.*$' >> >> >> >> This should be the most permissive possible regex on fail2ban, right? >> >> But here's the output of fail2ban-regex: >> >> >> >> $ fail2ban-regex "$line" "$regex" >> >> >> >> Running tests >> >> ============= >> >> >> >> Use failregex line : ^.*<HOST>.*$ >> >> Use single line : 2016/07/05 23:10:26 [error] 2359#0: *21 open() >> >> "/u... >> >> >> >> >> >> Results >> >> ======= >> >> >> >> Failregex: 0 total >> >> >> >> Ignoreregex: 0 total >> >> >> >> Date template hits: >> >> |- [# of hits] date format >> >> | [1] Year(?P<_sep>[-/.])Month(?P=_sep)Day >> >> 24hour:Minute:Second(?:,Microseconds)? >> >> `- >> >> >> >> Lines: 1 lines, 0 ignored, 0 matched, 1 missed [processed in 0.02 sec] >> >> |- Missed line(s): >> >> | 2016/07/05 23:10:26 [error] 2359#0: *21 open() >> >> >> >> "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657" >> >> failed (2: No such file or directory), client: 198.143.46.17, server: _, >> >> request: "GET >> >> /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657 HTTP/1.1", >> >> host: "www.appleipadwallpapers.com" >> >> `- >> >> >> >> >> >> (I'm new to fail2ban and I was worried my timestamp might have been >> >> nonstandard, but does the bit under "Date template hits" mean that I'm >> >> in the clear there?) By the way, fail2ban-testcases fails a few tests >> >> related to this: >> >> >> >> Regex for filter 'nginx-botsearch' has no samples: 2: '^\\[error\\] >> >> \\d+#\\d+: \\*\\d+ \\S+\\(\\) \\"\\S+\\" (failed|is not found) \\(2\\: >> >> No such file or directory\\), client\\: >> >> (?:::f{4,6}:)?(?P<host>[\\w\\-.^_]*\\w)\\, server\\: \\S*\\, request: >> >> \\"(GET|POST|HEAD) \\/\\S+ \\S+\\"\\, .*?$' >> >> >> >> Regex for filter 'nginx-http-auth' has no samples: 1: '^ \\[error\\] >> >> \\d+#\\d+: \\*\\d+ no user/password was provided for basic >> >> authentication, client: (?:::f{4,6}:)?(?P<host>[\\w\\-.^_]*\\w), server: >> >> \\S+, request: "\\S+ \\S+ HTTP/\\d+\\.\\d+", host: "\\S+"\\s*$' >> >> >> >> and so forth. Don't know if this specifically is relevant, but thought >> >> I'd mention it. Thanks all. >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San >> >> Francisco, CA to explore cutting-edge tech and listen to tech luminaries >> >> present their vision of the future. This family event has something for >> >> everyone, including kids. Get more information and register today. >> >> http://sdm.link/attshape >> >> _______________________________________________ >> >> Fail2ban-users mailing list >> >> Fail2ban-users@... >> >> https://lists.sourceforge.net/lists/listinfo/fail2ban-users >> >> >> >> >
signature.asc
Description: OpenPGP digital signature
------------------------------------------------------------------------------ Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San Francisco, CA to explore cutting-edge tech and listen to tech luminaries present their vision of the future. This family event has something for everyone, including kids. Get more information and register today. http://sdm.link/attshape
_______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users
