This was the logical end of a sequence of more general regexes to make
sure I wasn't losing my mind. Here's the one I started out with:
^\[error\] \d+#\d+: \*\d+ \S+\(\) \"\S+\" (failed|is not found) \(2\: No
such file or directory\), client\: <HOST>\, server\: \S*\, request\:
\"(GET|POST|HEAD) \/\S+ \S+\"\, .*?$
When you expand <HOST> out to |(?:::f{4,6}:)?(?P<host>\S+) |as the wiki
suggests (http://www.fail2ban.org/wiki/index.php/MANUAL_0_8#Filters),
you get
^\[error\] \d+#\d+: \*\d+ \S+\(\) \"\S+\" (failed|is not found) \(2\: No
such file or directory\), client\: (?:::f{4,6}:)?(?P<host>\S+)\,
server\: \S*\, request\: \"(GET|POST|HEAD) \/\S+ \S+\"\, .*?$
(I tried this one as well), which debuggex.com says should match.
Compiling this last regex in both Python 2 and Python 3, I get a match
when I run it against this line. It's not the over-general regex. The
timestamp, from fail2ban-regex output in my first, should also match. So
what gives?
> Date: Wed, 6 Jul 2016 21:43:33 -0400
> From: Zurd <[email protected]>
> Subject: Re: [Fail2ban-users] fail2ban refuses to match even most
> basic acceptable regex
> To: [email protected]
> Message-ID:
> <cafpujg7_4_ljjch0hwzjebkngjaou_hpizluuvo_k-lsy2x...@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> I am by no means expert but I can see that having a filter so simple and so
> small doesn't work. I don't think it's really a bug though as fail2ban
> could be compromised about 1 or 2 years ago and the regex had to be remade.
> I think having as little as possible of .* helps. Maybe an expert can
> comment on this? Isn't there a wiki page on how to create regex on the
> website of fail2ban? I can't seem to find anything.
>
> As for your regex, I can make it work like so below by just adding a few
> characters/words:
>
> $line=
> '2016/07/05 23:10:26 [error] 2359#0: *21 open()
> "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657"
> failed (2: No such file or directory), client: 198.143.46.17, server: _,
> request: "GET /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657
> HTTP/1.1", host: "www.appleipadwallpapers.com"'
>
> $regex=
> '^.*, client: <HOST>, server: _, request:.*$'
>
> Or just copy and paste this:
> fail2ban-regex '2016/07/05 23:10:26 [error] 2359#0: *21 open()
> "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657"
> failed (2: No such file or directory), client: 198.143.46.17, server: _,
> request: "GET /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657
> HTTP/1.1", host: "www.appleipadwallpapers.com"' '^.*, client: <HOST>,
> server: _, request:.*$'
>
>
>
>
> On Wed, Jul 6, 2016 at 4:15 PM, Alan Liddell <[email protected]>
> wrote:
>
>> Hi all,
>>
>> I checked the GitHub and asked on IRC (nobody around at the time) and
>> couldn't find anything like this. I'm running fail2ban 0.9.3 on Fedora
>> 24, Python 2.7.11/3.5.1, trying to check Nginx error logs for bots.
>> Here's the line:
>>
>> $ line='2016/07/05 23:10:26 [error] 2359#0: *21 open()
>>
>> "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657"
>> failed (2: No such file or directory), client: 198.143.46.17, server: _,
>> request: "GET
>> /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657 HTTP/1.1",
>> host: "www.appleipadwallpapers.com"'
>>
>> Here's the regex:
>>
>> $ regex='^.*<HOST>.*$'
>>
>> This should be the most permissive possible regex on fail2ban, right?
>> But here's the output of fail2ban-regex:
>>
>> $ fail2ban-regex "$line" "$regex"
>>
>> Running tests
>> =============
>>
>> Use failregex line : ^.*<HOST>.*$
>> Use single line : 2016/07/05 23:10:26 [error] 2359#0: *21 open()
>> "/u...
>>
>>
>> Results
>> =======
>>
>> Failregex: 0 total
>>
>> Ignoreregex: 0 total
>>
>> Date template hits:
>> |- [# of hits] date format
>> | [1] Year(?P<_sep>[-/.])Month(?P=_sep)Day
>> 24hour:Minute:Second(?:,Microseconds)?
>> `-
>>
>> Lines: 1 lines, 0 ignored, 0 matched, 1 missed [processed in 0.02 sec]
>> |- Missed line(s):
>> | 2016/07/05 23:10:26 [error] 2359#0: *21 open()
>>
>> "/usr/share/nginx/html/wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657"
>> failed (2: No such file or directory), client: 198.143.46.17, server: _,
>> request: "GET
>> /wallpaper/technology/Rendered-Blue-Cubes-iPad-Wallpaper/1657 HTTP/1.1",
>> host: "www.appleipadwallpapers.com"
>> `-
>>
>>
>> (I'm new to fail2ban and I was worried my timestamp might have been
>> nonstandard, but does the bit under "Date template hits" mean that I'm
>> in the clear there?) By the way, fail2ban-testcases fails a few tests
>> related to this:
>>
>> Regex for filter 'nginx-botsearch' has no samples: 2: '^\\[error\\]
>> \\d+#\\d+: \\*\\d+ \\S+\\(\\) \\"\\S+\\" (failed|is not found) \\(2\\:
>> No such file or directory\\), client\\:
>> (?:::f{4,6}:)?(?P<host>[\\w\\-.^_]*\\w)\\, server\\: \\S*\\, request:
>> \\"(GET|POST|HEAD) \\/\\S+ \\S+\\"\\, .*?$'
>>
>> Regex for filter 'nginx-http-auth' has no samples: 1: '^ \\[error\\]
>> \\d+#\\d+: \\*\\d+ no user/password was provided for basic
>> authentication, client: (?:::f{4,6}:)?(?P<host>[\\w\\-.^_]*\\w), server:
>> \\S+, request: "\\S+ \\S+ HTTP/\\d+\\.\\d+", host: "\\S+"\\s*$'
>>
>> and so forth. Don't know if this specifically is relevant, but thought
>> I'd mention it. Thanks all.
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
>> Francisco, CA to explore cutting-edge tech and listen to tech luminaries
>> present their vision of the future. This family event has something for
>> everyone, including kids. Get more information and register today.
>> http://sdm.link/attshape
>> _______________________________________________
>> Fail2ban-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/fail2ban-users
>>
>>
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Fail2ban-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fail2ban-users
