Hi Bill, I had tried fail2ban-regex and am aware of the epoch format but still there is an issue:
1) if I isolate the timestamp from the log entry (1516469849551) and test with
a dummy IP as follows, it fails
fail2ban-regex -v '1516469849551 1.2.3.4' '<HOST>'
Running tests
=============
Use failregex line : <HOST>
Use single line : 1516469849551 1.2.3.4
Results
=======
Failregex: 0 total
|- #) [# of hits] regular expression
| 1) [0] <HOST>
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [0] {^LN-BEG}ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|
?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] {^LN-BEG}(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] {^LN-BEG}(?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] {^LN-BEG}Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2)
%k:Minute:Second
| [0] {^LN-BEG}Day(?P<_sep>[-/])MON(?P=_sep)ExYear[
:]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
| [0] {^LN-BEG}Month/Day/ExYear:24hour:Minute:Second
| [0] {^LN-BEG}Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] {^LN-BEG}Epoch
| [0] {^LN-BEG}ExYear2ExMonthExDay ?24hour:Minute:Second
| [0] {^LN-BEG}MON Day, ExYear 12hour:Minute:Second AMPM
| [0] {^LN-BEG}ExYearExMonthExDay(?:T|
?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] {^LN-BEG}(?:Zone name )?(?:DAY )?MON Day
%k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] {^LN-BEG}(?:Zone offset )?(?:DAY )?MON Day
%k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] {^LN-BEG}TAI64N
| [0] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|
?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] (?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] (?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2) %k:Minute:Second
| [0] Day(?P<_sep>[-/])MON(?P=_sep)ExYear[
:]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
| [0] Month/Day/ExYear:24hour:Minute:Second
| [0] Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] Epoch
| [0] {^LN-BEG}24hour:Minute:Second
| [0] ^<Month/Day/ExYear2@24hour:Minute:Second>
| [0] ExYear2ExMonthExDay ?24hour:Minute:Second
| [0] MON Day, ExYear 12hour:Minute:Second AMPM
| [0] ^MON-Day-ExYear2 %k:Minute:Second
| [0] ExYearExMonthExDay(?:T|
?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] (?:Zone name )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?:
ExYear)?
| [0] (?:Zone offset )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?:
ExYear)?
| [0] TAI64N
`-
Lines: 1 lines, 0 ignored, 0 matched, 1 missed
[processed in 0.13 sec]
|- Missed line(s):
| 1516469849551 1.2.3.4
`-
2) same test but with a shorter epoch (it would appear Bitbucket adds the
milliseconds to the date/time field)
fail2ban-regex -v '1516469849 1.2.3.4' '<HOST>'
Running tests
=============
Use failregex line : <HOST>
Use single line : 1516469849 1.2.3.4
Results
=======
Failregex: 1 total
|- #) [# of hits] regular expression
| 1) [1] <HOST>
| 1.2.3.4 Sat Jan 20 17:37:29 2018
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] {^LN-BEG}Epoch
| [0] {^LN-BEG}ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|
?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] {^LN-BEG}(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] {^LN-BEG}(?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] {^LN-BEG}Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2)
%k:Minute:Second
| [0] {^LN-BEG}Day(?P<_sep>[-/])MON(?P=_sep)ExYear[
:]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
| [0] {^LN-BEG}Month/Day/ExYear:24hour:Minute:Second
| [0] {^LN-BEG}Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] {^LN-BEG}ExYear2ExMonthExDay ?24hour:Minute:Second
| [0] {^LN-BEG}MON Day, ExYear 12hour:Minute:Second AMPM
| [0] {^LN-BEG}ExYearExMonthExDay(?:T|
?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] {^LN-BEG}(?:Zone name )?(?:DAY )?MON Day
%k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] {^LN-BEG}(?:Zone offset )?(?:DAY )?MON Day
%k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] {^LN-BEG}TAI64N
| [0] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|
?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] (?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
| [0] (?:DAY )?MON Day ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2) %k:Minute:Second
| [0] Day(?P<_sep>[-/])MON(?P=_sep)ExYear[
:]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
| [0] Month/Day/ExYear:24hour:Minute:Second
| [0] Month-Day-ExYear %k:Minute:Second(?:\.Microseconds)?
| [0] Epoch
| [0] {^LN-BEG}24hour:Minute:Second
| [0] ^<Month/Day/ExYear2@24hour:Minute:Second>
| [0] ExYear2ExMonthExDay ?24hour:Minute:Second
| [0] MON Day, ExYear 12hour:Minute:Second AMPM
| [0] ^MON-Day-ExYear2 %k:Minute:Second
| [0] ExYearExMonthExDay(?:T|
?)Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [0] (?:Zone name )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?:
ExYear)?
| [0] (?:Zone offset )?(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?:
ExYear)?
| [0] TAI64N
`-
Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.04 sec]
This time I get a match on EPOCH date format as expected. So I need a date
pattern that will match epoch + 3 digits for the milliseconds part and I cant
seem to get this to work.
If I try matching the last 3 digits with various patterns they all fail:
fail2ban-regex -v --datepattern 'EPOCH\d+' '1516469849551 1.2.3.4' '<HOST>'
fail2ban-regex -v --datepattern 'EPOCH.%%f' '1516469849551 1.2.3.4' '<HOST>'
fail2ban-regex -v --datepattern 'EPOCH%%f' '1516469849551 1.2.3.4' '<HOST>'
So until I can get this epoch format to match I cant progress further.
Any ideas ?
thanks
> On 21 Jan 2018, at 01:02, Bill Shirley <[email protected]
> <mailto:[email protected]>> wrote:
>
> See what data patterns fail2ban is using. Run fail2ban-regex
> (change for your log file and filter) with the -v switch:
> fail2ban-regex -v /var/log/httpd/access_log
> /etc/fail2ban/filter.d/my_apache_access.conf
>
> I have a server using version0.9.3 which gives:
> Date template hits:
> |- [# of hits] date format
> | [128] (?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
> | [0] Year(?P<_sep>[-/.])Month(?P=_sep)Day
> 24hour:Minute:Second(?:,Microseconds)?
> | [0] Day(?P<_sep>[-/])Month(?P=_sep)(?:Year|Year2) 24hour:Minute:Second
> | [0] Day(?P<_sep>[-/])MON(?P=_sep)Year[
> :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
> | [0] Month/Day/Year:24hour:Minute:Second
> | [0] Month-Day-Year 24hour:Minute:Second\.Microseconds
> | [0] TAI64N
> | [0] Epoch
> | [0] Year-Month-Day[T ]24hour:Minute:Second(?:\.Microseconds)?(?:Zone
> offset)?
> | [0] ^24hour:Minute:Second
> | [0] ^<Month/Day/Year2@24hour:Minute:Second>
> | [0] ^Year2MonthDay ?24hour:Minute:Second
> | [0] MON Day, Year 12hour:Minute:Second AMPM
> | [0] ^MON-Day-Year2 24hour:Minute:Second
>
> I would think 'Epoch' would match but I can't find anything online that
> defines
> the date pattern.
>
> I had to add a datepattern= to my_apache_access filter when I upgraded
> to fail2ban 10.0 because they changed the date patterns requiring dates
> to be at the beginning of the line:
> # new date patterns for fail2ban-server-0.10.0-1
> #| [0] {^LN-BEG}ExYear(?P<_sep>[-/.])Month(?P=_sep)Day[T
> ]24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
> #| [0] {^LN-BEG}(?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?:
> ExYear)?
> #| [0] {^LN-BEG}(?:DAY )?MON Day ExYear
> 24hour:Minute:Second(?:\.Microseconds)?
> #| [0] {^LN-BEG}Day(?P<_sep>[-/])Month(?P=_sep)(?:ExYear|ExYear2)
> 24hour:Minute:Second
> #| [0] {^LN-BEG}Day(?P<_sep>[-/])MON(?P=_sep)ExYear[
> :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
> #| [0] {^LN-BEG}Month/Day/ExYear:24hour:Minute:Second
> #| [0] {^LN-BEG}Month-Day-ExYear 24hour:Minute:Second(?:\.Microseconds)?
> #| [0] {^LN-BEG}Epoch
> #| [0] {^LN-BEG}ExYear2ExMonthExDay ?24hour:Minute:Second
> #| [0] {^LN-BEG}MON Day, ExYear 12hour:Minute:Second AMPM
> #| [0] {^LN-BEG}ExYearExMonthExDay[T
> ]Ex24hourExMinuteExSecond(?:[.,]Microseconds)?(?:\s*Zone offset)?
> #| [0] {^LN-BEG}(?:Zone name )?(?:DAY )?MON Day
> 24hour:Minute:Second(?:\.Microseconds)?(?: ExYear)?
> #| [0] {^LN-BEG}(?:Zone offset )?(?:DAY )?MON Day
> 24hour:Minute:Second(?:\.Microseconds)?(?: ExYear)?
> #| [0] {^LN-BEG}TAI64N
> #| [0] {^LN-BEG}24hour:Minute:Second
> #| [0] ^<Month/Day/ExYear2@24hour:Minute:Second>
> #| [0] ^MON-Day-ExYear2 24hour:Minute:Second
>
> Ah, I finally found it:
> https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior
> <https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior>
>
> Bill
>
signature.asc
Description: Message signed with OpenPGP
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users
