From /var/log/httpd/access_log I can see:
198.20.70.114 - - [15/Jul/2016:04:05:57 +0100] "GET / HTTP/1.1" 200
1031 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/41.0.2228.0 Safari/537.36"
198.20.70.114 - - [15/Jul/2016:04:05:57 +0100] "GET /robots.txt
HTTP/1.1" 200 27 "-" "-"
198.20.70.114 - - [15/Jul/2016:04:05:57 +0100] "GET /sitemap.xml
HTTP/1.1" 404 272 "-" "-"
The same transaction appears in my error log, /var/log/httpd/error.log
as a single line:
[Fri Jul 15 04:05:57 2016] [error] [client 198.20.70.114] File does
not exist: /var/www/html/sitemap.xml
This is picked up by the apache-nohome jail.
I don't know apache as it comes pretty much pre-configured, but the
error log is specified in /etc/httpd/conf/httpd.conf as the parameter
ErrorLog. This is the relevant section of the file:
#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here. If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog logs/error_log
#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#
LogLevel warn
#
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
#
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
I am not an apache person, but I hope this helps.
Nick
On 15/07/2016 11:02, Mitchell Krog wrote:
> Hi Nick
>
> Would you mind greatly sharing your log format when you are back at your
> system? Would be interested to see what works better for you on Apache.
>
> My log format is very much the same as the OP and I often wonder why
> Fail2Ban doesn't pick up certain things to do with Apache.
>
> Kind Regards
> Mitch
>
>
> On 2016/07/15 11:48 AM, Nick Howitt wrote:
>> Silly question without my system in front of me to check, but is that
>> from your access log? Do you have a separate error log and do these
>> requests pop up there? My error log is natively in a completely
>> different format and good for fail2ban rules.
>>
>> Nick
>>
>> On 2016-07-15 10:39, Mohd Zainal Abidin wrote:
>>> I'm using http prefork.
>>>
>>> On Fri, Jul 15, 2016 at 5:34 PM, Mohd Zainal Abidin
>>> <[email protected]> wrote:
>>>
>>>> I'm not sure should block or not but if full path got other site
>>>> there.
>>>>
>>>> On Fri, Jul 15, 2016 at 5:32 PM, Alan Liddell
>>>> <[email protected]> wrote:
>>>>
>>>> If that's the verbatim output of your log, I'm pretty sure you'd
>>>> have to reconfigure how your web server writes its logs. Per the
>>>> manual:
>>>>
>>>> * In order for a log line to match your failregex, it actually has
>>>> to match in two parts: THE BEGINNING OF THE LINE HAS TO MATCH A
>>>> TIMESTAMP PATTERN OR REGEX, and the remainder of the line has to
>>>> match your failregex. If the failregex is anchored with a leading ^,
>>>> then the anchor refers to the start of the remainder of the line,
>>>> _after_ the timestamp and intervening whitespace
>>>>
>>>> Then if this pattern isn't in the usual filters, you'd have to
>>>> create one yourself. So if you rearranged it to output something
>>>> like
>>>> [15/Jul/2016:10:03:44 +0800] 27.111.213.117 "GET /2012/12/ HTTP/1.1"
>>>> 200 72434 "-" "Mozilla/4.0 (compatible;)"
>>>>
>>>> this regex should match it:
>>>>
>>>> ^ <HOST> "(GET|POST) /S+ HTTP/1.1" d+? d+? "-" "Mozilla/4.0
>>>> (compatible;)"$
>>>>
>>>> Not sure if the timestamp format is recognized. Maybe someone else
>>>> can chime in on this one?
>>>>
>>>> On Fri, Jul 15, 2016 at 3:07 AM, Mohd Zainal Abidin
>>>> <[email protected]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> How to block this kind of attack?
>>>>
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:32 +0800] "GET /2014/07/
>>>> HTTP/1.1" 200 70977 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:27 +0800] "GET /2007/05/
>>>> HTTP/1.1" 200 62797 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:33 +0800] "GET /2014/06/
>>>> HTTP/1.1" 200 72461 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:28 +0800] "GET /2006/12/
>>>> HTTP/1.1" 200 65124 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:34 +0800] "GET /2014/05/
>>>> HTTP/1.1" 200 72931 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:34 +0800] "GET /2014/04/
>>>> HTTP/1.1" 200 70848 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:32 +0800] "GET /xmlrpc.php?rsd
>>>> HTTP/1.1" 200 866 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:36 +0800] "GET /2014/02/
>>>> HTTP/1.1" 200 69820 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:36 +0800] "GET /2014/01/
>>>> HTTP/1.1" 200 74012 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET /2013/12/
>>>> HTTP/1.1" 200 74001 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET
>>>> /2007/10/page/2/ HTTP/1.1" 200 63882 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET
>>>> /2008/05/page/2/ HTTP/1.1" 200 63703 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET
>>>> /2008/04/page/2/ HTTP/1.1" 200 64863 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET
>>>> /2008/06/page/2/ HTTP/1.1" 200 64089 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET
>>>> /2007/12/page/2/ HTTP/1.1" 200 63587 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:29 +0800] "GET /2014/12/
>>>> HTTP/1.1" 200 73272 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:31 +0800] "GET /2006/11/
>>>> HTTP/1.1" 200 64642 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:37 +0800] "GET /2013/11/
>>>> HTTP/1.1" 200 68957 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:31 +0800] "GET /2006/09/
>>>> HTTP/1.1" 200 64719 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:38 +0800] "GET
>>>> /2008/01/page/2/ HTTP/1.1" 200 62711 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:38 +0800] "GET /2013/10/
>>>> HTTP/1.1" 200 70712 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:39 +0800] "GET
>>>> /2008/02/page/2/ HTTP/1.1" 200 64719 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:39 +0800] "GET
>>>> /2007/11/page/2/ HTTP/1.1" 200 64808 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:39 +0800] "GET /2013/09/
>>>> HTTP/1.1" 200 68252 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:32 +0800] "GET /2014/08/
>>>> HTTP/1.1" 200 69468 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:39 +0800] "GET /2013/08/
>>>> HTTP/1.1" 200 67360 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:41 +0800] "GET /2013/07/
>>>> HTTP/1.1" 200 70473 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:41 +0800] "GET /2013/06/
>>>> HTTP/1.1" 200 72604 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:35 +0800] "GET /2014/03/
>>>> HTTP/1.1" 200 68842 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET /2013/05/
>>>> HTTP/1.1" 200 74481 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET
>>>> /2007/09/page/2/ HTTP/1.1" 200 65605 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET
>>>> /2007/07/page/2/ HTTP/1.1" 200 64613 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET
>>>> /2007/08/page/2/ HTTP/1.1" 200 64851 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET
>>>> /2007/04/page/2/ HTTP/1.1" 200 65041 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET
>>>> /2007/06/page/2/ HTTP/1.1" 200 65219 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET
>>>> /2007/03/page/2/ HTTP/1.1" 200 66625 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:42 +0800] "GET /2013/03/
>>>> HTTP/1.1" 200 69079 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:43 +0800] "GET
>>>> /2007/01/page/2/ HTTP/1.1" 200 65362 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:43 +0800] "GET /2013/02/
>>>> HTTP/1.1" 200 71130 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:43 +0800] "GET
>>>> /2007/02/page/2/ HTTP/1.1" 200 65625 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:44 +0800] "GET
>>>> /2006/10/page/2/ HTTP/1.1" 200 64309 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:44 +0800] "GET /2013/01/
>>>> HTTP/1.1" 200 73073 "-" "Mozilla/4.0 (compatible;)"
>>>> 27.111.213.117 - - [15/Jul/2016:10:03:44 +0800] "GET /2012/12/
>>>> HTTP/1.1" 200 72434 "-" "Mozilla/4.0 (compatible;)"
>>>>
>>>> We getting this kind of attack from different ip last night. Our
>>>> website load goes to 100 and it become slow to response.
>>>>
>>>> --
>>>>
>>>> Thank you
>>>> ______________________
>>>>
>>>> Mohd Zainal Abidin
>>>>
>>>>
>>> ------------------------------------------------------------------------------
>>>> What NetFlow Analyzer can do for you? Monitors network bandwidth
>>>> and traffic
>>>> patterns at an interface-level. Reveals which users, apps, and
>>>> protocols are
>>>> consuming the most bandwidth. Provides multi-vendor support for
>>>> NetFlow,
>>>> J-Flow, sFlow and other flows. Make informed decisions using
>>>> capacity planning
>>>> reports.http://sdm.link/zohodev2dev [1]
>>>> _______________________________________________
>>>> Fail2ban-users mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/fail2ban-users [2]
>>>>
>>>> --
>>>>
>>>> ACL
>>> --
>>>
>>> Thank you
>>> ______________________
>>>
>>> Mohd Zainal Abidin
>>>
>>> --
>>>
>>> Thank you
>>> ______________________
>>>
>>> Mohd Zainal Abidin
>>>
>>>
>>> Links:
>>> ------
>>> [1] http://sdm.link/zohodev2dev
>>> [2] https://lists.sourceforge.net/lists/listinfo/fail2ban-users
>>>
>>> ------------------------------------------------------------------------------
>>> What NetFlow Analyzer can do for you? Monitors network bandwidth and
>>> traffic
>>> patterns at an interface-level. Reveals which users, apps, and
>>> protocols are
>>> consuming the most bandwidth. Provides multi-vendor support for
>>> NetFlow,
>>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>>> planning
>>> reports.http://sdm.link/zohodev2dev
>>>
>>> _______________________________________________
>>> Fail2ban-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/fail2ban-users
>> ------------------------------------------------------------------------------
>> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
>> patterns at an interface-level. Reveals which users, apps, and protocols are
>> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
>> J-Flow, sFlow and other flows. Make informed decisions using capacity
>> planning
>> reports.http://sdm.link/zohodev2dev
>> _______________________________________________
>> Fail2ban-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/fail2ban-users
>>
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity planning
> reports.http://sdm.link/zohodev2dev
> _______________________________________________
> Fail2ban-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/fail2ban-users
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
Fail2ban-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fail2ban-users
