Sorry, seems I didn't see your update.
Add an option:
request-timeout=60
to the WSGIDaemonProcess.
Set the value (in seconds) greater than you would expect your HTTP requests to
normally run.
What will happen is that when the average running time for all possible
concurrent requests exceeds that timeout value, the daemon process will be
forcibly restarted. This will have the effect of unblocking the process and a
new one will be started in its place. So acts as a fail safe to ensure your
application keeps running.
What this will also do is attempt to dump out Python stack traces for what all
the request handler threads were doing when the process is restarted. This will
hopefully allow you to work out why your request handlers are getting stuck, be
it they are getting stuck on a lock, or waiting on a backend service.
In short, your request handlers are getting stuck and not completing. Over time
these are building up and the thread pool for handling requests is exhausted
and so the process stops handling requests.
Graham
> On 7 Sep 2022, at 5:44 am, Juan Khawly <[email protected]> wrote:
>
> Any ideas?
>
> Thanks
>
> On Friday, September 2, 2022 at 8:47:47 AM UTC-4 Juan Khawly wrote:
> Hello Graham,
>
> I'm going to try to address your questions:
>
> Inside my Virtual Host
>
> Alias /static /data/home/user/project/frontend/build/static
> <Directory /data/home/user/project/frontend/build/static>
> Require all granted
> </Directory>
>
> <Directory /data/home/user/project/my_project>
> <Files wsgi.py>
> Require all granted
> </Files>
> </Directory>
>
> WSGIScriptAlias / /data/home/user/project/my_project/wsgi.py
> WSGIDaemonProcess my_project python-path=/data/home/user/project
> python-home=/data/home/user/environment/venv
> WSGIProcessGroup my_project
>
>
> Inside apache2.conf
>
> WSGIApplicationGroup %{GLOBAL}
>
> On the apache/error.log
> When I get the 503 on the access.log, these are the types of errors seen on
> the error.log
>
> One type of error
> [Thu Sep 01 04:22:21.520772 2022] [wsgi:error] [pid 3267:tid 140518453380864]
> [client 118.126.82.157:37722 <http://118.126.82.157:37722/>] Timeout when
> reading response headers from daemon process 'my_project':
> /data/home/project/my_project/my_project/wsgi.py
>
> Another type of error
> [Thu Sep 01 04:27:00.053558 2022] [wsgi:error] [pid 3267:tid 140518595991296]
> (11)Resource temporarily unavailable: [client 172.31.17.102:31880
> <http://172.31.17.102:31880/>] mod_wsgi (pid=3267): Unable to connect to WSGI
> daemon process ' my_project ' on '/var/run/apache2/wsgi.2385.1.1.sock' after
> multiple attempts as listener backlog limit was exceeded or the socket does
> not exist.
>
>
>
> Thanks,
> Juan Khawly
>
> On Thursday, September 1, 2022 at 5:54:43 PM UTC-4 Graham Dumpleton wrote:
> Would need to see the mod_wsgi configuration you are using to configure the
> WSGI application, including how WSGIDaemonProcess is configured and whether
> you are using WSGIApplicationGroup. Also, what errors are in the Apache error
> log when the 503 errors occur.
>
>
>> On 2 Sep 2022, at 4:57 am, Juan Khawly <[email protected] <>> wrote:
>>
>
>> Hello,
>>
>> I've been running into this problem for a while.
>>
>> CONTEXT
>>
>> I have an application developed in python (3.10), django 4.0.3, using
>> mod_wsgi and apache. The application is in a DEV environment and hosted in
>> AWS EC2. Currently, it does not receive traffic at all.
>>
>> Installation of Mod WSGI
>> apt-get install -y apache2-dev
>>
>> Setup out of the VENV
>> mod_wsgi-express install-module
>>
>> editing: /etc/apache2/mods-available/wsgi.load
>>
>> LoadModule wsgi_module
>> "/usr/lib/apache2/modules/mod_wsgi-py310.cpython-310-x86_64-linux-gnu.so
>> <http://mod_wsgi-py310.cpython-310-x86_64-linux-gnu.so/>"
>> WSGIPythonHome "/data/home/user/environment/venv"
>>
>> Module Enabled
>> a2enmod wsgi
>>
>> PROBLEM
>>
>> The application works perfect most of the time. Couple of times a week,
>> without traffic the apache server goes down into 503. Usually it is preceded
>> by a random request but it does not always happen that way. I am assuming
>> that is Slow DDOS but I want to make sure it is not miss configuration of
>> the WSGI.
>>
>> access.log example
>
>> <access.PNG>
>>
>> error.log example
>> I masked the internal routes
>>
>> This is one of the errors:
>> [Thu Sep 01 04:22:21.520772 2022] [wsgi:error] [pid 3267:tid
>> 140518453380864] [client 118.126.82.157:37722
>> <http://118.126.82.157:37722/>] Timeout when reading response headers from
>> daemon process 'XXXXX': /XXX/XXXX/XXXXX/XXXXX/XXXXXX/wsgi.py
>>
>> Another type of error:
>> [Thu Sep 01 04:22:21.520772 2022] [wsgi:error] [pid 3267:tid
>> 140518453380864] [client 118.126.82.157:37722
>> <http://118.126.82.157:37722/>] Timeout when reading response headers from
>> daemon process 'XXXXXXX': /XXX/XXXX/XXXXX/XXXXXX/XXXXXXX/wsgi.py
>>
>> SOLUTION
>>
>> If I restart the server, all works again until next failure.
>>
>> I've enabled the following modules, in case it is SlowDDOS
>> modreqtimeout
>> libapache2-mod-qos
>> libapache2-mod-security2.
>>
>> Any recommendation?
>>
>> Thanks,
>> Juan Khawly
>>
>>
>>
>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/modwsgi/3cc5285a-9943-4143-9b7f-5fa24e681c70n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/modwsgi/3cc5285a-9943-4143-9b7f-5fa24e681c70n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>> <access.PNG>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected]
> <mailto:[email protected]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/modwsgi/d837af27-86cb-4ef6-877a-40f52418a8d0n%40googlegroups.com
>
> <https://groups.google.com/d/msgid/modwsgi/d837af27-86cb-4ef6-877a-40f52418a8d0n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/modwsgi/8C6B57B1-4DF5-445B-9F25-47596D6BF9CB%40gmail.com.
