If we don't mention options like no.of process and no.of threads in WSGIDaemonProcess directive by default it will consider 1 process with 15 threads in mpm settings as event prefork right. Does the above configuration helps in serving 15 request concurrently. If so, then what happens when there are more no.of concurrent requests hit our web application. Does 15 threads means it serves 15 requests concurrently.
RajKumar On Wednesday 31 July 2024 at 12:47:09 UTC+5:30 RajKumar Ambadipelli wrote: > Thankyou for clarifying. > > RajKumar > > On Wednesday 31 July 2024 at 11:56:48 UTC+5:30 Graham Dumpleton wrote: > >> If you define a daemon process group, the number of processes defined for >> the group will always be started and running. >> >> With the way you have configured things your Python web application code >> is only loaded lazily by a process in a daemon process group when the first >> request arrives which is to be handled by that process. >> >> Thus, prior to any request being received, the memory footprint of the >> processes should be similar to that of running an empty Python interpreter. >> >> If your web applications are infrequently accessed and you want to >> minimise memory usage, then add to WSGIDaemonProcess the option: >> >> inactivity-timeout=30 >> >> See >> https://modwsgi.readthedocs.io/en/master/configuration-directives/WSGIDaemonProcess.html >> for >> details, but basically what it means is that the process will be restarted >> when idle and will revert memory usage by the process back to the base >> level. >> >> Do note however that if you Python web application takes a long time to >> load, then this may be noticeable to users if the Python web application >> code is going to have to be reloaded frequently. >> >> As to CPU usage, the process if not handling requests will be waiting on >> the socket on which requests are listening, so it should not be using any >> CPU at that time. >> >> There are other options to WSGIDaemonProcess you could play with to >> recycle the process periodically, but what might be appropriate really >> depends on your specific web application so I can't give concrete >> suggestions of what to use. >> >> Graham >> >> On 31 Jul 2024, at 2:55 PM, RajKumar Ambadipelli <[email protected]> >> wrote: >> >> What exactly I want is I want to know WSGIDaemonProcess which are ideal >> not receiving any requests and not serving response but only declared, How >> does this WSGIDaemonProcess effect my resources like memory and cpu. >> >> On Wednesday 31 July 2024 at 10:22:23 UTC+5:30 RajKumar Ambadipelli wrote: >> >>> Creating multiple virtualhost with different mod_wsgi daemons I meant >>> 'WSGIDaemonProcess' >>> and not using it that is it will not server and requests but only >>> declared does this effect my resources like memory and CPU. >>> On Monday 29 July 2024 at 12:45:21 UTC+5:30 Graham Dumpleton wrote: >>> >>>> Comes down to how much memory and CPU your machine has, plus how much >>>> traffic the sites get. So will depend and you will just have to see how it >>>> goes. >>>> >>>> On 29 Jul 2024, at 5:12 PM, RajKumar Ambadipelli <[email protected]> >>>> wrote: >>>> >>>> Yeah it is working fine what I did was >>>> >>>> step1. Unlinked existing symlink >>>> Step2. Create new symlink >>>> Step3. Managed permissions and shell context of files >>>> >>>> Therefore no >>>> reload or restart is needed. >>>> >>>> But I have doubt up to how many mod_wsgi daemons we can efficiently >>>> without any disturbance and up to what extent we can trust this deployment >>>> process using mod_wsgi in daemon mode for production environment. >>>> >>>> Thankyou, >>>> RajKumar >>>> On Thursday 25 July 2024 at 13:04:18 UTC+5:30 RajKumar Ambadipelli >>>> wrote: >>>> >>>>> Sorry that uoadmin is actually admin. >>>>> >>>>> Ok I will try this one. >>>>> >>>>> Thankyou, >>>>> RajKumar >>>>> >>>>> On Thursday 25 July 2024 at 12:59:13 UTC+5:30 Graham Dumpleton wrote: >>>>> >>>>>> You have two different directories which may complicate it a little, >>>>>> but what you can do is have something like: >>>>>> >>>>>> Listen 9002 <VirtualHost *:9002> >>>>>> ServerName test.myapp.com >>>>>> ErrorLog /var/log/webservice_error.log >>>>>> WSGIPassAuthorization On >>>>>> WSGIDaemonProcess Tes9002 >>>>>> python-path=/home/uoadmin/releases/test/students:/home/admin/releases/test/shared >>>>>> >>>>>> display-name=%{GROUP} >>>>>> WSGIProcessGroup Tes9002 WSGIApplicationGroup %{GLOBAL} >>>>>> WSGIScriptAlias / /home/admin/releases/test/students/conf/wsgi.py >>>>>> <Directory /home/admin/releases/test/students/conf> >>>>>> <Files wsgi.py> Require all granted </Files> >>>>>> </Directory> >>>>>> </VirtualHost> >>>>>> >>>>>> <VirtualHost *:9002> >>>>>> ServerName dev.myapp.com >>>>>> ErrorLog /var/log/webservice_error.log >>>>>> WSGIPassAuthorization On >>>>>> WSGIDaemonProcess Dev9002 >>>>>> python-path=/home/uoadmin/releases/dev/students:/home/admin/releases/dev/shared >>>>>> >>>>>> display-name=%{GROUP} >>>>>> WSGIProcessGroup Dev9002 WSGIApplicationGroup %{GLOBAL} >>>>>> WSGIScriptAlias / /home/admin/releases/dev/students/conf/wsgi.py >>>>>> <Directory /home/admin/releases/dev/students/conf> >>>>>> <Files wsgi.py> Require all granted </Files> >>>>>> </Directory> >>>>>> </VirtualHost> >>>>>> >>>>>> In this case /home/admin/releases/dev would actually be a symlink to >>>>>> the versioned directory. >>>>>> >>>>>> Not sure why have separate uoadmin directory, but also have >>>>>> /home/uoadmin/releases/dev as symlink to the versioned directory. >>>>>> >>>>>> To change what is used, remove/recreate symlinks so point to the >>>>>> directory for the new version. >>>>>> >>>>>> That the wsgi.py has changed should trigger a process restart if auto >>>>>> reloading is on (default). >>>>>> >>>>>> Alternatively could disable auto reloading and manually send SIGUSR1 >>>>>> to process to trigger a graceful reload after changing the symlink. >>>>>> >>>>>> That all said, I can't remember if you might have to configure Apache >>>>>> to tell it to allow following symlinks (Options FollowSymLinks). If >>>>>> it were static file handling would definitely be needed, but can't >>>>>> remember >>>>>> if would be required in this case where is WSGI application handling >>>>>> things. >>>>>> >>>>>> Graham >>>>>> >>>>>> On 25 Jul 2024, at 5:21 PM, RajKumar Ambadipelli <[email protected]> >>>>>> wrote: >>>>>> >>>>>> Ok, If I have only two virtualhosts all the time and I am going to >>>>>> change only python-path will that work with the direct signal using >>>>>> SIGUSR1. >>>>>> >>>>>> If the above is possible then I think I have some kind of solution. >>>>>> >>>>>> Thankyou, >>>>>> RajKumar >>>>>> >>>>>> On Thursday 25 July 2024 at 12:35:09 UTC+5:30 Graham Dumpleton wrote: >>>>>> >>>>>>> Are you always using the same two virtual host server names and just >>>>>>> updating the version number in the paths? >>>>>>> >>>>>>> On 25 Jul 2024, at 4:21 PM, RajKumar Ambadipelli <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Yes I am adding new virtual hosts when ever I want to release a new >>>>>>> version of that services lets say initially my virtualhost config will >>>>>>> be >>>>>>> like >>>>>>> >>>>>>> #Students Webservice Config >>>>>>> Listen 9002 <VirtualHost *:9002> >>>>>>> ServerName test.myapp.com >>>>>>> ErrorLog /var/log/webservice_error.log >>>>>>> WSGIPassAuthorization On >>>>>>> WSGIDaemonProcess Tes9002 python-path=/home/uoadmin/releases/1.0.0 >>>>>>> /students:/home/admin/releases/1.0.0/shared display-name=%{GROUP} >>>>>>> WSGIProcessGroup Tes9002 WSGIApplicationGroup %{GLOBAL} >>>>>>> WSGIScriptAlias / /home/admin/releases/1.0.0/students/conf/wsgi.py >>>>>>> <Directory /home/admin/releases/1.0.0/students/conf> >>>>>>> <Files wsgi.py> Require all granted </Files> >>>>>>> </Directory> >>>>>>> </VirtualHost> >>>>>>> >>>>>>> When i want to go for new releases the down part is appended to >>>>>>> above part >>>>>>> >>>>>>> <VirtualHost *:9002> >>>>>>> ServerName dev.myapp.com >>>>>>> ErrorLog /var/log/webservice_error.log >>>>>>> WSGIPassAuthorization On >>>>>>> WSGIDaemonProcess Dev9002 python- path=/home/uoadmin/releases/1.1.0 >>>>>>> /students:/home/admin/releases/1.1.0/shared display-name=%{GROUP} >>>>>>> WSGIProcessGroup Dev9002 WSGIApplicationGroup %{GLOBAL} >>>>>>> WSGIScriptAlias / /home/admin/releases/1.1.0/students/conf/wsgi.py >>>>>>> <Directory /home/admin/releases/1.1.0/students/conf> >>>>>>> <Files wsgi.py> Require all granted </Files> >>>>>>> </Directory> >>>>>>> </VirtualHost> >>>>>>> >>>>>>> >>>>>>> Now I am going to have two virtualhosts with two daemons 1st is >>>>>>> already recognized by apache server where as second one is not yet >>>>>>> recognized by apache server >>>>>>> >>>>>>> Thankyou, >>>>>>> RajKumar >>>>>>> >>>>>>> On Thursday 25 July 2024 at 11:40:31 UTC+5:30 Graham Dumpleton wrote: >>>>>>> >>>>>>>> What is the reason for doing the graceful restart? Is it because >>>>>>>> you are adding/removing virtual hosts, or making some other Apache >>>>>>>> config >>>>>>>> change. >>>>>>>> >>>>>>>> You do not need to do a complete Apache restart if just want to >>>>>>>> force a daemon process to restart, you can instead send the processes >>>>>>>> a >>>>>>>> signal directly. From memory it is SIGUSR1 that triggers a graceful >>>>>>>> restart >>>>>>>> of processes, but you will need to confirm that. >>>>>>>> >>>>>>>> On 25 Jul 2024, at 3:28 PM, RajKumar Ambadipelli < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> When i have 260 microservices those all are light weight >>>>>>>> applications using same python interpreter and with django rest api >>>>>>>> framework, and currently each application hosted on apache server >>>>>>>> usign >>>>>>>> mod_wsgi daemon mode and my main problem is while making changes to >>>>>>>> one of >>>>>>>> application virtualhost other ongoing daemons are distured as i need >>>>>>>> to >>>>>>>> reload or restart. >>>>>>>> All those 260 services very light weight each listen to http >>>>>>>> request on unique ports. >>>>>>>> >>>>>>>> ThankYou >>>>>>>> RajKumar >>>>>>>> >>>>>>>> On Tuesday 23 July 2024 at 16:37:42 UTC+5:30 Graham Dumpleton wrote: >>>>>>>> >>>>>>>>> One can optimise embedded mode for better performance, but I would >>>>>>>>> put a big caveat on that and say is only probably a good idea to >>>>>>>>> tackle if >>>>>>>>> you have the one web service. >>>>>>>>> >>>>>>>>> Running 260 micro services in one Apache httpd instance with >>>>>>>>> mod_wsgi sounds rather scary either way. >>>>>>>>> >>>>>>>>> If you use mod_wsgi daemon mode where each micro service is in its >>>>>>>>> own daemon process group (with a single process and small number of >>>>>>>>> threads), then you might get away with it if these aren't high volume >>>>>>>>> sites. That said, it is still a lot of managed daemon mode processes >>>>>>>>> and >>>>>>>>> not sure how well Apache will handle that, especially on restarts. >>>>>>>>> >>>>>>>>> Running them all in embedded mode would be a bad idea if each >>>>>>>>> needs a separate Python interpreter context because the Apache worker >>>>>>>>> process would be huge in size. If Apache httpd was configured for >>>>>>>>> prefork >>>>>>>>> MPM it would be even worse because you would have a potentially large >>>>>>>>> number of worker processes since all are single thread. You also run >>>>>>>>> a big >>>>>>>>> risk with micro services interfering with each other in strange ways >>>>>>>>> if >>>>>>>>> running in different sub interpreter contexts of the one process due >>>>>>>>> to how >>>>>>>>> Python imports C extensions, and process wide environment variables >>>>>>>>> work. >>>>>>>>> Various third party Python packages with C extensions will not even >>>>>>>>> work in >>>>>>>>> Python sub interpreters (eg., anything related to numpy). >>>>>>>>> >>>>>>>>> You definitely want event or worker MPM, but even then, for 260 >>>>>>>>> micro services, if they need separate Python interpreter context I >>>>>>>>> can't >>>>>>>>> really recommend it still because of size concerns for processes and >>>>>>>>> potential cross sub interpreter interference. >>>>>>>>> >>>>>>>>> So the question is whether when you said 260 micro services you >>>>>>>>> really mean independent web applications, or whether you just mean >>>>>>>>> you have >>>>>>>>> 260 different unique HTTP handlers as part of the one application, >>>>>>>>> and thus >>>>>>>>> in the same Python interpreter context. >>>>>>>>> >>>>>>>>> When people talk about such large number of micro services, >>>>>>>>> usually you would not be aiming to host them in a single Apache >>>>>>>>> instance >>>>>>>>> but would instead be looking at running something which can handle >>>>>>>>> things >>>>>>>>> at scale like Kubernetes and creating separate deployments for them >>>>>>>>> in >>>>>>>>> that, relying on the ingress routing Kubernetes provides to get >>>>>>>>> traffic to >>>>>>>>> the appropriate micro service. >>>>>>>>> >>>>>>>>> Graham >>>>>>>>> >>>>>>>>> On 23 Jul 2024, at 7:13 PM, RajKumar Ambadipelli < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>> mod_wsgi in embeded mode allows graceful restart, >>>>>>>>> What are the potential issues that I will face if I use mod_wsgi >>>>>>>>> in embedded mode instead of daemon mode, >>>>>>>>> I have to host around 260 python micro services. >>>>>>>>> >>>>>>>>> I have saw your blog on 'why are you using mod_wsgi in embedded >>>>>>>>> mode?' But, I unable to understand it very well in that you mentioned >>>>>>>>> if we >>>>>>>>> configure mpm settings correctly then mod_wsgi in embedded mode is >>>>>>>>> better >>>>>>>>> than daemon mode but not mentioned any configurations. >>>>>>>>> >>>>>>>>> Thanking you, >>>>>>>>> RajKumar >>>>>>>>> >>>>>>>>> On Tuesday 23 July 2024 at 13:04:50 UTC+5:30 Graham Dumpleton >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On 23 Jul 2024, at 4:09 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>>>> gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> I am using Apache Server with mod_wsgi for hosting my python >>>>>>>>>> django applications. Versions: Python 3.9.18 Server version: >>>>>>>>>> Apache/2.4.57 >>>>>>>>>> mod-wsgi==4.7.1 >>>>>>>>>> >>>>>>>>>> One of my application virtual host configuration with two >>>>>>>>>> different versions: >>>>>>>>>> >>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> So, When the source code is modified I can referesh the wsgi >>>>>>>>>> daemon using touch >>>>>>>>>> /home/uoadmin/releases/1.1.0/students/conf/wsgi.py touch >>>>>>>>>> /home/uoadmin/releases/1.0.0/students/conf/wsgi.py But when I added >>>>>>>>>> new >>>>>>>>>> virtualhost to the above configuration file or else when I modify >>>>>>>>>> above >>>>>>>>>> file the apache server unable to recognize modifications made the >>>>>>>>>> existing >>>>>>>>>> virtualhost or newly added virtualhost until doing apachectl >>>>>>>>>> graceful (or) >>>>>>>>>> apachectl restart (or) systemctl reload httpd but all the commands >>>>>>>>>> above >>>>>>>>>> killing the ongoing requests forcefully directly terminating them. >>>>>>>>>> >>>>>>>>>> How to handle above situation. >>>>>>>>>> >>>>>>>>>> I want to know how will apache server recognize modifications to >>>>>>>>>> virtualhost or newly added virtual host without reloading or >>>>>>>>>> restarting. >>>>>>>>>> >>>>>>>>>> It can't, Apache httpd requires you to perform a restart (reload) >>>>>>>>>> in order to read changes to the Apache configuration files. That is >>>>>>>>>> how it >>>>>>>>>> works. >>>>>>>>>> >>>>>>>>>> If above is not possible then is there anyway for restarting or >>>>>>>>>> reloading apache server gracefully that is without terminating or >>>>>>>>>> killing >>>>>>>>>> other ongoing requests or daemons while using apache server + >>>>>>>>>> mod_wsgi for >>>>>>>>>> serving python with django? >>>>>>>>>> >>>>>>>>>> Unfortunately not. The way Apache httpd manages the mod_wsgi >>>>>>>>>> daemon processes it will force a restart of those as well and even >>>>>>>>>> though >>>>>>>>>> Apache has a concept of graceful restart for it's own worker child >>>>>>>>>> processes, it doesn't extend that to managed process like the >>>>>>>>>> mod_wsgi >>>>>>>>>> daemon process and always restarts them immediately even when it is >>>>>>>>>> a >>>>>>>>>> graceful restart. There is nothing that can be done about this. >>>>>>>>>> >>>>>>>>>> The only way you could handle it if you need to be able to freely >>>>>>>>>> restart the main Apache server and have it not affect your Python >>>>>>>>>> web >>>>>>>>>> applications, is to run the Python web applications in distinct >>>>>>>>>> secondary >>>>>>>>>> web server processes and use the main Apache server to only proxy >>>>>>>>>> requests >>>>>>>>>> through to the secondary web servers. >>>>>>>>>> >>>>>>>>>> For the second web servers you could use mod_wsgi-express to make >>>>>>>>>> things easier, but you could also just not use mod_wsgi for the >>>>>>>>>> secondary >>>>>>>>>> web servers and use gunicorn or some other standalone Python >>>>>>>>>> WSGI/asyncio >>>>>>>>>> web server. >>>>>>>>>> >>>>>>>>>> Graham >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "modwsgi" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/modwsgi/d28663bc-a143-4e4f-949d-38e065c5ac9fn%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/modwsgi/d28663bc-a143-4e4f-949d-38e065c5ac9fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "modwsgi" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/modwsgi/1fffb2f7-ed8a-4d88-a52b-00e7e82e98d5n%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/modwsgi/1fffb2f7-ed8a-4d88-a52b-00e7e82e98d5n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "modwsgi" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/modwsgi/27697b57-c903-4881-bddd-691060d62b47n%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/modwsgi/27697b57-c903-4881-bddd-691060d62b47n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "modwsgi" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/modwsgi/0794a799-7f70-4d68-affe-b2b7a4f43529n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/modwsgi/0794a799-7f70-4d68-affe-b2b7a4f43529n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> >>>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "modwsgi" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/modwsgi/7edb7907-915b-41a6-a581-097ea1b87dc0n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/modwsgi/7edb7907-915b-41a6-a581-097ea1b87dc0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> >>>> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/modwsgi/618674de-ba2c-49df-8df9-10db6624df7fn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/modwsgi/618674de-ba2c-49df-8df9-10db6624df7fn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> >> -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/f08337f6-63a6-4fc4-a630-f437187b2b2en%40googlegroups.com.
