> We have a Django application with heavy workload. > > Now, we have a system with 3 servers and each server with 8 cores. > > Current deployment: > - Django is running through twisted. > - We have launched 16 twisted processes per server > - We use NGINX as load balancer between all nodes (48). > - Our application is a REST API. We don't use Django for user interface. > > This way we can distribute workload between all processors and I can > confirm is working ok when we increase activity. I see all processors > working and %CPU growing. This solution works in LINUX and Windows > (changing NGINX with APACHE for load balancing). > > I would like to do this with uwsgi but I cannot expand uwsgi to more than > one core. I have tried running uwsgi with 8 and 16 processes and I can > see > activity in one processor only. I have tried with cpu-affinity, processes, > workers, ... but I never get to use more than one CPU. > > Our processes have intensive python computation that must be made > synchronously. > > I would like to know if UWSGI is able to manage multiple processes > distributing workload between all system CPUs. > > Thanks in advance > _______________________________________________ >
This has nothing to do with uWSGI, but with how your kernel manages sockets and accept() syscall when multiple threads and processes are in place (more on this here: http://uwsgi-docs.readthedocs.org/en/latest/articles/SerializingAccept.html). The first thing to take in account is that the kernel could favour the same process (and cpu) over and over again to avoid cpu trash caches and other things, so you should trust the kernel the vast majority of time. I suppose you want to run your app with 16 processes per server so you will end with: uwsgi --processes 16 --master --socket xxx --wsgi-file xxx this will give you 16 processes sharing the same socket, and with the kernel passing requests to the best process candidate (and as we seen before this does not mean a different cpu everytime) You could distribute requests in a more "deterministic" way, adding --thunder-lock (even if its purpose is another one). In some scenario distributing requests between processes (processes, not cpus) could be a good approach. You can eventually map a process to be "attached" to a cpu (with cpu-affinity), but this will result in the vast majority of time in worst performance (and in virtualized setups this does not even work in consistent ways) So, it is the kernel that choose how to use your multiple cores, and if you see a single cpu doing the vast majority of work is because this will give you the best performance for the specific case. Btw, double check your configuration, i suppose with twisted you had every process bound to a different port and nginx doing the load balacing, while with uWSGI you have a port per server and the kernel distributing requests between processes bound to the same port. (you could eventually use uWSGI in the same way of twisted with one port per process but it is suboptimal since --thunder-lock is available) -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
