Thanks Graham for your prompt response. To answer your questions: - Yes using mod_wsgi-express. - onbuild: I am not wedded to this - I was just following the least line or resistance to port my application. What precisely are you recommending and what do I need to change (I am running docker-compose to orchestrate multiple wsgi containers behind haproxy acting as application selector). - The app is part of a jenkins build pipeline. The request normally takes around 60-120 seconds. In worst case it takes 6 hours (I know! - we have 100,000+ lines of XML across around 100 files to parse (submitted with the request) and to write to the database as a single transaction. It is almost impossible to split these up into small transactions and roll back if anything should fail. It would also leave the database in a "live" but inconsistent state until done. This has been running in full production for a couple of years now without issue so although not architecturally nice - it works. The dockerisation was part of providing more concurrency - We have 1 process (default) running on each wsgi container. - We have 5 threads (default) - Normally expect only1 concurrent request that writes to the database (most of the DB is write locked due to the transaction - so pointless trying more) We try to serialise access via jenkins pipelines and reference an unoccupied wsgi container - but expect a small number of concurrent database "read only client" accesses (20 worst case estimate).
Currently (bypassing the haproxy) and talking direct to a mod_wsgi container we are getting gateway timeouts. - I found .whiskey/server_args last night and added --include-file reference so my /tmp/mod_wsgi-localhost:80:0/http.conf now contains Include '/app/extra.conf' so I can add apache2 directives there. I also tried --socket-timeout and --request-timeout as you have subsequently recommended. It appears that Apache Timeout and --socket-timeout are the critical items that need to be increased. Whilst that has moved me forward a bit I am subsequently hitting other problems which result in wsgi process being abruptly restarted (with little or nothing reported in the apache logs) when sent test data taken from my production environment. I am seeing "Truncated or oversized response headers received from daemon process liocalhost:80" or simply "nothing reported" followed by "[MainThread] Returning wsgi_app" from my wsgi bootstrap as it restarts. Client see as 500 internal server error in this case. I am not sure if these relate to timeouts or how to debug further. They are very consistent in terms of time after the run starts (20 seconds). Dave -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
