See comments below.
> On 17 Aug 2021, at 7:26 am, Matt J <[email protected]> wrote:
>
> Hi Graham,
>
> Thank you for your response. I am now using mod_wsgi-express, and running
> into what I believe is a final issue - SSL errors. Please see this new
> configuration:
>
> conf.d/djangosite.conf - it can be noted that most, but not all other apps on
> this server use localhost in the proxy pass, but I believe it should be the
> hostname for our case. Maybe you can correct me on this.
> <Location "/apps/onequeue">
> ProxyPass "http://server01.corp.com:9889 <http://server01.corp.com:9889/>"
> retry=0
> ProxyPassReverse "http://server01.corp.com:9889
> <http://server01.corp.com:9889/>"
Note that based on this you are proxying to backend as HTTP.
> <IF "req('Authorization') =~ /^Bearer/">
> stuff
> </IF>
> <ELSE>
> stuff but with LDAP
> </ELSE>
>
> Require valid-user
> RequestHeader set X-Remote-User %{REMOTE_USER}s
> RequestHeader set X-Remote-Host %{REMOTE_HOST}s
> </Location>
>
> conf/httpd.conf - only relevant items shown (save a bunch of
> location-specific ldap crap)
> # LoadModule foo_module modules/mod_foo.so
> Include conf.modules.d/*.conf
> #MOD_WSGI installed from SOURCE using source python3.7 with shared flags
> LoadFile /deployment/tools/lib/libpython3.7m.so.1.0
You don't need this LoadFile line.
> #LoadModule wsgi_module modules/mod_wsgi.so
>
> #MOD_WSGI-EXPRESS installed from PyEnv with mod_wsgi installed via PIP
> LoadModule wsgi_module modules/mod_wsgi-py37.cpython-37m-x86_64-linux-gnu.so
> WSGIPythonHome "/deployment/apps/envs/djangosite-api"
You don't need to include mod_wsgi in the front end Apache anymore if the only
thing using mod_wsgi is now as a backend using mod_wsgi-express.
> Include conf.d/*.conf
>
> ServerName server01.corp.com:80 <http://server01.corp.com/>
>
> <VirtualHost *:80>
> Redirect permanent / https://server01.corp.com/
> <https://server01.corp.com/>
> </VirtualHost>
>
>
> conf.d/ssl.conf - All items shown (defaults may be commented)
>
> Listen 443 https
>
> SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
> SSLSessionCache shmcb:/run/httpd/sslcache(512000)
> SSLSessionCacheTimeout 300
>
> SSLRandomSeed startup file:/dev/urandom 256
> SSLRandomSeed connect builtin
> SSLCryptoDevice builtin
>
> <VirtualHost _default_:443>
>
> ErrorLog logs/ssl_error_log
> LogLevel warn
> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D"
> combined
> LogFormat "%h %l %u %t \"%r\" %>s %b" common
> LogFormat "%{Referer}i -> %U" referer
> LogFormat "%{User-agent}i" agent
> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I
> %O" combinedio
> CustomLog "|/sbin/rotatelogs /var/log/httpd/access.%Y%m%d%H%M%S 10M" combined
>
> SSLEngine on
>
> SSLProtocol all -SSLv2 -SSLv3
>
> SSLCipherSuite HIGH:3DES:!aNULL:!MD5:!SEED:!IDEA
>
> SSLCertificateFile /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
>
> SSLCertificateKeyFile /etc/pki/tls/private/server.key
>
> <Files ~ "\.(cgi|shtml|phtml|php3?)$">
> SSLOptions +StdEnvVars
> </Files>
> <Directory "/var/www/cgi-bin">
> SSLOptions +StdEnvVars
> </Directory>
>
> BrowserMatch "MSIE [2-5]" \
> nokeepalive ssl-unclean-shutdown \
> downgrade-1.0 force-response-1.0
> </VirtualHost>
>
>
>
> Steps Taken during Installation:
> export APXS=/usr/bin/apxs
> pip install --proxy=http://proxy.corp.com:80 <http://proxy.corp.com/>
> --trusted-host pypi.python.org <http://pypi.python.org/> --trusted-host
> files.pythonhosted.org <http://files.pythonhosted.org/> mod_wsgi
> /deployment/apps/envs/djangosite-api/bin/mod_wsgi-express install-module
> (output placed in above httpd.conf, I know this part is all fine)
You don't need to run 'mod_wsgi-express install-module' if using
mod_wsgi-express as backend. When mod_wsgi-express start-server is run it will
use version of mod_wsgi.so file installed in the Python virtual environment.
In other words, there doesn't need to be anything in the front end Apache for
mod_wsgi, only the config to set it up as a proxy which are just Apache
directives.
>
> Issue Occuring:
> Alas, we are at the command I am trying to use to test the server: (I have an
> identical variant for manage.py runmodwsgi that produces the same error).
>
> setup @
> mod_wsgi-express setup-server /deployment/source/djangosite/mysite/wsgi.py
> --reload-on-changes --port=9889 --https-port=443 --https-only
> --server-name=server01.corp.com <http://server01.corp.com/>
> --mount-point=/apps/djangosite --url-alias /apps/djangosite/static
> /deployment/source/djangosite/mysite/static/
> --ssl-certificate-file=/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
> --ssl-certificate-key-file=/etc/pki/tls/private/server.key --user=opc
> --group=cad
> --server-root=/deployment/source/djangosite/mysite/mod_wsgi-express
You are only proxying HTTP, not HTTPS, the backend doesn't need:
--https-port=443 --https-only --server-name=server01.corp.com
<http://server01.corp.com/>
--ssl-certificate-file=/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
--ssl-certificate-key-file=/etc/pki/tls/private/server.key
Your proxy is for /apps/onequeue but you are using
--mount-point=/apps/djangosite which doesn't match.
You are not using any of the options talked about in the two blog posts about
using mod_wsgi-express behind a proxy.
> start server @
> /deployment/source/mysite/mod_wsgi-express/apachectl start
> command output:
> (13)Permission denied: AH00072: make_sock: could not bind to address [::]:443
> (13)Permission denied: AH00072: make_sock: could not bind to address
> 0.0.0.0:443
> no listening sockets available, shutting down
> AH00015: Unable to open logs
>
> In this case it cannot use 443.
Correct.
>
> setup again @ (without specify --https-port and --https-only)
> mod_wsgi-express setup-server /deployment/source/djangosite/mysite/wsgi.py
> --reload-on-changes --port=9889 --server-name server01.corp.com
> <http://server01.corp.com/> --mount-point=/apps/djangosite --url-alias
> /apps/djangosite/static /deployment/source/djangosite/mysite/static/
> --ssl-certificate-file=/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
> --ssl-certificate-key-file=/etc/pki/tls/private/server.key --user opc
> --group cad
> --server-root=/deployment/source/djangosite/mysite/mod_wsgi-express
> --startup-log --access-log
As above, you don't need: --server-name server01.corp.com
<http://server01.corp.com/>
-ssl-certificate-file=/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
--ssl-certificate-key-file=/etc/pki/tls/private/server.key
> start server @
> /deployment/source/mysite/mod_wsgi-express/apachectl start
> web browser output: (chrome)
> [URL CHANGED BY BROWSER]
> https://server01.corp.com/apps/djangosite
> <https://server01.corp.com/apps/djangosite> ->
> https://server01.corp.com:9889/apps/djangosite
> <https://server01.corp.com:9889/apps/djangosite>
Your browser shouldn't see https://server01.corp.com:9889/apps/djangosite
<https://server01.corp.com:9889/apps/djangosite> it should only see
https://server01.corp.com/apps/djangosite
<https://server01.corp.com/apps/djangosite>
This the fact you are seeing the backend port is probably because you are
missing all the special options to mod_wsgi-express when using proxying
mentioned in:
http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html
<http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html>
http://blog.dscpl.com.au/2015/07/redirection-problems-when-proxying-to.html
<http://blog.dscpl.com.au/2015/07/redirection-problems-when-proxying-to.html>
It is very important you have the options as to which proxy headers to trust.
There is a chance your Django project code needs to change if it doesn't do URL
reconstruction properly with functions/template tags supplied by Django for the
purpose.
> server01.corp.com <http://server01.corp.com/> sent an invalid response.
> ERR_SSL_PROTOCOL_ERROR
>
If the backend is running on the same machine and not a separate machine on a
private network, we will also need to tell mod_wsgi-express to only listen on
127.0.0.1 and the proxy will need to use http://127.0.0.1:9889
<http://127.0.0.1:9889/> and not http://server01.corp.com:9889
<http://server01.corp.com:9889/> as the backend shouldn't be directly
contactable by the browser. I will need to double check what needs to be done
to ensure it only listens for connections on 127.0.0.1.
> In this case, I get an ambiguous ssl error. I do know that the browser
> changing my URL can be a proxy issue, but am unsure how to solve it in this
> case.
>
> So here my question is, how/what am I doing wrong to serve my app using
> mod_wsgi-express to upgrade insecure requests and use https behind an apache
> proxy?
>
First up you need those special options to mod_wsgi-express saying what proxy
headers you need to trust per the blog posts.
>
> As a side note, being able to use the apachectl start and stop with systemd
> will be a blessing. I'm not sure exactly how you are able to recreate/serve
> an apache module outside of apache like this but its very cool, very smart,
> and I cant wait to show my friends it working.
>
Am not a systemd expert, but you should be able to set up systemd with a config
that or start runs:
/deployment/source/mysite/mod_wsgi-express/apachectl start
and for stop runs:
/deployment/source/mysite/mod_wsgi-express/apachectl stop
> Along the way, I used these resources:
>
> https://pypi.org/project/mod-wsgi/ <https://pypi.org/project/mod-wsgi/>
> https://groups.google.com/g/modwsgi/c/nVNtX4zJGBY
> <https://groups.google.com/g/modwsgi/c/nVNtX4zJGBY>
> https://github.com/GrahamDumpleton/mod_wsgi/issues/411
> <https://github.com/GrahamDumpleton/mod_wsgi/issues/411>
> Aforementioned blog posts
>
> Thank you for your time,
> Matthew Johnson
>
>
> On Friday, August 13, 2021 at 4:13:18 PM UTC-6 Graham Dumpleton wrote:
> Since you tried gunicorn and your other sub sites do it, why not use Apache
> as a proxy to mod_wsgi-express?
>
> Use of mod_wsgi-express was already mentioned in that video you linked. You
> can find some blog posts specifically about the topic of using it behind
> Apache as a proxy at:
>
> http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html
> <http://blog.dscpl.com.au/2015/06/proxying-to-python-web-application.html>
> http://blog.dscpl.com.au/2015/07/redirection-problems-when-proxying-to.html
> <http://blog.dscpl.com.au/2015/07/redirection-problems-when-proxying-to.html>
>
> Just ignore that it talks about the mod_wsgi-express running in docker as
> that doesn't need to be the case.
>
> Finally, since part of the issue seems to relate to the mount point of the
> application, mod_wsgi-express does have a --mount-point option so that you
> can when it is behind the proxy still have it mounted as a sub path to make
> things potentially easier.
>
> If you are concerned about performance, your application is robust and you
> don't need the protections that daemon mode of mod_wsgi-express offers for
> auto recovery (which you aren't using in your current manual Apache setup
> with daemon mode), you can set the backend mod_wsgi-express to use embedded
> mode using --embedded-mode. This will eliminate some of the extra latency of
> using a proxy to backend Apache to daemon mode.
>
> Using mod_wsgi-express means that also not subject to using the older
> mod_wsgi version that RHEL ships with as installing it from PyPi instead.
>
> So as first step perhaps get your application working under mod_wsgi-express
> and see if it works and ask questions about anything you aren't quite sure
> about or can't get working.
>
> Also see:
>
> http://blog.dscpl.com.au/2015/04/using-modwsgi-express-with-django.html
> <http://blog.dscpl.com.au/2015/04/using-modwsgi-express-with-django.html>
> http://blog.dscpl.com.au/2015/04/integrating-modwsgi-express-as-django.html
> <http://blog.dscpl.com.au/2015/04/integrating-modwsgi-express-as-django.html>
>
> Graham
>
>
>> On 14 Aug 2021, at 2:40 am, Matt J <[email protected]
>> <applewebdata://6F440B21-9FB5-4B1D-8F23-5A199889C5EE>> wrote:
>>
>
>> Hello All,
>>
>> Firstly let me say that I have done my best over the past couple weeks to
>> find a solution to this problem, but have come up with no concrete
>> solutions, mostly because of limitations in my corporate environment. I've
>> found many helpful resources such as this one
>> https://www.youtube.com/watch?v=H6Q3l11fjU0
>> <https://www.youtube.com/watch?v=H6Q3l11fjU0>, but am coming straight to the
>> source for this question.
>>
>> How can I change the log location of mod_wsgi output (the app output)
>> without defining my app under a separate virtual host?
>>
>> I am hosting a Django Application using Python 3.7.10 on a RHEL7 server
>> using Apache version 2.4 and mod_wsgi version 4.8.0. This server uses only a
>> single virtual host to host multiple corporate applications (java, perl,
>> python etc), and this server hosts them typically using proxies and
>> locations under the single virtual host server01.corp.com:443
>> <http://server01.corp.com:443/>. Because domain name is important and cannot
>> be changed due to "its the way we have always done things", my django app
>> must be hosted at server01.corp.com <http://server01.corp.com/> and aliased
>> to a script path /apps/djangosite. Therefore, a conf.d for the site looks
>> like this:
>> -----------------------------------------------------------------
>> <Location "/apps/djangosite ">
>> #How all other apps are hosted on server01.corp.com
>> <http://server01.corp.com/> using a proxy
>> #ProxyPass "http://127.0.0.1:9889 <http://127.0.0.1:9889/>" retry=0
>> #ProxyPassReverse "http://127.0.0.1:9889 <http://127.0.0.1:9889/>"
>> <IF "req('Authorization') =~ /^Bearer/">
>> #stuff
>> </IF>
>> <ELSE>
>> #stuff but with ldap
>> </ELSE>
>>
>> Require valid-user
>> # Forward the REMOTE_USER env var as a request header
>> RequestHeader set X-Remote-User %{REMOTE_USER}s
>> RequestHeader set X-Remote-Host %{REMOTE_HOST}s
>>
>> #Add some variables for Django
>> #RequestHeader set X-Script-Name /apps/djangosite/
>> #RequestHeader set X-Forwarded-Script-Name /apps/djangosite/
>> </Location>
>>
>> #mod_wsgi Specific Entries
>> <Directory /fullpath/mysite/static>
>> Options FollowSymLinks Includes
>> Require all granted
>> </Directory>
>>
>> Alias /apps/djangosite/static/ /fullpath/mysite/static/
>>
>> <Directory /fullpath/mysite>
>> <Files wsgi.py>
>> Require all granted
>> </Files>
>> </Directory>
>>
>> WSGIDaemonProcess djangosite-api processes=2 threads=15
>> python-home=/fullpath/envs/djangosite_api/ user=corpuser
>> home=/fullpath/djangosite python-path=/fullpath:/fullpath:/fullpath
>> WSGIProcessGroup djangosite-api
>> WSGIScriptAlias /apps/djangosite /fullpath/mysite/wsgi.py
>> process-group=compute-api
>> WSGIRestrictEmbedded On
>> WSGIApplicationGroup %{GLOBAL}
>>
>> #pull mod_wsgi out of ssl_error_log
>> SetEnvIf Request_URI ^/apps/djangosite(/.*|$) djangosite-api
>> CustomLog /fullpath/logs/djangosite.debug.log combined env=djangosite-api
>> LogLevel debug wsgi:Trace8
>> -----------------------------------------------------------------
>>
>> Logging Directives in settings.py
>> LOGGING = {
>> 'version': 1,
>> 'disable_existing_loggers': False,
>> 'formatters': {
>> 'verbose': {
>> 'format': '{levelname} {asctime} {module} {process:d} {thread:d}
>> {message}',
>> 'style': '{',
>> },
>> },
>> 'handlers': {
>> 'file': {
>> 'level': 'DEBUG',
>> #'class':'logging.StreamHandler'
>> 'class': 'logging.FileHandler',
>> 'filename': ' /fullpath/logs/djangosite.debug.log '
>> }
>> },
>> 'root': {
>> 'handlers': ['file'],
>> 'level': 'DEBUG',
>> },
>> 'loggers': {
>> 'django': {
>> 'handlers': ['file'],
>> 'level': 'INFO',
>> 'propagate': True,
>> }
>> }
>> }
>>
>> And so here is the problem: With over 10 other deployed apps on
>> server01.corp.com <http://server01.corp.com/>, the ssl_error_log of this one
>> virtual host is pretty quick to bury messages, and I want to be able to get
>> my apps log in a specific location, rather then sifting through the hosts
>> log. If I was able to create another host, this wouldnt be an issue
>> obviously. I can get django debug info from the logging directive, and some
>> very slight request logging from setting the custom log above, but I will
>> never be able to get stack traces or wsgi startup info unless I go to the
>> ssl_error_log. I have read the warnings on a portable application using
>> stderr/stdout, and have tried to make the application as portable as the
>> development environment will allow. I cannot find a way to change the log
>> location of a specific module in apache. I tried gunicorn as a alternative
>> to mod_wsgi, but mod_wsgi has a cleaner setup for us and apache/gunicorn
>> struggled to host djangosite on anything other than the root /.
>>
>> Is there any way to circumvent this inability to create another virtualhost
>> while still having full control of logging?
>>
>> Thank you for your time,
>> Matt J
>>
>>
>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected]
>> <applewebdata://6F440B21-9FB5-4B1D-8F23-5A199889C5EE>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/modwsgi/fc14abce-0f43-4e35-a91e-05310847947an%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/modwsgi/fc14abce-0f43-4e35-a91e-05310847947an%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected]
> <mailto:[email protected]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/modwsgi/3abeb300-ad35-4e8c-b1ac-faf682f116f0n%40googlegroups.com
>
> <https://groups.google.com/d/msgid/modwsgi/3abeb300-ad35-4e8c-b1ac-faf682f116f0n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/modwsgi/F505DCB8-4E0C-4E8C-8DFD-2190A1B31C59%40gmail.com.
