On Sun, May 12, 2024 at 11:37 PM A McBain <[email protected]> wrote: > > On Sun, May 12, 2024 at 11:19 PM Graham Dumpleton > <[email protected]> wrote: > > > > The mod_wsgi.so specified by LoadFile is what dictates version (and in what > > location) of Python is being used. > > > > Right now you are using one under your home directory. I would not be > > surprised if it is compiled for the version of Python you built yourself > > from source code and that is causing the problem. > > > > Ensuring you have no virtual environments activated, run the following: > > > > pip3 uninstall mod_wsgi > > > > Run this multiple times until it says nothing to uninstall. > > > > Then try rebuilding/reinstall mod_wsgi again but when you run pip install > > use the --no-cache-dir option so it doesn't use old cached build. > > > > Then update LoadFile line to use whatever location it now gets installed to > > if you use a different location (eg., a dedicated virtual environment, or > > system Python directory. > > > > Ensure you stop and start Apache when changed. > > > > Graham > > OK I uninstalled the old one (there was only the .local copy). I ran > the install (with the flag) and it's now residing in the more expected > /usr/local/... python3.11 site-packages directory. > > Unfortunately no dice. > > I also did a full shutdown and boot. Nope. 😕
OK so for transparency it is working now! But heck if I know exactly why. I know I probably caused this mess and I'm not sure how to fix it exactly, so I'm just going to live in my mess for the time being, since this is just a home dev server, and proper projects go to an offsite server I purposefully haven't messed up. Also, thank you very much for helping me debug this, I really appreciate it. For anyone else I'll go through what I found in case it helps them with their own weird environments. Python versions on my server: 2.7.18 - the system one 3.11.1 - the one I compiled, only available at /usr/local/bin/python3.11 3.12.2 - the package/distro Python, and what is used when I do "python3 --version", available at /usr/bin/python3 and /usr/bin/python3.11 Inside the virtual environment folder of each app, which I'll just call Old App and New App, there's a pyvenv.cfg file. Old App appears to have been created with virtualenv, and New App was created with venv, but testing shows that doesn't matter. Old App had different file contents in pyvenv.cfg, presumably because it was created prior to installing the package Python (3.11.2): home = /usr/local/bin implementation = CPython version_info = 3.11.1.final.0 virtualenv = 20.17.1 include-system-site-packages = false base-prefix = /usr/local base-exec-prefix = /usr/local base-executable = /usr/local/bin/python3.11 New App had this in pyvenv.cfg (after I recreated it using virtualenv): home = /usr/bin implementation = CPython version_info = 3.11.2.final.0 virtualenv = 20.17.1+ds include-system-site-packages = false base-prefix = /usr base-exec-prefix = /usr base-executable = /usr/bin/python3 Once I changed /usr to /usr/local and python3 to python3.11 in the New App, the import error disappeared and left me only with logical errors due to mistakes in wsgi.prod.py and settings-prod.py leftover from debugging. Fixing those let the app start up just fine. Something weird is going on with the Python 3.11.2, or the interaction between it and the Python 3.11.1, but as per above since I have the app working I don't have the energy right this minute to untangle that. Thanks again, Alastair > > On 13 May 2024, at 4:10 PM, A McBain <[email protected]> wrote: > > > > On Sun, May 12, 2024 at 11:08 PM A McBain <[email protected]> wrote: > > > > > > > > > > On Sun, May 12, 2024 at 10:57 PM Graham Dumpleton > > <[email protected]> wrote: > > > > > > Ensure you do a complete stop and start of Apache and not just reload or > > restart in case Apache had cached a reference to a broken Python library > > variant from your self compiled build. > > > > If going to build your own Python from source code ensure you read. A bit > > dated, but still relevant. Ignore that talks about Docker, still applies > > for any Python build. > > > > Installing a custom Python version into a Docker image. > > blog.dscpl.com.au > > > > Also, where is LoadFile line in Apache picking up mod_wsgi.so from? > > > > Graham > > > > > > I don't intend to build from source again if I can help it. That was a > > PITA. 😄 I only did it way back since I believe at the time Python 3.11 > > wasn't available as a package for Devuan/Debian yet...? I honestly don't > > remember. Thank you for the link though if I ever do need. 🙂 > > > > I had been just doing "service apache2 restart" yeah, but I just tried a > > proper stop, then a start, and the state is the same. 😑 > > > > So best I can tell this is where I'm at: > > > > CLI-run Python, the older site, and new site, have identical paths except > > for the first entry (CLI = '', apps = their app dir) > > Both apps don't import cmath and a bunch of other modules but the CLI > > Python does > > All prefixes point to the correct virtual env root > > The old app still runs even after a full stop and start of Apache > > > > I apologize for all this. I don't have any idea what's going on and I've > > tried everything I can think of, what you've asked / suggested, and it's > > still breaking my brain. > > > > - Alastair > > > > > > Oh, the load command. Right, so that's at > > "/etc/apache2/mods-enabled/wsgi.load" > > > > The contents are this: > > > > #LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so > > LoadModule wsgi_module > > /home/asmcbain/.local/lib/python3.11/site-packages/mod_wsgi/server/mod_wsgi-py311.cpython-311-x86_64-linux-gnu.so > > > > On 13 May 2024, at 3:35 PM, A McBain <[email protected]> wrote: > > > > On Sun, May 12, 2024 at 10:22 PM A McBain <[email protected]> wrote: > > > > > > On Sun, May 12, 2024 at 10:05 PM A McBain <[email protected]> wrote: > > > > > > On Sun, May 12, 2024 at 9:47 PM Graham Dumpleton > > <[email protected]> wrote: > > > > > > > > > > On 13 May 2024, at 2:33 PM, A McBain <[email protected]> wrote: > > > > The old one is running on the same physical server, just with its own > > Apache config and own subdomain. As to having multiple things running, > > that's why I set WSGIApplicationGroup to %{SERVER} but just in case I > > changed it explicitly to "enfilade.asmcbain.net" (no effect, but shouldn't > > hurt anything to keep the change). > > > > > > The WSGIApplicationGroup directive sets which Python interpreter context is > > used. It is distinct from the daemon process group. > > > > You want each WSGI application instance to run in a different mod_wsgi > > daemon process group if using the main Python interpreter context > > (application group %{GLOBAL}). > > > > So what you want is for each VirtualHost to use a different named mod_wsgi > > dameon process group. > > > > <VirtualHost ...> > > ServerName site-1.example.com > > WSGIDaemonProcess site-1 \ > > user=asmcbain group=asmcbain \ > > display-name='%{GROUP}' \ > > lang='en_US.UTF-8' \ > > locale='en_US.UTF-8' \ > > threads=5 \ > > queue-timeout=45 \ > > socket-timeout=60 \ > > connect-timeout=15 \ > > request-timeout=60 \ > > inactivity-timeout=0 \ > > startup-timeout=15 \ > > deadlock-timeout=60 \ > > graceful-timeout=15 \ > > eviction-timeout=0 \ > > restart-interval=0 \ > > shutdown-timeout=5 \ > > maximum-requests=0 \ > > python-home=/home/asmcbain/enfilade/.env \ > > python-path=/home/asmcbain/enfilade > > WSGIProcessGroup site-1 > > WSGIApplicationGroup %{GLOBAL} > > WSGIScriptAlias / /home/asmcbain/enfilade/atcid/wsgi.py > > </VirtualHost> > > > > <VirtualHost ...> > > ServerName site-2.example.com > > WSGIDaemonProcess site-2 \ > > user=asmcbain group=asmcbain \ > > display-name='%{GROUP}' \ > > lang='en_US.UTF-8' \ > > locale='en_US.UTF-8' \ > > threads=5 \ > > queue-timeout=45 \ > > socket-timeout=60 \ > > connect-timeout=15 \ > > request-timeout=60 \ > > inactivity-timeout=0 \ > > startup-timeout=15 \ > > deadlock-timeout=60 \ > > graceful-timeout=15 \ > > eviction-timeout=0 \ > > restart-interval=0 \ > > shutdown-timeout=5 \ > > maximum-requests=0 \ > > python-home=/home/asmcbain/enfilade/.env \ > > python-path=/home/asmcbain/enfilade > > WSGIProcessGroup site-2 > > WSGIApplicationGroup %{GLOBAL} > > WSGIScriptAlias / /home/asmcbain/enfilade/atcid/wsgi.py > > </VirtualHost> > > > > So each site should have different name to WSGIDaemonProcess, with matching > > WSGIProcessGroup, and WSGIApplicationGroup should be %{GLOBAL} for both, > > not {SERVER}. > > > > The {SERVER} value has a different meaning and best avoiding it. > > > > So lets make sure each is running in separate daemon process groups. > > > > BTW, also make sure that somewhere outside of all VirtualHost definitions > > you add: > > > > WSGIRestrictEmbedded On > > > > > > OK, I updated both the old site and the new one to make > > WSGIApplicationGroup %{GLOBAL}. They always had different > > WSGIProcessGroup values. > > > > I also added WSGIRestrictEmbedded to the base Apache config. > > > > Apache restarts fine, and the old site loads up but the new one still > > prints the import math error even after the changes. 😕 > > > > I've attached the relevant bits to both. > > > > - Alastair > > > > > > Originally I'd compiled this Python 3.11 myself, but I've just double > > checked and verified Python 3.11 and the -dev package are from > > official packages and not my own manual build+install. > > > > I also checked that inside or outside of the virtual env (results are > > the same) I can run the Python interpreter shell and do: > > > > import math > > math.ceil(1.1) > > > > I don't get any errors. > > > > - Alastair > > > > > > Update to that. The modules loaded are different between the daemon > > one and the regular interpreter. > > > > Doing "print(sys.builtin_module_names)" results in the following for > > the daemon version: > > > > [Sun May 12 22:31:33.744235 2024] [wsgi:error] [pid 7430:tid > > 140134038087360] ('_abc', '_ast', '_codecs', '_collections', > > '_functools', '_imp', '_io', '_locale', '_operator', '_signal', > > '_sre', '_stat', '_string', '_symtable', '_thread', '_tokenize', > > '_tracemalloc', '_warnings', '_weakref', 'atexit', 'builtins', > > 'errno', 'faulthandler', > > 'gc', 'itertools', 'marshal', 'posix', 'pwd', 'sys', 'time', 'xxsubtype') > > > > And this substantially longer list for the one I just run from the CLI > > myself: > > > > ('_abc', '_ast', '_bisect', '_blake2', '_codecs', '_collections', > > '_csv', '_datetime', '_elementtree', '_functools', '_heapq', '_imp', > > '_io', '_locale', '_md5', '_opcode', '_operator', '_pickle', > > '_posixsubprocess', '_random', '_sha1', '_sha256', '_sha3', '_sha512', > > '_signal', '_socket', '_sre', '_stat', '_statistics', '_string', > > '_struct', '_symtable', '_thread', '_tokenize', '_tracemalloc', > > '_warnings', '_weakref', 'array', 'atexit', 'binascii', 'builtins', > > 'cmath', 'errno', 'faulthandler', 'fcntl', 'gc', 'grp', 'itertools', > > 'marshal', 'math', 'posix', 'pwd', 'pyexpat', 'select', 'spwd', 'sys', > > 'syslog', 'time', 'unicodedata', 'xxsubtype', 'zlib') > > > > Now I just have to figure out why. > > > > > > > > This will cause an error if mod_wsgi daemon process groups aren't being > > configured properly. > > > > Graham > > > > > > This is the result of the wsgi.prod.py printouts: > > > > [Sun May 12 21:29:11.788518 2024] [wsgi:error] [pid 5671:tid > > 140047682610880] sys.prefix '/home/asmcbain/enfilade/.env' > > [Sun May 12 21:29:11.788639 2024] [wsgi:error] [pid 5671:tid > > 140047682610880] sys.path ['/home/asmcbain/enfilade', > > '/usr/lib/python311.zip', '/usr/lib/python3.11', > > '/usr/lib/python3.11/lib-dynload', > > '/home/asmcbain/enfilade/.env/lib/python3.11/site-packages'] > > > > Alastair > > > > On Sun, May 12, 2024 at 9:24 PM Graham Dumpleton > > <[email protected]> wrote: > > > > > > > > On 13 May 2024, at 2:02 PM, A McBain <[email protected]> wrote: > > > > > > Hi, I looked at previous messages and others on StackOverflow but none seem > > to solve my issue. > > > > I have an app I wrote working perfectly fine under Python 3.11 with > > mod_wsgi and Apache 2. > > > > I did a bunch of development on the app (upgraded django, new features), > > and set up a new checkout of that on my server, with its own virtual > > environment (using venv). It uses effectively the same config (different > > subdomain) in Apache2 as the original older copy, but the new one fails > > with an import error while the old one is still chugging along. > > > > > > When you say "old one is still chugging along" can you confirm you are > > saying that the existing one is still running on the same server at the > > same time, or it is a completely new server you have set up. > > > > I double checked the instructions at > > https://docs.djangoproject.com/en/5.0/howto/deployment/wsgi/modwsgi/ > > > > I've also re-verified the mod_wsgi I installed (via pip) matches the Python > > version I'm using (both are Python 3.11). > > > > I also tried: > > > > Removing the python-path argument > > > > The mod_wsgi docs suggest I don't need that if I specify python-home? > > > > Removing python-path if it refers to site-packages directory of virtual > > environment specified to python-home is okay as path would be redundant. If > > python-path is used to tell it where your Django project is, that still > > needs to stay. > > > > Setting WSGIApplicationGroup to %{GLOBAL} > > > > That is recommended, but if you are hosting multiple WSGI applications on > > the same server, they would need to be delegated to different mod_wsgi > > daemon process groups. > > > > Unfortunately the error didn't change at all after trying those. > > > > I've attached the relevant apache2 config section which includes all the > > mod_wsgi-setup (the rest is just redirects, ssl stuff, aliases, etc.) > > > > I also attached the error from the log. It looks like it's trying to use > > the system Python instead of the one that exists in my .env (virtual > > environment) directory. > > > > > > The virtual environment doesn't have a copy of Python stdlib files so they > > are still imported from the system Python the virtual environment was > > created from. The virtual environment effectively only has its own > > site-packages directory. > > > > I'm banging my head as to why this worked before but not now so any help is > > much appreciated. Thank you! 🙂 > > > > Other details: > > > > wsgi.prod.py is the default wsgi.py, just modified to load settings.prod.py > > The server is Devuan Daedalus > > > > > > At the start of your WSGI script file add: > > > > import sys > > print('sys.prefix', repr(sys.prefix)) > > print('sys.path', repr(sys.path)) > > > > and check Apache logs for what paths it is using. > > > > Compare that to the working site. > > > > Post what you learn from that. > > > > Graham > > > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "modwsgi" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/modwsgi/IJp7zr6SjtY/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/7184C263-1522-4583-A98B-EC9806B0FA66%40gmail.com. > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "modwsgi" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/CAGV_ScocfNc%3DSv7W7Ha3MbrM_0wq43v8U0AM3JOuNtsfP9vK%3Dw%40mail.gmail.com. > > > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "modwsgi" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/modwsgi/IJp7zr6SjtY/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/41CA1CF3-8A06-46BF-A7AE-2CA74FCC2196%40gmail.com. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "modwsgi" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/CAGV_ScoxJu%3DxKwijvbPjE78BnZZD4s2ppRBT0-jCErCgdr8v%2Bg%40mail.gmail.com. > > > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "modwsgi" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/modwsgi/IJp7zr6SjtY/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/55163B95-DCAF-4129-BD81-4D477F4FC9F6%40gmail.com. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "modwsgi" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/CAGV_Scr3A_PxtgAtamReunLZ9zjG03OFG%2BZZFpQWV5aSzmo16Q%40mail.gmail.com. > > > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "modwsgi" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/modwsgi/IJp7zr6SjtY/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/modwsgi/F97967BE-2139-4E76-A9DC-4C7A9B3A397E%40gmail.com. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/CAGV_ScqPGkJ2r_UKL0rQHf6RHXLvfoSWrpmVk-tfc0gqCtvb%2BA%40mail.gmail.com.
