What do you get if you add:
import sys
import requests
print(type(sys.modules['requests'])
print(type(requests))
It looks a bit like some fancy is being done with delayed module loader as a
real module should have a __file__ attribute.
Graham
> On 29 Sep 2017, at 8:56 am, Chris Barton <[email protected]>
> wrote:
>
> Hi Graham,
>
> Here's what we're getting when we attempt to access the __file__ attribute of
> the imported requests library:
>
> [Thu Sep 28 15:37:30.406608 2017] [wsgi:info] [pid 9408:tid 139645478971136]
> mod_wsgi (pid=9408): Create interpreter
> 'dev.api.serviceoperations.mydomain.com|'.
> [Thu Sep 28 15:37:30.422457 2017] [wsgi:info] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] mod_wsgi (pid=9408, process='rest_api_v1',
> application='dev.api.serviceoperations.mydomain.com|'): Loading WSGI script
> '/var/www/sites/api/start.wsgi'.
> [Thu Sep 28 15:37:31.003398 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] mod_wsgi (pid=9408): Target WSGI script
> '/var/www/sites/api/start.wsgi' cannot be loaded as Python module.
> [Thu Sep 28 15:37:31.003458 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] mod_wsgi (pid=9408): Exception occurred processing
> WSGI script '/var/www/sites/api/start.wsgi'.
> [Thu Sep 28 15:37:31.003834 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] Traceback (most recent call last):
> [Thu Sep 28 15:37:31.003938 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] File "/var/www/sites/api/start.wsgi", line 11,
> in <module>
> [Thu Sep 28 15:37:31.003954 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] from rest_api_v1 import app as application
> [Thu Sep 28 15:37:31.003963 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] File
> "/var/www/sites/api/rest_api_v1/__init__.py", line 34, in <module>
> [Thu Sep 28 15:37:31.004012 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] print('requests.__file__', requests.__file__)
> [Thu Sep 28 15:37:31.004112 2017] [wsgi:error] [pid 9408:tid 139645478971136]
> [remote 10.6.70.171:49215] AttributeError: module 'requests' has no attribute
> '__file__'
> [Thu Sep 28 15:37:31.004266 2017] [headers:debug] [pid 9409:tid
> 139645425809152] mod_headers.c(899): AH01503: headers:
> ap_headers_error_filter()
>
>
> We've previously tried printing out all of the attributes of requests after
> it is imported and a severely truncated list of attributes is provided back:
> [
> "__doc__",
> "__loader__",
> "__name__",
> "__package__",
> "__path__",
> "__spec__"
> ]
>
> Here's the contents of requests.__path__ during runtime:
> [Thu Sep 28 15:54:21.196608 2017] [wsgi:error] [pid 11443:tid
> 140183901226752] requests.__path__ :
> _NamespacePath(['/var/www/sites/api/venv/lib/python3.6/site-packages/requests'])
>
> Manually running the Flask package I get the expected results from the print
> statement:
> (venv) root@ub50368:/var/www/sites/api# python rest_api_v1/
> requests.__file__
> /var/www/sites/api/venv/lib/python3.6/site-packages/requests/__init__.py
> * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
> * Restarting with stat
> requests.__file__
> /var/www/sites/api/venv/lib/python3.6/site-packages/requests/__init__.py
> * Debugger is active!
> * Debugger PIN: 217-497-265
>
>
>
> On Thursday, September 28, 2017 at 3:01:05 PM UTC-7, Graham Dumpleton wrote:
>
>> On 29 Sep 2017, at 5:51 am, Bill Freeman <[email protected] <javascript:>>
>> wrote:
>>
>> Do you have a module of your own named "requests" that is being confused,
>> somehow, with the installed package?
>> Or might you be importing something "as requests"?
>
> The way to determine that is to add somewhere in your code:
>
> import requests
>
> print('requests.__file__', requests.__file__)
>
> Then check the logs for where it says the module file is being imported.
>
> Graham
>
>>
>> On Thu, Sep 28, 2017 at 12:02 PM, Chris Barton
>> <[email protected] <javascript:>> wrote:
>> Hello,
>>
>> I've been working on the deployment of a Flask application using
>> mod_wsgi4.5.18 (compiled against Python3.6.1) and have been encountering
>> issues with the requests library that only seem to manifest when the Flask
>> app is run from behind Apache & mod_wsgi. When the app is invoked from the
>> virtualenv python interpreter it works flawlessly.
>>
>> The problem occurs in areas where we are attempting to use the requests
>> library to make a call to external APIs (BlueJeans & Slack) during the
>> handling of a request to our Flask app.
>>
>> Here are some pieces of our current configuration. We've been puzzled by
>> this for the past two weeks and do not seem to be getting any closer to a
>> solution. At this point the development team is asking to look at other
>> options such as Nginx & Gunicorn.
>>
>> Python traceback:
>> [Fri Sep 22 16:35:49.378817 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> [2017-09-22 16:35:49,377] ERROR in app: Exception on /conference [POST]
>> [Fri Sep 22 16:35:49.378844 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> Traceback (most recent call last):
>> [Fri Sep 22 16:35:49.378847 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/util/bluejeans.py", line 13, in
>> password_grant
>> [Fri Sep 22 16:35:49.378850 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> r = requests.post(BASE_URL + '/oauth2/token?Password', json=json)
>> [Fri Sep 22 16:35:49.378852 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> AttributeError: module 'requests' has no attribute 'post'
>> [Fri Sep 22 16:35:49.378855 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> [Fri Sep 22 16:35:49.378857 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> During handling of the above exception, another exception occurred:
>> [Fri Sep 22 16:35:49.378859 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> [Fri Sep 22 16:35:49.378861 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> Traceback (most recent call last):
>> [Fri Sep 22 16:35:49.378863 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/venv/lib/python3.6/site-packages/flask/app.py",
>> line 1982, in wsgi_app
>> [Fri Sep 22 16:35:49.378866 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> response = self.full_dispatch_request()
>> [Fri Sep 22 16:35:49.378868 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/venv/lib/python3.6/site-packages/flask/app.py",
>> line 1614, in full_dispatch_request
>> [Fri Sep 22 16:35:49.378871 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> rv = self.handle_user_exception(e)
>> [Fri Sep 22 16:35:49.378873 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File
>> "/var/www/sites/api/venv/lib/python3.6/site-packages/flask_cors/extension.py",
>> line 161, in wrapped_function
>> [Fri Sep 22 16:35:49.378875 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> return cors_after_request(app.make_response(f(*args, **kwargs)))
>> [Fri Sep 22 16:35:49.378877 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/venv/lib/python3.6/site-packages/flask/app.py",
>> line 1517, in handle_user_exception
>> [Fri Sep 22 16:35:49.378880 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> reraise(exc_type, exc_value, tb)
>> [Fri Sep 22 16:35:49.378882 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/venv/lib/python3.6/site-packages/flask/_compat.py",
>> line 33, in reraise
>> [Fri Sep 22 16:35:49.378884 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> raise value
>> [Fri Sep 22 16:35:49.378886 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/venv/lib/python3.6/site-packages/flask/app.py",
>> line 1612, in full_dispatch_request
>> [Fri Sep 22 16:35:49.378889 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> rv = self.dispatch_request()
>> [Fri Sep 22 16:35:49.378891 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/venv/lib/python3.6/site-packages/flask/app.py",
>> line 1598, in dispatch_request
>> [Fri Sep 22 16:35:49.378893 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> return self.view_functions[rule.endpoint](**req.view_args)
>> [Fri Sep 22 16:35:49.378895 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/endpoints/conference.py", line 11, in
>> create_conference
>> [Fri Sep 22 16:35:49.378898 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> response, status = conference.create_conference(request.json)
>> [Fri Sep 22 16:35:49.378902 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/logic/conference.py", line 35, in
>> create_conference
>> [Fri Sep 22 16:35:49.378904 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> userId, token = bjn_util.password_grant(bjn_config.USERNAME,
>> bjn_config.PASSWORD)
>> [Fri Sep 22 16:35:49.378907 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/util/bluejeans.py", line 24, in
>> password_grant
>> [Fri Sep 22 16:35:49.378909 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> logger.error(ex, application='Conference')
>> [Fri Sep 22 16:35:49.378911 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/util/logger.py", line 81, in error
>> [Fri Sep 22 16:35:49.378913 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> log(message, logType='error', application=application, traceback=traceback)
>> [Fri Sep 22 16:35:49.378916 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/util/logger.py", line 70, in log
>> [Fri Sep 22 16:35:49.378918 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> slack(message + '\\n' + traceback, 'https://hooks.slack.com/services/
>> <https://hooks.slack.com/services/><URL REMOVED>')
>> [Fri Sep 22 16:35:49.378920 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> File "/var/www/sites/api/rest_api_v1/util/logger.py", line 20, in slack
>> [Fri Sep 22 16:35:49.378923 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> response = requests.post(webhook_url, data=json.dumps(slack_data),
>> [Fri Sep 22 16:35:49.378926 2017] [wsgi:error] [pid 1708:tid
>> 139626802534144] [remote 10.6.70.171:50515 <http://10.6.70.171:50515/>]
>> AttributeError: module 'requests' has no attribute 'post'
>>
>>
>> Apache startup message showing mod_wsgi compiled for Python3.6:
>> [Wed Sep 27 16:13:25.690202 2017] [mpm_event:notice] [pid 63496:tid
>> 139773643462528] AH00489: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g
>> mod_wsgi/4.5.18 Python/3.6 configured -- resuming normal operations
>>
>> Python packages installed in the virtualenv:
>> (venv) root@ub50368:/var/www/sites/api# pip list
>> certifi (2017.7.27.1)
>> chardet (3.0.4)
>> click (6.7)
>> configparser (3.5.0)
>> Flask (0.12.2)
>> Flask-Cors (3.0.3)
>> Flask-MySQL (1.4.0)
>> idna (2.6)
>> itsdangerous (0.24)
>> Jinja2 (2.9.6)
>> MarkupSafe (1.0)
>> pip (9.0.1)
>> PyMySQL (0.7.11)
>> pytz (2017.2)
>> requests (2.18.4)
>> setuptools (36.5.0)
>> six (1.10.0)
>> urllib3 (1.22)
>> Werkzeug (0.12.2)
>> wheel (0.30.0)
>>
>> Globally installed Python packages:
>> root@ub50368:/var/www/sites/api# pip3.6 list
>> pip (9.0.1)
>> setuptools (28.8.0)
>> virtualenv (15.1.0)
>>
>> WSGI Directives from Apache2 configuration:
>> WSGIDaemonProcess rest_api_v1 user=api group=api home=/var/www/sites/api
>> python-home=/var/www/sites/api/venv
>> WSGIProcessGroup rest_api_v1
>> #WSGIApplicationGroup %{GLOBAL}
>> WSGIScriptAlias / /var/www/sites/api/start.wsgi
>>
>> Things we've tried:
>> Installing requests (& all pip packages) globally and disabling the
>> python-home parameter
>> Rebuilding the virtualenv
>> Installing requests directly from git/source
>> Using the activate_this.py -
>> http://flask.pocoo.org/docs/0.12/deploying/mod_wsgi/#working-with-virtual-environments
>>
>> <http://flask.pocoo.org/docs/0.12/deploying/mod_wsgi/#working-with-virtual-environments>
>> Embedding the requests source directly in the Flask app
>>
>> Does anyone have any idea of what could be going wrong, or how to dive
>> deeper into the problem?
>>
>> Thanks,
>>
>> Chris
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected] <javascript:>.
>> Visit this group at https://groups.google.com/group/modwsgi
>> <https://groups.google.com/group/modwsgi>.
>> For more options, visit https://groups.google.com/d/optout
>> <https://groups.google.com/d/optout>.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected] <javascript:>.
>> Visit this group at https://groups.google.com/group/modwsgi
>> <https://groups.google.com/group/modwsgi>.
>> For more options, visit https://groups.google.com/d/optout
>> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected]
> <mailto:[email protected]>.
> To post to this group, send email to [email protected]
> <mailto:[email protected]>.
> Visit this group at https://groups.google.com/group/modwsgi
> <https://groups.google.com/group/modwsgi>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
