[issue36753] Python modules not linking to libpython causes issues for RTLD_LOCAL system-wide
reimar added the comment: I'm not sure how I can explain it much better, I even wrote example code after all, but I'll try... > So if you are embedding python by dlopen'ing libpython.so Neither me nor Laszlo are using/embedding or otherwise involving Python (directly/intentionally at least). We just want to load some .so file. That .so itself might then use libpython. Or use a library that uses libpython. Or uses a library that uses a library that uses libpython. And how could we know whether SOME library down that dependency chain uses libpython or not? The result is that now EVERY SINGLE LIBRARY IN THE WHOLE SYSTEM needs to be loaded with RTLD_GLOBAL. Because a library 50 dependencies down that uses python might break otherwise and there's not really any way to know. Just to try be very clear: We are not users of libpython, we do not write or use any python code ourselves (except through indirect dependencies) and we are still hit by this issue. -- ___ Python tracker <https://bugs.python.org/issue36753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36753] Python modules not linking to libpython causes issues for RTLD_LOCAL system-wide
reimar added the comment: I guess one way this could be "solved" would be by libpython doing a dlopen on itself with RTLD_GLOBAL on Python initialization. I wouldn't bet that that wouldn't end up with some very interesting unintended consequences as well though. -- ___ Python tracker <https://bugs.python.org/issue36753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode
reimar added the comment: > In short, RTLD_LOCAL is not supported. I am sorry, this is not a workable stance. This does not just affect loading libpython directly. It also affects dlopen of a library that links to a library ... that links to libpython. For a developer is impossible to know if some library via dozens of dependencies might ever bring in libpython. Thus your stance is essentially equivalent to "nobody must ever use RTLD_LOCAL". I find it hard to consider that an acceptable "solution". -- nosy: +reimar ___ Python tracker <https://bugs.python.org/issue34814> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21536] extension built with a shared python cannot be loaded with a static python
reimar added the comment: As explained in Issue34814, this change not only breaks RTLD_LOCAL of libpython, but it breaks it in fact system-wide. It seems a bit much for Python to enforce a system-wide policy to not use RTLD_LOCAL just because of its own technical challenges, and thus I'd ask to reconsider the original change. -- nosy: +reimar ___ Python tracker <https://bugs.python.org/issue21536> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36753] Python modules not linking to libpython causes issues for RTLD_LOCAL system-wide
New submission from reimar :
Most affected platforms: RedHat and Debian, but with the changes from
issue21536 probably all Linux distributions will be affected.
issue34814 and issue21536 and
https://bugzilla.redhat.com/show_bug.cgi?id=1585201 make statements along the
lines of "In short, RTLD_LOCAL is not supported."
This might have been considered a reasonable stance because of the specific
example opening libpython directly.
However Python modules not linking to libpython also breaks things when
libpython is loaded in the most indirect ways via dlopen.
E.g. dlopen("libA.so", RTLD_LOCAL | RTLD_NOW)
libA might have linked against libB, libB against libC and libC might
optionally link against libpython.
As a developer generally cannot really know if some library might ever pull in
a most indirect reference to libpython, not supporting RTLD_LOCAL in Python
essentially means RTLD_LOCAL can NEVER EVER be used safely.
A test-case that will fail the import command when modules have not been linked
against libpython is attached (demonstrating only one layer of indirection, but
much more complex cases are of course possible).
You will need to adjust the (include, lib) paths in test.sh for your Python
version, it was written to demonstrate the issue against RedHat's modifications
of Python 2.7 (to my knowledge, RedHat and Debian has been affected by this
issue much longer than mainline Python).
While dlmopen is an alternative with similar behaviour to RTLD_LOCAL on recent
Linux versions for this case, it is not portable.
--
components: Library (Lib)
files: pytest.tar.gz
messages: 341094
nosy: reimar
priority: normal
severity: normal
status: open
title: Python modules not linking to libpython causes issues for RTLD_LOCAL
system-wide
type: behavior
Added file: https://bugs.python.org/file48291/pytest.tar.gz
___
Python tracker
<https://bugs.python.org/issue36753>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode
reimar added the comment: Sorry for my laziness. I opened Issue36753. -- ___ Python tracker <https://bugs.python.org/issue34814> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36753] Python modules not linking to libpython causes issues for RTLD_LOCAL system-wide
reimar added the comment: The modules not linking against libpython CAUSES this issue, thus Debian being affected even for old Python versions before issue21536. -- ___ Python tracker <https://bugs.python.org/issue36753> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
