[issue30910] Add -fexception to ppc64le build
Changes by Gustavo Serra Scalet : -- nosy: +gut ___ Python tracker <http://bugs.python.org/issue30910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30910] Add -fexception to ppc64le build
Gustavo Serra Scalet added the comment: -fexceptions documentation: https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fexceptions Highlight to: "you may need to enable this option when compiling C code that needs to interoperate properly with exception handlers written in C++" And this issue was detected when cpython is used like the following: .-. | A C++ code | <= Designed to be the catcher :-: | cpython | <= Should only pass exception up :-: | called C++ function | <= Raises a C++ exception '-' This is already working for X64, but not on PPC64LE (and possibly ARM64 and other architectures as well). For some reason the C++ unwinder can unwind C frames without special unwind information on X64. Maybe it can assume frame information on this architectures and it succeeds, but that's just pure luck in my opinion. ps: this patch is not v2.7 specific. It should be applied globally to other versions as well. It was merely reported to v2.7 as it was the python version that our c++ library was using. Maybe it'll be easier to backport this patch to other versions once it's in. -- ___ Python tracker <http://bugs.python.org/issue30910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30910] Add -fexception to ppc64le build
Gustavo Serra Scalet added the comment: > Why doesn't gcc make it the default behaviour? Because it "can produce significant data size overhead, although it does not affect execution" for C code. However it'd improves interoperability with C++, which I think it makes sense for a universal software like cpython. Or isn't interoperability with c++ a concern? I noticed you said that there are not tests for it currently, but I wonder the reason behind it. -- ___ Python tracker <http://bugs.python.org/issue30910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30910] Add -fexception to ppc64le build
Gustavo Serra Scalet added the comment: well, now I'm confused: what should we do with this change? Do we need to ask any other expert? -- ___ Python tracker <http://bugs.python.org/issue30910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30910] Add -fexception to ppc64le build
Gustavo Serra Scalet added the comment: > Throwing exceptions through CPython is totally unsupported, -fexceptions or > not. Like C++ code that's not exception-aware, CPython lacks any of the catch > handlers to properly clean up resources on unwind. wait wait. It's not expected that CPython would handle the exception at all! The only thing it's needed for CPython is to let the compiler generate extra information about how each function looks like in the stack (how the frame is structured, in DWARF language) in order for the unwinder to do his job. As documented, the code would not change its current behavior (no machine code is added or removed, so when debugging your function looks the same) but extra information is given in other for unwind to happen through those frames. -- ___ Python tracker <http://bugs.python.org/issue30910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30910] Add -fexception to ppc64le build
Gustavo Serra Scalet added the comment: Sorry, I didn't take a deeper analysis on what you said, I guess you wanted to point out this problem: .-. | A C++ code | <= Designed to be the catcher :-: | cpython | <= Malloc'd something but unwind will not free it :-: | called C++ function | <= Raises a C++ exception '-' I guess memory leak would be a consequence, yes. *Sigh* -- ___ Python tracker <http://bugs.python.org/issue30910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30102] improve performance of libSSL usage on hashing
Gustavo Serra Scalet added the comment: Is there any news on this issue? The PR 3112 also seems to be frozen at the moment. Is there some kind of code freeze happening at the moment that no reviews are taking place? -- ___ Python tracker <http://bugs.python.org/issue30102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30102] improve performance of libSSL usage on hashing
Gustavo Serra Scalet added the comment: No worries. I thank you also for reviewing all these changesets. I'm glad it worked in the end. -- ___ Python tracker <http://bugs.python.org/issue30102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30102] improve performance of libSSL usage on hashing
New submission from Gustavo Serra Scalet: To correctly pick the best algorithm for the current architecture, libssl needs to have OPENSSL_config(NULL) called as described on: https://wiki.openssl.org/index.php/Libcrypto_API This short change lead to a speedup of 50% on POWER8 when using hashlib.sha256 digest functionality as it now uses a SIMD approach that was already existing but not used by cpython. -- assignee: christian.heimes components: SSL messages: 291892 nosy: christian.heimes, gut priority: normal severity: normal status: open title: improve performance of libSSL usage on hashing type: enhancement versions: Python 2.7, Python 3.7 ___ Python tracker <http://bugs.python.org/issue30102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30102] improve performance of libSSL usage on hashing
Changes by Gustavo Serra Scalet : -- pull_requests: +1309 ___ Python tracker <http://bugs.python.org/issue30102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30102] improve performance of libSSL usage on hashing
Gustavo Serra Scalet added the comment:
> Christian Heimes added the comment:
>
> This small change also changes behavior of OpenSSL dramatically. What
> are your Python and OpenSSL versions?
I tested by compiling my own python3 (8aaf499) against its parent (d6d344d) For
python2, I made the same by compiling the 2.7.13 as a baseline and results were
consistent with the python I have on Ubuntu 17.10 (3.5.3 and 2.7.13).
> How did you profile the
> performance of the hashing functions?
I compared the timings between my patched vs patchless python versions by using
a 630MB file to calculate sha256 as:
perf stat -r 10 Python-2.7.13/python -c "import hashlib;
print(hashlib.sha256(open('ubuntu-16.10-server-ppc64el.iso','rb').read()).hexdigest())"
The speedup of ~50% is given due to patchless taking 3.082156387s and patched
taking 2.027484800s (errors are less than 1%)
I also noticed what changed on a code level by checking with gdb what is being
executed on the sha loop. After my patch I see altivec (SIMD) functions being
used on SHA main loop.
> If OpenSSL_config() really improves performance on relevant platforms,
> then we should be consistent and call it in _ssl.c, too.
I'll check that then. Thanks for the hint.
Ps: please note this behavior is noticed on a POWER8 machine. I'm not an
OpenSSL expert so I don't know if there are now more optimizations enabled on
other architectures as well.
--
___
Python tracker
<http://bugs.python.org/issue30102>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30102] improve performance of libSSL usage on hashing
Gustavo Serra Scalet added the comment: > Since OPENSSL_config() accepts a filename, maybe a first step would be > to expose the function as ssl.OPENSSL_config(filename) to allow user to > load *explicitly* a configuration file? ssl.OPENSSL_config() would call > OPENSSL_config(NULL). Would it work for you, Gustavo? It would work, I would just wait for such a decision. BTW, that interface is deprecated as we're discussing on the issue on GitHub: https://github.com/openssl/openssl/blob/cda3ae5bd0798c56fef5a5c1462d51ca1776504e/doc/crypto/OPENSSL_config.pod#notes I'll continue analyzing this issue there. -- ___ Python tracker <http://bugs.python.org/issue30102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
