Stefan Krah added the comment:
Note that my pybind11 is from GitHub master, it can also be a pybind11
issue.
It is interesting that you cannot reproduce your original issue with
3.6, so I'm reopening this issue.
I think we need a reproducer without pybind11 though, could you
tweak Pro
Change by Stefan Krah :
--
resolution: not a bug ->
stage: resolved ->
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list
Stefan Krah added the comment:
Regarding *my* issue, it could be anything, e.g. a missing call to
PyEval_InitThreads() in 3.6:
"Changed in version 3.7: This function is now called by Py_Initialize(), so you
don’t have to call it yourself anymore."
This is why we need to eliminat
Stefan Krah added the comment:
> With python 3.7.3 without https://github.com/python/cpython/pull/5278 works
> just fine.
Thanks, I'm now getting the same results as you. Looking at the smaller
test case, I also agree that it should work (as it did in 3.6).
New submission from Stefan Krah :
#39776 has shown that it is hard to understand the interaction between
ContextVars and threading in embedded scenarios.
I want to understand the code again, so I'm adding back a compile time
option to enable the thread local context that was present pri
Change by Stefan Krah :
--
assignee: -> skrah
components: +Extension Modules
stage: -> needs patch
type: -> behavior
versions: +Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.or
Stefan Krah added the comment:
Also, when I'm debugging things like #39776, I don't want to
switch between Python versions.
--
___
Python tracker
<https://bugs.python.o
Change by Stefan Krah :
--
keywords: +patch
pull_requests: +18064
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/18702
___
Python tracker
<https://bugs.python.org/issu
Stefan Krah added the comment:
New changeset 815280eb160af637e1347213659f9236adf78f80 by Stefan Krah in branch
'master':
bpo-39794: Add --without-decimal-contextvar (#18702)
https://github.com/python/cpython/commit/815280eb160af637e1347213659f92
Change by Stefan Krah :
--
pull_requests: +18072
pull_request: https://github.com/python/cpython/pull/18713
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +18074
pull_request: https://github.com/python/cpython/pull/18714
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 4d7012410cf4f91cbca4c406f4747289c2802333 by Stefan Krah in branch
'3.8':
[3.8] bpo-39794: Add --without-decimal-contextvar (GH-18702)
https://github.com/python/cpython/commit/4d7012410cf4f91cbca4c406f47472
Stefan Krah added the comment:
New changeset c4ca1f8f24118dc5c29e16118fb35a13963af290 by Stefan Krah in branch
'3.7':
[3.7] bpo-39794: Add --without-decimal-contextvar (GH-18702)
https://github.com/python/cpython/commit/c4ca1f8f24118dc5c29e16118fb35a
Change by Stefan Krah :
--
nosy: +boytsovea
___
Python tracker
<https://bugs.python.org/issue39794>
___
___
Python-bugs-list mailing list
Unsubscribe:
Stefan Krah added the comment:
For the people who find this issue via a search engine:
1) The defaults in 3.7, 3.8 and 3.9 are unchanged (you get
the ContextVar).
2) ./configure --without-decimal-contextvar enables the exact
TLS context code from 3.3-3.6. There is no new code
Stefan Krah added the comment:
> If someone builds an interpreter with this configure flag, does it break
> compatibility with anything that code may have started to expect as of 3.7?
Yes, anything that relies on the implicit context being coroutine-safe
does not work. The only tes
Stefan Krah added the comment:
To make things more clear (it is good that Gregory asked):
ONLY use this flag if you don't use coroutines at all
OR control ALL async libraries in your application.
Random async libraries from PyPI may rely on the
ContextVar and silently give incorrect re
Change by Stefan Krah :
--
keywords: +patch
pull_requests: +18101
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/18746
___
Python tracker
<https://bugs.python.org/issu
Stefan Krah added the comment:
I think the PR fixes the issue but I have to run longer tests still.
Threads created by PyGILState_Ensure() could have a duplicate tstate->id,
which confused the ContextVar caching machinery.
--
___
Python trac
Stefan Krah added the comment:
New changeset b3b9ade4a3d3fe00d933bcd8fc5c5c755d1024f9 by Stefan Krah in branch
'master':
bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746)
https://github.com/python/cpython/commit/b3b9ade4a3d3fe00d933bcd8fc5
Change by Stefan Krah :
--
pull_requests: +18107
pull_request: https://github.com/python/cpython/pull/18752
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +18108
pull_request: https://github.com/python/cpython/pull/18753
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 5a92f42d8723ee865be80f028d402204649da15d by Stefan Krah in branch
'3.8':
bpo-39776: Lock ++interp->tstate_next_unique_id. (GH-18746) (#18746) (#18752)
https://github.com/python/cpython/commit/5a92f42d8723ee865be80f028d4
Stefan Krah added the comment:
New changeset 852aee69f49c654a03ad1f64d90a78ba8848e2c6 by Stefan Krah in branch
'3.7':
bpo-39776: Lock ++interp->tstate_next_unique_id (GH-18746)
https://github.com/python/cpython/commit/852aee69f49c654a03ad1f64d90
Change by Stefan Krah :
--
title: Crash in decimal module in heavy-multithreaded scenario ->
PyContextVar_Get(): crash due to race condition in updating tstate->id
___
Python tracker
<https://bugs.python.org/i
Change by Stefan Krah :
--
assignee: -> skrah
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscrib
Stefan Krah added the comment:
Since many duplicate tstate ids are generated in the test case
before the crash happens, I guess a set of tstates with the same
id silently uses the cached context of the "winner" tstate.
This can lead to incorrect results without noticing.
-
Change by Stefan Krah :
Added file: https://bugs.python.org/file48945/pydecimal_cases.zip
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bug
Stefan Krah added the comment:
Also _pydecimal was affected. This is a modified version of Evgeny's
test case:
from _pydecimal import *
from time import sleep
from random import randint
import sys
sys.setswitchinterval(0.001)
def usleep(x):
sleep(x/100.0)
class Test:
Stefan Krah added the comment:
Setting to release blocker, but please move to deferred again
if a release is almost finished.
--
nosy: +lukasz.langa, ned.deily
priority: deferred blocker -> release blocker
___
Python tracker
<
Stefan Krah added the comment:
#39776 is fixed, but _pydecimal was also affected, see msg363266.
--
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
stage: patch review -> resolved
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Un
Change by Stefan Krah :
--
resolution: -> fixed
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscrib
Stefan Krah added the comment:
Thanks Ned, closing then.
Evgeny, please reopen if you see it again (I ran the tests for about
20 min, way above the usual reproduction time of 1 min).
Thanks for the very instructive test case!
--
status: open -> clo
Stefan Krah added the comment:
Concerning memoryview, I've looked at this very briefly: memoryview
does not pack values directly, it casts first, which is legal:
#define PACK_SINGLE(ptr, src, type) \
do { \
t
Stefan Krah added the comment:
So which memoryview test (unless it is using the struct module)
relies on UB?
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
Okay, in memoryview the cast tests can trigger UB checks. memoryview assumes
that bool is packed correctly, so just casting does not work.
Casting anything to bool is of course a bit silly anyway.
diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py
Stefan Krah added the comment:
I checked that NumPy also packs correctly:
>>> import numpy as np
>>> x = np.array([0,1,2,3], dtype=np.bool)
>>> x.tobytes()
b'\x00\x01\x01\x01'
So I vote for not handling incorrectly packed values and removing
&quo
Stefan Krah added the comment:
The docs for native mode (we now always assume C99):
"The '?' conversion code corresponds to the _Bool type defined by C99."
The memoryview tests that fail are essentially auto-generated and not
prescriptive. They just happened to work wi
Stefan Krah added the comment:
The memcpy() is NOT a hack and performs exactly the same operation
as casting the pointer and then dereferencing, only in a manner that
avoids unaligned accesses.
On platforms like x86 the memcpy() is optimized to a simple assignment.
Casting the pointer and
Change by Stefan Krah :
--
pull_requests: +18312
pull_request: https://github.com/python/cpython/pull/18964
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 1ae9cde4b2323235b5f9ff4bc76e4175a2257172 by Stefan Krah in branch
'master':
bpo-39689: Do not test undefined casts to _Bool (GH-18964)
https://github.com/python/cpython/commit/1ae9cde4b2323235b5f9ff4bc76e41
Stefan Krah added the comment:
New changeset 636eecc16229432ec8e26e6da287c52f3ca3 by Miss Islington (bot)
in branch '3.7':
bpo-39689: Do not test undefined casts to _Bool (GH-18964) (#18965)
https://github.com/python/cpython/commit/636eecc16229432ec8e26e6da2
Stefan Krah added the comment:
New changeset f8ce3e2dae277baa2ef92e8a3e935953dc6c3f39 by Miss Islington (bot)
in branch '3.8':
bpo-39689: Do not test undefined casts to _Bool (GH-18964) (#18966)
https://github.com/python/cpython/commit/f8ce3e2dae277baa2ef92e8a3e9359
Stefan Krah added the comment:
memoryview only supports the native format, so I've disabled the
(wrong) test that casts arrays with arbitrary values to _Bool. So
memoryview is done.
IMO the problem in _struct is that it swaps the x->unpack function
for the native one, which does
Change by Stefan Krah :
--
pull_requests: +18317
pull_request: https://github.com/python/cpython/pull/18969
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
> This looks like a "new feature/improvement". Why was this code backported to
> a stable version?
Thanks for the lecture. This is an esoteric case between bugfix and
feature that only occurs with very large context precisions.
If Bloomberg
Stefan Krah added the comment:
> This code has introduced a regression in AIX in Python 3.7.7
Also this is a rather bold statement since probably no one has ever
run _decimal on AIX with MAX_PREC.
--
___
Python tracker
<https://bugs.pyth
Stefan Krah added the comment:
> Well, I just did and I can confirm that reverting the 3.7 backport fixes the
> problem.
If you are fortunate enough to have access to an AIX system, I guess
you have to find out why POWER6 AIX 3.8 and PPC64 AIX 3.8 apparently
work on
Stefan Krah added the comment:
Hi Michael, in case you have built 3.7.7 on AIX, have you observed
any problems with test_decimal?
--
nosy: +Michael.Felt
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
These flags worked for xlc when snakebite was still up:
./configure CC=xlc_r AR="ar -X64" CFLAGS="-q64 -qmaxmem=7" LDFLAGS="-q64"
-qmaxmem was always finicky, I remember segfaults too (AIX
Stefan Krah added the comment:
BTW, if you are compiling with xlc and there"s no xlc buildbot,
perhaps a company-that-shall-not-be-named should provide one.
I mean, it's not okay to complain about a regression and then
mention xlc about 10 m
Change by Stefan Krah :
--
assignee: -> skrah
___
Python tracker
<https://bugs.python.org/issue39576>
___
___
Python-bugs-list mailing list
Unsubscrib
Change by Stefan Krah :
--
assignee: -> skrah
keywords: -3.7regression, patch
___
Python tracker
<https://bugs.python.org/issue39576>
___
___
Python-bugs-lis
Stefan Krah added the comment:
A focused issue report would include at least the following:
1) Acknowledge that gcc builds work on the AIX buildbots (a
fact that has been entirely ignored so far).
2) State the exact regression: msg364373 (which was also ignored,
are you using
Stefan Krah added the comment:
It is also instructive how Granlund views xlc:
https://gmplib.org/list-archives/gmp-bugs/2010-November/002119.html
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
"The **exact** regression is that I could run the test suite without any crash
or freeze on this AIX system
with 3.7.6 and I cannot in 3.7.7. At the very least this is due to the fact
that there is a new test that crashes/hangs."
In other words, c
Stefan Krah added the comment:
Sorry, I'm reacting like Granlund now and close. These discussions
lead nowhere.
--
status: open -> closed
___
Python tracker
<https://bugs.python.org
Stefan Krah added the comment:
> So it's clear that something has to change. IMO, preserving (2) and relaxing
> (1) is the more useful choice.
But not in this issue I think. GH-18969 is a minimal change that
*removes* UB for the standard sizes.
UB for the native type
Stefan Krah added the comment:
I think this issue should be about fixing the tests so that people
looking at the sanitizer buildbots can move on.
GH-18969 fixes "?" and "!?", which clearly used wrong
semantics with the new compiler behavior. This should be an
uncontrov
Stefan Krah added the comment:
Since xlc has elementary bugs like
https://github.com/openssl/openssl/issues/10624
it may be worth checking out if this is an optimizer bug with -q64.
--
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
The lower MAX_PREC for 32-bit could be the reason.
On the other hand, historically, suncc and xlc have had a lot of
problems with the 64-bit build. The winner is suncc, which seriously
miscompiled libmpdec without a whole litany of flags:
https
Stefan Krah added the comment:
> Wouldn't having less static types slow down startup time?
Yes, and not only startup time:
https://bugs.python.org/issue15722
--
nosy: +skrah
___
Python tracker
<https://bugs.python.org
Stefan Krah added the comment:
> Or maybe _decimal_state_global was used in "hot code".
Yes, _decimal has problems here that most modules don't have.
Modules like atexit are obviously fine. :)
I just posted it as an example why one should be a bit cautious.
> The PEP
Stefan Krah added the comment:
Has anything emerged xlc-wise? Traceback, does it work --without-pymalloc?
I have the feeling that (in many OSS projects) the more complex xlc
issues never get resolved after the initial report.
So I'm contemplating to do the same as
Stefan Krah added the comment:
You can also set the decimal.FloatOperation trap to avoid accidental
errors:
>>> from decimal import *
>>> c = getcontext()
>>> Decimal(4.6) * 100
Decimal('459.9644728632120')
>>> c.traps[FloatOperation]
New submission from Stefan Krah :
Newer icc version require -fwrapv:
https://software.intel.com/en-us/forums/intel-c-compiler/topic/849064
--
components: Build
messages: 365976
nosy: skrah
priority: normal
severity: normal
stage: needs patch
status: open
title: Add -fwrapv for new icc
New submission from Stefan Krah :
50e6e991781db761c496561a995541ca8d83ff87 causes or exposes a
leak. Possibly the leak was there before but showed up under
"still reachable".
Now it is "definitely lost", so tstate->interp->ceval.pending
needs to be cleaned up.
==11
Stefan Krah added the comment:
Since I just had a similar incident with the Intel compiler, I just
want to point out that compiling CPython still requires -fwrapv.
I've mentioned this before in AIX issues, and no one has pointed
out the corresponding xlc flag.
https://gcc.gnu.org/lega
Change by Stefan Krah :
--
nosy: -skrah
___
Python tracker
<https://bugs.python.org/issue40244>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Stefan Krah :
Synopsis: There are no relevant new features for _decimal, but it would be too
much work/error prone to have divergent code in libmpdec-2.5.0 and Python 3.9,
especially for the Linux distributions.
I'll release libmpdec-2.5.0/libmpdec++-2.5.0 in a mon
Change by Stefan Krah :
--
keywords: +patch
pull_requests: +19872
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/20652
___
Python tracker
<https://bugs.python.org/issu
New submission from Stefan Krah :
Again code coverage prevents the merge button from going green:
https://travis-ci.org/github/python/cpython/jobs/695095365
"The job exceeded the maximum time limit for jobs, and has been terminated."
Why can this not run on buildbot.
Stefan Krah added the comment:
New changeset 087d612efebe7c64e5f079b07e0454111859830e by Stefan Krah in branch
'master':
bpo-40874: Update to libmpdec-2.5.0 (GH-20652)
https://github.com/python/cpython/commit/087d612efebe7c64e5f079b07e0454
Stefan Krah added the comment:
It finally went through after Travis-CI hanging for about an hour.
--
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/i
Change by Stefan Krah :
--
versions: +Python 3.10
___
Python tracker
<https://bugs.python.org/issue40874>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Stefan Krah :
There appears to be an xlc buildbot with libmpdec failures.
libmpdec uses C99 extern inline semantics. From the brief period that I had
access to xlc I remember that xlc was quite picky about C99.
Actually all of Python uses C99. So I think xlc_r needs to be
Stefan Krah added the comment:
New changeset 83bff88b4b16fb30491faa9263bbd6f3df4bab56 by Miss Islington (bot)
in branch '3.9':
bpo-40874: Update to libmpdec-2.5.0 (GH-20652)
https://github.com/python/cpython/commit/83bff88b4b16fb30491faa9263bbd6
New submission from Stefan Krah :
>From test_decimal:
test_xor (test.test_decimal.PyIBMTestCases) ... ==17597== Invalid read of size 1
==17597==at 0x64A7E2: newline_in_string (pegen.c:940)
==17597==by 0x64A84E: bad_single_statement (pegen.c:958)
==17597==by 0x64A
New submission from Stefan Krah :
./configure --with-valgrind:
Objects/unicodeobject.c: In function ‘unicode_release_interned’:
Objects/unicodeobject.c:15672:26: error: lvalue required as left operand of
assignment
Py_REFCNT(s) += 1;
^
Objects
New submission from Stefan Krah :
Also in test_decimal, there's a small leak here:
==10040== 24 bytes in 1 blocks are definitely lost in loss record 549 of 5,095
==10040==at 0x4C2DE56: malloc (vg_replace_malloc.c:299)
==10040==by 0x643B33: fstring_compile_expr (parse_string.
New submission from Stefan Krah :
I'm opening a separate issue to prevent #40521 from getting too big.
Lists and tuples sometimes leak (starting 69ac6e58f and later):
==1445== 56 bytes in 1 blocks are definitely lost in loss record 1,542 of 4,898
==1445==at 0x4C2DE56: m
Stefan Krah added the comment:
> It's not a regression. It's just that bpo-40521 helped Valgrind to detect
> such bug :-)
Yes, I suspected that previously reachable global objects were just unreachable
after the new free lists. Th
Stefan Krah added the comment:
I think I'm going to revert this for 3.7 and 3.8 -- not because of xlc (it is
almost certainly a compiler or missing flag error), but because coordination
with the Linux distributions is a mess, see #40874.
I really want the system libmpdec to be the sa
Change by Stefan Krah :
--
pull_requests: +19948
pull_request: https://github.com/python/cpython/pull/20743
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +19949
pull_request: https://github.com/python/cpython/pull/20744
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 706de4e5a4b21880c67f6b90e3a2147a258d6fc5 by Stefan Krah in branch
'3.8':
[3.8] Revert bpo-39576: Clarify the word size for the 32-bit build. (GH-20743)
https://github.com/python/cpython/commit/706de4e5a4b21880c67f6b90e3a214
Stefan Krah added the comment:
New changeset c0b79450bc9e93105799528151c48d25af8240a3 by Stefan Krah in branch
'3.7':
[3.7] Revert bpo-39576: Clarify the word size for the 32-bit build. (GH-20744)
https://github.com/python/cpython/commit/c0b79450bc9e93105799528151c48d
Change by Stefan Krah :
--
pull_requests: +19950
pull_request: https://github.com/python/cpython/pull/20745
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +19951
pull_request: https://github.com/python/cpython/pull/20746
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 32c1fb07e6f2ded90e5dd24d4b46b7aa7a795d2e by Stefan Krah in branch
'3.8':
[3.8] Revert bpo-39576: docs: set context for decimal arbitrary precision
arithmetic (GH-20745)
https://github.com/python/cpyt
Stefan Krah added the comment:
New changeset 9bd891920a5186b7d02281ea9966225efa0ceba1 by Stefan Krah in branch
'3.7':
[3.7] Revert bpo-39576: docs: set context for decimal arbitrary precision
arithmetic (GH-20746)
https://github.com/python/cpyt
Change by Stefan Krah :
--
pull_requests: +19952
pull_request: https://github.com/python/cpython/pull/20747
___
Python tracker
<https://bugs.python.org/issue39
Change by Stefan Krah :
--
pull_requests: +19953
pull_request: https://github.com/python/cpython/pull/20748
___
Python tracker
<https://bugs.python.org/issue39
Stefan Krah added the comment:
New changeset 22faf6ad3bcc0ae478a9a3e2d8e35888d88d6ce8 by Stefan Krah in branch
'3.7':
[3.7] Revert bpo-39576: Prevent memory error for overly optimistic precisions
(GH-20748)
https://github.com/python/cpython/commit/22faf6ad3bcc0ae478a9a3e2d8e358
Stefan Krah added the comment:
New changeset 0f5a28f834bdac2da8a04597dc0fc5b71e50da9d by Stefan Krah in branch
'3.8':
[3.8] Revert bpo-39576: Prevent memory error for overly optimistic precisions
(GH-20747)
https://github.com/python/cpython/commit/0f5a28f834bdac2da8a04597dc0fc5
Stefan Krah added the comment:
The 3.9 change (see #40874) works successfully on all buildbots, except for the
32-bit xlc bot which should use c99_r.
Additionally, it has been tested with the latest gcc/clang/icc/cl.exe, static
analyzers and clang-tidy.
It survives brute force allocation
Stefan Krah added the comment:
s/icc/Python compiled with icc/
--
___
Python tracker
<https://bugs.python.org/issue40923>
___
___
Python-bugs-list mailin
Stefan Krah added the comment:
It would be easier to understand this issue if you provide the exact
command line.
icc also segfaults without -fwrapv, see #40223, so that may be the
cause here, too.
--
nosy: +skrah
___
Python tracker
<ht
Stefan Krah added the comment:
Yes, the generated Python crashes (as the link I gave shows). I'm using the
command line, so I can't be of any assistance with Parallel Studio XE.
--
___
Python tracker
<https://bugs.python.o
601 - 700 of 3396 matches
Mail list logo
