Hi Stefano,
Tested as you asked, and then bisected it. Short answer: upstream >= 3.13.13 is
fine, and you are missing a related patch. I think I have found it.
You are missing gh-130555, commit d567f451e157: "[3.13] gh-130555: Fix
use-after-free in dict.clear() with embedded values (gh-145268) (#145430)".
deb13u3 carries debian/patches/traverse-managed-dicts.patch, which is
gh-130327, commit 702d08578394 ("Always traverse managed dictionaries, even
when inline values are available"). Upstream shipped both of those in 3.13.13.
You took the first and not the second.
They are not independent. gh-130555 is a prerequisite for gh-130327:
- gh-130327 makes the GC always traverse a split table's values, including
embedded/inline ones. It removes the !mp->ma_values->embedded guard in
dict_traverse.
- Without gh-130555, clear_lock_held() still clears an embedded-values dict
with Py_CLEAR(oldvalues->values[i]) in a loop, and only sets oldvalues->size =
0 afterwards. Py_CLEAR decrefs, which can run arbitrary code and re-enter the
GC while the value slots still hold dangling pointers and size is still
non-zero.
- So the now-unconditional traverse walks freed pointers. That is exactly what
we see: the cores die in _PyObject_GC_UNTRACK / _PyGCHead_SET_PREV with a
misaligned object pointer, and in pymalloc_alloc.
- gh-130555 introduces clear_embedded_values(), which NULLs every slot and sets
size = 0 first, then decrefs from a local array. A re-entrant traverse can then
never observe a dangling slot.
So applying gh-130327 alone does not merely fail to fix something - it
introduces the use-after-free. deb13u2 does not have this crash; deb13u3 does.
The bisection. Each row is three full runs of the same test suite under
pytest-xdist -n 30, all interpreters built with gcc 14.2.0 and an identical
./configure, all with greenlet 3.5.3 and SQLAlchemy 2.0.51:
Debian 3.13.5-2+deb13u3 18, 22 crashed
workers
upstream 3.13.12 (has neither patch) 0, 0, 0
upstream 3.13.14 (has both) 0, 0, 0
upstream 3.13.5 + gh-130327 only (= your backport) 16, 10, 14
upstream 3.13.5 + gh-130327 + gh-130555 0, 0, 0
The last two rows are the useful ones: same tarball, same compiler, same flags,
one commit apart.
Worth noting for severity: this is silent heap corruption in a stable release,
not just a crash. We only caught it because a 30-way parallel test suite turns
it into reliable segfaults. A single-process workload will more often corrupt
quietly than die.
Happy to test a deb13u4 before you upload, if that is useful - I can turn a
build around the same day.
Thanks for the quick reply.
Martin
> On 13. Jul 2026, at 14:34, Stefano Rivera <[email protected]> wrote:
>
> Hi Martin (2026.07.13_06:58:31_+0000)
>> Since upgrading python3.13 from 3.13.5-2+deb13u2 to 3.13.5-2+deb13u3, an
>> application test suite that uses SQLAlchemy's asyncio ORM (and therefore
>> greenlet) segfaults reliably. The interpreter dies inside the cyclic GC and
>> inside pymalloc, at random places, in a different test on every run.
>
> Urgh, sorry about that.
>
>> The same code, the same wheels, and the same greenlet version are stable on
>> upstream CPython 3.13.5 -- the identical upstream version Debian ships -- so
>> the regression appears to come from the Debian patch set in deb13u3 rather
>> than from CPython 3.13.5 itself.
>
> Can you test this against upstream >= 3.13.13? That includes the same patch.
>
> This change is also in >= 3.14.4 and >= 3.15.0a7.
>
> If it's working in these versions, but not with our backported patch, then
> we're clearly missing a related patch, and we must find it.
>
> If it's not working with the latest upstream releases, then there's an
> upstream bug here that we should take upstream.
>
> Stefano
>
> --
> Stefano Rivera
> http://tumbleweed.org.za/
> +1 415 683 3272
