[Pau Freixes <pfrei...@gmail.com>]
> Recently I've been facing a really weird bug where a Python program
> was randomly segfaulting during the finalization, the program was
> using some C extensions via Cython.
There's nothing general that can be said that would help. These
things require excruciating details to resolve.
It's generally the case that these things are caused by code missing
some aspect of correct use of Python's C API. Which can be subtle!
It's very rare that such misuse is found in the core, but not so rare
in extensions.
Here's the most recent. Be aware that it's a very long report, and -
indeed - is stuffed to the gills with excruciating details ;-)
https://bugs.python.org/issue38006
In that case a popular C extension wasn't really "playing by the
rules", but we changed CPython's cyclic garbage collector to prevent
it from segfaulting anyway.
So, one thing you didn't tell us: which version of Python were you
using? If not the most recent (3.8.1), try again with that (which
contains the patches discussed in the issue report above).
> Debugging the issue I realized that during the deallocation of one of
> the Python objects the deallocation function was trying to release a
> pointer that was surprisingly assigned to NULL. The pointer was at
> the same time held by another Python object that was an attribute of
> the Python object that had the deallocation function, something like
> this:
>
> ...
>
> So my question would be, could CPython deallocate the objects during
> the finalization step without considering the dependencies between
> objects?
No, it could not. BUT. But but but. If C code isn't playing by all
the rules, it's possible that CPython doesn't _know_ what the
dependencies actually are. CPython can only know about pointers that
the user's C API calls tell CPython about. In the report above, the C
extension didn't tell CPython about some pointers in cycles at all.
Curiously, for most kinds of garbage collectors, failing to inform the
collector about pointers is massively & widely & very reliably
catastrophic. If it doesn't see a pointer, it can conclude that a
live object is actually trash. But CPython's cyclic collector follows
pointers to prove objects _are_ trash, not to prove that they're
alive. So when CPython isn't told about a pointer, it usually "fails
soft", believing a trash object is actually alive instead. That can
cause a memory leak, but usually not a crash.
But not always. In the presence of weakrefs too, believing a trash
object is actually alive can cause some kinds of finalization to be
done "too late", which can cause a crash when refcounting discovers
that the object actually is trash. No, you're not going to get a
simple example of that ;-) DIg through the issue report above.
> If this is not the right list to make this kind of questions, just let
> me know what would be the best place for making this kind of questions
Open an issue report instead? Discussions won't solve it, alas.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/WTH2LH3S52CN435B6I273BJ3QJW5GTES/
Code of Conduct: http://python.org/psf/codeofconduct/
