On Tue, 2021-01-19 at 13:31 +0000, Mark Shannon wrote: > Hi everyone, > > It's time for yet another PEP :) > > Fortunately, this one is a small one that doesn't change much. > It's aim is to make the VM more robust. > > Abstract > ======== > > This PEP proposes that machine stack overflow is treated differently > from runaway recursion. This would allow programs to set the maximum > recursion depth to fit their needs and provide additional safety > guarantees. > > The following program will run safely to completion: > > sys.setrecursionlimit(1_000_000) > > def f(n): > if n: > f(n-1) > > f(500_000) > > The following program will raise a StackOverflow, without causing a > VM > crash: > > sys.setrecursionlimit(1_000_000) > > class X: > def __add__(self, other): > return self + other > > X() + 1 >
This is appreciated! I recently spend quite a bit of time trying to solve a StackOverflow like this in NumPy (and was unable to fully resolve it). Of course the code triggering it was bordering on malicious, but it would be nice if it was clear how to not segfault. Just some questions/notes: * We currently mostly use `Py_EnterRecursiveCall()` in situations where we need to safe-guard against "almost python" recursions. For example an attribute lookup that returns `self`, or a list containing itself. In those cases the python recursion limit seems a bit nicer (lower and easier to understand). I am not sure it actually matters much, but my question is: Are we sure we want to replace all (or even many) C recursion checks? * Assuming we swap `Py_EnterRecursiveCall()` logic, I am wondering if a new `StackOverflow` exception name is useful. It may create two names for almost identical Python code: If you unpack a list containing itself compared to a mapping implementing `__getitem__` in Python you would get different exceptions. * `Py_CheckStackDepthWithHeadRoom()` is usually not necessary, because `Py_CheckStackDepth()` would leave plenty of headroom for typical clean-up? Can we assume that DECREF's (i.e. list, tuple), will never check the depth, so head-room is usually not necessary? This is all good, but I am not immediately sure when `Py_CheckStackDepthWithHeadRoom()` would be necessary (There are probably many cases where it clearly is, but is it ever for fairly simple code?). What happens if the maximum stack depth is reached while a `StackOverflow` exception is already set? Will the current "watermark" mechanism remain, or could there be a simple rule that an uncleared `StackOverflow` exception ensures some additional head-room? Cheers, Sebastian > ----------- > > The full PEP can be found here: > https://www.python.org/dev/peps/pep-0651 > > As always, comments are welcome. > > Cheers, > Mark. > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/ZY32N43YZJM3WYXSVD7OCGVNDGPR6DUM/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N456CVKWZ3E3VKPOE2DZMFLVSMOK5BSF/ Code of Conduct: http://python.org/psf/codeofconduct/
