Re: [Python-Dev] Issue #23903 - stable API is incomplete
2016-12-21 2:52 GMT+01:00 Steve Dower : > _PyBytes_DecodeEscape > _PyDebug_PrintTotalRefs > _PyThreadState_Current > _PyTrash_thread_deposit_object > _PyTrash_thread_destroy_chain > _PyUnicode_DecodeUnicodeEscape > _Py_AddToAllObjects > _Py_ForgetReference > _Py_GetRefTotal > _Py_HashSecret_Initialized > _Py_NegativeRefcount > _Py_NewReference > _Py_PrintReferenceAddresses > _Py_PrintReferences > _Py_RefTotal These functions are private. Would it be possible to not export them? Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
On 21.12.16 11:50, Victor Stinner wrote: 2016-12-21 2:52 GMT+01:00 Steve Dower : _PyBytes_DecodeEscape _PyDebug_PrintTotalRefs _PyThreadState_Current _PyTrash_thread_deposit_object _PyTrash_thread_destroy_chain _PyUnicode_DecodeUnicodeEscape _Py_AddToAllObjects _Py_ForgetReference _Py_GetRefTotal _Py_HashSecret_Initialized _Py_NegativeRefcount _Py_NewReference _Py_PrintReferenceAddresses _Py_PrintReferences _Py_RefTotal These functions are private. Would it be possible to not export them? Private functions used in public macros (like _Py_NewReference) should be exported. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
2016-12-21 14:06 GMT+01:00 Serhiy Storchaka : >> These functions are private. Would it be possible to not export them? > > Private functions used in public macros (like _Py_NewReference) should be > exported. Ah, _Py_NewReference is used in the PyObject_INIT(op, typeobj) *macro*, right. IMO it's an issue with our public API: for the stable ABI, we should replace such macro with a function which hides implementation details. Example from pystate.h: #ifdef Py_BUILD_CORE PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current; # define PyThreadState_GET() \ ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) #else # define PyThreadState_GET() PyThreadState_Get() #endif Ok, now why should _Py_PrintReferences() function be exported? This private function is only called from Py_FinalizeEx(). It is not used in a macro. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PySlice_GetIndicesEx annd stable ABI: bikeshedding
Three months ago we discussed about an issue with PySlice_GetIndicesEx(). (https://mail.python.org/pipermail/python-dev/2016-August/145901.html) The problem was that PySlice_GetIndicesEx() takes the size of the sequence, but the size of the sequence can be changed when call custom __index__() methods inside PySlice_GetIndicesEx(). The solution is to split PySlice_GetIndicesEx() into two functions: one function convert Python objects to C integers by calling __index__() methods, other function takes the size of the sequence and adjusts indices, it is atomic from Python view. The code if (PySlice_GetIndicesEx(item, length, &start, &stop, &step, &slicelength) < 0) return -1; should be replaced with if (foo(item, &start, &stop, &step) < 0) return -1; slicelength = bar(&start, &stop, step, length); PySlice_GetIndicesEx() can be converted to a macro calling these two functions. It would be enough just recompile an extension to make it invulnerable to the original bug. But there is a problem. New functions should be added to stable ABI. This means that we should design names and signatures of these functions even if don't make them a part of public API. Let's start bikeshedding. What are your ideas about names and the order of arguments of two following functions? 1. Takes a slice object, returns it's start, stop and step as Py_ssize_t values. Can fail. 2. Takes slice's start, stop, step, and the length of the sequence (all are Py_ssize_t), returns adjusted start, stop, and the length of sliced subsequence (as Py_ssize_t). Always success. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
"Ok, now why should _Py_PrintReferences() function be exported?" It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is excluded from the headers (my list was automatically generated). And ideally, private functions that are deliberately exported would have comments, or if they're new, a version check on Py_LIMITED_API. Top-posted from my Windows Phone -Original Message- From: "Victor Stinner" Sent: 12/21/2016 6:25 To: "Serhiy Storchaka" Cc: "Python Dev" Subject: Re: [Python-Dev] Issue #23903 - stable API is incomplete 2016-12-21 14:06 GMT+01:00 Serhiy Storchaka : >> These functions are private. Would it be possible to not export them? > > Private functions used in public macros (like _Py_NewReference) should be > exported. Ah, _Py_NewReference is used in the PyObject_INIT(op, typeobj) *macro*, right. IMO it's an issue with our public API: for the stable ABI, we should replace such macro with a function which hides implementation details. Example from pystate.h: #ifdef Py_BUILD_CORE PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current; # define PyThreadState_GET() \ ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current)) #else # define PyThreadState_GET() PyThreadState_Get() #endif Ok, now why should _Py_PrintReferences() function be exported? This private function is only called from Py_FinalizeEx(). It is not used in a macro. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
On 21.12.16 17:41, Steve Dower wrote: "Ok, now why should _Py_PrintReferences() function be exported?" It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is excluded from the headers (my list was automatically generated). And ideally, private functions that are deliberately exported would have comments, or if they're new, a version check on Py_LIMITED_API. Seconded. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
On Dec 21, 2016 7:43 AM, "Steve Dower" wrote: "Ok, now why should _Py_PrintReferences() function be exported?" It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is excluded from the headers (my list was automatically generated). It sounds like the opt-out approach isn't working very well, and maybe an opt-in approach should be considered instead? I recognize that the way C headers work makes this difficult, but it seems like something needs to change. Or maybe the test suite should error out if any unexpected symbols appear in the stable ABI? -n ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
2016-12-21 17:21 GMT+01:00 Nathaniel Smith : > It sounds like the opt-out approach isn't working very well, and maybe an > opt-in approach should be considered instead? I recognize that the way C > headers work makes this difficult, but it seems like something needs to > change. I proposed something different: "Python 3.7: remove all private C functions from the Python C API?" https://mail.python.org/pipermail/python-dev/2016-September/146386.html Create subdirectories in Include/ to define private functions in different files. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
"maybe the test suite should error out if any unexpected symbols appear in the stable ABI?" This or on build (personally I prefer this sort of validation at build time, but I know others would prefer to defer it). We have a script now that can extract all the right functions, though I think it'll only work in the source tree as it relies on clinic. But that should make it fairly straightforward to spit out a list and compare it to a checked in list. At the same time, we have a problem in the current release, which is the functions I listed earlier. I would really like to fix that without blocking on getting the right long-term fix (since the immediate fix only affects one file in the Windows distribution, though it has implications for supportability). Top-posted from my Windows Phone -Original Message- From: "Nathaniel Smith" Sent: 12/21/2016 8:22 To: "Steve Dower" Cc: "Serhiy Storchaka" ; "Victor Stinner" ; "Python Dev" Subject: Re: [Python-Dev] Issue #23903 - stable API is incomplete On Dec 21, 2016 7:43 AM, "Steve Dower" wrote: "Ok, now why should _Py_PrintReferences() function be exported?" It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is excluded from the headers (my list was automatically generated). It sounds like the opt-out approach isn't working very well, and maybe an opt-in approach should be considered instead? I recognize that the way C headers work makes this difficult, but it seems like something needs to change. Or maybe the test suite should error out if any unexpected symbols appear in the stable ABI? -n___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #23903 - stable API is incomplete
There's a difference between "private", "stable for 3.x" and "stable for all 3" though. It's the third category that's getting too many functions added without due consideration. Top-posted from my Windows Phone -Original Message- From: "Victor Stinner" Sent: 12/21/2016 8:40 To: "Nathaniel Smith" Cc: "Steve Dower" ; "Serhiy Storchaka" ; "Python Dev" Subject: Re: [Python-Dev] Issue #23903 - stable API is incomplete 2016-12-21 17:21 GMT+01:00 Nathaniel Smith : > It sounds like the opt-out approach isn't working very well, and maybe an > opt-in approach should be considered instead? I recognize that the way C > headers work makes this difficult, but it seems like something needs to > change. I proposed something different: "Python 3.7: remove all private C functions from the Python C API?" https://mail.python.org/pipermail/python-dev/2016-September/146386.html Create subdirectories in Include/ to define private functions in different files. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PySlice_GetIndicesEx annd stable ABI: bikeshedding
On Wed, 21 Dec 2016 at 06:52 Serhiy Storchaka wrote: > [SNIP] > Let's start bikeshedding. What are your ideas about names and the order > of arguments of two following functions? > > 1. Takes a slice object, returns it's start, stop and step as Py_ssize_t > values. Can fail. > start, stop, step > > 2. Takes slice's start, stop, step, and the length of the sequence (all > are Py_ssize_t), returns adjusted start, stop, and the length of sliced > subsequence (as Py_ssize_t). Always success. > length, start, stop, step No specific ideas on names. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
