09.11.17 04:08, Raymond Hettinger пише:
On Nov 8, 2017, at 8:30 AM, Serhiy Storchaka <storch...@gmail.com> wrote:
Macros Py_SETREF and Py_XSETREF were introduced in 3.6 and backported to all
maintained versions ([1] and [2]). Despite their names they are private. I
think that they are enough stable now and would be helpful in third-party code.
Are there any objections against adding them to the stable C API? [3]
I have mixed feeling about this. You and Victor seem to really like these
macros, but they have been problematic for me. I'm not sure whether it is a
conceptual issue or a naming issue, but the presence of these macros impairs my
ability to read code and determine whether the refcounts are correct. I
usually end-up replacing the code with the unrolled macro so that I can count
the refs across all the code paths.
If the problem is with naming, what names do you prefer? This already
was bikeshedded (I insisted on discussing names before introducing the
macros), but may now you have better ideas?
The current code contains 212 usages of Py_SETREF and Py_XSETREF. Maybe
10% of them correctly used temporary variables before introducing these
macros, and these macros just made the code shorter. But in the rest of
cases the refcount was decremented before setting new value. Not always
this caused a problem, but it is too hard to prove that using such code
is safe in every concrete case. Unrolling all these invocations will
make the code larger and more cumbersome, and it is hard to do
automatically.
The other issue is that when there are multiple occurrences of these macros for
multiple variables, it interferes with my best practice of deferring all
decrefs until the data structures are in a fully consistent state. Any one of
these can cause arbitrary code to run. I greatly prefer putting all the
decrefs at the end to increase my confidence that it is okay to run other code
that might reenter the current code. Pure python functions effectively have
this built-in because the locals all get decreffed at the end of the function
when a return-statement is encountered. That practice helps me avoid hard to
spot re-entrancy issues.
I agree with you. If you need to set two or more attributes
synchronously, Py_SETREF will not help you. This should be clearly
explained in the documentation.
Several subsequent Py_SETREFs may be an error. When I created my patches
for using Py_SETREF I encountered several such cases and used different
code for them. Maybe still there is not completely correct code, but in
any case it is better now than before introducing Py_SETREF.
But in many case you need to set only one attribute or different
attributes are not tightly related.
Lastly, I think we should have a preference to not grow the stable C API.
Bigger APIs are harder to learn and remember, not so much for you and Victor
who use these frequently, but for everyone else who has to lookup all the
macros whose function isn't immediately self-evident.
I agree with you again. But these macros are pretty helpful. They allow
to write the safer code easy. And they are more used than any other C
API addition in 3.5, 3.6, and 3.7.
3.7:
Py_X?SETREF -- 216
Py_UNREACHABLE -- 65
Py_RETURN_RICHCOMPARE -- 15
PyImport_GetModule -- 28
PyTraceMalloc_(T|Unt)rack -- 9
PyOS_(BeforeFork|AfterFork_(Parent|Child)) -- 24
Py_tss_NEEDS_INIT -- 6
PyInterpreterState_GetID -- 3
3.6:
PySlice_Unpack -- 22
PySlice_AdjustIndices -- 22
PyErr_SetImportErrorSubclass -- 3
PyErr_ResourceWarning -- 6
PyOS_FSPath -- 9
Py_FinalizeEx -- 21
Py_MEMBER_SIZE -- 4
3.5:
PyCodec_NameReplaceErrors -- 3
PyErr_FormatV -- 6
PyCoro_New -- 3
PyCoro_CheckExact -- 14
PyModuleDef_Init -- 30
PyModule_FromDefAndSpec2? -- 10
PyModule_ExecDef -- 3
PyModule_SetDocString -- 4
PyModule_AddFunctions -- 3
PyNumber_MatrixMultiply -- 4
PyNumber_InPlaceMatrixMultiply -- 4
Py_DecodeLocale -- 25
Py_EncodeLocale -- 11
Some older macros:
Py_STRINGIFY -- 15
Py_ABS -- 54
Py_MIN -- 66
Py_MAX - 56
The above number include declaration and definition, hence remove 2 or 3
per name for getting the number of usages.
Only added in 3.4 Py_UNUSED beats them (291) because it is used in
generated Argument Clinic code.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
