[issue33199] PyDict_Copy() can leave 'ma_version_tag' uninitialized
New submission from pdox : PyDict_Copy leaves 'ma_version_tag' uninitialized when the dictionary being copied has a split table. -- components: Interpreter Core messages: 314768 nosy: pdox priority: normal severity: normal status: open title: PyDict_Copy() can leave 'ma_version_tag' uninitialized type: behavior versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue33199> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31596] expose pthread_getcpuclockid in time module
New submission from pdox: time.clock_gettime() makes it possible to retrieve the thread-specific cpu-time clock for the current thread using time.CLOCK_THREAD_CPUTIME_ID. However, it is currently not possible in Python to retrieve the thread-specific clock for other threads. Exposing pthread_getcpuclockid() makes this possible. -- components: Extension Modules messages: 303059 nosy: pdox priority: normal pull_requests: 3756 severity: normal status: open title: expose pthread_getcpuclockid in time module type: enhancement versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31596> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31622] Make threading.get_ident() return an opaque type
New submission from pdox : Currently, Python exposes "thread identifiers" as unsigned long values across multiple modules: threading.get_ident() -> Returns an integer thread id sys._current_frames() -> Dictionary keys are integer thread ids signal.pthread_kill() -> Accepts an arbitrary integer thread id In reality, OS level thread identifiers are opaque types. On a system with pthreads, the thread_id that Python provides is merely pthread_t casted to unsigned long. This works today, but is in violation of the standard, and could break on systems with exotic pthread_t. There is also the ability to introduce undefined behavior, such as sending a signal to an invalid thread id: >>> signal.pthread_kill(42, signal.SIGHUP) Segmentation fault (core dumped) Changing the thread identifiers to an opaque type (which also tracks the validity of the thread id, for blocking use of expired system thread ids) will solve these two issues. -- components: Library (Lib) messages: 303291 nosy: pdox priority: normal severity: normal status: open title: Make threading.get_ident() return an opaque type type: enhancement versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31622> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31622] Make threading.get_ident() return an opaque type
pdox added the comment: If we don't want to change the type of get_ident(), there is another option. We could have PyThread keep track of which thread ids (e.g. pthread_t) values are valid (meaning, the thread is still running). This could be tracked in a global data structure (protected by a mutex) internal to PyThread. This can be done automatically for threads created by PyThread, and threads where Python is running (in between PyGILState_Ensure/PyGILState_Release, or other entry functions). If PyThread had this ability, then pthread_kill, etc. could raise an exception when the thread_id is invalid. The tricky part would be non-Python-created threads. There two obvious options there... one would be to provide explicit PyThread_Register/PyThread_Unregister functions that a thread could call to make itself known to PyThread. The other, would be to allow the use of unsafe thread_id's, as long as an override flag is provided. (e.g. pthread_kill(thread_id, sig, allow_unsafe=True). To take the abstraction one step further, we could completely hide the OS-level thread identifier inside PyThread, and instead generate our own ids, guaranteeing that they will always be integers. (and perhaps also guaranteeing a higher level of uniqueness) (This is not so unusual, as pthread_t is itself an abstraction, unrelated to kernel-level thread ids. On linux, the kernel identifies threads using globally unique TIDs, which can be fetched with SYS_gettid. This value does not match pthread_self(). With GLIBC, the value of pthread_t is (usually) a pointer to the bottom of the stack for the thread, which holds a thread descriptor (placed there by pthread_create). For this reason, pthread doesn't mesh well with threads that it didn't create (e.g., calling pthread_self in a thread which was created with clone(), will not do the right thing). pthread_kill is essentially a wrapper around tkill(), which first resolves the pthread_t into a TID.) -- ___ Python tracker <https://bugs.python.org/issue31622> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31596] expose pthread_getcpuclockid in time module
pdox added the comment: This looks specific to FreeBSD and s390x. Those platforms might not provide the same cpu-time clock consistency guarantees as Linux+glibc+x86. Would it be ok to just disable the ordering check for those systems? -- ___ Python tracker <https://bugs.python.org/issue31596> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31622] Make threading.get_ident() return an opaque type
pdox added the comment: I don't see much enthusiasm or agreement here, so I'm closing for now. -- resolution: -> postponed stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue31622> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31785] Move instruction code blocks to separate file
New submission from pdox : I'd like to move all instruction code (the code inside a TARGET(NAME) block) from Python/ceval.c to a new file, Python/instructions.h. The new file will contain only instruction bodies (prefixed by related helper functions/macros). Eval-context macros (e.g. TARGET, DISPATCH, TOP, PUSH, POP, etc) will not be moved to instructions.h, but will be expected to be available (defined by the #includer). ceval.c will define the eval-context macros in the same way, and #include "instructions.h", inside the body of _PyEval_EvalFrameDefault. The code emitted should remain exactly the same. The benefit of this change, is that it becomes easy to produce alternative implementations of EvalFrame which reuse the same instruction code, but with changes to the evaluation context or dispatch mechanism. In particular, after this change, I hope to experiment with adding a cross-platform subroutine-threading code evaluator. (for performance testing) -- components: Interpreter Core messages: 304370 nosy: pdox priority: normal severity: normal status: open title: Move instruction code blocks to separate file type: enhancement versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31785> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31785] Move instruction code from ceval.c to a separate file
Change by pdox : -- title: Move instruction code blocks to separate file -> Move instruction code from ceval.c to a separate file ___ Python tracker <https://bugs.python.org/issue31785> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31785] Move instruction code from ceval.c to a separate file
pdox added the comment: rhettinger, helper functions used by the instruction code would also be moved into instructions.h (e.g. call_function, do_call_core, cmp_outcome, etc). Done properly, there should be very little need to move between instructions.h and ceval.c to understand what is happening. The contract between the two files has to be very explicit. The only dependencies instructions.h would have on ceval.c would be to receive the evaluator macros (for manipulating the stack & locals, flow control, triggering error condition, etc). A strict (non-leaky) abstraction interface is necessary for being able to re-use the instruction code in an alternate evaluator. -- ___ Python tracker <https://bugs.python.org/issue31785> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31791] Ensure that all PyTypeObject fields are set to non-NULL defaults
New submission from pdox :
Ensure that every function pointer in every PyTypeObject is set to a non-NULL
default, to spare the cost of checking for NULL with every use. As a basic
example, consider PyNumber_Negative:
PyObject *
PyNumber_Negative(PyObject *o)
{
PyNumberMethods *m;
if (o == NULL) {
return null_error();
}
m = o->ob_type->tp_as_number;
if (m && m->nb_negative)
return (*m->nb_negative)(o);
return type_error("bad operand type for unary -: '%.200s'", o);
}
If "tp_as_number" and "nb_negative" were always guaranteed non-NULL, then the
function could omit the second if statement, and invoke the function pointer
directly. To maintain the existing behavior, the default nb_negative function
would be set to the following:
PyObject* nb_negative_default(PyObject *o) {
return type_error("bad operand type for unary -: '%.200s'", o);
}
This removes two NULL-checks from the PyNumber_Negative. Many other operators
and builtins would be able to benefit in the same way.
--
components: Interpreter Core
messages: 304420
nosy: pdox
priority: normal
severity: normal
status: open
title: Ensure that all PyTypeObject fields are set to non-NULL defaults
type: performance
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue31791>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31791] Ensure that all PyTypeObject fields are set to non-NULL defaults
pdox added the comment: I believe it would also make sense to inline the 'as' structures (tp_as_async, tp_as_number, tp_as_sequence, tp_as_mapping, and tp_as_buffer), instead of having them be pointers. This would save one pointer dereference with each use. But this would have API compatibility consequences, so for this change it is off the table. -- ___ Python tracker <https://bugs.python.org/issue31791> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31791] Ensure that all PyTypeObject fields are set to non-NULL defaults
pdox added the comment: serhiy.storchaka: 1) Where tp_as_number would normally be NULL, instead it would point to a fixed PyNumberMethods structure containing the default functions. This would make the memory increase negligible, as all non-number types would use the same structure. 2) If this is behavior we want to formally support, then we should provide macros for it. (all they must do is compare against the default pointer value, rather than NULL). Are there particular extension(s) you suspect may be doing this? 3) This has to be handled on a case-by-case basis. I would not remove inlined optimizations. If there are some fields that truly benefit from remaining NULL, those can be left alone. I would like to focus on the functions for which the common case (in code without type errors) is to make the indirect call. -- ___ Python tracker <https://bugs.python.org/issue31791> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31857] Make the behavior of USE_STACKCHECK deterministic
New submission from pdox : USE_STACKCHECK is a Windows-only feature which provides additional safety against C stack overflow by periodically calling PyOS_CheckStack to determine whether the current thread is too close to the end of the stack. The way USE_STACKCHECK ensures that PyOS_CheckStack is called frequently is surprising. It does this by artificially decrementing _Py_CheckRecursionLimit with every call to Py_EnterRecursiveCall: #ifdef USE_STACKCHECK /* With USE_STACKCHECK, we artificially decrement the recursion limit in order to trigger regular stack checks in _Py_CheckRecursiveCall(), except if the "overflowed" flag is set, in which case we need the true value of _Py_CheckRecursionLimit for _Py_MakeEndRecCheck() to function properly. */ # define _Py_MakeRecCheck(x) \ (++(x) > (_Py_CheckRecursionLimit += PyThreadState_GET()->overflowed - 1)) #else # define _Py_MakeRecCheck(x) (++(x) > _Py_CheckRecursionLimit) #endif _Py_CheckRecursionLimit defaults to 1000, and is constant when USE_STACKCHECK is off. (except when changed by Py_SetRecursionLimit) With a single thread, each call to Py_EnterRecursiveCall causes _Py_CheckRecursionLimit to decrement by 1 until it equals the current recursion depth, at which point _Py_CheckRecursiveCall is triggered, resetting _Py_CheckRecursionLimit back to the default. This could be anywhere from 500 to 1000 calls depending on the recursion depth. With multiple threads, the behavior may be more chaotic, as each thread will be decrementing _Py_CheckRecursionLimit independently. I propose that instead, the call to PyOS_CheckStack is triggered in a predictable fashion, using a separate counter for each thread, so that it occurs every N'th call to Py_EnterRecursiveCall. (N = 64 seems reasonable, as PyOS_CheckStack guarantees 2048 * sizeof(void*) bytes remaining on the C stack). -- components: Windows messages: 304854 nosy: paul.moore, pdox, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Make the behavior of USE_STACKCHECK deterministic type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31857> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31857] Make the behavior of USE_STACKCHECK deterministic
Change by pdox : -- keywords: +patch pull_requests: +4069 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31857> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31921] Bring together logic for entering/leaving a frame in frameobject.c
New submission from pdox : This is a refactor to move all the code involved in entering a frame (making it the active frame for the current tstate) and leaving a frame (removing it as an active frame, possibly destructing it or making it GC tracked) into private functions in frameobject.c. The underlying goal of this change is to make it possible to add (as a performance feature) stack-allocated frames, in addition to the usual heap-allocated ones. -- components: Interpreter Core messages: 305408 nosy: pdox priority: normal severity: normal status: open title: Bring together logic for entering/leaving a frame in frameobject.c type: performance versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31921> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31921] Bring together logic for entering/leaving a frame in frameobject.c
Change by pdox : -- keywords: +patch pull_requests: +4195 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31921> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31921] Bring together logic for entering/leaving a frame in frameobject.c
pdox added the comment: Raymond, this is not an experiment, but the beginning of a sustained effort to improve locality during execution, to make more effective use of CPU cache lines. Rather than discuss the details of this change one PR at a time, I will close this issue for now, and open a discussion on python-dev. -- stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue31921> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31921] Bring together logic for entering/leaving a frame in frameobject.c
Change by pdox : -- resolution: -> rejected ___ Python tracker <https://bugs.python.org/issue31921> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
