Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
On Mon, 18 Jun 2018 19:49:28 +0200 Stefan Behnel wrote: > Victor Stinner schrieb am 18.06.2018 um 15:09: > > I tried two options to add support for FASTCALL on calling an object: > > add a flag in tp_flags and reuse tp_call, or add a new tp_fastcall > > slot. I failed to implement correctly any of these two options. > > > > There are multiple issues with tp_fastcall: > > > > * ABI issue: it's possible to load a C extension using the old ABI, > > without tp_fastcall: it's not possible to write type->tp_fastcall on > > such type. This limitation causes different issues. > > Not a problem if we rededicate the unused (since Py3.0) "tp_print" slot for > it. On the topic of the so-called old ABI (which doesn't really exist), I would like to merge https://github.com/python/cpython/pull/4944 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
On 2018-06-18 15:09, Victor Stinner wrote: There are multiple issues with tp_fastcall: Personally, I think that you are exaggerating these issues. Below, I'm writing the word FASTCALL to refer to tp_fastcall in your patch as well as my C call protocol in the PEP-in-progress. * ABI issue: it's possible to load a C extension using the old ABI, without tp_fastcall: it's not possible to write type->tp_fastcall on such type. This limitation causes different issues. It's not hard to check for FASTCALL support and have a case distinction between using tp_call and FASTCALL. * If tp_call is modified, tp_fastcall may be outdated. I plan to support FASTCALL only for extension types. Those cannot be changed from Python. If it turns out that FASTCALL might give significant benefits also for heap types, we can deal with those modifications: we already need to deal with such modifications anyway for existing slots like __call__. * Many public functions of the C API still requires the tuple and dict to pass positional and keyword arguments, so a compatibility layer is required to types who only want to implement FASTCALL. Related issue: what is something calls tp_call with (args: tuple, kwargs: dict)? Crash or call a compatibility layer converting arguments to FASTCALL calling convention? You make it sound as if such a "compatibility layer" is a big issue. You just need one C API function to put in the tp_call slot which calls the object instead using FASTCALL. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
On 19 June 2018 at 16:12, INADA Naoki wrote: > > On Tue, Jun 19, 2018 at 2:56 PM Jeroen Demeyer wrote: >> >> On 2018-06-18 16:55, INADA Naoki wrote: >> > Speeding up most python function and some bultin functions was very >> > significant. >> > But I doubt making some 3rd party call 20% faster can make real >> > applications significant faster. >> >> These two sentences are almost contradictory. I find it strange to claim >> that a given optimization was "very significant" in specific cases while >> saying that the same optimization won't matter in other cases. > > > It's not contradictory because there is basis: > > In most real world Python application, number of calling Python methods or > bulitin functions are much more than other calls. > > For example, optimization for bulitin `tp_init` or `tp_new` by FASTCALL was > rejected because it's implementation is complex and it's performance gain is > not significant enough on macro benchmarks. > > And I doubt number of 3rd party calls are much more than calling builtin > tp_init or tp_new. I don't think this assumption is correct, as scientific Python software spends a lot of time calling other components in the scientific Python stack, and bypassing the core language runtime entirely. However, they're using the CPython C API's function calling abstractions to do it, and those are currently expensive (frustratingly so, when the caller, the callee, *and* the interpreter implementation defining the call abstraction layer are all implemented in C). Hence Jeroen's PEPs to make the FASTCALL API a generally available one. That's quite different from the situation with object constructors, where a whole lot of applications will get to the point of having a relatively stable working set of objects, and then see the rate of object creation slow down markedly. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
That's why I suggested to add new benchmark. 2018年6月19日(火) 22:22 Ivan Levkivskyi : > On 19 June 2018 at 13:02, Nick Coghlan wrote: > >> On 19 June 2018 at 16:12, INADA Naoki wrote: >> > >> > On Tue, Jun 19, 2018 at 2:56 PM Jeroen Demeyer >> wrote: >> >> >> >> On 2018-06-18 16:55, INADA Naoki wrote: >> >> > Speeding up most python function and some bultin functions was very >> >> > significant. >> >> > But I doubt making some 3rd party call 20% faster can make real >> >> > applications significant faster. >> >> >> >> These two sentences are almost contradictory. I find it strange to >> claim >> >> that a given optimization was "very significant" in specific cases >> while >> >> saying that the same optimization won't matter in other cases. >> > >> > >> > It's not contradictory because there is basis: >> > >> > In most real world Python application, number of calling Python >> methods or >> > bulitin functions are much more than other calls. >> > >> > For example, optimization for bulitin `tp_init` or `tp_new` by FASTCALL >> was >> > rejected because it's implementation is complex and it's performance >> gain is >> > not significant enough on macro benchmarks. >> > >> > And I doubt number of 3rd party calls are much more than calling builtin >> > tp_init or tp_new. >> >> I don't think this assumption is correct, as scientific Python >> software spends a lot of time calling other components in the >> scientific Python stack, and bypassing the core language runtime >> entirely. >> >> > A recent Python survey by PSF/JetBrains shows that almost half of current > Python > users are using it for data science/ML/etc. For all these people most of > the time is spent > on calling C functions in extensions. > > -- > Ivan > > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
On 19 June 2018 at 13:02, Nick Coghlan wrote: > On 19 June 2018 at 16:12, INADA Naoki wrote: > > > > On Tue, Jun 19, 2018 at 2:56 PM Jeroen Demeyer > wrote: > >> > >> On 2018-06-18 16:55, INADA Naoki wrote: > >> > Speeding up most python function and some bultin functions was very > >> > significant. > >> > But I doubt making some 3rd party call 20% faster can make real > >> > applications significant faster. > >> > >> These two sentences are almost contradictory. I find it strange to claim > >> that a given optimization was "very significant" in specific cases while > >> saying that the same optimization won't matter in other cases. > > > > > > It's not contradictory because there is basis: > > > > In most real world Python application, number of calling Python > methods or > > bulitin functions are much more than other calls. > > > > For example, optimization for bulitin `tp_init` or `tp_new` by FASTCALL > was > > rejected because it's implementation is complex and it's performance > gain is > > not significant enough on macro benchmarks. > > > > And I doubt number of 3rd party calls are much more than calling builtin > > tp_init or tp_new. > > I don't think this assumption is correct, as scientific Python > software spends a lot of time calling other components in the > scientific Python stack, and bypassing the core language runtime > entirely. > > A recent Python survey by PSF/JetBrains shows that almost half of current Python users are using it for data science/ML/etc. For all these people most of the time is spent on calling C functions in extensions. -- Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575 (Unifying function/method classes) update
2018-06-19 13:58 GMT+02:00 Jeroen Demeyer : > Personally, I think that you are exaggerating these issues. I'm not trying to convince you to abandon the idea. I would be happy to be able to use FASTCALL in more cases! I just tried to explain why I chose to abandon my idea. FASTCALL is cute on tiny microbenchmarks, but I'm not sure that having spent almost one year on it was worth it :-) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] C-level calling (was: PEP 575 (Unifying function/method classes) update)
Victor Stinner schrieb am 19.06.2018 um 16:59: > 2018-06-19 13:58 GMT+02:00 Jeroen Demeyer : >> Personally, I think that you are exaggerating these issues. > > I'm not trying to convince you to abandon the idea. I would be happy > to be able to use FASTCALL in more cases! I just tried to explain why > I chose to abandon my idea. > > FASTCALL is cute on tiny microbenchmarks, but I'm not sure that having > spent almost one year on it was worth it :-) Fastcall is actually nice, also because it has a potential to *simplify* several things with regard to calling Python objects from C. Thanks for implementing it, Victor. Just to add another bit of background on top of the current discussion, there is an idea around, especially in the scipy/big-data community, (and I'm not giving any guarantees here that it will lead to a PEP + implementation, as it depends on people's workload) to design a dedicated C level calling interface for Python. Think of it as similar to the buffer interface, but for calling arbitrary C functions by bypassing the Python call interface entirely. Objects that wrap some kind of C function (and there are tons of them in the CPython world) would gain C signature meta data, maybe even for overloaded signatures, and C code that wants to call them could validate that meta data and call them as native C calls. But that is a rather big project to undertake, and I consider Jeroen's new PEP also a first step in that direction. Stefan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
