[Numpy-discussion] A bite of C++
Hi folks, https://github.com/numpy/numpy/pull/19713 showcases what *could* be a first step toward getting rid of generated C code within numpy, in favor of some C++ code, coupled with a single macro trick. Basically, templated code is an easy and robust way to replace generated code (the C++ compiler becomes the code generator when instantiating code), and a single X-macro takes care of the glue with the C world. Some changes in distutils were needed to cope with C++-specific flags, and extensions that consist in mixed C and C++ code. I've kept the change as minimal as possible to ease the (potential) transition and keep the C++ code close to the C code. This led to less idiomatic C++ code, but I value a "correct first" approach. There's an on-going effort by seiko2plus to remove that C layer, I acknowledge this would bring even more C++ code, but that looks orthogonal to me (and a very good second step!) All lights are green for the PR, let's assume it's a solid ground for discussion :-) So, well, what do you think? Should we go forward? Potential follow-ups : - do we want to use -nostdlib, to be sure we don't bring any C++ runtime dep? - what about -fno-exception, -fno-rtti? - coding style? - (I'm-not-a-farseer-I-don-t-know-all-topics) ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A bite of C++
On Wed, Aug 25, 2021 at 05:50:49PM -0500, Sebastian Berg wrote: > On Wed, 2021-08-25 at 17:48 +0200, Serge Guelton wrote: > > Hi folks, > > > > https://github.com/numpy/numpy/pull/19713 showcases what *could* be a > > first step > > toward getting rid of generated C code within numpy, in favor of some > > C++ code, > > coupled with a single macro trick. > > > > Basically, templated code is an easy and robust way to replace > > generated code > > (the C++ compiler becomes the code generator when instantiating > > code), and a > > single X-macro takes care of the glue with the C world. Hi Sebastian and thanks for the feedback. > I am not a C++ export, and really have to get to used to this code. So > I would prefer if some C++ experts can look at it and give feedback. I don't know if I'm a C++ expert, but I've a decent background with that language. I'll try to give as much clarification as I can. > This will be a bit harder to read for me than our `.c.src` code for a > while. But on the up-side, I am frustrated by my IDE not being able to > deal with the `c.src` templating. > One reaction reading the X-macro trick is that I would be more > comfortable with a positive list rather than block-listing. It just > felt a bit like too much magic and I am not sure how good it is to > assume we usually want to export everything (for one, datetimes are > pretty special). > > Even if it is verbose, I would not mind if we just list everything, so > long we have short-hands for all-integers, all-float, all-inexact, etc. There has been similar comments on the PR, I've reverted to an explicit listing. > > > > Some changes in distutils were needed to cope with C++-specific > > flags, and > > extensions that consist in mixed C and C++ code. > > > > > Potential follow-ups : > > > > - do we want to use -nostdlib, to be sure we don't bring any C++ > > runtime dep? > > What does this mean for compatibility? It sounds reasonable to me for > now if it increases systems we can run on, but I really don't know. It basically means less packaging issues as one doesn't need to link with the standard C++ library. It doesn't prevent from using some headers, but remove some aspect of the language. If numpy wants to use C++ as a preprocessor on steorids, that's fine. If Numpy wants to embrace more of C++, it's a bad idea (e.g. no new operator) > > - what about -fno-exception, -fno-rtti? > > How do C++ exceptions work at run-time? What if I store a C++ function > pointer that raises an exception and use it from a C program? Does it > add run-time overhead, do we need that `no-exception` to define that > our functions are actually C "calling convention" in this regard?! Exception add runtime overhead and imply larger binaries. If an exception is raised at C++ level and not caught at C++ level, there going to unwind the whole C stack and then call a default handler that terminates the program. > Run-time calling convention changes worry me, because I am not sure C++ > exception have a place in the current or even future ABI. All our > current API use a `-1` return value for exceptions. > > This is just like Python's type slots, so there must be "off the > shelve" approaches for this? > > Embracing C++ exceptions seems a bit steep to me right now, unless I am > missing something awesome? I totally second your opinion. In the spirit of C++ as a preprocessor on steroids, I don't see why exception would be needed. > I will note that a lot of the functions that we want to template like > this, are – and should be – accessible as public API (i.e. you can ask > NumPy to give you the function pointer). As of now, I've kept the current C symbol names, which requires a thin foward to the C++ implementation. I would be glad to remove those, but I think it's a nice second step, something that could be done once the custom preprocessor has been removed. ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Small update on conversion of %.c.src to %.cpp
Hi Folks, As already discussed a few months ago [1], I've started to move some *.src files to C++, based on the idea that the custom template engine of Numpy is very similar to C++ templates. This is a short update on the on-going translation. I've already converted the following sources: - selection.c.src - mergesort.c.src - heapsort.c.src - timsort.c.src - binsearch.c.src - clip.c.src - radixsort.c.src One can get an idea of the conversion work through these two PRs: - A typical conversion: https://github.com/numpy/numpy/pull/20897 - C/C++ layer: https://github.com/numpy/numpy/pull/20968 (this is probably the most technical part) I've also submitted this work for a Small Development Grant to NumFOCUS. I'm planning to finish the conversion whatever the result, but that would still be great ;-) See ++ Serge [1] https://mail.python.org/archives/list/numpy-discussion@python.org/thread/BLCIC2WMJQ5VT6HJSUW4V5TNGQ36JQXI/#BLCIC2WMJQ5VT6HJSUW4V5TNGQ36JQXI ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Team Summit at SciPy 2022
On Tue, Jul 19, 2022 at 12:29:40PM -0700, Sebastian Berg wrote: > Hi all, > > Last week we had a team meeting at the SciPy conference with part of > the team calling in remotely. The main discussions were about SIMD and > updating our Roadmap. > > Most discussion should simply lead to an updated roadmap, but if anyone > is curious, the meeting notes are archived here: > > https://github.com/numpy/archive/blob/main/other_meetings/2022-07-12-SciPy2022_Team_Summit.md Hi folks, I hope it doesn't come too late, but the "templated universal SIMD system" looks very similar to what has been done in xsimd [0]. xsimd is used by Krita, pythran, xtensor and has been gaining quite a lot of momentum since its 8.0.0 release. Joining effort here would probably be beneficial to the whole ecosystem... does that make any sense? Serge [0] https://github.com/xtensor-stack/xsimd ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Adding bit_count ufunc
On Fri, Nov 25, 2022 at 08:09:02PM +0100, Sebastian Berg wrote: > Thanks for bringing this up again. The Python method exists and it > seems like relatively basic functionality. > > Overall, I am slightly in favor of adding the ufunc. So if nobody > voices an opinion that it doesn't seem a good fit for NumPy, I would be > happy to move forward with it. > > - Sebastian > > > PS: One of my main concern would be if we were to add many bitwise > functions, in which case a `bitwise` namespace might be nice. But I am > not convinced that should stop us here. Technically speaking, bitwise_and, birwise_or, bitwise_xor and bitwise_not already exist and popcount is widely spread, it already has its compiler builtin under the name of __builtin_popcount ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
