[Numpy-discussion] Re: An article on numpy data types
On Wed, Dec 29, 2021 at 9:59 AM Charles R Harris wrote: > On Tue, Dec 28, 2021 at 2:54 PM Warren Weckesser < > warren.weckes...@gmail.com> wrote: > >> On 12/28/21, Lev Maximov wrote: >> > On Tue, Dec 28, 2021 at 3:43 PM Evgeni Burovski >> > >> > wrote: >> > >> >> Very nice overview! >> >> >> >> One question and one suggestion: >> >> >> >> 1. Is integer wraparound guaranteed for signed ints, or is it an >> >> implementation detail? For unsigned ints, sure, it's straight from a C >> >> standard; what about signed types however. >> >> >> > Signed ints wraparound in just the same way as unsigned, both in C and >> in >> > NumPy. Added an illustration. >> >> Overflow of *signed* ints in the C language is *undefined behavior*. >> In practice, most compilers might do what you expect, but the >> wrap-around behavior is not guaranteed and should not be relied on. >> >> Warren >> >> > There used to be one's complement machines. I expect Numpy will break > badly if it is compiled for anything that doesn't use two's complement. > Chuck > Yes, according to C standard signed integer overflow is undefined behavior. So, does NumPy guarantee wraparound for signed ints overflow? (at least provided that the platform is two's complement) There is an open issue "Document behavior of casts and overflows for signed integer types" #17982 https://github.com/numpy/numpy/issues/17982 There is some discussion, but no definitive answer. As a side note, Rust has both checked/unchecked wraparound arithmetic and saturated arithmetic as specialized methods: pub const fn saturating_add(self, rhs: u32) -> u32 pub fn saturating_add_signed(self, rhs: i32) -> u32 (experimental) pub const fn saturating_mul(self, rhs: u32) -> u32 etc. Best wishes, Lev > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: lev.maxi...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: An article on numpy data types
On Thu, Dec 30, 2021 at 4:12 AM Lev Maximov wrote: > On Wed, Dec 29, 2021 at 9:59 AM Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> On Tue, Dec 28, 2021 at 2:54 PM Warren Weckesser < >> warren.weckes...@gmail.com> wrote: >> >>> On 12/28/21, Lev Maximov wrote: >>> > On Tue, Dec 28, 2021 at 3:43 PM Evgeni Burovski >>> > >>> > wrote: >>> > >>> >> Very nice overview! >>> >> >>> >> One question and one suggestion: >>> >> >>> >> 1. Is integer wraparound guaranteed for signed ints, or is it an >>> >> implementation detail? For unsigned ints, sure, it's straight from a C >>> >> standard; what about signed types however. >>> >> >>> > Signed ints wraparound in just the same way as unsigned, both in C and >>> in >>> > NumPy. Added an illustration. >>> >>> Overflow of *signed* ints in the C language is *undefined behavior*. >>> In practice, most compilers might do what you expect, but the >>> wrap-around behavior is not guaranteed and should not be relied on. >>> >>> Warren >>> >>> >> There used to be one's complement machines. I expect Numpy will break >> badly if it is compiled for anything that doesn't use two's complement. >> > > Chuck >> > > Yes, according to C standard signed integer overflow is undefined > behavior. > > So, does NumPy guarantee wraparound for signed ints overflow? > (at least provided that the platform is two's complement) > > There is an open issue > "Document behavior of casts and overflows for signed integer types" #17982 > https://github.com/numpy/numpy/issues/17982 > There is some discussion, but no definitive answer. > > As a side note, Rust has both checked/unchecked wraparound arithmetic and > saturated arithmetic > as specialized methods: > > pub const fn saturating_add(self, rhs: u32) -> u32 > pub fn saturating_add_signed(self, rhs: i32) -> u32 (experimental) > pub const fn saturating_mul(self, rhs: u32) -> u32 > etc. > > There have been discussions about overflow behavior. The main problem is performance when there is no hardware support. There used to be architectures that offered that (VAX), but it has fallen out of favor. NumPy doesn't have an official policy that I know of, but it is currently pretty much two's complement with overflow wrap. Which is not to say that things will never change, but it isn't a priority. Chuck ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: An article on numpy data types
On Fri, Dec 31, 2021 at 12:12 AM Charles R Harris wrote: > > > On Thu, Dec 30, 2021 at 4:12 AM Lev Maximov wrote: > >> On Wed, Dec 29, 2021 at 9:59 AM Charles R Harris < >> charlesr.har...@gmail.com> wrote: >> >>> On Tue, Dec 28, 2021 at 2:54 PM Warren Weckesser < >>> warren.weckes...@gmail.com> wrote: >>> On 12/28/21, Lev Maximov wrote: > On Tue, Dec 28, 2021 at 3:43 PM Evgeni Burovski > > wrote: > >> Very nice overview! >> >> One question and one suggestion: >> >> 1. Is integer wraparound guaranteed for signed ints, or is it an >> implementation detail? For unsigned ints, sure, it's straight from a C >> standard; what about signed types however. >> > Signed ints wraparound in just the same way as unsigned, both in C and in > NumPy. Added an illustration. Overflow of *signed* ints in the C language is *undefined behavior*. In practice, most compilers might do what you expect, but the wrap-around behavior is not guaranteed and should not be relied on. Warren >>> There used to be one's complement machines. I expect Numpy will break >>> badly if it is compiled for anything that doesn't use two's complement. >>> >> >> Chuck >>> >> >> Yes, according to C standard signed integer overflow is undefined >> behavior. >> >> So, does NumPy guarantee wraparound for signed ints overflow? >> (at least provided that the platform is two's complement) >> >> There is an open issue >> "Document behavior of casts and overflows for signed integer types" #17982 >> https://github.com/numpy/numpy/issues/17982 >> There is some discussion, but no definitive answer. >> >> As a side note, Rust has both checked/unchecked wraparound arithmetic and >> saturated arithmetic >> as specialized methods: >> >> pub const fn saturating_add(self, rhs: u32) -> u32 >> pub fn saturating_add_signed(self, rhs: i32) -> u32 (experimental) >> pub const fn saturating_mul(self, rhs: u32) -> u32 >> etc. >> >> > There have been discussions about overflow behavior. The main problem is > performance when there is no hardware support. There used to be > architectures that offered that (VAX), but it has fallen out of favor. > NumPy doesn't have an official policy that I know of, but it is currently > pretty much two's complement with overflow wrap. Which is not to say that > things will never change, but it isn't a priority. > Yes, that reflects my expectations. My primary concern right now is how to formulate this to the readers of the article so as not to mislead them. Can they rely on the wraparound for the signed ints in NumPy? Or is it rather 'use at your own risk'? Btw, is signed integer wrapping covered by regressions? Lev > > Chuck > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: lev.maxi...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Proposal for new function to determine if a float contains an integer
Hi, I wrote a reference implementation for a C ufunc, `isint`, which returns True for integers and False for non-integers, found here: https://github.com/madphysicist/isint_ufunc. The idea came from a Stack Overflow question of mine, which has gotten a fair number of views and even some upvotes: https://stackoverflow.com/q/35042128/2988730. The current "recommended" solution is to use ``((x % 1) == 0)``. This is slower and more cumbersome because of the math operations and the temporary storage. My version returns a single array of booleans with no intermediaries, and is between 5 and 40 times faster, depending on the type and size of the input. If you are interested in taking a look, there is a suite of tests and a small benchmarking script that compares the ufunc against the modulo expression. The entire thing currently works with bit twiddling on an appropriately converted integer representation of the number. It assumes a standard IEEE754 representation for float16, float32, float64. The extended 80-bit float128 format gets some special treatment because of the explicit integer bit. Complex numbers are currently integers only if they are real and integral. Integer types (including bool) are always integers. Time and text raise TypeErrors, since their integerness is meaningless. If a consensus forms that this is something appropriate for numpy, I will need some pointers on how to package up C code properly. This was an opportunity for me to learn to write a basic ufunc. I am still a bit confused about where code like this would go, and how to harness numpy's code generation. I put comments in my .c and .h file showing how I would expect the generators to look, but I'm not sure where to plug something like that into numpy. It would also be nice to test on architectures that have something other than a 80-bit extended long double instead of a proper float128 quad-precision number. Please let me know your thoughts. Regards, - Joe ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] No credits left in travis for 1.22.0 release
Hi All, We need to get more credits in order to run MacPython/numpy-wheels builds on travis. Anyone recall how to go about this? Chuck ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: No credits left in travis for 1.22.0 release
On Thu, Dec 30, 2021 at 6:26 PM Charles R Harris wrote: > Hi All, > > We need to get more credits in order to run MacPython/numpy-wheels builds > on travis. Anyone recall how to go about this? > > Chuck > MacPython has used 800550 of 80 credits. NumPy has used 625860 of 80 credits, so should be good for a couple of months. SciPy has one credit left, but doesn't use travis. I think the MacPython credits covers everyone who uses multibuild and travis, so NumPy is likely not the only one affected. Chuck ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Proposal for new function to determine if a float contains an integer
At least some of the commenters on that StackOverflow page need a slightly stronger check: not only is_integer(x), but also "np.iinfo(dtype).min <= x <= np.info(dtype).max" for some particular dtype. i.e. "Can I losslessly set these values into the array I already have?" On Thu, Dec 30, 2021 at 4:34 PM Joseph Fox-Rabinovitz < jfoxrabinov...@gmail.com> wrote: > Hi, > > I wrote a reference implementation for a C ufunc, `isint`, which returns > True for integers and False for non-integers, found here: > https://github.com/madphysicist/isint_ufunc. The idea came from a Stack > Overflow question of mine, which has gotten a fair number of views and even > some upvotes: https://stackoverflow.com/q/35042128/2988730. The current > "recommended" solution is to use ``((x % 1) == 0)``. This is slower and > more cumbersome because of the math operations and the temporary storage. > My version returns a single array of booleans with no intermediaries, and > is between 5 and 40 times faster, depending on the type and size of the > input. > > If you are interested in taking a look, there is a suite of tests and a > small benchmarking script that compares the ufunc against the modulo > expression. The entire thing currently works with bit twiddling on an > appropriately converted integer representation of the number. It assumes a > standard IEEE754 representation for float16, float32, float64. The extended > 80-bit float128 format gets some special treatment because of the explicit > integer bit. Complex numbers are currently integers only if they are real > and integral. Integer types (including bool) are always integers. Time and > text raise TypeErrors, since their integerness is meaningless. > > If a consensus forms that this is something appropriate for numpy, I will > need some pointers on how to package up C code properly. This was an > opportunity for me to learn to write a basic ufunc. I am still a bit > confused about where code like this would go, and how to harness numpy's > code generation. I put comments in my .c and .h file showing how I would > expect the generators to look, but I'm not sure where to plug something > like that into numpy. It would also be nice to test on architectures that > have something other than a 80-bit extended long double instead of a proper > float128 quad-precision number. > > Please let me know your thoughts. > > Regards, > > - Joe > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: jbrockmen...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Proposal for new function to determine if a float contains an integer
Is adding arbitrary optional parameters a thing with ufuncs? I could easily add upper and lower bounds checks. On Thu, Dec 30, 2021, 20:56 Brock Mendel wrote: > At least some of the commenters on that StackOverflow page need a slightly > stronger check: not only is_integer(x), but also "np.iinfo(dtype).min <= x > <= np.info(dtype).max" for some particular dtype. i.e. "Can I losslessly > set these values into the array I already have?" > > > > On Thu, Dec 30, 2021 at 4:34 PM Joseph Fox-Rabinovitz < > jfoxrabinov...@gmail.com> wrote: > >> Hi, >> >> I wrote a reference implementation for a C ufunc, `isint`, which returns >> True for integers and False for non-integers, found here: >> https://github.com/madphysicist/isint_ufunc. The idea came from a Stack >> Overflow question of mine, which has gotten a fair number of views and even >> some upvotes: https://stackoverflow.com/q/35042128/2988730. The current >> "recommended" solution is to use ``((x % 1) == 0)``. This is slower and >> more cumbersome because of the math operations and the temporary storage. >> My version returns a single array of booleans with no intermediaries, and >> is between 5 and 40 times faster, depending on the type and size of the >> input. >> >> If you are interested in taking a look, there is a suite of tests and a >> small benchmarking script that compares the ufunc against the modulo >> expression. The entire thing currently works with bit twiddling on an >> appropriately converted integer representation of the number. It assumes a >> standard IEEE754 representation for float16, float32, float64. The extended >> 80-bit float128 format gets some special treatment because of the explicit >> integer bit. Complex numbers are currently integers only if they are real >> and integral. Integer types (including bool) are always integers. Time and >> text raise TypeErrors, since their integerness is meaningless. >> >> If a consensus forms that this is something appropriate for numpy, I will >> need some pointers on how to package up C code properly. This was an >> opportunity for me to learn to write a basic ufunc. I am still a bit >> confused about where code like this would go, and how to harness numpy's >> code generation. I put comments in my .c and .h file showing how I would >> expect the generators to look, but I'm not sure where to plug something >> like that into numpy. It would also be nice to test on architectures that >> have something other than a 80-bit extended long double instead of a proper >> float128 quad-precision number. >> >> Please let me know your thoughts. >> >> Regards, >> >> - Joe >> ___ >> NumPy-Discussion mailing list -- numpy-discussion@python.org >> To unsubscribe send an email to numpy-discussion-le...@python.org >> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ >> Member address: jbrockmen...@gmail.com >> > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: jfoxrabinov...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: No credits left in travis for 1.22.0 release
On 31/12/21 3:50 am, Charles R Harris wrote: On Thu, Dec 30, 2021 at 6:26 PM Charles R Harris mailto:charlesr.har...@gmail.com>> wrote: Hi All, We need to get more credits in order to run MacPython/numpy-wheels builds on travis. Anyone recall how to go about this? Chuck MacPython has used 800550 of 80 credits. NumPy has used 625860 of 80 credits, so should be good for a couple of months. SciPy has one credit left, but doesn't use travis. I think the MacPython credits covers everyone who uses multibuild and travis, so NumPy is likely not the only one affected. Chuck I requested more OpenSource credits for MacPython. The MacPython credits cover many of the wheel-building projects, the main repos (numpy/numpy) are under different accounts. I last reqeusted credits for MacPython in October. A bit more information: Travis-ci.com bills differently for each platform. For x86, they bill 10 credits per minute on linux 20 credits per minute on windows 50 credits per minute on macos aarch64 (arm linux), ppc, and s390x are **not billed** against credits. I think this is because other companies provide the actual hardware runners. Some of the projects under MacPython like matplotlib have migrated off travis and use cibuildwheel in their main repo [0], using qemu for arm64. Others like numpy-wheels and openblas-libs moved their x86 builds to other providers, their travis builds actually consume 0 credits. h5py-wheels, scipy-wheels, pandas-wheels and others are still consuming credits for their x86 builds. If all the projects moved to other providers it would save opening requests for more credits, admittedly it is easier to request credits than to move the build :). I could also be more proactive in tracking the credit situation and request a renewal before they run out, I will set a reminder to do so 2 1/2 months from now. Matti [0] https://github.com/matplotlib/matplotlib/blob/main/.github/workflows/cibuildwheel.yml ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Who is the tallest and strongest Chinese basketball player?
Yao Ming is considered to be the wonderful as well as a best chinese basketball player. He is an executive as well as a professional https://www.justallstar.com/20-fun-facts-about-yao-ming-and-how-he-re-shapes-chinese-basketball/";>chinese basketball player. Yao ming was born in shanghai and he is considered to be a very powerful as well as a professional among all the other players of the national basketball association team. He also won a national basketball championship in the final year. Yao ming has a height of 6 feets and 3 inches. ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Who is the tallest and strongest Chinese basketball player?
On 31/12/21 6:14 am, Just allstar wrote: Yao Ming ... Sorry, I hit the wrong button when filtering spam off the list. Matti ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: No credits left in travis for 1.22.0 release
I'll need to check with the other maintainers (see https://github.com/h5py/h5py/issues/2026), but I believe h5py shouldn't need the x86 builds any more (I'm less sure about the other systems)? James On Fri, 31 Dec 2021 at 17:03, Matti Picus wrote: > > > On 31/12/21 3:50 am, Charles R Harris wrote: > > > > > > On Thu, Dec 30, 2021 at 6:26 PM Charles R Harris > > mailto:charlesr.har...@gmail.com>> wrote: > > > > Hi All, > > > > We need to get more credits in order to run MacPython/numpy-wheels > > builds on travis. Anyone recall how to go about this? > > > > Chuck > > > > > > MacPython has used 800550 of 80 credits. NumPy has used 625860 of > > 80 credits, so should be good for a couple of months. SciPy has > > one credit left, but doesn't use travis. I think the MacPython credits > > covers everyone who uses multibuild and travis, so NumPy is likely not > > the only one affected. > > > > Chuck > > > I requested more OpenSource credits for MacPython. The MacPython credits > cover many of the wheel-building projects, the main repos (numpy/numpy) > are under different accounts. I last reqeusted credits for MacPython in > October. > > > A bit more information: > > Travis-ci.com bills differently for each platform. > > For x86, they bill > > 10 credits per minute on linux > > 20 credits per minute on windows > > 50 credits per minute on macos > > aarch64 (arm linux), ppc, and s390x are **not billed** against credits. > I think this is because other companies provide the actual hardware runners. > > > Some of the projects under MacPython like matplotlib have migrated off > travis and use cibuildwheel in their main repo [0], using qemu for arm64. > > > Others like numpy-wheels and openblas-libs moved their x86 builds to > other providers, their travis builds actually consume 0 credits. > > > h5py-wheels, scipy-wheels, pandas-wheels and others are still consuming > credits for their x86 builds. If all the projects moved to other > providers it would save opening requests for more credits, admittedly it > is easier to request credits than to move the build :). I could also be > more proactive in tracking the credit situation and request a renewal > before they run out, I will set a reminder to do so 2 1/2 months from now. > > > Matti > > > [0] > https://github.com/matplotlib/matplotlib/blob/main/.github/workflows/cibuildwheel.yml > > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: aragilar+nu...@gmail.com ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
