[Numpy-discussion] Re: How is "round to N decimal places" defined for binary floating point numbers?
(Not a NumPy-dev, but here goes. Probably some of the discussions are "too basic", but hard to say which.) I would take it that round x to N radix-R digits means round_to_integer(x * R**N)/R**N (ignoring floating-point issues) Now, we can discuss what round to integer means. There, the major difference among round-to-nearest-approaches is how to handle ties (sort of what your example shows, although it is more a consequence of a limitation than a different rounding mode). Even with decimal representations you will need to define which of the supported rounding modes are used, something that is typically not accessible for users (in Python not accessible at all AFAICT). Based on standard school rounding (to nearest), tie to even is the norm (and is the default in IEEE 754), which seems to be what NumPy states that it does https://numpy.org/doc/stable/reference/generated/numpy.round.html "For values exactly halfway between rounded decimal values, NumPy rounds to the nearest even value. Thus 1.5 and 2.5 round to 2.0, -0.5 and 0.5 round to 0.0, etc." For use cases, I believe something like representing currency with, typically, two decimal digits, is a pretty good use case. However, it is hard to argue against your claim that one should probably use decimal representations instead. Still, for approximately correct computation, typically much faster as decimal support is scarce in hardware, where the need is more or less to get "good looking numbers" it still makes sense. Hence, one can probably say that it will be inexact anyway and maybe the discussion in the docs is enough. (For your example, NumPy is actually more correct than CPython. There are probably cases the other way around as well though.) One may think that an additional optional argument, specifying rounding mode, could make sense. However, since the purpose of np.round seems to be performance (as otherwise one could just have wrapped/imitated Python's round), this may remove parts of the benefit. I can also recommend the thread pointed out in the previous answer. (Didn't read it before starting writing this.) Quite a lot of interesting discussions, especially the legacy argument. BR Oscar Gustafsson (FWIW, I suggested that NumPy should be able to round to a given number of bits, or arbitrary base, primarily as a way to fake (short) fixed-point representations, but that was turned down as not really wanted as part of the core library. So there is a separate library for NumPy-like fixed-point numbers in development, with "all" possible rounding/quantization methods supported. Anyone can feel free to contact me for more info, but I will announce it, once in a usable enough state.) Den tors 28 dec. 2023 kl 22:46 skrev Stefano Miccoli via NumPy-Discussion < numpy-discussion@python.org>: > I have always been puzzled about how to correctly define the python > built-in `round(number, ndigits)` when `number` is a binary float and > `ndigits` is greater than zero. > Apparently CPython and numpy disagree: > >>> round(2.765, 2) > 2.77 > >>> np.round(2.765, 2) > 2.76 > > My question for the numpy devs are: > - Is there an authoritative source that explains what `round(number, > ndigits)` means when the digits are counted in a base different from the > one used in the floating point representation? > - Which was the first programming language to implement an intrinsic > function `round(number, ndigits)` where ndgits are always decimal, > irrespective of the representation of the floating point number? (I’m not > interested in algorithms for printing a decimal representation, but in > languages that allow to store and perform computations with the rounded > value.) > - Is `round(number, ndigits)` a useful function that deserves a rigorous > definition, or is its use limited to fuzzy situations, where accuracy can > be safely traded for speed? > > Personally I cannot think of sensible uses of `round(number, ndigits)` for > binary floats: whenever you positively need `round(number, ndigits)`, you > should use a decimal floating point representation. > > Stefano___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: oscar.gustafs...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] ANN: APyTypes 0.1
Dear all, as a consequence of a discussion here https://mail.python.org/archives/list/numpy-discussion@python.org/message/WANLFEYLKRSYSOIG4A4HFLFUHKF5EPZA/ we are now please to announce the first release of APyTypes - algorithmic data types, both fixed- and floating-point, in Python. The library aims to be a drop-in replacement for NumPy, but with the difference that you can exactly specify the numerical format (and rounding mode). It is fair to say that there is a bit of work left until that, but the scalar and array data types and basic arithmetic operations are working. The library can be installed using: pip install apytypes (please let us know if your platform is not supported). For a comparison with other libraries see: https://apytypes.github.io/apytypes/comparison.html (general documentation at https://apytypes.github.io/ repo at https://github.com/apytypes/apytypes ). It should be noted that this is not a suitable replacement if you just want to use a different, existing, dtype, e.g. from ml_dtypes, this is a completely flexible solution for any dtype with complete control of rounding/quantization, including stochastic quantization. BR Oscar Gustafsson (I will not post additional announcements here, but as this was an outcome of a discussion on the mailing list, I thought it may be of interest to at least some members.) ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] New feature: binary (arbitrary base) rounding
Hi all, I hope this is the correct way to propose a new feature. https://github.com/numpy/numpy/issues/22522 Currently, the around-function supports rounding to a given number of decimal digits. It is often quite convenient to be able to round to a given number of binary digits to mimic fixed-point representations. Currently this can be readily achieved using e.g. fractional_bits = 5 scale = 2**fractional_bits x_round = np.around(x*scale)/scale However, it would be more convenient (and probably faster) to provide dedicated support for it. I see a few different ways to obtain this: 1. Provide a separate function (binaryround?) 2. Provide a base argument to around which defaults to 10. 3. Provide a quant(ization) function where the argument is the step-size. (For completeness, one may think of having multiple quantization modes, not just rounding) Any opinions? BR Oscar Gustafsson ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding
Den tis 8 nov. 2022 kl 11:44 skrev Sebastian Berg < sebast...@sipsolutions.net>: > On Thu, 2022-11-03 at 11:37 +0100, Oscar Gustafsson wrote: > > Hi all, > > > > I hope this is the correct way to propose a new feature. > > https://github.com/numpy/numpy/issues/22522 > > > > Thanks for the proposal. I don't have much of an opinion on this and > right now I am mainly wondering whether there is prior art which can > inform us that this is relatively widely useful? > Well, if designing hardware it is typically more useful than decimal rounding. Not sure what to bring up as prior art, but the whole idea is to use it to emulate fixed-point behavior. So quantizing/rounding a floating-point value to the nearest fixed-point value with a given number of fractional bits. bround(0.37, 3) = 0.011 = 0.375. Also, see ac_fixed below which is part of Algorithmic C types, ac_types, https://hlslibs.org/ supported in most high-level synthesis tools, see e.g. https://www.intel.com/content/www/us/en/develop/documentation/oneapi-fpga-optimization-guide/top/quick-reference/algorithmic-c-data-types.html > Currently, the around-function supports rounding to a given number of > > decimal digits. It is often quite convenient to be able to round to a > > given > > number of binary digits to mimic fixed-point representations. > > Currently > > this can be readily achieved using e.g. > > > > fractional_bits = 5 > > scale = 2**fractional_bits > > x_round = np.around(x*scale)/scale > > > > However, it would be more convenient (and probably faster) to provide > > dedicated support for it. > > > This seems more like a a place for some bit-fiddling, but I am not > sure. Also a question is whether rounding modes make sense and what > you want (truncate, round to even?). > The "correct" way would of course be to do this using integers or ac_fixed https://github.com/hlslibs/ac_types/blob/master/include/ac_fixed.h You are correct that one would like to support different rounding modes (although for fixed-point one usually do not care as much about that as for floating-point, standard "add half LSB" rounding and truncation are typically enough). I didn't want to bring that up at this early stage. > I suspect that this would be more something for a project similar to > Warrens ufunclab: > > https://github.com/WarrenWeckesser/ufunclab > > I.e. written as a NumPy ufunc, but not in NumPy iself. > > You may be correct. I'll have a look at that approach for a relatively simple way to obtain it. Not sure how straightforward it is, but if one could supply the different ac_*-types as custom data types to NumPy that would be a great way of doing it. However, from a quick check it seems like it is only possible to add custom types which are "structs" of existing types. ac_typed are that in some sense (everything is internally represented as integer-types), but probably not close enough for a simple implementation. BR Oscar Gustafsson > > - Sebastian > > > > > > I see a few different ways to obtain this: > > 1. Provide a separate function (binaryround?) > > 2. Provide a base argument to around which defaults to 10. > > 3. Provide a quant(ization) function where the argument is the step- > > size. > > (For completeness, one may think of having multiple quantization > > modes, not > > just rounding) > > > > Any opinions? > > > > BR Oscar Gustafsson > > ___ > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > To unsubscribe send an email to numpy-discussion-le...@python.org > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > Member address: sebast...@sipsolutions.net > > > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: oscar.gustafs...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding
> > I'm not an expert, but I never encountered rounding floating point numbers > in bases different from 2 and 10. > I agree that this is probably not very common. More a possibility if one would supply a base argument to around. However, it is worth noting that Matlab has the quant function, https://www.mathworks.com/help/deeplearning/ref/quant.html which basically supports arbitrary bases (as a special case of an even more general approach). So there may be other use cases (although the example basically just implements around(x, 1)). BR Oscar Gustafsson ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding
Den tors 10 nov. 2022 kl 13:10 skrev Sebastian Berg < sebast...@sipsolutions.net>: > On Thu, 2022-11-10 at 11:08 +0100, Oscar Gustafsson wrote: > > > > > > I'm not an expert, but I never encountered rounding floating point > > > numbers > > > in bases different from 2 and 10. > > > > > > > I agree that this is probably not very common. More a possibility if > > one > > would supply a base argument to around. > > > > However, it is worth noting that Matlab has the quant function, > > https://www.mathworks.com/help/deeplearning/ref/quant.html which > > basically > > supports arbitrary bases (as a special case of an even more general > > approach). So there may be other use cases (although the example > > basically > > just implements around(x, 1)). > > > To be honest, hearing hardware design and data compression does make me > lean towards it not being mainstream enough that inclusion in NumPy > really makes sense. But happy to hear opposing opinions. > Here I can easily argue that "all" computations are limited by finite word length and as soon as you want to see the effect of any type of format not supported out of the box, it will be beneficial. (Strictly, it makes more sense to quantize to a given number of bits than a given number of decimal digits, as we cannot represent most of those exactly.) But I may not do that. > It would be nice to have more of a culture around ufuncs that do not > live in NumPy. (I suppose at some point it was more difficult to do C- > extension, but that is many years ago). > I do agree with this though. And this got me realizing that maybe what I actually would like to do is to create an array-library with fully customizable (numeric) data types instead. That is, sort of, the proper way to do it, although the proposed approach is indeed simpler and in most cases will work well enough. (Am I right in believing that it is not that easy to piggy-back custom data types onto NumPy arrays? Something different from using object as dtype or the "struct-like" custom approach using the existing scalar types.) BR Oscar Gustafsson ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding
Thanks! That does indeed look like a promising approach! And for sure it would be better to avoid having to reimplement the whole array-part and only focus on the data types. (If successful, my idea of a project would basically solve all the custom numerical types discussed, bfloat16, int2, int4 etc.) I understand that the following is probably a hard question to answer, but is it expected that there will be work done on this in the "near" future to fill any holes and possibly become more stable? For context, the current plan on my side is to propose this as a student project for the spring, so primarily asking for planning and describing the project a bit better. BR Oscar Den tors 10 nov. 2022 kl 15:13 skrev Sebastian Berg < sebast...@sipsolutions.net>: > On Thu, 2022-11-10 at 14:55 +0100, Oscar Gustafsson wrote: > > Den tors 10 nov. 2022 kl 13:10 skrev Sebastian Berg < > > sebast...@sipsolutions.net>: > > > > > On Thu, 2022-11-10 at 11:08 +0100, Oscar Gustafsson wrote: > > > > > > > > > > I'm not an expert, but I never encountered rounding floating > > > > > point > > > > > numbers > > > > > in bases different from 2 and 10. > > > > > > > > > > > > > I agree that this is probably not very common. More a possibility > > > > if > > > > one > > > > would supply a base argument to around. > > > > > > > > However, it is worth noting that Matlab has the quant function, > > > > https://www.mathworks.com/help/deeplearning/ref/quant.html which > > > > basically > > > > supports arbitrary bases (as a special case of an even more > > > > general > > > > approach). So there may be other use cases (although the example > > > > basically > > > > just implements around(x, 1)). > > > > > > > > > To be honest, hearing hardware design and data compression does > > > make me > > > lean towards it not being mainstream enough that inclusion in NumPy > > > really makes sense. But happy to hear opposing opinions. > > > > > > > Here I can easily argue that "all" computations are limited by finite > > word > > length and as soon as you want to see the effect of any type of > > format not > > supported out of the box, it will be beneficial. (Strictly, it makes > > more > > sense to quantize to a given number of bits than a given number of > > decimal > > digits, as we cannot represent most of those exactly.) But I may not > > do > > that. > > > > > > > It would be nice to have more of a culture around ufuncs that do > > > not > > > live in NumPy. (I suppose at some point it was more difficult to > > > do C- > > > extension, but that is many years ago). > > > > > > > I do agree with this though. And this got me realizing that maybe > > what I > > actually would like to do is to create an array-library with fully > > customizable (numeric) data types instead. That is, sort of, the > > proper way > > to do it, although the proposed approach is indeed simpler and in > > most > > cases will work well enough. > > > > (Am I right in believing that it is not that easy to piggy-back > > custom data > > types onto NumPy arrays? Something different from using object as > > dtype or > > the "struct-like" custom approach using the existing scalar types.) > > NumPy is pretty much fully customizeable (beyond just numeric data > types). > Admittedly, to not have weird edge cases and have more power you have > to use the new API (NEP 41-43 [1]) and that is "experimental" and may > have some holes. > "Experimental" doesn't mean it is expected to change significantly, > just that you can't ship your stuff broadly really. > > The holes may matter for some complicated dtypes (custom memory > allocation, parametric...). But at this point many should be rather > fixable, so before you do your own give NumPy a chance? > > - Sebastian > > > [1] https://numpy.org/neps/nep-0041-improved-dtype-support.html > > > > > BR Oscar Gustafsson > > ___ > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > To unsubscribe send an email to numpy-discussion-le...@python.org > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > Member address: sebast...@sipsolutions.net > > > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: oscar.gustafs...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
