[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Oscar Gustafsson]

Den tis 8 nov. 2022 kl 11:44 skrev Sebastian Berg <
sebast...@sipsolutions.net>:

> On Thu, 2022-11-03 at 11:37 +0100, Oscar Gustafsson wrote:
> > Hi all,
> >
> > I hope this is the correct way to propose a new feature.
> > https://github.com/numpy/numpy/issues/22522
> >
>
> Thanks for the proposal.  I don't have much of an opinion on this and
> right now I am mainly wondering whether there is prior art which can
> inform us that this is relatively widely useful?
>

Well, if designing hardware it is typically more useful than decimal
rounding. Not sure what to bring up as prior art, but the whole idea is to
use it to emulate fixed-point behavior. So quantizing/rounding a
floating-point value to the nearest fixed-point value with a given number
of fractional bits. bround(0.37, 3) = 0.011 = 0.375. Also, see ac_fixed
below which is part of Algorithmic C types, ac_types, https://hlslibs.org/
supported in most high-level synthesis tools, see e.g.
https://www.intel.com/content/www/us/en/develop/documentation/oneapi-fpga-optimization-guide/top/quick-reference/algorithmic-c-data-types.html

> Currently, the around-function supports rounding to a given number of
> > decimal digits. It is often quite convenient to be able to round to a
> > given
> > number of binary digits to mimic fixed-point representations.
> > Currently
> > this can be readily achieved using e.g.
> >
> > fractional_bits = 5
> > scale = 2**fractional_bits
> > x_round = np.around(x*scale)/scale
> >
> > However, it would be more convenient (and probably faster) to provide
> > dedicated support for it.
>
>
> This seems more like a a place for some bit-fiddling, but I am not
> sure.  Also a question is whether rounding modes make sense and what
> you want (truncate, round to even?).
>

The "correct" way would of course be to do this using integers or ac_fixed
https://github.com/hlslibs/ac_types/blob/master/include/ac_fixed.h

You are correct that one would like to support different rounding modes
(although for fixed-point one usually do not care as much about that as for
floating-point, standard "add half LSB" rounding and truncation are
typically enough). I didn't want to bring that up at this early stage.


> I suspect that this would be more something for a project similar to
> Warrens ufunclab:
>
> https://github.com/WarrenWeckesser/ufunclab
>
> I.e. written as a NumPy ufunc, but not in NumPy iself.
>
> You may be correct. I'll have a look at that approach for a
relatively simple way to obtain it.

Not sure how straightforward it is, but if one could supply the different
ac_*-types as custom data types to NumPy that would be a great way of doing
it. However, from a quick check it seems like it is only possible to add
custom types which are "structs" of existing types. ac_typed are that in
some sense (everything is internally represented as integer-types), but
probably not close enough for a simple implementation.

BR Oscar Gustafsson


>
> - Sebastian
>
>
> >
> > I see a few different ways to obtain this:
> > 1. Provide a separate function (binaryround?)
> > 2. Provide a base argument to around which defaults to 10.
> > 3. Provide a quant(ization) function where the argument is the step-
> > size.
> > (For completeness, one may think of having multiple quantization
> > modes, not
> > just rounding)
> >
> > Any opinions?
> >
> > BR Oscar Gustafsson
> > ___
> > NumPy-Discussion mailing list -- numpy-discussion@python.org
> > To unsubscribe send an email to numpy-discussion-le...@python.org
> > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> > Member address: sebast...@sipsolutions.net
>
>
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: oscar.gustafs...@gmail.com
>
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Stefano Miccoli]

On 8 Nov 2022, at 15:32, 
numpy-discussion-requ...@python.org 
wrote:

Thanks for the proposal.  I don't have much of an opinion on this and
right now I am mainly wondering whether there is prior art which can
inform us that this is relatively widely useful?

Base-2 (bit) rounding is implemented in Numcodecs 

 in the context of data compression.
As pointed out by M. Klöwer et al. in  
 "Rounding bits without real 
information to zero facilitates lossless compression algorithms and encodes the 
uncertainty within the data itself."

I'm not an expert, but I never encountered rounding floating point numbers in 
bases different from 2 and 10.

Stefano
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Oscar Gustafsson]

>
> I'm not an expert, but I never encountered rounding floating point numbers
> in bases different from 2 and 10.
>

I agree that this is probably not very common. More a possibility if one
would supply a base argument to around.

However, it is worth noting that Matlab has the quant function,
https://www.mathworks.com/help/deeplearning/ref/quant.html which basically
supports arbitrary bases (as a special case of an even more general
approach). So there may be other use cases (although the example basically
just implements around(x, 1)).

BR Oscar Gustafsson
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Sebastian Berg]

On Thu, 2022-11-10 at 11:08 +0100, Oscar Gustafsson wrote:
> > 
> > I'm not an expert, but I never encountered rounding floating point
> > numbers
> > in bases different from 2 and 10.
> > 
> 
> I agree that this is probably not very common. More a possibility if
> one
> would supply a base argument to around.
> 
> However, it is worth noting that Matlab has the quant function,
> https://www.mathworks.com/help/deeplearning/ref/quant.html which
> basically
> supports arbitrary bases (as a special case of an even more general
> approach). So there may be other use cases (although the example
> basically
> just implements around(x, 1)).


To be honest, hearing hardware design and data compression does make me
lean towards it not being mainstream enough that inclusion in NumPy
really makes sense.  But happy to hear opposing opinions.

It would be nice to have more of a culture around ufuncs that do not
live in NumPy.  (I suppose at some point it was more difficult to do C-
extension, but that is many years ago).

- Sebastian


> 
> BR Oscar Gustafsson
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: sebast...@sipsolutions.net


___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Oscar Gustafsson]

Den tors 10 nov. 2022 kl 13:10 skrev Sebastian Berg <
sebast...@sipsolutions.net>:

> On Thu, 2022-11-10 at 11:08 +0100, Oscar Gustafsson wrote:
> > >
> > > I'm not an expert, but I never encountered rounding floating point
> > > numbers
> > > in bases different from 2 and 10.
> > >
> >
> > I agree that this is probably not very common. More a possibility if
> > one
> > would supply a base argument to around.
> >
> > However, it is worth noting that Matlab has the quant function,
> > https://www.mathworks.com/help/deeplearning/ref/quant.html which
> > basically
> > supports arbitrary bases (as a special case of an even more general
> > approach). So there may be other use cases (although the example
> > basically
> > just implements around(x, 1)).
>
>
> To be honest, hearing hardware design and data compression does make me
> lean towards it not being mainstream enough that inclusion in NumPy
> really makes sense.  But happy to hear opposing opinions.
>

Here I can easily argue that "all" computations are limited by finite word
length and as soon as you want to see the effect of any type of format not
supported out of the box, it will be beneficial. (Strictly, it makes more
sense to quantize to a given number of bits than a given number of decimal
digits, as we cannot represent most of those exactly.)  But I may not do
that.


> It would be nice to have more of a culture around ufuncs that do not
> live in NumPy.  (I suppose at some point it was more difficult to do C-
> extension, but that is many years ago).
>

I do agree with this though. And this got me realizing that maybe what I
actually would like to do is to create an array-library with fully
customizable (numeric) data types instead. That is, sort of, the proper way
to do it, although the proposed approach is indeed simpler and in most
cases will work well enough.

(Am I right in believing that it is not that easy to piggy-back custom data
types onto NumPy arrays? Something different from using object as dtype or
the "struct-like" custom approach using the existing scalar types.)

BR Oscar Gustafsson
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Sebastian Berg]

On Thu, 2022-11-10 at 14:55 +0100, Oscar Gustafsson wrote:
> Den tors 10 nov. 2022 kl 13:10 skrev Sebastian Berg <
> sebast...@sipsolutions.net>:
> 
> > On Thu, 2022-11-10 at 11:08 +0100, Oscar Gustafsson wrote:
> > > > 
> > > > I'm not an expert, but I never encountered rounding floating
> > > > point
> > > > numbers
> > > > in bases different from 2 and 10.
> > > > 
> > > 
> > > I agree that this is probably not very common. More a possibility
> > > if
> > > one
> > > would supply a base argument to around.
> > > 
> > > However, it is worth noting that Matlab has the quant function,
> > > https://www.mathworks.com/help/deeplearning/ref/quant.html which
> > > basically
> > > supports arbitrary bases (as a special case of an even more
> > > general
> > > approach). So there may be other use cases (although the example
> > > basically
> > > just implements around(x, 1)).
> > 
> > 
> > To be honest, hearing hardware design and data compression does
> > make me
> > lean towards it not being mainstream enough that inclusion in NumPy
> > really makes sense.  But happy to hear opposing opinions.
> > 
> 
> Here I can easily argue that "all" computations are limited by finite
> word
> length and as soon as you want to see the effect of any type of
> format not
> supported out of the box, it will be beneficial. (Strictly, it makes
> more
> sense to quantize to a given number of bits than a given number of
> decimal
> digits, as we cannot represent most of those exactly.)  But I may not
> do
> that.
> 
> 
> > It would be nice to have more of a culture around ufuncs that do
> > not
> > live in NumPy.  (I suppose at some point it was more difficult to
> > do C-
> > extension, but that is many years ago).
> > 
> 
> I do agree with this though. And this got me realizing that maybe
> what I
> actually would like to do is to create an array-library with fully
> customizable (numeric) data types instead. That is, sort of, the
> proper way
> to do it, although the proposed approach is indeed simpler and in
> most
> cases will work well enough.
> 
> (Am I right in believing that it is not that easy to piggy-back
> custom data
> types onto NumPy arrays? Something different from using object as
> dtype or
> the "struct-like" custom approach using the existing scalar types.)

NumPy is pretty much fully customizeable (beyond just numeric data
types).
Admittedly, to not have weird edge cases and have more power you have
to use the new API (NEP 41-43 [1]) and that is "experimental" and may
have some holes.
"Experimental" doesn't mean it is expected to change significantly,
just that you can't ship your stuff broadly really.

The holes may matter for some complicated dtypes (custom memory
allocation, parametric...). But at this point many should be rather
fixable, so before you do your own give NumPy a chance?

- Sebastian


[1] https://numpy.org/neps/nep-0041-improved-dtype-support.html

> 
> BR Oscar Gustafsson
> ___
> NumPy-Discussion mailing list -- numpy-discussion@python.org
> To unsubscribe send an email to numpy-discussion-le...@python.org
> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
> Member address: sebast...@sipsolutions.net


___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: New feature: binary (arbitrary base) rounding

[Francesc Alted]

On Tue, Nov 8, 2022 at 11:43 AM Sebastian Berg 
wrote:

>
> I suspect that this would be more something for a project similar to
> Warrens ufunclab:
>
> https://github.com/WarrenWeckesser/ufunclab
>
> I.e. written as a NumPy ufunc, but not in NumPy iself.
>

Hey, ufunclab is really cool and inspiring!  I just wonder why this has not
been announced more broadly :-)

-- 
Francesc Alted
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com