Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Fri, Oct 7, 2016 at 6:12 PM, Charles R Harris wrote: > Hi All, > > The time for NumPy 1.12.0 approaches and I like to have a final decision on > the treatment of integers to negative integer powers with the `**` operator. > The two alternatives looked to be > > Raise an error for arrays and numpy scalars, including 1 and -1 to negative > powers. > > Pluses > > Backward compatible > Allows common powers to be integer, e.g., arange(3)**2 > Consistent with inplace operators > Fixes current wrong behavior. > Preserves type > > > Minuses > > Integer overflow > Computational inconvenience > Inconsistent with Python integers > > > Always return a float > > Pluses > > Computational convenience > > > Minuses > > Loss of type > Possible backward incompatibilities > Not applicable to inplace operators I guess I could be wrong, but I think the backwards incompatibilities are going to be *way* too severe to make option 2 possible in practice. -n -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sat, Oct 8, 2016 at 4:40 AM, Nathaniel Smith wrote: > On Fri, Oct 7, 2016 at 6:12 PM, Charles R Harris > wrote: > > Hi All, > > > > The time for NumPy 1.12.0 approaches and I like to have a final decision > on > > the treatment of integers to negative integer powers with the `**` > operator. > > The two alternatives looked to be > > > > Raise an error for arrays and numpy scalars, including 1 and -1 to > negative > > powers. > > > > Pluses > > > > Backward compatible > > Allows common powers to be integer, e.g., arange(3)**2 > > Consistent with inplace operators > > Fixes current wrong behavior. > > Preserves type > > > > > > Minuses > > > > Integer overflow > > Computational inconvenience > > Inconsistent with Python integers > > > > > > Always return a float > > > > Pluses > > > > Computational convenience > > > > > > Minuses > > > > Loss of type > > Possible backward incompatibilities > > Not applicable to inplace operators > > I guess I could be wrong, but I think the backwards incompatibilities > are going to be *way* too severe to make option 2 possible in > practice. > > Backwards compatibility is also a major concern for me. Here are my current thoughts - Add an fpow ufunc that always converts to float, it would not accept object arrays. - Raise errors in current power ufunc (**), for ints to negative ints. The power ufunc will change in the following ways - +1, -1 to negative ints will error, currently they work - n > 1 ints to negative ints will error, currently warn and return zero - 0 to negative ints will error, they currently return the minimum integer The `**` operator currently calls the power ufunc, leave that as is for backward almost compatibility. The remaining question is numpy scalars, which we can make either compatible with Python, or with NumPy arrays. I'm leaning towards NumPy array compatibility mostly on account of type preservation and the close relationship between zero dimensionaly arrays and scalars. The fpow function could be backported to NumPy 1.11 if that would be helpful going forward. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sat, Oct 8, 2016 at 6:59 AM, Charles R Harris wrote: > > > On Sat, Oct 8, 2016 at 4:40 AM, Nathaniel Smith wrote: >> >> On Fri, Oct 7, 2016 at 6:12 PM, Charles R Harris >> wrote: >> > Hi All, >> > >> > The time for NumPy 1.12.0 approaches and I like to have a final decision >> > on >> > the treatment of integers to negative integer powers with the `**` >> > operator. >> > The two alternatives looked to be >> > >> > Raise an error for arrays and numpy scalars, including 1 and -1 to >> > negative >> > powers. >> > >> > Pluses >> > >> > Backward compatible >> > Allows common powers to be integer, e.g., arange(3)**2 >> > Consistent with inplace operators >> > Fixes current wrong behavior. >> > Preserves type >> > >> > >> > Minuses >> > >> > Integer overflow >> > Computational inconvenience >> > Inconsistent with Python integers >> > >> > >> > Always return a float >> > >> > Pluses >> > >> > Computational convenience >> > >> > >> > Minuses >> > >> > Loss of type >> > Possible backward incompatibilities >> > Not applicable to inplace operators >> >> I guess I could be wrong, but I think the backwards incompatibilities >> are going to be *way* too severe to make option 2 possible in >> practice. >> > > Backwards compatibility is also a major concern for me. Here are my current > thoughts > > Add an fpow ufunc that always converts to float, it would not accept object > arrays. Maybe call it `fpower` or even `float_power`, for consistency with `power`? > Raise errors in current power ufunc (**), for ints to negative ints. > > The power ufunc will change in the following ways > > +1, -1 to negative ints will error, currently they work > n > 1 ints to negative ints will error, currently warn and return zero > 0 to negative ints will error, they currently return the minimum integer > > The `**` operator currently calls the power ufunc, leave that as is for > backward almost compatibility. The remaining question is numpy scalars, > which we can make either compatible with Python, or with NumPy arrays. I'm > leaning towards NumPy array compatibility mostly on account of type > preservation and the close relationship between zero dimensionaly arrays and > scalars. Sounds good to me. I agree that we should prioritize within-numpy consistency over consistency with Python. > The fpow function could be backported to NumPy 1.11 if that would be helpful > going forward. I'm not a big fan of this kind of backport. Violating the "bug-fixes-only" rule makes it hard for people to understand our release versions. And it creates the situation where people can write code that they think requires numpy 1.11 (because it works with their numpy 1.11!), but then breaks on other people's computers (because those users have 1.11.(x-1)). And if there's some reason why people aren't willing to upgrade to 1.12 for new features, then probably better to spend energy addressing those instead of on putting together 1.11-and-a-half releases. -n -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sat, Oct 8, 2016 at 9:12 AM, Nathaniel Smith wrote: > On Sat, Oct 8, 2016 at 6:59 AM, Charles R Harris > wrote: > > > > > > On Sat, Oct 8, 2016 at 4:40 AM, Nathaniel Smith wrote: > >> > >> On Fri, Oct 7, 2016 at 6:12 PM, Charles R Harris > >> wrote: > >> > Hi All, > >> > > >> > The time for NumPy 1.12.0 approaches and I like to have a final > decision > >> > on > >> > the treatment of integers to negative integer powers with the `**` > >> > operator. > >> > The two alternatives looked to be > >> > > >> > Raise an error for arrays and numpy scalars, including 1 and -1 to > >> > negative > >> > powers. > >> > > >> > Pluses > >> > > >> > Backward compatible > >> > Allows common powers to be integer, e.g., arange(3)**2 > >> > Consistent with inplace operators > >> > Fixes current wrong behavior. > >> > Preserves type > >> > > >> > > >> > Minuses > >> > > >> > Integer overflow > >> > Computational inconvenience > >> > Inconsistent with Python integers > >> > > >> > > >> > Always return a float > >> > > >> > Pluses > >> > > >> > Computational convenience > >> > > >> > > >> > Minuses > >> > > >> > Loss of type > >> > Possible backward incompatibilities > >> > Not applicable to inplace operators > >> > >> I guess I could be wrong, but I think the backwards incompatibilities > >> are going to be *way* too severe to make option 2 possible in > >> practice. > >> > > > > Backwards compatibility is also a major concern for me. Here are my > current > > thoughts > > > > Add an fpow ufunc that always converts to float, it would not accept > object > > arrays. > > Maybe call it `fpower` or even `float_power`, for consistency with `power`? > > > Raise errors in current power ufunc (**), for ints to negative ints. > > > > The power ufunc will change in the following ways > > > > +1, -1 to negative ints will error, currently they work > > n > 1 ints to negative ints will error, currently warn and return zero > > 0 to negative ints will error, they currently return the minimum integer > > > > The `**` operator currently calls the power ufunc, leave that as is for > > backward almost compatibility. The remaining question is numpy scalars, > > which we can make either compatible with Python, or with NumPy arrays. > I'm > > leaning towards NumPy array compatibility mostly on account of type > > preservation and the close relationship between zero dimensionaly arrays > and > > scalars. > > Sounds good to me. I agree that we should prioritize within-numpy > consistency over consistency with Python. > > > The fpow function could be backported to NumPy 1.11 if that would be > helpful > > going forward. > > I'm not a big fan of this kind of backport. Violating the > "bug-fixes-only" rule makes it hard for people to understand our > release versions. And it creates the situation where people can write > code that they think requires numpy 1.11 (because it works with their > numpy 1.11!), but then breaks on other people's computers (because > those users have 1.11.(x-1)). And if there's some reason why people > aren't willing to upgrade to 1.12 for new features, then probably > better to spend energy addressing those instead of on putting together > 1.11-and-a-half releases. > The power ufunc is updated in https://github.com/numpy/numpy/pull/8127. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
Hello, I think it should be consistent with Python3. So, it should give back a float. Best regards, Krisztian On Sat, Oct 8, 2016 at 3:12 AM, Charles R Harris wrote: > Hi All, > > The time for NumPy 1.12.0 approaches and I like to have a final decision > on the treatment of integers to negative integer powers with the `**` > operator. The two alternatives looked to be > > > *Raise an error for arrays and numpy scalars, including 1 and -1 to > negative powers.* > *Pluses* > >- Backward compatible >- Allows common powers to be integer, e.g., arange(3)**2 >- Consistent with inplace operators >- Fixes current wrong behavior. >- Preserves type > > > *Minuses* > >- Integer overflow >- Computational inconvenience >- Inconsistent with Python integers > > > *Always return a float * > > *Pluses* > >- Computational convenience > > > *Minuses* > >- Loss of type >- Possible backward incompatibilities >- Not applicable to inplace operators > > > > Thoughts? > > Chuck > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sat, Oct 8, 2016 at 1:31 PM, Krisztián Horváth wrote: > Hello, > > I think it should be consistent with Python3. So, it should give back a > float. > > Best regards, > Krisztian > > Can't do that and also return integers for positive powers. It isn't possible to have behavior completely compatible with python for arrays: can't have mixed type returns, can't have arbitrary precision integers. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
Sorry, I was not clear enough. I meant that the second option (always float) would be more coherent with Python3. On Oct 8, 2016 9:36 PM, "Charles R Harris" wrote: On Sat, Oct 8, 2016 at 1:31 PM, Krisztián Horváth wrote: > Hello, > > I think it should be consistent with Python3. So, it should give back a > float. > > Best regards, > Krisztian > > Can't do that and also return integers for positive powers. It isn't possible to have behavior completely compatible with python for arrays: can't have mixed type returns, can't have arbitrary precision integers. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
Well, testing under windows 64 bit, Python 3.5.2, positive powers of integers give integers and negative powers of integers give floats. So, do you want to raise an exception when taking a negative power of an element of an array of integers? Because not doing so would be inconsistent with raising the exception when applying the same operation to the array. Clearly things are broken now (I get zeros when calculating negative powers of numpy arrays of integers others than 1), but that behavior was consistent with python itself under python 2.x because the division of two integers was an integer. That does not hold under Python 3.5 where the division of two integers is a float. You have offered either to raise an exception or to always return a float (i.e. even with positive exponents). You have never offered to be consistent with what Python does. This last option would be my favorite. If it cannot be implemented, then I would prefer always float. At least one would be consistent with something and we would not invent yet another convention. On 08.10.2016 21:36, Charles R Harris wrote: On Sat, Oct 8, 2016 at 1:31 PM, Krisztián Horváth wrote: Hello, I think it should be consistent with Python3. So, it should give back a float. Best regards, Krisztian Can't do that and also return integers for positive powers. It isn't possible to have behavior completely compatible with python for arrays: can't have mixed type returns, can't have arbitrary precision integers. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sat, Oct 8, 2016 at 1:40 PM, V. Armando Sole wrote: > Well, testing under windows 64 bit, Python 3.5.2, positive powers of > integers give integers and negative powers of integers give floats. So, do > you want to raise an exception when taking a negative power of an element of > an array of integers? Because not doing so would be inconsistent with > raising the exception when applying the same operation to the array. > > Clearly things are broken now (I get zeros when calculating negative powers > of numpy arrays of integers others than 1), but that behavior was consistent > with python itself under python 2.x because the division of two integers was > an integer. That does not hold under Python 3.5 where the division of two > integers is a float. Even on Python 2, negative powers gave floats: >>> sys.version_info sys.version_info(major=2, minor=7, micro=12, releaselevel='final', serial=0) >>> 2 ** -2 0.25 > You have offered either to raise an exception or to always return a float > (i.e. even with positive exponents). You have never offered to be consistent > with what Python does. This last option would be my favorite. If it cannot > be implemented, then I would prefer always float. At least one would be > consistent with something and we would not invent yet another convention. Numpy tries to be consistent with Python when it makes sense, but this is only one of several considerations. The use cases for numpy objects are different from the use cases for Python scalar objects, so we also consistently deviate in cases when that makes sense -- e.g., numpy bools are very different from Python bools (Python barely distinguishes between bools and integers, because they don't need to; indexing makes the distinction much more important to numpy), numpy integers are very different from Python integers (Python's arbitrary-width integers provide great semantics, but don't play nicely with large fixed-size arrays), numpy pays much more attention to type consistency between inputs and outputs than Python does (again because of the extra constraints imposed by working with memory-intensive type-consistent arrays), etc. For python, 2 ** 2 -> int, 2 ** -2 -> float. But numpy can't do this, because then 2 ** np.array([2, -2]) would have to be both int *and* float, which it can't be. Not a problem that Python has. Or we could say that the output is int if all the inputs are positive, and float if any of them are negative... but then that violates the numpy principle that output dtypes should be determined entirely by input dtypes, without peeking at the actual values. (And this rule is very important for avoiding nasty surprises when you run your code on new inputs.) And then there's backwards compatibility to consider. As mentioned, we *could* deviate from Python by making ** always return float... but this would almost certainly break tons and tons of people's code that is currently doing integer ** positive integer and expecting to get an integer back. Which is something we don't do without very careful weighing of the trade-offs, and my intuition is that this one is so disruptive we probably can't pull it off. Breaking working code needs a *very* compelling reason. -n -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] delete pixel from the raster image with specific range value
any idea how to delete pixel from the raster image with specific range value using numpy/scipy or gdal? for example i have a raster image with the 5 class : 1. 0-100 2. 100-200 3. 200-300 4. 300-500 5. 500-1000 and i want to delete class 1 range value or maybe i want to delete class 1,2,4,5 if i need only class 3 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
but then that violates the numpy > principle that output dtypes should be determined entirely by input > dtypes, without peeking at the actual values. (And this rule is very > important for avoiding nasty surprises when you run your code on new > inputs.) > At division you get back an array of floats. >>> y = np.int64([1,2,4]) >>> y/1 array([ 1., 2., 4.]) >>> y/y array([ 1., 1., 1.]) Why is it different, if you calculate the power of something? > And then there's backwards compatibility to consider. As mentioned, we > *could* deviate from Python by making ** always return float... but > this would almost certainly break tons and tons of people's code that > is currently doing integer ** positive integer and expecting to get an > integer back. Which is something we don't do without very careful > weighing of the trade-offs, and my intuition is that this one is so > disruptive we probably can't pull it off. Breaking working code needs > a *very* compelling reason. > This is a valid reasoning. But it could be solved with raising an exception to warn the users for the new behaviour. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sat, Oct 8, 2016 at 3:18 PM, Krisztián Horváth wrote: > > > >> but then that violates the numpy >> principle that output dtypes should be determined entirely by input >> dtypes, without peeking at the actual values. (And this rule is very >> important for avoiding nasty surprises when you run your code on new >> inputs.) > > At division you get back an array of floats. > y = np.int64([1,2,4]) y/1 > array([ 1., 2., 4.]) y/y > array([ 1., 1., 1.]) > > Why is it different, if you calculate the power of something? The difference is that Python division always returns float. Python int ** int sometimes returns int and sometimes returns float, depending on which particular integers are used. We can't be consistent with Python because Python isn't consistent with itself. >> >> And then there's backwards compatibility to consider. As mentioned, we >> *could* deviate from Python by making ** always return float... but >> this would almost certainly break tons and tons of people's code that >> is currently doing integer ** positive integer and expecting to get an >> integer back. Which is something we don't do without very careful >> weighing of the trade-offs, and my intuition is that this one is so >> disruptive we probably can't pull it off. Breaking working code needs >> a *very* compelling reason. > > This is a valid reasoning. But it could be solved with raising an exception > to warn the users for the new behaviour. That is generally the best conservative strategy for making a backwards incompatible change like this: instead of going straight to the new behavior, first make it raise an error, and then once people have had time to stop depending on the old behavior, then you can add the new behavior. But in this case if we were going to make int ** int return float, this rule would mean that we have to make int ** int always raise an error for a few years, i.e. remove integer power support from numpy altogether. That's a non-starter. -n -- Nathaniel J. Smith -- https://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] delete pixel from the raster image with specific range value
What do you mean delete? Set to zero or NaN? You want an (N-1) dimensional array of all the acceptable values from the N dimensional array? Elliot On Oct 8, 2016 5:11 PM, "Xristos Xristoou" wrote: > any idea how to delete pixel from the raster image with > specific range value using numpy/scipy or gdal? > > for example i have a raster image with the > 5 class : > > 1. 0-100 > 2. 100-200 > 3. 200-300 > 4. 300-500 > 5. 500-1000 > > and i want to delete class 1 range value > or maybe i want to delete class 1,2,4,5 if i need only class 3 > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Integers to negative integer powers, time for a decision.
On Sun, Oct 9, 2016 at 4:12 AM, Nathaniel Smith wrote: > On Sat, Oct 8, 2016 at 6:59 AM, Charles R Harris > wrote: > > > > > > On Sat, Oct 8, 2016 at 4:40 AM, Nathaniel Smith wrote: > >> > >> On Fri, Oct 7, 2016 at 6:12 PM, Charles R Harris > >> wrote: > >> > Hi All, > >> > > >> > The time for NumPy 1.12.0 approaches and I like to have a final > decision > >> > on > >> > the treatment of integers to negative integer powers with the `**` > >> > operator. > >> > The two alternatives looked to be > >> > > >> > Raise an error for arrays and numpy scalars, including 1 and -1 to > >> > negative > >> > powers. > >> > > >> > Pluses > >> > > >> > Backward compatible > >> > Allows common powers to be integer, e.g., arange(3)**2 > >> > Consistent with inplace operators > >> > Fixes current wrong behavior. > >> > Preserves type > >> > > >> > > >> > Minuses > >> > > >> > Integer overflow > >> > Computational inconvenience > >> > Inconsistent with Python integers > >> > > >> > > >> > Always return a float > >> > > >> > Pluses > >> > > >> > Computational convenience > >> > > >> > > >> > Minuses > >> > > >> > Loss of type > >> > Possible backward incompatibilities > >> > Not applicable to inplace operators > >> > >> I guess I could be wrong, but I think the backwards incompatibilities > >> are going to be *way* too severe to make option 2 possible in > >> practice. > >> > > > > Backwards compatibility is also a major concern for me. Here are my > current > > thoughts > > > > Add an fpow ufunc that always converts to float, it would not accept > object > > arrays. > > Maybe call it `fpower` or even `float_power`, for consistency with `power`? > > > Raise errors in current power ufunc (**), for ints to negative ints. > > > > The power ufunc will change in the following ways > > > > +1, -1 to negative ints will error, currently they work > > n > 1 ints to negative ints will error, currently warn and return zero > > 0 to negative ints will error, they currently return the minimum integer > > > > The `**` operator currently calls the power ufunc, leave that as is for > > backward almost compatibility. The remaining question is numpy scalars, > > which we can make either compatible with Python, or with NumPy arrays. > I'm > > leaning towards NumPy array compatibility mostly on account of type > > preservation and the close relationship between zero dimensionaly arrays > and > > scalars. > > Sounds good to me. I agree that we should prioritize within-numpy > consistency over consistency with Python. > +1 sounds good to me too. > > > The fpow function could be backported to NumPy 1.11 if that would be > helpful > > going forward. > > I'm not a big fan of this kind of backport. Violating the > "bug-fixes-only" rule makes it hard for people to understand our > release versions. And it creates the situation where people can write > code that they think requires numpy 1.11 (because it works with their > numpy 1.11!), but then breaks on other people's computers (because > those users have 1.11.(x-1)). And if there's some reason why people > aren't willing to upgrade to 1.12 for new features, then probably > better to spend energy addressing those instead of on putting together > 1.11-and-a-half releases. > Agreed, this is not something we want to backport. Ralf > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
