[Python-Dev] Deprecating float.is_integer()
I searched usages of is_integer() on GitHub and have found that it is used *only* in silly code like (x/5).is_integer(), (x**0.5).is_integer() (or even (x**(1/3)).is_integer()) and in loops like: i = 0 while i < 20: if i.is_integer(): print(i) i += 0.1 (x/5).is_integer() is an awful way of determining the divisibility by 5. It returns wrong result for large integers and some floats. (x % 5 == 0) is a more clear and reliable way (or PEP 8 compliant (not x % 5)). Does anybody know examples of the correct use of float.is_integer() in real programs? For now it looks just like a bug magnet. I suggest to deprecate it in 3.7 or 3.8 and remove in 3.9 or 3.10. If you even need to test if a float is an exact integer, you could use (not x % 1.0). It is even faster than x.is_integer(). ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Wed, Mar 21, 2018 at 4:42 AM Steven D'Aprano wrote: > > > > Could float et al. add an __index__ method that would return a ValueError > > if the value was not an integer? > > That would allow us to write things like: > > "abcdefgh"[5.0] > > which is one of the things __index__ was invented to prevent. I’m not so sure — it was invented to prevent using e.g. 6.1 as an index, which int(I) would allow. More specifically, it was invented to Allow true integers that aren’t a python int ( like numpy int types). But, in fact, it is common to use floating point computation to compute an index — though usually one would make a conscious choice between round() and floor() and ceil() when doing so. Passing floor(a_float) as an index is a perfectly reasonable thing to do. But Guidos point is well taken — Having __index__ fail based on value is setting people up for bugs down the line. However, it seems use of is_integer() on a float is setting people up for exactly the same sorts of bugs. Another example is that pow() functions sometimes swap to an exact > algorithm if the power is an int. There's no particular reason why > x**n and x**n.0 ought to be different, but they are: > > py> 123**10 > 792594609605189126649 > > py> 123**10.0 > 7.925946096051892e+20 I think this is exactly like the __index__ use case. If the exponent is a literal, use what you mean. If the exponent is a computed float, then you really don’t want a different result depending on whether the computed value is exactly an integer or one ULP off. The user should check/convert to an integer with a method appropriate to the problem at hand. If it wasn’t too heavyweight, it might be nice to have some sort of flag on floats indicating whether they really ARE an integer, rather than happen to be: -Created from an integer literal - created from an integer object - result of floor(), ceil() or round() Any others? But that would be too heavyweight, and not that useful. In short, is_integer() is an attractive nuisance. -CHB PS: for the power example, the “right” solution is to have two operators: integer power and float power, like we do for float vs floor division. No, it’s not worth it in this case, but having it be value dependent would be worse than type dependent. > > On the other hand, some might argue that by passing 10.0 as the power, I > am specifically requesting a float implementation and result. I don't > wish to argue in favour of either position, but just to point out that > it is sometimes reasonable to want to know whether a float represents an > exact integer value or not. > > > > -- > Steve > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
> > > Does anybody know examples of the correct use of float.is_integer() in > real programs? For now it looks just like a bug magnet. I suggest to > deprecate it in 3.7 or 3.8 and remove in 3.9 or 3.10. +1 It really doesn’t appear to be the right solution for any problem. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] What is the purpose of NEXT_BLOCK()?
There is the NEXT_BLOCK() macro in compile.c. It creates a new block, creates an implicit jump from the current block to the new block, and sets it as the current block. But why it is used? All seems working if remove NEXT_BLOCK(). If there was a need of NEXT_BLOCK() (if it reduces the computational complexity of compilation or allows some optimizations), it should be documented, and we should analyze the code and add missed NEXT_BLOCK() where they are needed, and perhaps add new tests. Otherwise it can be removed. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On 14 March 2018 at 08:29, Tim Peters wrote:
> [Tim]
> >> An obvious way to extend it is for Fraction() to look for a special
> >> method too, say "_as_integer_ratio()".
>
> [Greg Ewing]
> > Why not __as_integer_ratio__?
>
> Because. at this point, that would be beating a dead horse ;-)
>
I'm not so sure about that, as if we define a protocol method for it, then
we'd presumably also define an "operator.as_integer_ratio" function, and
that function could check __index__ in addition to checking the new
protocol method.
For example:
def as_integer_ratio(n):
# Automatically accept true integers
if hasattr(n, "__index__"):
return (n.__index__(), 1)
# New reserved protocol method
if hasattr(n, "__integer_ratio__"):
return n.__integer_ratio__()
# Historical public protocol method
if hasattr(n, "as_integer_ratio"):
return n.as_integer_ratio()
# Check for lossless integer conversion
try:
int_n = int(n)
except TypeError:
pass
else:
if int_n == n:
return (int_n, 1)
raise TypeError(f"{type(n)} does not support conversion to an
integer ratio")
Similarly, on the "operator.is_integer" front:
def is_integer(n):
# Automatically accept true integers
if hasattr(n, "__index__"):
return True
# New reserved protocol method
if hasattr(n, "__is_integer__"):
return n.__is_integer__()
# Historical public protocol method
if hasattr(n, "is_integer"):
return n.is_integer()
# As a last resort, check for lossless int conversion
return int(n) == n
Cheers,
Nick.
P.S. I've suggested "operator" as a possible location, since that's where
we put "operator.index", and it's a low level module that doesn't bring in
any transitive dependencies. However, putting these protocol wrappers
somewhere else (e.g. in "math" or "numbers") may also make sense.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Wed, Mar 21, 2018 at 10:31:19AM +, Chris Barker wrote: > On Wed, Mar 21, 2018 at 4:42 AM Steven D'Aprano wrote: > > > Could float et al. add an __index__ method that would return a ValueError > > > if the value was not an integer? > > > > That would allow us to write things like: > > > > "abcdefgh"[5.0] > > > > which is one of the things __index__ was invented to prevent. > > I’m not so sure — it was invented to prevent using e.g. 6.1 as an index, > which int(I) would allow. As would int(6.0). If we wanted 6.0 to be accepted as an index, then floats would already have an __index__ method :-) [...] > But Guidos point is well taken — Having __index__ fail based on value is > setting people up for bugs down the line. > > However, it seems use of is_integer() on a float is setting people up for > exactly the same sorts of bugs. I don't think so. You aren't going to stop people from testing whether a float is an integer. (And why should you? It isn't *wrong* to do so. Some floats simply are integer valued.) All you will do is force them to write code which is even worse than what they have now. One wrong solution: int(x) == x That can raise ValueError and OverflowError, but at least it is somewhat understandable. Serhiy suggested that people should use the cryptic: (not x % 1.0) but that's hardly self-documenting: its not obvious what it does or how it works. Suppose I see that snippet in a code review, and let's suppose I recognise it and aren't totally perplexed by it. Will it pass the review? I have to make a decision: - will it fail when x is an INF or NAN? - does it give the correct results when x is negative? - does this suffer from rounding errors that could affect the result? - what if x is not a float, but a Decimal, a Fraction or an int too big to convert to a float? None of the answers are obvious at a glance. In fact, Serhiy's suggestion is not correct when x is not a float: py> from fractions import Fraction py> x =Fraction(1) + Fraction(1, 10**500) # certainly not an integer py> x.denominator == 1 # sadly Fraction doesn't support is_integer False py> not x % 1.0 True > Another example is that pow() functions sometimes swap to an exact > > algorithm if the power is an int. There's no particular reason why > > x**n and x**n.0 ought to be different, but they are: > > > > py> 123**10 > > 792594609605189126649 > > > > py> 123**10.0 > > 7.925946096051892e+20 > > > I think this is exactly like the __index__ use case. If the exponent is a > literal, use what you mean. Naturally. I already eluded to that in my earlier post. Nevertheless, this is just an example, and we shouldn't expect that the power will be a literal. I'm just illustrating the concept. > If the exponent is a computed float, then you > really don’t want a different result depending on whether the computed > value is exactly an integer or one ULP off. I don't think you actually mean to say that. I'm pretty sure that we *do* want different results if the exponent differs from an integer by one ULP. After all, that's what happens now: py> x = 25 py> x**1.0 25.0 py> x**(1.0+(2**-52)) # one ULP above 25.018 py> x**(1.0-(2**-53)) # one ULP below 24.99 I don't want to change the behaviour of pow(), but we shouldn't dismiss the possibility of some other numeric function wanting to treat values N.0 and N the same. Let's say, an is_prime(x) function that supports floats as well as ints: is_prime(3.0) # True is_prime(3.1) # False If the argument x.is_integer() returns True, then we convert to an int and test for primality. If not, then it's definitely not prime. > The user should check/convert to an integer with a method appropriate to > the problem at hand. Oh, you mean something like x.is_integer()? I agree! *wink* > If it wasn’t too heavyweight, it might be nice to have some sort of flag on > floats indicating whether they really ARE an integer, rather than happen to > be: > > -Created from an integer literal > - created from an integer object > - result of floor(), ceil() or round() I don't understand this. You seem to be saying that none of the following are "really" integer valued: float(10) floor(10.1) ceil(10.1) round(10.1) If they're not all exactly equal to the integer 10, what on earth should they equal? -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] ThreadedProcessPoolExecutor
Hi, I've made a custom concurrent.futures.Executor mixing the ProcessPoolExecutor and ThreadPoolExecutor. I've published it here: https://github.com/nilp0inter/threadedprocess This executor is very similar to a ProcessPoolExecutor, but each process in the pool have it's own ThreadPoolExecutor inside. The motivation for this executor is mitigate the problem we have in a project were we have a very large number of long running IO bounded tasks, that have to run concurrently. Those long running tasks have sparse CPU bounded operations. To resolve this problem I considered multiple solutions: 1. Use asyncio to run the IO part as tasks and use a ProcessPoolExecutor to run the CPU bounded operations with "run_in_executor". Unfortunately the CPU operations depends on a large memory context, and using a ProcessPoolExecutor this way force the parent process to picklelize all the context to send it to the task, and because the context is so large, this operation is itself very CPU demanding. So it doesn't work. 2. Executing the IO/CPU bounded operations in different processes with multiprocessing.Process. This actually works, but the number of idle processes in the system is too large, resulting in a bad memory footprint. 3. Executing the IO/CPU bounded operations in threads. This doesn't work because the sum of all CPU operations saturate the core where the Python process is running and the other cores are wasted doing nothing. So I coded the ThreadedProcessPoolExecutor that helped me maintaining the number of processes under control (I just have one process per CPU core) allowing me to have a very high concurrency (hundreds of threads per process). I have a couple of questions: The first one is about the license. Given that I copied the majority of the code from the concurrent.futures library, I understand that I have to publish the code under the PSF LICENSE. Is this correct? My second question is about the package namespace. Given that this is an concurrent.futures.Executor subclass I understand that more intuitive place to locate it is under concurrent.futures. Is this a suitable use case for namespace packages? Is this a good idea? Best regards, Roberto ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
I searched 6M LoC of Python code at Dropbox and found only three uses. They seem legit. Two are about formatting a number that's given as a float, deciding whether to print a float as 42 or 3.14. The third is attempting a conversion from float to integer where a non-integer must raise a specific exception (the same function also supports a string as long as it can be parsed as an int). I don't doubt we would get by if is_integer() was deprecated. On Wed, Mar 21, 2018 at 3:31 AM, Chris Barker wrote: > >> Does anybody know examples of the correct use of float.is_integer() in >> real programs? For now it looks just like a bug magnet. I suggest to >> deprecate it in 3.7 or 3.8 and remove in 3.9 or 3.10. > > > +1 > > It really doesn’t appear to be the right solution for any problem. > > -CHB > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What is the purpose of NEXT_BLOCK()?
Maybe spelunking in the Python 2 branch will help? It seems it was introduced in 2005 by Jeremy Hylton with this comment: /* The distinction between NEW_BLOCK and NEXT_BLOCK is subtle. (I'd like to find better names.) NEW_BLOCK() creates a new block and sets it as the current block. NEXT_BLOCK() also creates an implicit jump from the current block to the new block. */ That comment (and NEW_BLOCK()) are no longer found in the Python 3 source. On Wed, Mar 21, 2018 at 3:52 AM, Serhiy Storchaka wrote: > There is the NEXT_BLOCK() macro in compile.c. It creates a new block, > creates an implicit jump from the current block to the new block, and sets > it as the current block. > > But why it is used? All seems working if remove NEXT_BLOCK(). If there was > a need of NEXT_BLOCK() (if it reduces the computational complexity of > compilation or allows some optimizations), it should be documented, and we > should analyze the code and add missed NEXT_BLOCK() where they are needed, > and perhaps add new tests. Otherwise it can be removed. > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ThreadedProcessPoolExecutor
Roberto, That looks like an interesting class. I presume you're intending to publish this as a pip package on PyPI.python.org? I'm no lawyer, but I believe you can license your code under a new license (I recommend BSD) as long as you keep a copy and a mention of the PSF license in your distribution as well. (Though perhaps you could structure your code differently and inherit from the standard library modules rather than copying them?) In terms of the package namespace, do not put it in the same namespace as standard library code! It probably won't work and will cause world-wide pain and suffering for the users of your code. Invent your project name and use that as a top-level namespace, like everyone else. :-) Good luck with your project, --Guido On Wed, Mar 21, 2018 at 8:03 AM, Roberto Martínez < robertomartin...@gmail.com> wrote: > Hi, > > I've made a custom concurrent.futures.Executor mixing the > ProcessPoolExecutor and ThreadPoolExecutor. > > I've published it here: > > https://github.com/nilp0inter/threadedprocess > > This executor is very similar to a ProcessPoolExecutor, but each process > in the pool have it's own ThreadPoolExecutor inside. > > The motivation for this executor is mitigate the problem we have in a > project were we have a very large number of long running IO bounded tasks, > that have to run concurrently. Those long running tasks have sparse CPU > bounded operations. > > To resolve this problem I considered multiple solutions: > >1. Use asyncio to run the IO part as tasks and use a >ProcessPoolExecutor to run the CPU bounded operations with >"run_in_executor". Unfortunately the CPU operations depends on a large >memory context, and using a ProcessPoolExecutor this way force the parent >process to picklelize all the context to send it to the task, and because >the context is so large, this operation is itself very CPU demanding. So it >doesn't work. >2. Executing the IO/CPU bounded operations in different processes with >multiprocessing.Process. This actually works, but the number of idle >processes in the system is too large, resulting in a bad memory footprint. >3. Executing the IO/CPU bounded operations in threads. This doesn't >work because the sum of all CPU operations saturate the core where the >Python process is running and the other cores are wasted doing nothing. > > So I coded the ThreadedProcessPoolExecutor that helped me maintaining the > number of processes under control (I just have one process per CPU core) > allowing me to have a very high concurrency (hundreds of threads per > process). > > I have a couple of questions: > > The first one is about the license. Given that I copied the majority of > the code from the concurrent.futures library, I understand that I have to > publish the code under the PSF LICENSE. Is this correct? > > My second question is about the package namespace. Given that this is an > concurrent.futures.Executor subclass I understand that more intuitive place > to locate it is under concurrent.futures. Is this a suitable use case for > namespace packages? Is this a good idea? > > Best regards, > Roberto > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
As requested on the bug tracker, I've submitted a pull request for is_integer() support on the other numeric types. https://github.com/python/cpython/pull/6121 These are the tactics I used to implement it: - float: is_integer() already exists, so no changes - int: return True - Real: return x == int(x). Although Real doesn't explicitly support conversation to int with __int__, it does support conversion to int with __trunc__. The int constructor falls back to using __trunc__. - Rational (also inherited by Fraction): return x.denominator == 1 as Rational requires that all numbers must be represented in lowest form. - Integral: return True - Decimal: expose the existing dec_mpd_isinteger C function to Python as is_integer() ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
Thank you! As you may or may not have noticed in a different thread, we're going through a small existential crisis regarding the usefulness of is_integer() -- Serhiy believes it is not useful (and even an attractive nuisance) and should be deprecated. OTOH the existence of dec_mpd_isinteger() seems to validate to me that it actually exposes useful functionality (and every Python feature can be abused, so that alone should not be a strong argument for deprecation). On Wed, Mar 21, 2018 at 1:33 AM, Robert Smallshire wrote: > As requested on the bug tracker, I've submitted a pull request for > is_integer() support on the other numeric types. > https://github.com/python/cpython/pull/6121 > > These are the tactics I used to implement it: > > - float: is_integer() already exists, so no changes > > - int: return True > > - Real: return x == int(x). Although Real doesn't explicitly support > conversation to int with __int__, it does support conversion to int with > __trunc__. The int constructor falls back to using __trunc__. > > - Rational (also inherited by Fraction): return x.denominator == 1 as > Rational requires that all numbers must be represented in lowest form. > > - Integral: return True > > - Decimal: expose the existing dec_mpd_isinteger C function to Python as > is_integer() > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What is the purpose of NEXT_BLOCK()?
21.03.18 17:13, Guido van Rossum пише: Maybe spelunking in the Python 2 branch will help? It seems it was introduced in 2005 by Jeremy Hylton with this comment: /* The distinction between NEW_BLOCK and NEXT_BLOCK is subtle. (I'd like to find better names.) NEW_BLOCK() creates a new block and sets it as the current block. NEXT_BLOCK() also creates an implicit jump from the current block to the new block. */ That comment (and NEW_BLOCK()) are no longer found in the Python 3 source. It explains what NEXT_BLOCK() does, but not why it was needed to do this. NEW_BLOCK() was never used at all. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
On Wed, Mar 21, 2018 at 3:08 PM, Guido van Rossum wrote: > I searched 6M LoC of Python code at Dropbox and found only three uses. > They seem legit. Two are about formatting a number that's given as a float, > deciding whether to print a float as 42 or 3.14. The third is attempting a > conversion from float to integer where a non-integer must raise a specific > exception (the same function also supports a string as long as it can be > parsed as an int). > > I don't doubt we would get by if is_integer() was deprecated. > > Since code that's been deleted can't have bugs, +1. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Mar 21, 2018 05:40, "Steven D'Aprano" wrote: I don't want to change the behaviour of pow(), but we shouldn't dismiss the possibility of some other numeric function wanting to treat values N.0 and N the same. Let's say, an is_prime(x) function that supports floats as well as ints: is_prime(3.0) # True is_prime(3.1) # False For me this is an argument against is_integer() rather than for it :-). is_prime(float) should *obviously*[1] be a TypeError. Primality is only meaningfully defined over the domain of integers, and this is a case where operator.index is exactly what you want. Of course it's just an example, and perhaps there are other, better examples. But it makes me nervous that this is the best example you could quickly come up with. -n [1] Warning: I am not Dutch. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
Here's an excerpted (and slightly simplified for consumption here) usage of
float.is_integer() from the top of a function which does some
convolution/filtering in a geophysics application. I've mostly seen it used
in guard clauses in this way to reject either illegal numeric arguments
directly, or particular combinations of arguments as in this case:
def filter_convolve(x, y, xf, yf, stride=1, padding=1):
x_out = (x - xf + 2*padding) / stride + 1
y_out = (y - yf + 2*padding) / stride + 1
if not (x_out.is_integer() and y_out.is_integer()):
raise ValueError("Invalid convolution filter_convolve({x},
{y}, {xf}, {yf}, {stride}, {padding})"
.format(x=x, y=y, xf=xf, yf=yf,
stride=stride, padding=padding))
x_out = int(x_out)
y_out = int(y_out)
# ...
Of course, there are other ways to do this check, but the approach here is
obvious and easy to comprehend.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
I'd prefer to see `float.is_integer` stay. There _are_ occasions when one wants to check that a floating-point number is integral, and on those occasions, using `x.is_integer()` is the one obvious way to do it. I don't think the fact that it can be misused should be grounds for deprecation. As far as real uses: I didn't find uses of `is_integer` in our code base here at Enthought, but I did find plenty of places where it _could_ reasonably have been used, and where something less readable like `x % 1 == 0` was being used instead. For evidence that it's generally useful: it's already been noted that the decimal module uses it internally. The mpmath package defines its own "isint" function and uses it in several places: see https://github.com/fredrik-johansson/mpmath/blob/2858b1000ffdd8596defb50381dcb83de2b6/mpmath/ctx_mp_python.py#L764. MPFR also has an mpfr_integer_p predicate: http://www.mpfr.org/mpfr-current/mpfr.html#index-mpfr_005finteger_005fp. A concrete use-case: suppose you wanted to implement the beta function ( https://en.wikipedia.org/wiki/Beta_function) for real arguments in Python. You'll likely need special handling for the poles, which occur only for some negative integer arguments, so you'll need an is_integer test for those. For small positive integer arguments, you may well want the accuracy advantage that arises from computing the beta function in terms of factorials (giving a correctly-rounded result) instead of via the log of the gamma function. So again, you'll want an is_integer test to identify those cases. (Oddly enough, I found myself looking at this recently as a result of the thread about quartile definitions: there are links between the beta function, the beta distribution, and order statistics, and the (k-1/3)/(n+1/3) expression used in the recommended quartile definition comes from an approximation to the median of a beta distribution with integral parameters.) Or, you could look at the SciPy implementation of the beta function, which does indeed do the C equivalent of is_integer in many places: https://github.com/scipy/scipy/blob/11509c4a98edded6c59423ac44ca1b7f28fba1fd/scipy/special/cephes/beta.c#L67 In sum: it's an occasionally useful operation; there's no other obvious, readable spelling of the operation that does the right thing in all cases, and it's _already_ in Python! In general, I'd think that deprecation of an existing construct should not be done lightly, and should only be done when there's an obvious and significant benefit to that deprecation. I don't see that benefit here. -- Mark ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ThreadedProcessPoolExecutor
El mié., 21 mar. 2018 a las 16:23, Guido van Rossum () escribió: > Roberto, > > That looks like an interesting class. I presume you're intending to > publish this as a pip package on PyPI.python.org? > > Precisely. I'm no lawyer, but I believe you can license your code under a new license > (I recommend BSD) as long as you keep a copy and a mention of the PSF > license in your distribution as well. (Though perhaps you could structure > your code differently and inherit from the standard library modules rather > than copying them?) > I am using inheritance as much as I can. But due to some functions being at the module level, instead of being Executor methods (for the sake of being pickelizable, I suppose); I am being forced to copy some of them just to modify a couple of lines. > In terms of the package namespace, do not put it in the same namespace as > standard library code! It probably won't work and will cause world-wide > pain and suffering for the users of your code. Invent your project name and > use that as a top-level namespace, like everyone else. :-) > > Ok, I don't want to cause world-wide pain (yet). Thank you! Best regards, Roberto > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
I've been using and teaching python for close to 20 years and I never noticed that x.is_integer() exists until this thread. I would say the "one obvious way" is less than obvious. On the other hand, `x == int(x)` is genuinely obvious... and it immediately suggests the probably better `math.isclose(x, int(x))` that is what you usually mean. On Wed, Mar 21, 2018, 2:08 PM Mark Dickinson wrote: > I'd prefer to see `float.is_integer` stay. There _are_ occasions when one > wants to check that a floating-point number is integral, and on those > occasions, using `x.is_integer()` is the one obvious way to do it. I don't > think the fact that it can be misused should be grounds for deprecation. > > As far as real uses: I didn't find uses of `is_integer` in our code base > here at Enthought, but I did find plenty of places where it _could_ > reasonably have been used, and where something less readable like `x % 1 == > 0` was being used instead. For evidence that it's generally useful: it's > already been noted that the decimal module uses it internally. The mpmath > package defines its own "isint" function and uses it in several places: see > https://github.com/fredrik-johansson/mpmath/blob/2858b1000ffdd8596defb50381dcb83de2b6/mpmath/ctx_mp_python.py#L764. > MPFR also has an mpfr_integer_p predicate: > http://www.mpfr.org/mpfr-current/mpfr.html#index-mpfr_005finteger_005fp. > > A concrete use-case: suppose you wanted to implement the beta function ( > https://en.wikipedia.org/wiki/Beta_function) for real arguments in > Python. You'll likely need special handling for the poles, which occur only > for some negative integer arguments, so you'll need an is_integer test for > those. For small positive integer arguments, you may well want the accuracy > advantage that arises from computing the beta function in terms of > factorials (giving a correctly-rounded result) instead of via the log of > the gamma function. So again, you'll want an is_integer test to identify > those cases. (Oddly enough, I found myself looking at this recently as a > result of the thread about quartile definitions: there are links between > the beta function, the beta distribution, and order statistics, and the > (k-1/3)/(n+1/3) expression used in the recommended quartile definition > comes from an approximation to the median of a beta distribution with > integral parameters.) > > Or, you could look at the SciPy implementation of the beta function, which > does indeed do the C equivalent of is_integer in many places: > https://github.com/scipy/scipy/blob/11509c4a98edded6c59423ac44ca1b7f28fba1fd/scipy/special/cephes/beta.c#L67 > > In sum: it's an occasionally useful operation; there's no other obvious, > readable spelling of the operation that does the right thing in all cases, > and it's _already_ in Python! In general, I'd think that deprecation of an > existing construct should not be done lightly, and should only be done when > there's an obvious and significant benefit to that deprecation. I don't see > that benefit here. > > -- > Mark > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
On Wed, Mar 21, 2018 at 11:14 AM, David Mertz wrote: > I've been using and teaching python for close to 20 years and I never > noticed that x.is_integer() exists until this thread. I would say the "one > obvious way" is less than obvious. > > On the other hand, `x == int(x)` is genuinely obvious... and it > immediately suggests the probably better `math.isclose(x, int(x))` that is > what you usually mean. > We can argue about this forever, but I don't think I would have come up with that either when asked "how to test a float for being a whole number". I would probably have tried "x%1 == 0" which is terrible. I like to have an API that doesn't have the pitfalls of any of the "obvious" solutions that numerically naive people would come up with, and x.is_integer() is it. Let's keep it. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
[David Mertz] > I've been using and teaching python for close to 20 years and I never > noticed that x.is_integer() exists until this thread. Except it was impossible to notice across most of those years, because it didn't exist across most of those years ;-) > I would say the "one obvious way" is less than obvious. When it was introduced, it _became_ the one obvious way. > On the other hand, `x == int(x)` is genuinely obvious.. But a bad approach: it can raise OverflowError (for infinite x); it can raise ValueError (for x a NaN); and can waste relative mountains of time creating huge integers, e.g., >>> int(1e306) 117216064596736454828831087825013238982328892017892380671244575047987920451875459594568606138861698291060311049225532948520696938805711440650122628514669428460356992624968028329550689224175284346730060716088829214255439694630119794546505512415617982143262670862918816362862119154749127262208 In Python 2, x == math.floor(x) was much better on the latter count, but not in Python 3 (math.floor used to return a float, but returns an int now). As to Serhiy's `not x % 1.0`, after 5 minutes I gave up trying to prove it's always correct. Besides infinities and NaNs, there's also that Python's float mod can be surprising: >>> (-1e-20) % 1.0 1.0 There isn't a "clean" mathematical definition of what Python's float % does, which is why proof is strained. In general, the "natural" result is patched when and if needed to maintain that x == y*(x//y) + x%y is approximately true. The odd % result above is a consequence of that, and that (-1e-20) // 1.0 is inarguably -1.0. > and it immediately suggests the probably better `math.isclose(x, int(x))` that > is what you usually mean. Even in some of the poor cases Serhiy found, that wouldn't be a lick better. For example, math.isclose(x/5, int(x/5)) is still a plain wrong way to check whether x is divisible by 5. >>> x = 1e306 >>> math.isclose(x/5, int(x/5)) True >>> x/5 == int(x/5) True >>> int(x) % 5 3 The problem there isn't how "is it an integer?" is spelled, it's that _any_ way of spelling "is it an integer?" doesn't answer the question they're trying to answer. They're just plain confused about how floating point works. The use of `.is_integer()` (however spelled!) isn't the cause of that, it's a symptom. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
On Wed, Mar 21, 2018 at 3:02 PM, Tim Peters wrote:
> [David Mertz]
> > I've been using and teaching python for close to 20 years and I never
> > noticed that x.is_integer() exists until this thread.
>
> Except it was impossible to notice across most of those years, because
> it didn't exist across most of those years ;-)
>
That's probably some of the reason. I wasn't sure if someone used the time
machine to stick it back into Python 1.4.
> > On the other hand, `x == int(x)` is genuinely obvious..
>
> But a bad approach: it can raise OverflowError (for infinite x); it
> can raise ValueError (for x a NaN);
These are the CORRECT answers! Infinity neither is nor is not an integer.
Returning a boolean as an answer is bad behavior; I might argue about
*which* exception is best, but False is not a good answer to
`float('inf').is_integer()`. Infinity is neither in the Reals nor in the
Integers, but it's just as much the limit of either.
Likewise Not-a-Number isn't any less an integer than it is a real number
(approximated by a floating point number). It's NOT a number, which is
just as much not an integer.
> and can waste relative mountains
> of time creating huge integers, e.g.,
>
True enough. But it's hard to see where that should matter. No floating
point number on the order of 1e306 is sufficiently precise as to be an
integer in any meaningful sense. If you are doing number theory with
integers of that size (or larger is perfectly fine too) the actual test is
`isinstance(x, int)`. Using a float is just simply wrong for the task to
start with, whether or not those bits happen to represent something
Integral... the only case where you should see this is
"measuring/estimating something VERY big, very approximately."
For example, this can be true (even without reaching inf):
>>> x.is_integer()
True
>>> (math.sqrt(x**2)).is_integer()
False
The problem there isn't how "is it an integer?" is spelled, it's that
> _any_ way of spelling "is it an integer?" doesn't answer the question
> they're trying to answer. They're just plain confused about how
> floating point works. The use of `.is_integer()` (however spelled!)
> isn't the cause of that, it's a symptom.
>
Agreed!
--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons. Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
On Wed, Mar 21, 2018 at 8:49 PM, David Mertz wrote: > For example, this can be true (even without reaching inf): > > >>> x.is_integer() > True > >>> (math.sqrt(x**2)).is_integer() > False > If you have a moment to share it, I'd be interested to know what value of `x` you used to achieve this, and what system you were on. This can't happen under IEEE 754 arithmetic. -- Mark ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Wed, Mar 21, 2018 at 09:46:06AM -0700, Nathaniel Smith wrote: [...] > For me this is an argument against is_integer() rather than for it :-). > is_prime(float) should *obviously*[1] be a TypeError. Primality is only > meaningfully defined over the domain of integers And 3.0 is an integer. Just because it is float *object* does not mean it is not an integer *value*. Do not mistake the leaky abstraction of multiple numeric types for the mathematical number three. Primality-related functions are not limited to integers. For example, the prime counting function is defined on the reals: https://en.wikipedia.org/wiki/Prime-counting_function and there's no reason not to extend the domain of is_prime to any real. "Practicality beats purity" -- why should the result be different just because the input has a ".0" at the end? Mathematically it doesn't: the answer to something like "Is 3.0 a prime?" is a clear Yes, not "I'm sorry, I don't understand the question!" which an exception would imply. As programmers, there is always a tension between the leaky abstraction of our numeric types, and the mathematical equality of: 3 == 3.0 == 9/3 == 3+0j etc. The decision on whether to be more or less restrictive on the *types* a function accepts is up to the individual developer. Having decided to be *less* restrictive, an is_integer method would be useful. For what it's worth, Wolfram|Alpha gives inconsistant results. It allows testing of rationals for primality: "Is 9/3 a prime?" evaluates as true, but: "Is 3.0 a prime?" gets parsed as "Is 3 a prime number?" and yet evaluates as false. A clear bug for software using a natural-language interface and intended to be used by students and non-mathematicans. > and this is a case where > operator.index is exactly what you want. It is exactly not what I want. > Of course it's just an example, and perhaps there are other, better > examples. But it makes me nervous that this is the best example you could > quickly come up with. I actually had to work hard to come up with an example as simple and understandable as primality testing. The first example I thought of was Bessel functions of the 1st and 2nd kind with arbitrary real-valued orders, where you *absolutely* do want order 3.0 (float) and order 3 (int) to be precisely the same. But I avoided giving it because I thought it would be too technical and it would intimidate people. I thought that the prime number example would be easier to understand. Next time I want to make a point, I'll go for argument by intimidation. *wink* -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
Note: this is a top-posted essay much more about floating-point
philosophy than about details. Details follow from the philosophy,
and if philosophies don't match the desired details will never match
either.
Understanding floating point requires accepting that they're a funky
subset of rational numbers, augmented with some oddballs (NaNs,
"infinities", minus zero). At best the reals are a vague inspiration,
and floats have their own terminology serving their actual nature.
Thinking about reals instead is often unhelpful.
For example, it's bog standard terminology to call all IEEE-754 values
that aren't infinities or NaNs "finite". Which, by no coincidence, is
how Python's math.isfinite() discriminates. Within the finites -
which are all rational numbers - the distinction between integers and
non-integers is obvious, but only after you're aware of it and give it
some thought. Which most people aren't and don't - but that's no
reason to prevent the rest of us from getting work done ;-)
This isn't anything new in Python - it's as old as floating-point.
For example, look up C's ancient "modf" function (which breaks a
float/double into its "integer" and "fractional" parts, and treats all
finite floats of sufficiently large magnitude as having fractional
parts of 0.0 - because they in are fact exact integers).
The idea that floats are "just approximations - so all kinds of slop
is acceptable and all kinds of fear inescapable" went out of style
when IEEE-754 was introduced. That standard codified an alternative
view: that functions on floats should behave as if their inputs were
_exactly_ correct, and - given that - produce the closest
representable value to the infinitely precise result. That proved to
be extremely valuable in practice, allowing the development of
shorter, faster, more robust, and more accurate numerical algorithms.
The trend ever since has been to do more & more along those lines,
from trig functions doing argument reduction as if pi were represented
with infinite precision, to adding single-rounding dot product
primitives (all again acting as if all the inputs were exactly
correct).
Since that approach has been highly productive in real life, it's the
one I favor. Arguments like "no floating point number on the order of
1e306 is sufficiently precise as to be an integer in any meaningful
sense" don't even start to get off the ground in that approach. Maybe
in 1970 ;-) You can have no actual idea of whether 1e306 is exactly
right or off by a factor of a million just from staring at it, and
real progress has been made by assuming all values are exactly what
they appear to be, then acting accordingly. If you want to model that
some values are uncertain, that's fine, but then you need something
like interval arithmetic instead.
>From that fundamental "take floats exactly at face value" view, what
.is_integer() should do for floats is utterly obvious: there is no
possible argument about whether a given IEEE-754 float is or is not an
integer, provided you're thinking about IEEE-754 floats (and not,
e.g., about mathematical reals), and making even a tiny attempt to
honor the spirit of the IEEE-754 standard.
Whether that's _useful_ to you depends on the application you're
writing at the time. The advantage of the philosophy is that it often
gives clear guidance about what implementations "should do"
regardless, and following that guidance has repeatedly proved to be a
boon to those writing numerical methods. And, yes, also a pain in the
ass ;-)
--- nothing new below ---
On Wed, Mar 21, 2018 at 3:49 PM, David Mertz wrote:
> On Wed, Mar 21, 2018 at 3:02 PM, Tim Peters wrote:
>>
>> [David Mertz]
>> > I've been using and teaching python for close to 20 years and I never
>> > noticed that x.is_integer() exists until this thread.
>>
>> Except it was impossible to notice across most of those years, because
>> it didn't exist across most of those years ;-)
>
>
> That's probably some of the reason. I wasn't sure if someone used the time
> machine to stick it back into Python 1.4.
>
>>
>> > On the other hand, `x == int(x)` is genuinely obvious..
>>
>> But a bad approach: it can raise OverflowError (for infinite x); it
>> can raise ValueError (for x a NaN);
>
>
> These are the CORRECT answers! Infinity neither is nor is not an integer.
> Returning a boolean as an answer is bad behavior; I might argue about
> *which* exception is best, but False is not a good answer to
> `float('inf').is_integer()`. Infinity is neither in the Reals nor in the
> Integers, but it's just as much the limit of either.
>
> Likewise Not-a-Number isn't any less an integer than it is a real number
> (approximated by a floating point number). It's NOT a number, which is just
> as much not an integer.
>
>>
>> and can waste relative mountains
>> of time creating huge integers, e.g.,
>
>
> True enough. But it's hard to see where that should matter. No floating
> point number on the order of 1e306 is sufficientl
Re: [Python-Dev] Deprecating float.is_integer()
[David Mertz ]
>> For example, this can be true (even without reaching inf):
>>
>> >>> x.is_integer()
>> True
>> >>> (math.sqrt(x**2)).is_integer()
>> False
[Mark Dickinson ]
> If you have a moment to share it, I'd be interested to know what value of
> `x` you used to achieve this, and what system you were on. This can't happen
> under IEEE 754 arithmetic.
I expect it might happen under one of the directed rounding modes
(like "to +infinity").
But under 754 binary round-nearest/even arithmetic, it's been formally
proved that sqrt(x*x) == x exactly for all non-negative finite x such
that x*x neither overflows nor underflows (and .as_integer() has
nothing to do with that very strong result):
https://hal.inria.fr/hal-01148409/document
OTOH, the paper notes that it's not necessarily true for IEEE decimal
arithmetic; e.g.,
>>> import decimal
>>> decimal.getcontext().prec = 4
>>> (decimal.Decimal("31.66") ** 2).sqrt() # result is 1 ulp smaller
Decimal('31.65')
>>> decimal.getcontext().prec = 5
>>> (decimal.Decimal("31.660") ** 2).sqrt() # result is 1 ulp larger
Decimal('31.661')
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
> [Mark Dickinson ] >> If you have a moment to share it, I'd be interested to know what value of >> `x` you used to achieve this, and what system you were on. This can't happen >> under IEEE 754 arithmetic. > > I expect it might happen under one of the directed rounding modes > (like "to +infinity"). PyPy (5.8): x = 1e300 x.is_integer() True math.sqrt(x**2).is_integer() False x**2 inf (It gives an OverflowError on my CPython installs.) I believe this is allowed, and Python is not required to raise OverflowError here: https://docs.python.org/3.6/library/exceptions.html#OverflowError says: > for historical reasons, OverflowError is sometimes raised for integers that > are outside a required range. Because of the lack of standardization of > floating point exception handling in C, most floating point operations are > not checked -- Devin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
On Wed, Mar 21, 2018 at 11:43 PM, Tim Peters wrote: > Note: this is a top-posted essay much more about floating-point > philosophy than about details. Details follow from the philosophy, > and if philosophies don't match the desired details will never match > either. > but of course :-) > From that fundamental "take floats exactly at face value" view, what > .is_integer() should do for floats is utterly obvious: sure -- but I don't think anyone is arguing that -- the question is whether the function should exist -- and that means not "how should it work?" or "is it clearly and appropriately defined?" but rather, "is it the "right" thing to do in most cases, when deployed by folks that haven't thought deeply about floating point. > Whether that's _useful_ to you depends on the application you're > writing at the time. exactly. I think pretty much all the real world code that's been shown here for using .is_integer() is really about type errors (issues). The function at hand really wants integer inputs -- but wants to allow the user to be sloppy and provide a float type that happens to be an int. Given Python's duck-typing nature, maybe that's a good thing? I know I really discourage dynamic type checking Also, every example has been for small-ish integers -- exponents, factorials, etc -- not order 1e300 -- or inf or NaN, etc. Finally, the use-cases where the value that happens-to-be-an-int is computed via floating point -- .is_integer() is probably the wrong check -- you probably want isclose(). The other use-cases: and floor() and ceil() and round() all produce actual integers -- so no need for that anymore. All this points to: we don't need .is_integer All the being said -- the standard for depreciation is much higher bar than not-adding-it-in-the-first-place. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
[Devin Jeanpierre ] > PyPy (5.8): > x = 1e300 > x.is_integer() > True > math.sqrt(x**2).is_integer() > False > x**2 > inf I think you missed that David said "even without reaching inf" (you did reach inf), and that I said "such that x*x neither overflows nor underflows". Those are technical words related to IEEE-754: your x*x sets the IEEE overflow flag, although CPython may or may not raise the Python OverflowError exception. > > (It gives an OverflowError on my CPython installs.) > > I believe this is allowed, and Python is not required to raise > OverflowError here: > https://docs.python.org/3.6/library/exceptions.html#OverflowError > says: > >> for historical reasons, OverflowError is sometimes raised for integers that >> are outside a required range. Because of the lack of standardization of >> floating point exception handling in C, most floating point operations are >> not checked You can avoid the OverflowError (but not the IEEE overflow condition!) under CPython by multiplying instead: >>> x = 1e300 >>> x*x inf ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Wed, Mar 21, 2018 at 12:38 PM, Steven D'Aprano wrote: > In fact, Serhiy's suggestion is not correct when x is not a float: > This is a key point -- see some of Tim's posts -- like ot or not, you probably need to know when you are working with a float and when you aren't -- and what the implications of that are. I think the key pint is this: where does the logic of whether a given object represents an integer belong? At first glance, it clearly belongs with the type -- floats know how they are represented, as do fractions and decimals -- they can determine it in an efficient and clearly defined way. However, my argument is that while an integer-valued float is clearly defined in the binary representation, what "is this an integer?" means is actually use-case dependent, and thus the user should be deciding how to do it (i.e. with isclose() or int(x) == x or > If the exponent is a computed float, then you > > really don’t want a different result depending on whether the computed > > value is exactly an integer or one ULP off. > > I don't think you actually mean to say that. I'm pretty sure that we > *do* want different results if the exponent differs from an integer by > one ULP. yes -- poorly worded -- I mean you want the slightly different result, not a different type and algorithm - i.e continuity if you had a range of slightly less than an integer to slightly more than an integer. > The user should check/convert to an integer with a method appropriate to > > the problem at hand. > > Oh, you mean something like x.is_integer()? I agree! > > *wink* > That's the point -- is_integer may or may not be appropriate, and whether it is is a function of use-case, not type. > If it wasn’t too heavyweight, it might be nice to have some sort of flag > on > > floats indicating whether they really ARE an integer, rather than happen > to > > be: > > > > -Created from an integer literal > > - created from an integer object > > - result of floor(), ceil() or round() > > I don't understand this. > poorly worked again -- I shoud not write these on a phone > You seem to be saying that none of the following are "really" integer > valued: > > float(10) > floor(10.1) > ceil(10.1) > round(10.1) > I meant those are the ones that ARE really integer valued. turns out that py3 returns an actual int for all of those other than float() (of course) anyway, so that's pretty much been done -- and no need for is_integer() well, it's been fun, but looks like it's sticking around. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Wed, Mar 21, 2018 at 4:12 PM, Guido van Rossum wrote: > Thank you! As you may or may not have noticed in a different thread, we're > going through a small existential crisis regarding the usefulness of > is_integer() -- Serhiy believes it is not useful (and even an attractive > nuisance) and should be deprecated. OTOH the existence of > dec_mpd_isinteger() seems to validate to me that it actually exposes useful > functionality (and every Python feature can be abused, so that alone should > not be a strong argument for deprecation). > if not deprecated, then do we add it to all the other numeric types? Which was the original suggestion, yes? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
Ok. I'm wrong on that example.
On Wed, Mar 21, 2018, 9:11 PM Tim Peters wrote:
> [David Mertz ]
> >> For example, this can be true (even without reaching inf):
> >>
> >> >>> x.is_integer()
> >> True
> >> >>> (math.sqrt(x**2)).is_integer()
> >> False
>
> [Mark Dickinson ]
> > If you have a moment to share it, I'd be interested to know what value of
> > `x` you used to achieve this, and what system you were on. This can't
> happen
> > under IEEE 754 arithmetic.
>
> I expect it might happen under one of the directed rounding modes
> (like "to +infinity").
>
> But under 754 binary round-nearest/even arithmetic, it's been formally
> proved that sqrt(x*x) == x exactly for all non-negative finite x such
> that x*x neither overflows nor underflows (and .as_integer() has
> nothing to do with that very strong result):
>
> https://hal.inria.fr/hal-01148409/document
>
> OTOH, the paper notes that it's not necessarily true for IEEE decimal
> arithmetic; e.g.,
>
> >>> import decimal
> >>> decimal.getcontext().prec = 4
> >>> (decimal.Decimal("31.66") ** 2).sqrt() # result is 1 ulp smaller
> Decimal('31.65')
>
> >>> decimal.getcontext().prec = 5
> >>> (decimal.Decimal("31.660") ** 2).sqrt() # result is 1 ulp larger
> Decimal('31.661')
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
[Chris Barker ] > ... > ... "is it the "right" thing to do in most cases, when deployed by folks > that haven't thought deeply about floating point. Gimme a break ;-) Even people who _believe_ they've thought about floating point still litter the bug tracker with >>> .1 + .2 0.30004 "bug reports". .is_integer() is easy to explain compared to that - and you have to go out of your way to use it. > ... > I think pretty much all the real world code that's been shown here for using > .is_integer() is really about type errors (issues). The function at hand > really wants integer inputs -- but wants to allow the user to be sloppy and > provide a float type that happens to be an int. Given Python's duck-typing > nature, maybe that's a good thing? I know I really discourage dynamic type > checking So you identified a use case. One you don't approve of (nor do I), but not strongly enough to demand they suffer instead ;-) > Also, every example has been for small-ish integers -- exponents, > factorials, etc -- not order 1e300 -- or inf or NaN, etc. > > Finally, the use-cases where the value that happens-to-be-an-int is computed > via floating point -- .is_integer() is probably the wrong check -- you > probably want isclose(). Everyone who has implemented a production math library can recall cases where the functionality was needed. Here, that includes at least Stefan Krah and me. You could also follow the link from Mark Dickinson to SciPy's implementation of the beta function. In every case I've needed the functionality, isclose() would have been utterly useless. Behold: >>> (-1.0) ** 3.0 -1.0 >>> (-1.0) ** 3.0001 # different result _type_ (-1-3.142007854859299e-12j) >>> math.isclose(3.0, 3.0001) True And another showing that the same functionality is needed regardless of how large the power: >>> (-1.0) ** 1e300 # an even integer power 1.0 When implementing an externally defined standard, when it says "and if such-and-such is an integer ...", it _means_ exactly an integer, not "or a few ULP away from an integer". IEEE pow()-like functions bristle with special cases for integers. >>> (-math.inf) ** 3.1 inf >>> (-math.inf) ** 3.0 # note: this one has a negative result (odd integer >>> power) -inf >>> (-math.inf) ** 2.9 inf > ... > All this points to: we don't need .is_integer I'll grant that you don't think you need it. So don't use it ;-) > All the being said -- the standard for depreciation is much higher bar than > not-adding-it-in-the-first-place. I would not have added it as a method to begin with - but I agree with Guido that it doesn't reach the bar for deprecation. The only examples of "bad" uses we saw were from people still so naive about floating-point behavior that they'll easily fall into other ways to get it wrong. What we haven't seen: a single person here saying "you know, I think _I'd_ be seduced into misusing it!". It's not _inherently_ confusing at all. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Symmetry arguments for API expansion
On Wed, Mar 21, 2018 at 6:48 PM, Chris Barker wrote: > On Wed, Mar 21, 2018 at 4:12 PM, Guido van Rossum > wrote: > >> Thank you! As you may or may not have noticed in a different thread, >> we're going through a small existential crisis regarding the usefulness of >> is_integer() -- Serhiy believes it is not useful (and even an attractive >> nuisance) and should be deprecated. OTOH the existence of >> dec_mpd_isinteger() seems to validate to me that it actually exposes useful >> functionality (and every Python feature can be abused, so that alone should >> not be a strong argument for deprecation). >> > > if not deprecated, then do we add it to all the other numeric types? Which > was the original suggestion, yes? > Yes. That's a pronouncement, so we can end this thread (and more importantly the other). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
OK, we'll keep float.is_integer(), and that's a pronouncement, so that we can hopefully end this thread soon. It should also be added to int. After all that's what started this thread, with the observation that mypy and PEP 484 consider an int valid whenever a float is expected. Since PEP 484 and mypy frown upon the numeric tower I don't care too much about whether it's added it there (or to Decimal) but given that we're keeping float.is_integer() and adding int.is_integer(), I don't see what's gained by not adding it to the numeric tower and Decimal either. Numpy can do what it likes, it doesn't play by the same rules as the stdlib anyway. On Wed, Mar 21, 2018 at 7:29 PM, Tim Peters wrote: > [Chris Barker ] > > ... > > ... "is it the "right" thing to do in most cases, when deployed by folks > > that haven't thought deeply about floating point. > > Gimme a break ;-) Even people who _believe_ they've thought about > floating point still litter the bug tracker with > > >>> .1 + .2 > 0.30004 > > "bug reports". .is_integer() is easy to explain compared to that - > and you have to go out of your way to use it. > > > ... > > I think pretty much all the real world code that's been shown here for > using > > .is_integer() is really about type errors (issues). The function at hand > > really wants integer inputs -- but wants to allow the user to be sloppy > and > > provide a float type that happens to be an int. Given Python's > duck-typing > > nature, maybe that's a good thing? I know I really discourage dynamic > type > > checking > > So you identified a use case. One you don't approve of (nor do I), > but not strongly enough to demand they suffer instead ;-) > > > > Also, every example has been for small-ish integers -- exponents, > > factorials, etc -- not order 1e300 -- or inf or NaN, etc. > > > > Finally, the use-cases where the value that happens-to-be-an-int is > computed > > via floating point -- .is_integer() is probably the wrong check -- you > > probably want isclose(). > > Everyone who has implemented a production math library can recall > cases where the functionality was needed. Here, that includes at > least Stefan Krah and me. You could also follow the link from Mark > Dickinson to SciPy's implementation of the beta function. > > In every case I've needed the functionality, isclose() would have been > utterly useless. Behold: > > >>> (-1.0) ** 3.0 > -1.0 > >>> (-1.0) ** 3.0001 # different result _type_ > (-1-3.142007854859299e-12j) > >>> math.isclose(3.0, 3.0001) > True > > And another showing that the same functionality is needed regardless > of how large the power: > > >>> (-1.0) ** 1e300 # an even integer power > 1.0 > > When implementing an externally defined standard, when it says "and if > such-and-such is an integer ...", it _means_ exactly an integer, not > "or a few ULP away from an integer". IEEE pow()-like functions > bristle with special cases for integers. > > >>> (-math.inf) ** 3.1 > inf > >>> (-math.inf) ** 3.0 # note: this one has a negative result (odd integer > power) > -inf > >>> (-math.inf) ** 2.9 > inf > > > > ... > > All this points to: we don't need .is_integer > > I'll grant that you don't think you need it. So don't use it ;-) > > > > All the being said -- the standard for depreciation is much higher bar > than > > not-adding-it-in-the-first-place. > > I would not have added it as a method to begin with - but I agree with > Guido that it doesn't reach the bar for deprecation. The only > examples of "bad" uses we saw were from people still so naive about > floating-point behavior that they'll easily fall into other ways to > get it wrong. What we haven't seen: a single person here saying "you > know, I think _I'd_ be seduced into misusing it!". It's not > _inherently_ confusing at all. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
Tim Peters wrote: from trig functions doing argument reduction as if pi were represented with infinite precision, That sounds like an interesting trick! Can you provide pointers to any literature describing how it's done? Not doubting it's possible, just curious. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating float.is_integer()
[Tim] >> from trig functions doing argument reduction as if pi were represented >> with infinite precision, [Greg Ewing ] > That sounds like an interesting trick! Can you provide > pointers to any literature describing how it's done? > > Not doubting it's possible, just curious. As I recall, when it was first done a "lazy" routine produced as many bits of pi as a given argument required, doing gonzo arbitrary precision arithmetic. Later, computer-aided analysis based on continued fraction expansions identified the worst possible case across all IEEE doubles (& singles). For example, it's possible in reasonable time to find the IEEE double that comes closest to being an exact integer multiple of pi/4 (or whatever other range you want to reduce to). Then it's only necessary to precompute pi to as many bits as needed to handle the worst case. In practice, falling back to that is necessary only for "large" arguments, and the usual double-precision numeric tricks suffice for smaller arguments. Search the web for "trig argument reduction" for whatever the state of the art may be today ;-) For actual code, FDLIBM does "as if infinite precision" trig argument reduction, using a precomputed number of pi bits sufficient to handle the worst possible IEEE double case, and is available for free from NETLIB: http://www.netlib.org/fdlibm/ The code is likely to be baffling, though, as there's scant explanation. Reading a paper or two first would be a huge help. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Do we have vlookup function
Do we have vlookup function which can be used in dataframe same as used in excel. Can you please provide the pointer for the same? Thanks!!! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
