Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Antoon Pardon]

Op 20-07-16 om 07:42 schreef Steven D'Aprano:
> Floating point maths is hard, thinking carefully about what you are doing and 
> whether it is appropriate to use == or a fuzzy almost-equal comparison, or if 
> equality is the right way at all.
>
> "But thinking is hard, can't you just tell me the answer?"
>
> No. But I can give some guidelines:
>
> Floating point arithmetic is deterministic, it doesn't just randomly mix in 
> error out of spite or malice. So in principle, you can always estimate the 
> rounding error from any calculation -- and sometimes there is none.

I would like to see a practical example of such an outcome.

> Arithmetic on integer-values (e.g. 1.0) is always exact, up to a limit of 
> either 2**53 or approximately 1e53, I forget which. (That's why most 
> Javascript 
> programmers fail to notice that they don't have an integer type.) So long as 
> you're using operations that only produce integer values from integer 
> arguments 
> (such as + - * // but not / ) then all calculations are exact. It is a waste 
> of 
> time to do:
>
> x = 2.0
> y = x*1002.0
> is_equal(y, 2004.0, 1e-16)
>
> when you can just do y == 2004.0.

But why perforem integer arithmetics in floats, isn't that a waste of time too?
I really see no reason to use floats if you know all your results will be 
integers.

-- 
Antoon.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What exactly is "exact" (was Clean Singleton Docstrings)

[Marko Rauhamaa]

Random832 :

> On Tue, Jul 19, 2016, at 18:17, Marko Rauhamaa wrote:
>> I'd love it if batteries were priced per joule, or even per
>> kilowatt-hour.
>
> Typically their capacity is labeled in amp-hours.

Did you really see that labeled on the (nonrechargeable AA) battery?

> You have to know your devices to know if the voltage difference
> between different battery types (Which ranges from 1.2 to 1.7 for
> nominal-1.5 AAA/AA/C/D cells, and may be lower under load) is
> significant or not, and in what direction.

Well, your amp hours will be shittier with a lower voltage. That's why
reporting joules would be more honest.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Steven D'Aprano]

On Wednesday 20 July 2016 16:45, Lawrence D’Oliveiro wrote:

> I was trying something like
>
> ctx.dashes = ((0.1, 0.03, 0.03, 0.03), 0)
>
> and wondering why it wasn’t working...

And so are we. Since you've already solved the problem, maybe you could 
enlighten us?

 This makes no sense to me at all.  You appear to be trying to create a
 tuple, which contains a tuple and an integer.  You then say it doesn't
 work, but imply that using __slots__ fixes the problem.  So please explain
 exactly what you were trying to achieve, the exact error you got, with the
 complete traceback, and how using __slots__ fixed the problem.
>>>
>>> No traceback. The lines were simply coming out solid, instead of dashed.
>> 
>> And __slots__ fixed the problem how, exactly?
> 
> The Context attribute that controls the dash settings is called “dash”, not
> “dashes”.

Ah, finally, an actually *useful* reply. Honestly Lawrence, couldn't you have 
just said this right at the start, instead of having us drag the answer out of 
you over multiple email exchanges?

If you had said right at the beginning "I have a habit of mistyping attribute 
names, and using __slots__ ensures I get an immediate error" then we would have 
understood you. Instead, you waste our time, *and yours*, leading us up the 
garden path by implying that you fixed a display bug by changing something 
like:

class Context(object):
def __init__(self):
self.dashes = something

to:

class Context(object):
__slots__ = ("dashes",)
def __init__(self):
self.dashes = something

Perhaps now you understand why you gave us the impression of cargo-cult 
debugging.




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Peter Otten]

Lawrence D’Oliveiro wrote:

> On Wednesday, July 20, 2016 at 6:19:45 PM UTC+12, Chris Angelico wrote:
>>
>> On Wed, Jul 20, 2016 at 9:58 AM, Lawrence D’Oliveiro wrote:
>>>
>>> On Wednesday, July 20, 2016 at 9:24:57 AM UTC+12, bream...@gmail.com
>>> wrote:

 On Tuesday, July 19, 2016 at 3:54:12 AM UTC+1, Lawrence D’Oliveiro
 wrote:
>
> On Tuesday, July 19, 2016 at 11:12:52 AM UTC+12, bream...@gmail.com
> wrote:
>>
>> On Monday, July 18, 2016 at 10:48:15 PM UTC+1, Lawrence D’Oliveiro
>> wrote:
>>>
>>> 
>>> When you have lots of read/write properties, I find __slots__ to be
>>> a good idea.
>>
>> Please explain why, thank you.
>
> I was trying something like
>
> ctx.dashes = ((0.1, 0.03, 0.03, 0.03), 0)
>
> and wondering why it wasn’t working...

 This makes no sense to me at all.  You appear to be trying to create a
 tuple, which contains a tuple and an integer.  You then say it doesn't
 work, but imply that using __slots__ fixes the problem.  So please
 explain exactly what you were trying to achieve, the exact error you
 got, with the complete traceback, and how using __slots__ fixed the
 problem.
>>>
>>> No traceback. The lines were simply coming out solid, instead of dashed.
>> 
>> And __slots__ fixed the problem how, exactly?
> 
> The Context attribute that controls the dash settings is called “dash”,
> not “dashes”.
> 
>> This sounds like the sort of cargo cult debugging that I'd expect of PHP
>> programmers ("I put addslashes around everything and now it works, so in
>> future I'll use addslashes everywhere"), but around here, we're better
>> than that.
> 
> OK, boss.

pylint can detect candidates for accidental attribute creation:

$ cat attrib.py
class Context:
def __init__(self):
self.dashes = None

ctx = Context()
ctx.dasehs = ("foo", "bar")
$ pylint attrib | grep dasehs
W:  6, 4: Attribute 'dasehs' defined outside __init__ (attribute-defined-
outside-init)


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Antoon Pardon :
> But why perforem integer arithmetics in floats,

Conceptual and practical simplificity.

> isn't that a waste of time too?

Probably not, especially compared with the overhead of boxing.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Lawrence D’Oliveiro]

On Wednesday, July 20, 2016 at 7:26:36 PM UTC+12, Peter Otten wrote:

> pylint can detect candidates for accidental attribute creation:

And __slots__ will prevent them outright.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Lawrence D’Oliveiro]

On Tuesday, July 19, 2016 at 9:48:15 AM UTC+12, I wrote:

> When you have lots of read/write properties, I find __slots__ to be a good
> idea.

Let me amend that. When you have *any* read/write properties, I find __slots__ 
to be a good idea.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Peter Otten]

Lawrence D’Oliveiro wrote:

> On Wednesday, July 20, 2016 at 7:26:36 PM UTC+12, Peter Otten wrote:
> 
>> pylint can detect candidates for accidental attribute creation:
> 
> And __slots__ will prevent them outright.

And attributes added intentionally.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Steven D'Aprano]

On Wed, 20 Jul 2016 06:50 pm, Lawrence D’Oliveiro wrote:

> On Wednesday, July 20, 2016 at 7:26:36 PM UTC+12, Peter Otten wrote:
> 
>> pylint can detect candidates for accidental attribute creation:
> 
> And __slots__ will prevent them outright.


As well as those added intentionally.

Sometimes I wonder what it is that you see in Python and why you use it.
You're frequently complaining about features of the language. You go out of
your way to write in the style of Java/C/whatever in Python, which means
you get all the disadvantages of writing verbose and inflexible static
Java/C/whatever PLUS the inefficiency of dynamic Python, so the worst of
both worlds.

I genuinely don't see what you get out of using Python. Would you care to
explain? There must be something you like about it, even if it isn't the
ability to add attributes to (nearly) arbitrary objects without declaring
them, or the duck-typing of booleans.




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Steven D'Aprano]

On Wed, 20 Jul 2016 05:09 pm, Antoon Pardon wrote:

> Op 20-07-16 om 07:42 schreef Steven D'Aprano:
>> Floating point maths is hard, thinking carefully about what you are doing
>> and whether it is appropriate to use == or a fuzzy almost-equal
>> comparison, or if equality is the right way at all.
>>
>> "But thinking is hard, can't you just tell me the answer?"
>>
>> No. But I can give some guidelines:
>>
>> Floating point arithmetic is deterministic, it doesn't just randomly mix
>> in error out of spite or malice. So in principle, you can always estimate
>> the rounding error from any calculation -- and sometimes there is none.
> 
> I would like to see a practical example of such an outcome.


[steve@ando ~]$ cd /home/steve/python/python-dev/3.4/Lib/test/
[steve@ando test]$ grep self.assertEqual test_statistics.py | wc -l
95


Not all of the 95 examples of using assertEqual apply to float values, but a
good proportion of them do. And if I were a better computer scientist,
there would probably be a lot more assertEquals in my code. A lot of the
time that I do a fuzzy comparison its because I'm too lazy or not good
enough to get a better result.

I am not a good computer scientist. But Bruce Dawson *is* a good computer
scientist:

https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/

Quote:

Conventional wisdom says that you should never compare two floats
for equality – you should always use an epsilon. Conventional
wisdom is wrong.

I’ve written in great detail about how to compare floating-point
values using an epsilon, but there are times when it is just not
appropriate. Sometimes there really is an answer that is correct,
and in those cases anything less than perfection is just sloppy.

So yes, I’m proudly comparing floats to see if they are equal.



>> Arithmetic on integer-values (e.g. 1.0) is always exact, up to a limit of
>> either 2**53 or approximately 1e53, I forget which. (That's why most
>> Javascript programmers fail to notice that they don't have an integer
>> type.) So long as you're using operations that only produce integer
>> values from integer arguments (such as + - * // but not / ) then all
>> calculations are exact. It is a waste of time to do:
>>
>> x = 2.0
>> y = x*1002.0
>> is_equal(y, 2004.0, 1e-16)
>>
>> when you can just do y == 2004.0.
> 
> But why perforem integer arithmetics in floats, isn't that a waste of time
> too? I really see no reason to use floats if you know all your results
> will be integers.

In Python, it's probably neither harmful nor useful. The cost of dealing
with boxed values (objects rather than low-level machine values) will
probably outweigh any performance gain one way or the other.

But in lower-level languages, you might find that floating point arithmetic
is faster than integer arithmetic, if you can pass the work on to a FPU
instead of a (slow?) CPU. Or not. It depends on the machine.

Or you might be using a language like Javascript, which intentionally has
only floats for numbers. That's okay, you can still perform exact integer
arithmetic, so long as you stay within the bounds of ±2**16.

Not even in Javascript do you need to write something like this:

x = 0.0
for i in range(20):
x += 1.0

assert abs(x - 20.0) <= 1e-16




-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What exactly is "exact" (was Clean Singleton Docstrings)

[alister]

On Wed, 20 Jul 2016 02:09:58 +0300, Marko Rauhamaa wrote:

> Ian Kelly :
> 
>> Ah, the machinations that users of imperial units have to endure.
> 
> Europeans often mistakenly believe that Americans haven't yet adopted
> the SI units. They have:
> 
>  - the length of a ski is measured in centimeters
> 
>  - the width of film and the diameter of a gun barrel are measured in
>millimeters
> 
>  - the wavelength of visible light is measured in nanometers
> 
>  - the volume of a soda bottle is measured in liters
> 
>  - a quantity of confiscated narcotics is measured in kilograms
> 
>  - daily allowance of a vitamin is measured in milligrams
> 
>  - the pitch of a note is measured in hertz
> 
>  - a short timespan is measured in seconds
> 
> 
> So the problem Europeans have with the situation is *not* the lack of
> adoption of new units. Rather, it is the refusal of giving up the
> traditional units.
> 
> The Anglo-American culture is notorious for its eagerness to absorb
> foreign influences. What it abhors is reductionism. There is always room
> for more words, ideas, mores, inventions, but nothing is ever deprecated
> with a simple edict.
> 
> 
> Marko

One of my biggest questions since the Brexit vote is can we g back to 
using imperial weights & measures (please).




-- 
Egotist:  A person of low taste, more interested in himself than in me.
-- Ambrose Bierce
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Steven D'Aprano :

> I am not a good computer scientist. But Bruce Dawson *is* a good
> computer scientist:
>
> https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-f
> loatsso-test-them-all/
>
> Quote:
>
> Conventional wisdom says that you should never compare two floats
> for equality – you should always use an epsilon. Conventional
> wisdom is wrong.
>
> I’ve written in great detail about how to compare floating-point
> values using an epsilon, but there are times when it is just not
> appropriate. Sometimes there really is an answer that is correct,
> and in those cases anything less than perfection is just sloppy.
>
> So yes, I’m proudly comparing floats to see if they are equal.

The point of view in the linked article is very different from that of
most application programming that makes use of floating-point numbers.

Yes, if what you are testing or developing is a numeric or mathematical
package, you should test its numeric/mathematical soundness to the bit.
However, in virtually any other context, you have barely any use for
a floating-point equality comparison because:

 1. Floating-point numbers are an approximation of *real numbers*. Two
independently measured real numbers are never equal because under
any continuous probability distribution, the probability of any
given real number is zero. Only continuous ranges can have nonzero
probabilities.

 2. Floating-point numbers are *imperfect approximations* of real
numbers. Even when real numbers are derived exactly, floating-point
operations may introduce "lossy compression artifacts" that have to
be compensated for in application programs.

What you have to do exactly to compensate for these challenges depends
on the application, and is very easy to get wrong. However, if an
application programmer is using == to compare two floating-point data
values, it is almost certainly a mistake.

> Or you might be using a language like Javascript, which intentionally
> has only floats for numbers. That's okay, you can still perform exact
> integer arithmetic, so long as you stay within the bounds of ±2**16.
>
> Not even in Javascript do you need to write something like this:
>
> x = 0.0
> for i in range(20):
> x += 1.0
>
> assert abs(x - 20.0) <= 1e-16

Correct because Javascript makes an exactness guarantee of its integers
(I imagine).

In Python, I think it would usually be bad style to rely even on:

   1.0 + 1.0 == 2.0

It is very difficult to find a justification for that assumption in
Python's specifications. What we have:

   Floating-point numbers are represented in computer hardware as base 2
   (binary) fractions.
   https://docs.python.org/3/tutorial/floatingpoint.html>

   almost all platforms map Python floats to IEEE-754 “double precision”
   https://docs.python.org/3/tutorial/floatingpoint.html#represent
   ation-error>

   numbers.Real (float)
 These represent machine-level double precision floating point
 numbers. You are at the mercy of the underlying machine
 architecture (and C or Java implementation) for the accepted range
 and handling of overflow.
   https://docs.python.org/3/reference/datamodel.html#the-standar
   d-type-hierarchy>


I believe a Python implementation that would have:

   1.0 + 1.0 != 2.0

would not be in violation of Python's data model. In fact, even:

   1.0 != 1.0

might be totally conformant. For example, we could have a new underlying
real-number technology that stored the value in an *analogous* format
(say, an ultra-precise voltage level) and performed the calculations
using some fast, analogous circuitry.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What exactly is "exact" (was Clean Singleton Docstrings)

[Random832]

On Wed, Jul 20, 2016, at 03:16, Marko Rauhamaa wrote:
> Random832 :
> > Typically their capacity is labeled in amp-hours.
> 
> Did you really see that labeled on the (nonrechargeable AA) battery?

Sorry, I must have imagined that. Anyway, my point was that the reality
is too complicated to easily assign a number to - that they don't even
try supports that. Duracell's datasheets don't even have a single
number, just a bunch of line graphs. 

> > You have to know your devices to know if the voltage difference
> > between different battery types (Which ranges from 1.2 to 1.7 for
> > nominal-1.5 AAA/AA/C/D cells, and may be lower under load) is
> > significant or not, and in what direction.
> 
> Well, your amp hours will be shittier with a lower voltage.

Define "shittier". An incandescent flashlight (which consumes less power
at lower voltage) will last longer, but won't be as bright. If it's
still acceptably bright, that's not worse.

> That's why
> reporting joules would be more honest.

And *my* point is that the number of joules depends on how the battery
is used. And different types of batteries are optimized for different
applications.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Angelico]

On Wed, Jul 20, 2016 at 11:54 PM, Marko Rauhamaa  wrote:
>  2. Floating-point numbers are *imperfect approximations* of real
> numbers. Even when real numbers are derived exactly, floating-point
> operations may introduce "lossy compression artifacts" that have to
> be compensated for in application programs.

This is the kind of black FUD that has to be fought off. What
"compression artifacts" are introduced? The *only* lossiness in IEEE
binary floating-point arithmetic is rounding. (This is the bit where
someone like Steven is going to point out that there's something else
as well.) Unless you are working with numbers that require more
precision than you have available, the result should be perfectly
accurate. And there are other systems far less 'simple'. Can you
imagine this second assertion failing?

assert x <= y # if not, swap the values
assert x <= (x+y)/2 <= y

Because it can with decimal.Decimal, due to the way rounding happens in decimal.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Marko Rauhamaa]

Chris Angelico :

> On Wed, Jul 20, 2016 at 11:54 PM, Marko Rauhamaa  wrote:
>>  2. Floating-point numbers are *imperfect approximations* of real
>> numbers. Even when real numbers are derived exactly,
>> floating-point operations may introduce "lossy compression
>> artifacts" that have to be compensated for in application
>> programs.
>
> This is the kind of black FUD that has to be fought off. What
> "compression artifacts" are introduced? The *only* lossiness in IEEE
> binary floating-point arithmetic is rounding.

You are joining me in spreading the FUD. Yes, the immediate lossiness is
rounding, but the effects of that rounding can result in atrocious
accumulative errors in numeric calculations.

> Unless you are working with numbers that require more precision than
> you have available, the result should be perfectly accurate.

Whoa, hold it there! Catastrophic cancellation (https://en.wikipedia.org/wiki/Loss_of_significance>) is not a myth:

   >>> 0.2 / (0.2 - 0.1)
   2.0
   >>> 0.2 / ((2e15 + 0.2) - (2e15 + 0.1))
   0.8

You can fall victim to the phenomenon when you collect statistics over a
long time. The cumulative sum of a measurement can grow very large,
which causes the naïve per-second rate calculation to become
increasingly bogus.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

reversed(enumerate(x))

[Ian Kelly]

I had occasion to write something like this:

for i, n in reversed(enumerate(x)): pass

Of course this fails with "TypeError: argument to reversed() must be a
sequence". I ended up using this instead:

for i, n in zip(reversed(range(len(x))), reversed(x)): pass

This works but is extraordinarily ugly and not terribly clear. I think
it's less confusing however than:

for i, n in zip(range(len(x)-1, -1, -1), reversed(n)): pass

How would you write this?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: reversed(enumerate(x))

[Chris Angelico]

On Thu, Jul 21, 2016 at 3:42 AM, Ian Kelly  wrote:
> I had occasion to write something like this:
>
> for i, n in reversed(enumerate(x)): pass
>
> Of course this fails with "TypeError: argument to reversed() must be a
> sequence". I ended up using this instead:
>
> for i, n in zip(reversed(range(len(x))), reversed(x)): pass

At the cost of coalescing the enumeration, you could:

for i, n in reversed(list(enumerate(x))): pass

It's reasonably clean but less efficient.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: reversed(enumerate(x))

[Brendan Abel]

You could create your own generator that wraps enumerate

def reverse_enumerate(iterable):
for i, val in enumerate(reversed(iterable)):
yield len(iterable) - 1 - i, val

for i, val in reverse_enumerate(x):
...

On Wed, Jul 20, 2016 at 10:42 AM, Ian Kelly  wrote:

> I had occasion to write something like this:
>
> for i, n in reversed(enumerate(x)): pass
>
> Of course this fails with "TypeError: argument to reversed() must be a
> sequence". I ended up using this instead:
>
> for i, n in zip(reversed(range(len(x))), reversed(x)): pass
>
> This works but is extraordinarily ugly and not terribly clear. I think
> it's less confusing however than:
>
> for i, n in zip(range(len(x)-1, -1, -1), reversed(n)): pass
>
> How would you write this?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: reversed(enumerate(x))

[Steven D'Aprano]

On Thu, 21 Jul 2016 03:46 am, Chris Angelico wrote:

> On Thu, Jul 21, 2016 at 3:42 AM, Ian Kelly  wrote:
>> I had occasion to write something like this:
>>
>> for i, n in reversed(enumerate(x)): pass
>>
>> Of course this fails with "TypeError: argument to reversed() must be a
>> sequence". I ended up using this instead:
>>
>> for i, n in zip(reversed(range(len(x))), reversed(x)): pass
> 
> At the cost of coalescing the enumeration, you could:
> 
> for i, n in reversed(list(enumerate(x))): pass
> 
> It's reasonably clean but less efficient.


Less efficient than what? 

reversed() only operates on sequences, so it can't operate on arbitrary
iterators of unknown length? Possibly of indefinite length?

def iter():
while random.random() > 0.01:
yield random.random()

So in the most general case, you have to form a list before you can reverse
it.

Personally, I think your version is the most straightforward and obvious
solution that works on anything. (Well, perhaps not on infinite iterators.)
Yes, you have to convert x into a list, but that's in general the only way
to use reversed() anyway. If your needs are not too great, or simplicity of
code is more important than 

The "best" solution might be some more work: you might convert x into a list
only if it's an iterator, then iterate over it in reverse:


def sequencefy(x):
if x is iter(x):
return list(x)
return x

def enumerate_down(it):
seq = sequencefy(it)
n = len(seq) - 1
for item in reversed(seq):
yield (n, item)
n -= 1

for i, item = enumerate_down(x):
...



An advantage of this is that it works well with lazy sequences like
(x)range. There's no need to build up a huge list of (index, item) pairs
before reversing it.


-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: reversed(enumerate(x))

[Random832]

On Wed, Jul 20, 2016, at 13:42, Ian Kelly wrote:
> I had occasion to write something like this:
> 
> for i, n in reversed(enumerate(x)): pass
> 
> How would you write this?

I'd write my own version of enumerate with a step argument, and call
enumerate(reversed(x), start=len(x), step=-1)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: reversed(enumerate(x))

[Chris Angelico]

On Thu, Jul 21, 2016 at 5:13 AM, Steven D'Aprano  wrote:
> On Thu, 21 Jul 2016 03:46 am, Chris Angelico wrote:
>
>> On Thu, Jul 21, 2016 at 3:42 AM, Ian Kelly  wrote:
>>> I had occasion to write something like this:
>>>
>>> for i, n in reversed(enumerate(x)): pass
>>>
>>> Of course this fails with "TypeError: argument to reversed() must be a
>>> sequence". I ended up using this instead:
>>>
>>> for i, n in zip(reversed(range(len(x))), reversed(x)): pass
>>
>> At the cost of coalescing the enumeration, you could:
>>
>> for i, n in reversed(list(enumerate(x))): pass
>>
>> It's reasonably clean but less efficient.
>
>
> Less efficient than what?

Than something that let you "enumerate the reverse" of something. Consider:

# Here's what we want to happen.
x = ["foo", "bar", "spam"]
for i, n in reversed(enumerate(x)): # ENABLE MINDREADING MAGIC
print(i, n)

2 spam
1 bar
0 foo

So we could do this by first reversing the sequence, and then
*enumerating down*. But enumerate() doesn't allow that - it has a
'start' parameter, but no 'step'. Let's pretend it did.

for i, n in enumerate(reversed(x), len(x)-1, -1):
# same result

The very cleanest, IMO, would be something like this:

def enumerate_reversed(seq):
n = len(seq)
for obj in reversed(seq):
n -= 1
yield n, obj

> reversed() only operates on sequences, so it can't operate on arbitrary
> iterators of unknown length? Possibly of indefinite length?

Of course not; but it doesn't necessarily require that the sequence
exist as a list, much less as two. My simple and naive example will
take any sequence, pair values with their indices, coalesce the result
as a list, and then iterate backward through that list. Imagine if the
sequence in question were range(1<<256), for instance. You can't
list(enumerate(x)) that, but you can certainly reversed() it and get
its length... well, okay, apparently len(range(1<<256)) is illegal in
CPython, but len(range(1<<63-1)) is fine. You shouldn't need to list()
that.

So yes, my naive version is less memory efficient for large lists.

> Personally, I think your version is the most straightforward and obvious
> solution that works on anything. (Well, perhaps not on infinite iterators.)
> Yes, you have to convert x into a list, but that's in general the only way
> to use reversed() anyway. If your needs are not too great, or simplicity of
> code is more important than

I agree, which is why I suggested it.

> The "best" solution might be some more work: you might convert x into a list
> only if it's an iterator, then iterate over it in reverse:
>
>
> def sequencefy(x):
> if x is iter(x):
> return list(x)
> return x
>
> def enumerate_down(it):
> seq = sequencefy(it)
> n = len(seq) - 1
> for item in reversed(seq):
> yield (n, item)
> n -= 1
>
> for i, item = enumerate_down(x):
> ...
>
>
>
> An advantage of this is that it works well with lazy sequences like
> (x)range. There's no need to build up a huge list of (index, item) pairs
> before reversing it.

Yeah, or the simpler version that I used. All depends how much
complexity you really need.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

apache not ale to execute a vbscript with python

[vineeth menneni]

Hi, 

I am trying to call a external vbscript through subprocess.call() it works fine 
with my local, but when i try to test apache says that no such file or 
directory. The problem here is that the vbscript should generate a .xlsx file 
and it is not being created. So apache gives a warning no such file  or 
directory. Can you suggest any solutions. 

Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Lawrence D’Oliveiro]

On Thursday, July 21, 2016 at 12:11:09 AM UTC+12, Steven D'Aprano wrote:

> [long irrelevant rant deleted]

Just because I pointed out what a load of nonsense you were spouting about 
__slots__, by giving a counterexample of their usefulness? Man, your pride must 
be hurt...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Lawrence D’Oliveiro]

On Wednesday, July 20, 2016 at 9:16:30 PM UTC+12, Peter Otten wrote:
>
> Lawrence D’Oliveiro wrote:
> 
>> On Wednesday, July 20, 2016 at 7:26:36 PM UTC+12, Peter Otten wrote:
>> 
>>> pylint can detect candidates for accidental attribute creation:
>> 
>> And __slots__ will prevent them outright.
> 
> And attributes added intentionally.

You mean, being able to dynamically add new attributes to an object?

Probably not a good idea to mix that with read/write properties...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What exactly is "exact" (was Clean Singleton Docstrings)

[Gregory Ewing]

Random832 wrote:

Well, your amp hours will be shittier with a lower voltage.


Define "shittier". An incandescent flashlight (which consumes less power
at lower voltage) will last longer, but won't be as bright. If it's
still acceptably bright, that's not worse.


I think the point is that the cell will deliver (at most)
the same number of amp-hours at high load as low load,
but if the voltage is lower, those amp-hours will
represent less energy delivered to the load. An amp-hour
that doesn't give you as much energy as you were expecting
could be described as "shitty". :-)

On top of that, the cell might deliver less amp-hours at
high load, for bonus shittiness.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: reversed(enumerate(x))

[Michael Selik]

> On Jul 20, 2016, at 12:42 PM, Ian Kelly  wrote:
> 
>for i, n in reversed(enumerate(x)): pass
> 
> fails with "TypeError: argument to reversed() must be a sequence".

So make it a sequence:
for i, n in reversed(list(enumerate(x))): pass

If ``x`` is very large, you can use your zip/range technique to save memory.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[breamoreboy]

On Wednesday, July 20, 2016 at 10:48:23 PM UTC+1, Lawrence D’Oliveiro wrote:
> On Thursday, July 21, 2016 at 12:11:09 AM UTC+12, Steven D'Aprano wrote:
> 
> > [long irrelevant rant deleted]
> 
> Just because I pointed out what a load of nonsense you were spouting about 
> __slots__, by giving a counterexample of their usefulness? Man, your pride 
> must be hurt...

I hereby request that the moderators take this idiot offline as he's managed to 
insult two highly respected members of this community in no time at all, those 
of course being Steven D'Aprano and Peter Otten.

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Chris Angelico]

On Thu, Jul 21, 2016 at 8:11 AM, Lawrence D’Oliveiro
 wrote:
> On Wednesday, July 20, 2016 at 9:16:30 PM UTC+12, Peter Otten wrote:
>>
>> Lawrence D’Oliveiro wrote:
>>
>>> On Wednesday, July 20, 2016 at 7:26:36 PM UTC+12, Peter Otten wrote:
>>>
 pylint can detect candidates for accidental attribute creation:
>>>
>>> And __slots__ will prevent them outright.
>>
>> And attributes added intentionally.
>
> You mean, being able to dynamically add new attributes to an object?
>
> Probably not a good idea to mix that with read/write properties...

This is exactly what Steven was talking about. If you can't handle
dynamic read-write attributes (whether they're properties or simple
attributes is orthogonal to this), why even use Python? You can get a
linter to tell you about misspelled attribute names, which has a huge
advantage over __slots__ in that it can give you the information prior
to actually hitting that line of code (prior to even running the
program - and some editors have integrated linters, so you don't even
need to consciously run the linter, you just see the results as you
type). That gives you virtually the same benefit as C/Java style of
coding, where you're forced to declare all your members in the class;
but as an added bonus, you can *dynamically add attributes*! Don't
think that's useful? Well, the other day I wanted to add a wrapper
around the constructor for someone else's object to stash a bit of
extra information into it (to track down a resource leak - turned out
there was an ever-growing collection of objects, because they weren't
being removed from that object), and if I'd been using Java, the
response would have been "no way does that even make sense". Thanks to
dynamic code, I could do what I needed to and track down the leak. If
the author of that object had used __slots__, though, I'd have been
utterly stuck.

Using __slots__ basically takes your object down to the level of a
Java one. That's generally fine for something immutable (you really
don't need to be adding attributes to the integer 42), but even with
immutables, I've sometimes wanted to add hidden attributes; with
mutable objects, it's common enough that you want to leave the option
open unless you have good reason not to. And "my editor sucks" is not
a good reason; nor is "my debugging skills suck".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can't add variables to instances of built-in classes

[Steven D'Aprano]

On Thu, 21 Jul 2016 08:11 am, Lawrence D’Oliveiro wrote:

> On Wednesday, July 20, 2016 at 9:16:30 PM UTC+12, Peter Otten wrote:
>>
>> Lawrence D’Oliveiro wrote:
>> 
>>> On Wednesday, July 20, 2016 at 7:26:36 PM UTC+12, Peter Otten wrote:
>>> 
 pylint can detect candidates for accidental attribute creation:
>>> 
>>> And __slots__ will prevent them outright.
>> 
>> And attributes added intentionally.
> 
> You mean, being able to dynamically add new attributes to an object?
> 
> Probably not a good idea to mix that with read/write properties...

Why not?


If you're going to make a comment, would you please include some actual
significant information in your post? Dropping obtuse hints and incomplete
answers are not useful, and they give the strong impression that you are
only replying in order to gain a smug sense of superiority rather than to
disseminate knowledge and understanding.

Remember that other people cannot know the thought in your mind, they can
only judge by the actual words you have written, and if those words
standing alone do not explain your thinking, then people will:

(1) fail to understand the information you are trying to pass on;

(2) imagine that you are intentionally trying to be difficult; and

(3) characterise you as an insufferable, pretentious wanker who thinks that
giving mysterious, unintelligible responses makes him out to be ever-so
smarter than everyone else.


Poor quality answer:

"Probably not a good idea to mix that with read/write properties..."

Better:

"Probably not a good idea to mix that with read/write properties, for the
following reasons: [succinct explanation of the reasons why one should
avoid mixing ordinary attributes and properties]. Here's a longer
discussion [link to URL]."





-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What exactly is "exact" (was Clean Singleton Docstrings)

[Steven D'Aprano]

On Wed, 20 Jul 2016 11:24 pm, alister wrote:

> One of my biggest questions since the Brexit vote is can we g back to
> using imperial weights & measures (please).

I suppose you might as well -- there's no more empire, no more jobs or
houses, and once the financial traders leave London there'll be no more
money. When Scotland and Northern Ireland leaves there'll be no more United
Kingdom either, and then you and the Welsh can sit around all day trying to
remember how many cubic chains in a hogshead and the number of pints to a
firkin, and blaming the perfidious French and cabbage-eating Huns for
kicking you out of the EU against your will.

England Prevails!



-- 
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Wednesday, July 20, 2016 at 11:13:05 AM UTC+5:30, Steven D'Aprano wrote:
> On Tuesday 19 July 2016 14:58, Rustom Mody wrote:
> 
> > So I again ask: You say «"Never compare floats for equality" is a pernicious
> > myth»
> 
> It is the word *never* which makes it superstition. If people said "Take care 
> with using == for floats, its often not what you want" I would have no 
> argument 
> with the statement.
> 
> I'd even (reluctantly) accept "usually not what you want". But "never" is out-
> and-out cargo-cult programming.

You seem to not understand the realities of teaching.
You (teacher in general) cannot say a saga; only epigrams
You cannot transmit wisdom (even if you own some) just a bit of 
savviness/cleverness.

So let me ask the question again differently:
How many errors happen by people not using ε-neighborhood checks instead of == 
checks
How many errors happen by the opposite (mis)use?

IOW “myth”... ok “pernicious myth” Not

BTW APL whose main domain of application is scientific chooses to enshrine
this —equality is ε-neighborhood checking not exact equality checking — into 
its builtin ‘==’

And 
http://www.dyalog.com/uploads/documents/Papers/tolerant_comparison/tolerant_comparison.htm

ε is spelt ⎕ct (Comparison Tolerance)
And of course == is spelt =
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Chris Angelico]

On Thu, Jul 21, 2016 at 3:28 PM, Rustom Mody  wrote:
> ε is spelt ⎕ct (Comparison Tolerance)
> And of course == is spelt =

spelt is spelled spelled. Unless, of course, you mean the wheat variety.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Wednesday, July 20, 2016 at 8:29:25 PM UTC+5:30, Marko Rauhamaa wrote:
> Chris Angelico :
> 
> > On Wed, Jul 20, 2016 at 11:54 PM, Marko Rauhamaa  wrote:
> >>  2. Floating-point numbers are *imperfect approximations* of real
> >> numbers. Even when real numbers are derived exactly,
> >> floating-point operations may introduce "lossy compression
> >> artifacts" that have to be compensated for in application
> >> programs.
> >
> > This is the kind of black FUD that has to be fought off. What
> > "compression artifacts" are introduced? The *only* lossiness in IEEE
> > binary floating-point arithmetic is rounding.
> 
> You are joining me in spreading the FUD. Yes, the immediate lossiness is
> rounding, but the effects of that rounding can result in atrocious
> accumulative errors in numeric calculations.
> 
> > Unless you are working with numbers that require more precision than
> > you have available, the result should be perfectly accurate.
> 
> Whoa, hold it there! Catastrophic cancellation ( https://en.wikipedia.org/wiki/Loss_of_significance>) is not a myth:

Whose lead para starts:


| Catastrophic cancellation… The effect is that the number of accurate 
| (significant) digits in the result is reduced unacceptably. Ways to avoid 
this 
| effect are studied in numerical analysis.

I would go a step further:
The field of numerical analysis came into existence only because this fact
multiplied by the fact that computers do their (inaccurate ≠ inexact) 
computations
billions of times faster than we do
makes significance a very significant problem!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Rustom Mody]

On Thursday, July 21, 2016 at 11:05:28 AM UTC+5:30, Chris Angelico wrote:
> On Thu, Jul 21, 2016 at 3:28 PM, Rustom Mody  wrote:
> > ε is spelt ⎕ct (Comparison Tolerance)
> > And of course == is spelt =
> 
> spelt is spelled spelled. Unless, of course, you mean the wheat variety.

Love it!
Though not everyone agrees (including Australians!)
http://english.stackexchange.com/questions/5712/spelt-vs-spelled

Anyway... Ive been collecting quines/self-referential statements for classes
on Theory of computation.

Like these from Douglas Hofstadter.
To which I’ll add yours

You cant have “your cake” and spell it “too”

You cant have your use and mention it too

If this sentence were in Chinese it would say something else

.siht ekil ti gnidaer eb d'uoy ,werbeH ni erew ecnetnes siht fI
-- 
https://mail.python.org/mailman/listinfo/python-list

Python packages listed in PyPI

[Rayne via Python-list]

Hi,
May I know if the Python packages listed on the PyPI page 
(https://pypi.python.org/pypi) are OS-independent? That is, do I download and 
install the same package on both Windows and Linux systems?
Thank you.
Regards,Rayne
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python packages listed in PyPI

[Pete Forman]

Rayne  writes:

> May I know if the Python packages listed on the PyPI page
> (https://pypi.python.org/pypi) are OS-independent? That is, do I
> download and install the same package on both Windows and Linux

The simple answer is yes. pip install will find the appropriate
implementation for your OS and version of Python.

If the package is pure Python then it will be the same across OS. If
there is some binary content then PyPI will deliver the correct wheel
(precompiled) or C/C++ source. The latter generally builds OOTB on
Linux but needs extra work on Windows as that does not bundle a
compiler.

-- 
Pete Forman
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python packages listed in PyPI

[Rayne via Python-list]

Thanks! One more question: Does "pip install" require Internet to work? Or are 
all implementations already contained in the packages and so do not require 
additional downloads?

  From: Pete Forman 
 To: python-list@python.org 
 Sent: Thursday, July 21, 2016 2:19 PM
 Subject: Re: Python packages listed in PyPI
   
Rayne  writes:

> May I know if the Python packages listed on the PyPI page
> (https://pypi.python.org/pypi) are OS-independent? That is, do I
> download and install the same package on both Windows and Linux

The simple answer is yes. pip install will find the appropriate
implementation for your OS and version of Python.

If the package is pure Python then it will be the same across OS. If
there is some binary content then PyPI will deliver the correct wheel
(precompiled) or C/C++ source. The latter generally builds OOTB on
Linux but needs extra work on Windows as that does not bundle a
compiler.

-- 
Pete Forman
-- 
https://mail.python.org/mailman/listinfo/python-list


   
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Floating point equality [was Re: What exactly is "exact" (was Clean Singleton Docstrings)]

[Steven D'Aprano]

On Thursday 21 July 2016 15:28, Rustom Mody wrote:

> On Wednesday, July 20, 2016 at 11:13:05 AM UTC+5:30, Steven D'Aprano wrote:
>> On Tuesday 19 July 2016 14:58, Rustom Mody wrote:
>> 
>> > So I again ask: You say «"Never compare floats for equality" is a
>> > pernicious myth»
>> 
>> It is the word *never* which makes it superstition. If people said "Take
>> care with using == for floats, its often not what you want" I would have no
>> argument with the statement.
>> 
>> I'd even (reluctantly) accept "usually not what you want". But "never" is
>> out- and-out cargo-cult programming.
> 
> You seem to not understand the realities of teaching.
> You (teacher in general) cannot say a saga; only epigrams

Is "Don't use exact equality unless you know what you're doing" enough of an 
epigram for you?


> You cannot transmit wisdom (even if you own some) just a bit of
> savviness/cleverness.

Maybe so, but that's no excuse for transmitting outright misinformation and 
superstition. In physics, dealing with motion in the presence of energy losses 
is hard, and beginning and intermediate levels of physics will generally 
explicitly or implicitly ignore friction and air resistance. But physics 
teachers do not teach that "air resistance doesn't exist; you mustn't try to 
take friction into account". They teach that it is a simplification.




> So let me ask the question again differently:
> How many errors happen by people not using ε-neighborhood checks instead of
> == checks How many errors happen by the opposite (mis)use?

Precisely 35, and 17812, respectively.


> IOW “myth”... ok “pernicious myth” Not
> 
> BTW APL whose main domain of application is scientific chooses to enshrine
> this —equality is ε-neighborhood checking not exact equality checking — into
> its builtin ‘==’

I have a lot of respect for Ken Iverson, and a lot of admiration for language 
designers willing to experiment with alternate paradigms.

But keeping in mind that in APL, if you set ⎕ct to 0 you get an exact 
comparison, can you find any quotes from Iverson saying that you should *never* 
perform exact equality comparisons?



> And
> 
http://www.dyalog.com/uploads/documents/Papers/tolerant_comparison/tolerant_comparison.htm

Nice resource, thank you.


> ε is spelt ⎕ct (Comparison Tolerance)
> And of course == is spelt =




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python packages listed in PyPI

[Ben Finney]

Rayne via Python-list  writes:

> May I know if the Python packages listed on the PyPI page
> (https://pypi.python.org/pypi) are OS-independent?

There is a Trove classifier, “Operating System”, which allows the
package maintainer to declare one or more specific operating systems.

For example, a package might have the trove classifiers
“Operating System :: MacOS :: MacOS X” and
“Operating System :: POSIX”. That is a positive assertion from the
package maintainer that those operating systems are supported.

> That is, do I download and install the same package on both Windows
> and Linux systems?

You should use ‘pip’ to decide which specific file to download for your
platform.

-- 
 \ “How wonderful that we have met with a paradox. Now we have |
  `\some hope of making progress.” —Niels Bohr |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python packages listed in PyPI

[Steven D'Aprano]

On Thursday 21 July 2016 16:28, Rayne wrote:

> Thanks! One more question: Does "pip install" require Internet to work?

Yes, you must have internet access.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python packages listed in PyPI

[Ben Finney]

(Please don't top-post; instead, interleave your responses and trim
material you're not responding to.)

Rayne via Python-list  writes:

> Thanks! One more question: Does "pip install" require Internet to
> work?

Yes, ‘pip’ resolves the dependencies, download the files, and installs
them.

> Or are all implementations already contained in the packages and so do
> not require additional downloads?

I don't understand what difference you intend between “implementations”
and “packages”.

-- 
 \“Pinky, are you pondering what I'm pondering?” “Wuh, I think |
  `\   so, Brain, but burlap chafes me so.” —_Pinky and The Brain_ |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list