Struggeling with collections

[Faling Dutchman]

Hey folks,

I am just starting off in python, but have good knowledge of both Java and C#. 
Now is the problem that I need to have multiple instances of one dictionary, 
that is not a problem if you know how many, but now, it is an unknown amount.

Some background info:

I am making a library for an API. This library must be easy to use for the 
people who are going to use it. So I am making the models for the data, the 
connections and so on, so they just have to fill in the gaps. In C# and Java I 
did it with objects, but they do not work alike in python, or at least that is 
what I have found.

If I do this:

class Item:
def __init__(self, id, productId, quantity, pageCount, files, option, 
metadata):
self.id = id
self.productId = productId
self.quantity = quantity
self.pageCount = pageCount
self.files = files
self.option = option
self.metadata = metadata

itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
print(itm)

it prints: <__main__.Item object at 0x02EBF3B0>

So that is not usefull to me. There can be an infinite amount of objects of 
Item, and it needs to be easy accessable, just like 
for i in items
print(i)

and it has to show all the parameters of the class Item and not say "ive got an 
object  at this memory address, have a nice day"

I hope my question is clear.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __del__: when to use it? What happens when you SystemExit/NameError wrt del? Method vs function calls.

[Steven D'Aprano]

On Monday 07 March 2016 17:13, Veek. M wrote:

> import foo
> class Bar(object):
> def __del__(self, foo=foo):
> foo.bar()# Use something in module foo
> 
> ### Why the foo=foo? import foo, would increment the ref-count for
> object 'foo' 

Yes. And then at shutdown, the module globals get deleted so it gets 
decremented again and then garbage collected.

You cannot tell what order objects will be deleted, so you cannot tell 
whether your instance will be deleted before or after foo. If it happens 
before foo, then __del__ will run and you will be fine. If it happens after 
foo, then foo is gone and your __del__ method will fail.


> so why go to all the effort of passing it in to every
> instance via the local stack (ref-to-object-foo)?

So that every instance has a guaranteed and reliable reference to foo that 
cannot be garbage collected before that instance.

Here is a sketch of what happens without the foo=foo:

import foo  # refcount = 1
x = Bar()  # make an instance
y = Bar()  # and another instance
...
z = Bar()  # another instance
...
...
del z  # z gets garbage collected, everything is fine
...
sys.exit()  # Shutdown starts, x and y still exist
# Python starts garbage collecting objects
# in some unknown order...
...
...
del x  # x gets garbage collected, foo still exists so everything is fine
del foo  # refcount = 0, so foo gets garbage collected
del y  # y gets garbage collected, but foo is gone!




Here is a sketch of what happens with the foo=foo:

import foo  # refcount = 1
x = Bar()  # make an instance, foo refcount = 2
y = Bar()  # and another instance, foo refcount = 3
...
z = Bar()  # another instance, foo refcount = 4
...
...
del z  # z gets garbage collected, foo refcount = 3
...
sys.exit()  # Shutdown starts, x and y still exist
# Python starts garbage collecting objects
# in some unknown order...
...
...
del x  # x gets garbage collected, foo refcount = 2
del y  # y gets garbage collected, foo refcount = 1
del foo  # refcount = 0, so foo gets garbage collected



This guarantees that so long as there are any instances yet to be garbage 
collected, foo cannot be be garbage collected. Only after the last of those 
instances are gone will foo be garbage collected.



> Text for 1,2 from Beazley:
> It’s important to note that in some cases the __del__() method might not
> be invoked at program termination.This can occur if circular references
> exist between objects (in which case objects may be allocated but
> accessible from no known name-space).Although Python’s garbage collector
> can reclaim unused circular references dur-ing execution, it isn’t
> normally invoked on program termination.
> 
> ### which begs the question, why not on program termination? 

I'm afraid I don't know. Perhaps read the source code? 


> How bad can
> it be since you are terminating anyway. __del__ seems like a total waste
> product method 

I won't say "total" waste product, but automatic destructor methods are only 
good for very simple use-cases where you don't care that they are called in 
a non-deterministic fashion.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Vincent Vande Vyvre]

Le 07/03/2016 09:24, Faling Dutchman a écrit :

Hey folks,

I am just starting off in python, but have good knowledge of both Java and C#. 
Now is the problem that I need to have multiple instances of one dictionary, 
that is not a problem if you know how many, but now, it is an unknown amount.

Some background info:

I am making a library for an API. This library must be easy to use for the 
people who are going to use it. So I am making the models for the data, the 
connections and so on, so they just have to fill in the gaps. In C# and Java I 
did it with objects, but they do not work alike in python, or at least that is 
what I have found.

If I do this:

class Item:
 def __init__(self, id, productId, quantity, pageCount, files, option, 
metadata):
 self.id = id
 self.productId = productId
 self.quantity = quantity
 self.pageCount = pageCount
 self.files = files
 self.option = option
 self.metadata = metadata

itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
print(itm)

it prints: <__main__.Item object at 0x02EBF3B0>

So that is not usefull to me. There can be an infinite amount of objects of 
Item, and it needs to be easy accessable, just like
for i in items
 print(i)

and it has to show all the parameters of the class Item and not say "ive got an 
object  at this memory address, have a nice day"

I hope my question is clear.


The classes have a dict

Python 3.2.3 (default, Jun 18 2015, 21:46:42)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class Item:
... def __init__(self, id, productId, quantity, pageCount, files, 
option, metadata):

... self.id = id
... self.productId = productId
... self.quantity = quantity
... self.pageCount = pageCount
... self.files = files
... self.option = option
... self.metadata = metadata
...
>>> i = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
>>> i.__dict__
{'files': 'asdf', 'option': {'ads': 55, 'asdf': 3}, 'pageCount': 1, 
'metadata': None, 'productId': None, 'id': 1, 'quantity': 1}

>>>

--
https://mail.python.org/mailman/listinfo/python-list

How to get which attribute causes the AttributeError except inspecting strings?

[ZhangXiang]

In python3, when I write code like this:

try:
fields = [getattr(Product, field) for field in fields.split(',')]
except AttributeError as e:
raise HTTPError(...)

I want to raise a new type of error giving a string telling the user which 
attribute is not valid. But I don't see any method I can use to get the 
attribute name except inspecting e.args[0].

Could anyone give me a hint? Maybe I miss something. 

By the way, I don't quite want to change my code to a for-loop so I can access 
the field variable in exception handling.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Steven D'Aprano]

On Monday 07 March 2016 19:24, Faling Dutchman wrote:

> Hey folks,
> 
> I am just starting off in python, but have good knowledge of both Java and
> C#. Now is the problem that I need to have multiple instances of one
> dictionary, that is not a problem if you know how many, but now, it is an
> unknown amount.

Whenever you have an unpredictable number of some object, *any* object, you 
should turn to a list, or a dict.

"I need some integers, but I don't know if there will be 3 or 5."

Wrong solution:

a = 23
b = 24
c = 25
if foo:
d = 26
e = 27
# later...
if foo:
print(a, b, c, d, e)
else:
print(a, b, c)



Right solution:


integers = [23, 24, 25]
if foo:
integers.extend([26, 27])
# later...
print(*foo)


It doesn't matter whether you are dealing with ints, floats, strings, dicts, 
lists, sets, or your own custom-built objects. If you don't know in advance 
how many you want, put them in a list. Or a dict.



> Some background info:
> 
> I am making a library for an API. This library must be easy to use for the
> people who are going to use it. 

Define "easy to use".



> If I do this:
> 
> class Item:
> def __init__(self, id, productId, quantity, pageCount, files, option,
> metadata):
> self.id = id
> self.productId = productId
> self.quantity = quantity
> self.pageCount = pageCount
> self.files = files
> self.option = option
> self.metadata = metadata
> 
> itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
> print(itm)
> 
> it prints: <__main__.Item object at 0x02EBF3B0>
> 
> So that is not usefull to me.

Then create a better __repr__ or __str__ method. All you are seeing is the 
default representation inherited from object. If you don't like it, override 
it.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Faling Dutchman]

Op maandag 7 maart 2016 09:49:51 UTC+1 schreef Vincent Vande Vyvre:
> Le 07/03/2016 09:24, Faling Dutchman a écrit :
> > Hey folks,
> > I am just starting off in python, but have good knowledge of both Java and 
> > C#. Now is the problem that I need to have multiple instances of one 
> > dictionary, that is not a problem if you know how many, but now, it is an 
> > unknown amount.
> > Some background info:
> > I am making a library for an API. This library must be easy to use for the 
> > people who are going to use it. So I am making the models for the data, the 
> > connections and so on, so they just have to fill in the gaps. In C# and 
> > Java I did it with objects, but they do not work alike in python, or at 
> > least that is what I have found.
> > If I do this:
> > class Item:
> >  def __init__(self, id, productId, quantity, pageCount, files, option, 
> > metadata):
> >  self.id = id
> >  self.productId = productId
> >  self.quantity = quantity
> >  self.pageCount = pageCount
> >  self.files = files
> >  self.option = option
> >  self.metadata = metadata
> > itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
> > print(itm)
> > it prints: <__main__.Item object at 0x02EBF3B0>
> > So that is not usefull to me. There can be an infinite amount of objects of 
> > Item, and it needs to be easy accessable, just like
> > for i in items
> >  print(i)
> > and it has to show all the parameters of the class Item and not say "ive 
> > got an object  at this memory address, have a nice day"
> > I hope my question is clear.
> The classes have a dict
> Python 3.2.3 (default, Jun 18 2015, 21:46:42)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> class Item:
> ... def __init__(self, id, productId, quantity, pageCount, files, 
> option, metadata):
> ... self.id = id
> ... self.productId = productId
> ... self.quantity = quantity
> ... self.pageCount = pageCount
> ... self.files = files
> ... self.option = option
> ... self.metadata = metadata
> ...
>  >>> i = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
>  >>> i.__dict__
> {'files': 'asdf', 'option': {'ads': 55, 'asdf': 3}, 'pageCount': 1, 
> 'metadata': None, 'productId': None, 'id': 1, 'quantity': 1}
>  >>>

Thanks, that is at least part of the problem solved. The rest I can figure out 
myself though. This was the biggest hurdle.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Gary Herron]

On 03/07/2016 12:24 AM, Faling Dutchman wrote:

Hey folks,

I am just starting off in python, but have good knowledge of both Java and C#. 
Now is the problem that I need to have multiple instances of one dictionary, 
that is not a problem if you know how many, but now, it is an unknown amount.

Some background info:

I am making a library for an API. This library must be easy to use for the 
people who are going to use it. So I am making the models for the data, the 
connections and so on, so they just have to fill in the gaps. In C# and Java I 
did it with objects, but they do not work alike in python, or at least that is 
what I have found.

If I do this:

class Item:
 def __init__(self, id, productId, quantity, pageCount, files, option, 
metadata):
 self.id = id
 self.productId = productId
 self.quantity = quantity
 self.pageCount = pageCount
 self.files = files
 self.option = option
 self.metadata = metadata

itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
print(itm)

it prints: <__main__.Item object at 0x02EBF3B0>

So that is not usefull to me. There can be an infinite amount of objects of 
Item, and it needs to be easy accessable, just like
for i in items
 print(i)

and it has to show all the parameters of the class Item and not say "ive got an 
object  at this memory address, have a nice day"

I hope my question is clear.


It's not clear in the slightest.  In fact there isn't even a question 
here.  Ignoring everything about numbers of objects and libraries and 
access (of what?), it seems you want an object of type Item to be able 
to print itself nicely.Please correct me if I've got that wrong.


You can do so by defining a member __str__ (or __repr__).  Here's a 
small example.  The returned formatted string can be as simple or 
complex as you wish.


>>> class C:
...   def __init__(self, a, b):
... self.a = a
... self.b = b
...   def __str__(self):
... return "C(a={}, b={})".format(self.a, self.b)  # Modify to suit 
your needs.

...
>>> print(C(1,2))
C(a=1, b=2)
>>>


Gary Herron







--
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

--
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Jussi Piitulainen]

Faling Dutchman writes:

> I am just starting off in python, but have good knowledge of both Java
> and C#. Now is the problem that I need to have multiple instances of

...

> it prints: <__main__.Item object at 0x02EBF3B0>
>
> So that is not usefull to me. There can be an infinite amount of

...

> and it has to show all the parameters of the class Item and not say
> "ive got an object at this memory address, have a nice day"

Others have now told you that your class has a special method that
defines the string representation of instances. What you are seeing is
the default. You can specify your own.

I just want to add that Java does it the same way. Surely this hasn't
changed? (I've never known C#, so no comment on that.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Faling Dutchman]

Op maandag 7 maart 2016 10:31:48 UTC+1 schreef Jussi Piitulainen:
> Faling Dutchman writes:
> > I am just starting off in python, but have good knowledge of both Java
> > and C#. Now is the problem that I need to have multiple instances of
> ...
> > it prints: <__main__.Item object at 0x02EBF3B0>
> >
> > So that is not usefull to me. There can be an infinite amount of
> ...
> > and it has to show all the parameters of the class Item and not say
> > "ive got an object at this memory address, have a nice day"
> Others have now told you that your class has a special method that
> defines the string representation of instances. What you are seeing is
> the default. You can specify your own.
> I just want to add that Java does it the same way. Surely this hasn't
> changed? (I've never known C#, so no comment on that.)


At least, I've never encountered it, but well, the __dict__ is very usefull, as 
I said, I'm verry new to python, but I love programming in it. I hope to get 
way better in it as I am now :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Peter Otten]

ZhangXiang wrote:

> In python3, when I write code like this:
> 
> try:
> fields = [getattr(Product, field) for field in fields.split(',')]
> except AttributeError as e:
> raise HTTPError(...)
> 
> I want to raise a new type of error giving a string telling the user which
> attribute is not valid. But I don't see any method I can use to get the
> attribute name except inspecting e.args[0].
> 
> Could anyone give me a hint? Maybe I miss something.
> 
> By the way, I don't quite want to change my code to a for-loop so I can
> access the field variable in exception handling.

It seems that the name of the attribute is not made available. You have to 
parse the error message or provide the name yourself:

>>> def mygetattr(object, name):
... try:
... return getattr(object, name)
... except AttributeError as err:
... err.name = name
... raise
... 
>>> class Product:
... foo = 42
... 
>>> fields = "foo,bar"
>>> try:
... fields = [mygetattr(Product, field) for field in fields.split(",")]
... except AttributeError as err:
... print(err)
... print("Missing attribute:", err.name)
... 
type object 'Product' has no attribute 'bar'
Missing attribute: bar

Raising a subclass instead of mutating and reraising the original 
AttributeError is probably cleaner...

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Xiang Zhang]

On Monday, March 7, 2016 at 5:49:48 PM UTC+8, Peter Otten wrote:
> ZhangXiang wrote:
> 
> > In python3, when I write code like this:
> > 
> > try:
> > fields = [getattr(Product, field) for field in fields.split(',')]
> > except AttributeError as e:
> > raise HTTPError(...)
> > 
> > I want to raise a new type of error giving a string telling the user which
> > attribute is not valid. But I don't see any method I can use to get the
> > attribute name except inspecting e.args[0].
> > 
> > Could anyone give me a hint? Maybe I miss something.
> > 
> > By the way, I don't quite want to change my code to a for-loop so I can
> > access the field variable in exception handling.
> 
> It seems that the name of the attribute is not made available. You have to 
> parse the error message or provide the name yourself:
> 
> >>> def mygetattr(object, name):
> ... try:
> ... return getattr(object, name)
> ... except AttributeError as err:
> ... err.name = name
> ... raise
> ... 
> >>> class Product:
> ... foo = 42
> ... 
> >>> fields = "foo,bar"
> >>> try:
> ... fields = [mygetattr(Product, field) for field in fields.split(",")]
> ... except AttributeError as err:
> ... print(err)
> ... print("Missing attribute:", err.name)
> ... 
> type object 'Product' has no attribute 'bar'
> Missing attribute: bar
> 
> Raising a subclass instead of mutating and reraising the original 
> AttributeError is probably cleaner...

Yes. It is a way to achieving that. But is it reasonable to add an attribute to 
AttributeError so we can easily get the attribute name?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Tony van der Hoff]

On 06/03/16 14:41, Steven D'Aprano wrote:

On Sun, 6 Mar 2016 10:34 pm, Tony van der Hoff wrote:


Hi, I've been experimenting with a short test program under python 2.7
and python 3.4.2. It's a simple read from file, and locate a word therein.

I get the (subjective) impression that python2  is slightly faster than
python3. Is that correct? Is there any documentation to support this?


I believe that, overall, Python 3 is still slightly slower than Python 2,
but it's a near thing. Have a look at the latest performance benchmarks:

https://speed.python.org/comparison/

Eyeballing the graph, I estimate that the latest 3.x version is probably
about 10% slower overall, although different benchmarks show different
speeds.


Excellent, just what I was looking for. Thank you very much, Steven.

--
Tony van der Hoff| mailto:t...@vanderhoff.org
Buckinghamshire, England |
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[jmp]

On 03/07/2016 09:46 AM, ZhangXiang wrote:

In python3, when I write code like this:

try:
 fields = [getattr(Product, field) for field in fields.split(',')]
except AttributeError as e:
 raise HTTPError(...)

I want to raise a new type of error giving a string telling the user which 
attribute is not valid. But I don't see any method I can use to get the 
attribute name except inspecting e.args[0].

Could anyone give me a hint? Maybe I miss something.

By the way, I don't quite want to change my code to a for-loop so I can access 
the field variable in exception handling.



Hi,

It is strange to morph an AttributeError in an HTTPError, anyway, you 
could write


try:
 fields = [getattr(Product, field) for field in fields.split(',')]
except AttributeError as e:
 raise HTTPError(str(e))

Or if you really need to write some custom message, you could parse the 
attribute error message for the attribute name.



# I've not tested this code !!
import re
try:
 fields = [getattr(Product, field) for field in fields.split(',')]
except AttributeError as e:
 # e.message should look like "class Foo has no attribute 'a'"
 attr = re.search(r"'(\w+)'", e.message).group(1)
 raise HTTPError('Custom message for the attribute %s' % attr)


Tbh I don't like it but it could do the job until someone raises you an 
attribute error with a different string pattern.


jm

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 06/03/2016 11:34, Tony van der Hoff wrote:

Hi, I've been experimenting with a short test program under python 2.7
and python 3.4.2. It's a simple read from file, and locate a word therein.

I get the (subjective) impression that python2  is slightly faster than
python3. Is that correct? Is there any documentation to support this?


I've also found that 3 was consistently slower than 2 on various 
benchmarks. Perhaps 10 to 20% slower (also 3.4 vs. 2.7).


One example (processing a jpeg file), both with CPython on Windows[1]:

Python 3.4:6.8 seconds
Python 2.7:5.4 seconds

But then:

PyPy:  2.1 seconds [2]

PyPy I think is only compatible with Python 3 code (there are a few 
other issues too).


([1] Windows Python I believe is a little slower than on Linux, for the 
same version of Python and on the same hardware. I think due to CPython 
being compiled with MSVC not gcc. But it would affect 2 and 3 equally.)


([2] With larger data, PyPy gets progressively faster compared with 
normal Python, as it can optimise big loops better. This test was with 
0.5 Mpixels of data)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Marko Rauhamaa]

BartC :

> I've also found that 3 was consistently slower than 2 on various
> benchmarks. Perhaps 10 to 20% slower (also 3.4 vs. 2.7).

Python doesn't exist for performance-critical parts of your solution.
Also, Python programs tend to take huge amounts of space.

Where that is a problem, use some other programming language like C.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __del__: when to use it? What happens when you SystemExit/NameError wrt del? Method vs function calls.

[Veek. M]

Steven D'Aprano wrote:

> On Monday 07 March 2016 17:13, Veek. M wrote:
> 
>> import foo
>> class Bar(object):
>> def __del__(self, foo=foo):
>> foo.bar()# Use something in module foo
>> 
>> ### Why the foo=foo? import foo, would increment the ref-count for
>> object 'foo'
> 
> Yes. And then at shutdown, the module globals get deleted so it gets
> decremented again and then garbage collected.
> 
> You cannot tell what order objects will be deleted, so you cannot tell
> whether your instance will be deleted before or after foo. If it
> happens before foo, then __del__ will run and you will be fine. If it
> happens after foo, then foo is gone and your __del__ method will fail.
> 
> 
>> so why go to all the effort of passing it in to every
>> instance via the local stack (ref-to-object-foo)?
> 
> So that every instance has a guaranteed and reliable reference to foo
> that cannot be garbage collected before that instance.
> 
> Here is a sketch of what happens without the foo=foo:
> 
> import foo  # refcount = 1
> x = Bar()  # make an instance
> y = Bar()  # and another instance
> ...
> z = Bar()  # another instance
> ...
> ...
> del z  # z gets garbage collected, everything is fine
> ...
> sys.exit()  # Shutdown starts, x and y still exist
> # Python starts garbage collecting objects
> # in some unknown order...
> ...
> ...
> del x  # x gets garbage collected, foo still exists so everything is
> fine
> del foo  # refcount = 0, so foo gets garbage collected
> del y  # y gets garbage collected, but foo is gone!
> 
> 
> 
> 
> Here is a sketch of what happens with the foo=foo:
> 
> import foo  # refcount = 1
> x = Bar()  # make an instance, foo refcount = 2
> y = Bar()  # and another instance, foo refcount = 3
> ...
> z = Bar()  # another instance, foo refcount = 4
> ...
> ...
> del z  # z gets garbage collected, foo refcount = 3
> ...
> sys.exit()  # Shutdown starts, x and y still exist
> # Python starts garbage collecting objects
> # in some unknown order...
> ...
> ...
> del x  # x gets garbage collected, foo refcount = 2
> del y  # y gets garbage collected, foo refcount = 1
> del foo  # refcount = 0, so foo gets garbage collected
> 
> 
> 
> This guarantees that so long as there are any instances yet to be
> garbage collected, foo cannot be be garbage collected. Only after the
> last of those instances are gone will foo be garbage collected.
> 
> 
> 
>> Text for 1,2 from Beazley:
>> It’s important to note that in some cases the __del__() method might
>> not be invoked at program termination.This can occur if circular
>> references exist between objects (in which case objects may be
>> allocated but accessible from no known name-space).Although Python’s
>> garbage collector can reclaim unused circular references dur-ing
>> execution, it isn’t normally invoked on program termination.
>> 
>> ### which begs the question, why not on program termination?
> 
> I'm afraid I don't know. Perhaps read the source code?
> 
> 
>> How bad can
>> it be since you are terminating anyway. __del__ seems like a total
>> waste product method
> 
> I won't say "total" waste product, but automatic destructor methods
> are only good for very simple use-cases where you don't care that they
> are called in a non-deterministic fashion.
> 
> 
> 
Hi thanks! I kind of posted and your reply came through. Got it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 11:11, Marko Rauhamaa wrote:

BartC :


I've also found that 3 was consistently slower than 2 on various
benchmarks. Perhaps 10 to 20% slower (also 3.4 vs. 2.7).


Python doesn't exist for performance-critical parts of your solution.
Also, Python programs tend to take huge amounts of space.


I'm not complaining. For my purposes (implementing a similar language 
that competes with Python for speed), I welcome the slower speed of 
Python 3!


(Although competing with CPython is too easy. PyPy is more of a problem. 
With the Jpeg benchmark I mentioned, I can beat PyPy up to 6Mpix, but 
then PyPy starts to get faster. At 80Mpix, PyPy is 60% faster.)


--
Bartc

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Andrew Jaffe]

On 06/03/2016 14:41, Steven D'Aprano wrote:

On Sun, 6 Mar 2016 10:34 pm, Tony van der Hoff wrote:


Hi, I've been experimenting with a short test program under python 2.7
and python 3.4.2. It's a simple read from file, and locate a word therein.

I get the (subjective) impression that python2  is slightly faster than
python3. Is that correct? Is there any documentation to support this?


I believe that, overall, Python 3 is still slightly slower than Python 2,
but it's a near thing. Have a look at the latest performance benchmarks:

https://speed.python.org/comparison/

Eyeballing the graph, I estimate that the latest 3.x version is probably
about 10% slower overall, although different benchmarks show different
speeds.



Dumb question, and this probably isn't the place for it, but the three 
Pythons being tested are:


 64-bit CPython on Li... latest in branch '2.7'
 64-bit CPython on Li... latest in branch '3.5'
 64-bit CPython on Li... latest

I understand that the first two are the released 2.7 and 3.5 versions; 
is the third the most recent 3.x build?


Andrew



--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Fabien]

On 03/07/2016 12:38 PM, BartC wrote:


(Although competing with CPython is too easy. PyPy is more of a problem.
With the Jpeg benchmark I mentioned, I can beat PyPy up to 6Mpix, but
then PyPy starts to get faster. At 80Mpix, PyPy is 60% faster.)


Just out of curiosity: are you also competing against numpy/scipy?
--
https://mail.python.org/mailman/listinfo/python-list

Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

[Veek. M]

import socket
class Client(object):
 def __init__(self,addr):
  self.server_addr = addr
  self.sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
  self.sock.connect(addr)

 def __getstate__(self):
  return self.server_addr
 
 def __setstate__(self,value):  
  self.server_addr = value
  self.sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
  self.sock.connect(self.server_addr)
-

We'd use it like so:
x = Client(192.168.0.1)
import pickle
pickle.dump(x) #getstate gets called and returns IP for pickling.

#program exits, we restart it
x=Client(None)
x = pickle.load(fh) #Is the IP passed to setstate??
x.__setstate__(self, unpickledIP) ???
Is this correct?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Regex: Perl to Python

[Fillmore]

Big thank you to everyone who offered their help!

On 03/06/2016 11:38 PM, Fillmore wrote:




--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 12:19, Fabien wrote:

On 03/07/2016 12:38 PM, BartC wrote:


(Although competing with CPython is too easy. PyPy is more of a problem.
With the Jpeg benchmark I mentioned, I can beat PyPy up to 6Mpix, but
then PyPy starts to get faster. At 80Mpix, PyPy is 60% faster.)


Just out of curiosity: are you also competing against numpy/scipy?


No, I only compare basic language functions. I understand that Python 
depends on complex built-in functions, and external libraries such as 
numpy, for it to be used viably. But I'm also interested in using such 
languages directly.


Take the jpeg benchmark. Of course both Python and my language are 
hopelessly slow and impractical compared with a C implementation, but 
this is still a useful test (and in fact the interpreted version was 
used to more easily develop a streamlined decoder that was then 
back-ported to C, doubling its speed).


(The Python version of that program is here:
http://pastebin.com/cHx3UhQb. It should work with any Python.)

For example, suppose you wanted to do crude animation by loading a 
series of small 640x480 images. Loading such an image gives you the 
following frame rates, excluding the extra overheads of displaying the 
results:


Python 2.7:0.33 fps
Python 3.4:0.25 fps
PyPy:  0.55 fps (don't known what version)
Mine:  3.77 fps

You can't really call the first three animated (more like a slide show), 
but 4 fps just about cuts it!


Sometimes pushing an interpreted language a bit further can be useful 
(hence the existence of the PyPy project).


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 12:25 AM, BartC  wrote:
> No, I only compare basic language functions. I understand that Python
> depends on complex built-in functions, and external libraries such as numpy,
> for it to be used viably. But I'm also interested in using such languages
> directly.
>
> Take the jpeg benchmark. Of course both Python and my language are
> hopelessly slow and impractical compared with a C implementation, but this
> is still a useful test (and in fact the interpreted version was used to more
> easily develop a streamlined decoder that was then back-ported to C,
> doubling its speed).
>
> (The Python version of that program is here:
> http://pastebin.com/cHx3UhQb. It should work with any Python.)

Actually, it won't work with any Python - not if it gets a broken
file. Your abortjpeg() function doesn't work :)

But what you have there is a messy mixture of old-style classes used
as if they were C structs, array.array() as if it were a C array, and
utterly uncommented code that looks like it was ported from, again, C.
You're not going to prove anything useful about Python - any version
of Python - by using it badly. Try implementing JPEG decoding in a
more Pythonic way - or, better still, use pillow and don't write that
kind of code yourself at all.

Benchmarking Py2 vs Py3 on that kind of code is like trying to figure
out whether it's easier to drag a 747 by your teeth or your toes.
Yeah, you might get a result, but it's a useless one.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

importing

[Tony van der Hoff]

I thought I understood this, but apparently not:
Under py3:

1. "import tkinter" imports the whole module into the name space. Any 
access to names therein must be prefixed with the module name.

ie top = tkinter.Tk()
But tkinter.messagebox.showwarning()  errors with "module has no 
attribute 'messagebox'"


2. "from tkinter import *" loads the name space from the module into the 
program name space. No need to prefix the module name onto the attribute 
name. Pollutes the name space, but makes typing easier.

ie top = Tk()
But messagebox.showwarning() still errors.

3. in either of the above cases, if I add "from tkinter import 
messagebox, the attribute resolves correctly.


I imagined that the "*" form implied "load the lot". Evidently, my 
understanding is lacking. Will somebody please put me straight, or give 
me a reference to some definitive documentation?


--
Tony van der Hoff| mailto:t...@vanderhoff.org
Buckinghamshire, England |
--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Tim Golden]

On 07/03/2016 15:54, Tony van der Hoff wrote:
> I thought I understood this, but apparently not:
> Under py3:
> 
> 1. "import tkinter" imports the whole module into the name space. Any
> access to names therein must be prefixed with the module name.
> ie top = tkinter.Tk()
> But tkinter.messagebox.showwarning()  errors with "module has no
> attribute 'messagebox'"
> 
> 2. "from tkinter import *" loads the name space from the module into the
> program name space. No need to prefix the module name onto the attribute
> name. Pollutes the name space, but makes typing easier.
> ie top = Tk()
> But messagebox.showwarning() still errors.
> 
> 3. in either of the above cases, if I add "from tkinter import
> messagebox, the attribute resolves correctly.
> 
> I imagined that the "*" form implied "load the lot". Evidently, my
> understanding is lacking. Will somebody please put me straight, or give
> me a reference to some definitive documentation?
> 

What you're seeing is that "messagebox" is a module inside the "tkinter"
package. And modules are not automatically made available under their
parent packages.

That is: a module's attributes are part of its namespace; but a
package's modules aren't part of its.

TJG
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Ian Kelly]

On Mon, Mar 7, 2016 at 8:54 AM, Tony van der Hoff  wrote:
> I thought I understood this, but apparently not:
> Under py3:
>
> 1. "import tkinter" imports the whole module into the name space. Any access
> to names therein must be prefixed with the module name.
> ie top = tkinter.Tk()
> But tkinter.messagebox.showwarning()  errors with "module has no attribute
> 'messagebox'"

tkinter.messagebox is a module inside the tkinter package. Importing
tkinter doesn't automatically import tkinter.messagebox also.

>
> 2. "from tkinter import *" loads the name space from the module into the
> program name space. No need to prefix the module name onto the attribute
> name. Pollutes the name space, but makes typing easier.
> ie top = Tk()
> But messagebox.showwarning() still errors.

This is still just loading names from the tkinter module, not its submodules.

> 3. in either of the above cases, if I add "from tkinter import messagebox,
> the attribute resolves correctly.

This is the first place where you actually instruct Python to import
tkinter.messagebox.
-- 
https://mail.python.org/mailman/listinfo/python-list

ANN: eGenix pyOpenSSL Distribution 0.13.14

[eGenix Team: M.-A. Lemburg]

ANNOUNCING

   eGenix.com pyOpenSSL Distribution

Version 0.13.14

An easy-to-install and easy-to-use distribution
of the pyOpenSSL Python interface for OpenSSL -
   available for Windows, Mac OS X and Unix platforms


This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.14.html


INTRODUCTION

The eGenix.com pyOpenSSL Distribution includes everything you need to
get started with SSL in Python.

It comes with an easy-to-use installer that includes the most recent
OpenSSL library versions in pre-compiled form, making your application
independent of OS provided OpenSSL libraries:

http://www.egenix.com/products/python/pyOpenSSL/

pyOpenSSL is an open-source Python add-on that allows writing SSL/TLS-
aware network applications as well as certificate management tools:

https://launchpad.net/pyopenssl/

OpenSSL is an open-source implementation of the SSL/TLS protocol:

http://www.openssl.org/


NEWS

This new release of the eGenix.com pyOpenSSL Distribution includes the
following updates:

New in OpenSSL
--

 * Updated included OpenSSL libraries from OpenSSL 1.0.1r to
   1.0.1s. See https://www.openssl.org/news/secadv/20160301.txt ​for a
   complete list of changes. The following fixes are relevant for
   pyOpenSSL applications:

   - CVE-2016-0800 (DROWN attack) A cross-protocol attack was
 discovered that could lead to decryption of TLS sessions by using
 a server supporting SSLv2 and EXPORT cipher suites as a
 Bleichenbacher RSA padding oracle.

 As additional result of this attack, the default OpenSSL
 configuration no longer includes the SSLv2 protocol support
 starting with 1.0.1s.

   - Several low priority issues related to memory leaks.

 * Disabled SSLv2 support in all our OpenSSL library builds (no-ssl2).

 * Disabled TLS compression in all our OpenSSL library builds
   (no-comp). This may lead to problems with other libraries that
   still expect to find these APIs. pyOpenSSL itself does not use
   them.

 * Updated the Mozilla CA root bundle to version 2016-03-01. Nothing
   much changed, except the date of the bundle file.

Please see the product changelog for the full set of changes.

http://www.egenix.com/products/python/pyOpenSSL/changelog.html


pyOpenSSL / OpenSSL Binaries Included
-

In addition to providing sources, we make binaries available that
include both pyOpenSSL and the necessary OpenSSL libraries for all
supported platforms: Windows, Linux, Mac OS X and FreeBSD, for x86 and
x64.

To simplify installation, we have uploaded a web installer to PyPI
which will automatically choose the right binary for your platform, so
a simple

pip install egenix-pyopenssl

will get you the package with OpenSSL libraries installed. Please see
our installation instructions for details:

http://www.egenix.com/products/python/pyOpenSSL/#Installation

We have also added .egg-file distribution versions of our eGenix.com
pyOpenSSL Distribution for Windows, Linux and Mac OS X to the
available download options. These make setups using e.g. zc.buildout
and other egg-file based installers a lot easier.


DOWNLOADS

The download archives and instructions for installing the package can
be found at:

http://www.egenix.com/products/python/pyOpenSSL/


UPGRADING

Before installing this version of pyOpenSSL, please make sure that
you uninstall any previously installed pyOpenSSL version. Otherwise,
you could end up not using the included OpenSSL libs.

___
SUPPORT

Commercial support for these packages is available from eGenix.com.
Please see

http://www.egenix.com/services/support/

for details about our support offerings.


MORE INFORMATION

For more information about the eGenix pyOpenSSL Distribution, licensing
and download instructions, please visit our web-site or write to
sa...@egenix.com.

About eGenix (http://www.egenix.com/):

eGenix is a Python software project, consulting and product
company delivering expert services and professional quality
products for companies, Python users and developers. We specialize
in database driven applications, large scale software designs and
integration.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Mar 07 2016)
>>> Python Projects, Coaching and

Re: importing

[Chris Angelico]

On Tue, Mar 8, 2016 at 2:54 AM, Tony van der Hoff  wrote:
> I thought I understood this, but apparently not:
> Under py3:
>
> 1. "import tkinter" imports the whole module into the name space. Any access
> to names therein must be prefixed with the module name.
> ie top = tkinter.Tk()
> But tkinter.messagebox.showwarning()  errors with "module has no attribute
> 'messagebox'"

Your description is of a general module, but tkinter is actually a
package. Packages are slightly different, in that their modules might
not be loaded automatically. You can also do this:

import tkinter.messagebox

which is a syntax that works ONLY with packages. [1]

> 2. "from tkinter import *" loads the name space from the module into the
> program name space. No need to prefix the module name onto the attribute
> name. Pollutes the name space, but makes typing easier.
> ie top = Tk()
> But messagebox.showwarning() still errors.
>
> I imagined that the "*" form implied "load the lot". Evidently, my
> understanding is lacking. Will somebody please put me straight, or give me a
> reference to some definitive documentation?

Sorta-kinda. The star import means "load all the obvious names". Even
with non-package modules, it won't always import everything - if
there's a list of names in __all__, only those names will  be
imported. Generally, this means you don't get nested module references
(eg because a module did "import sys"), but it can do anything else as
well.

The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.

ChrisA

[1] Modulo os.path, which is done by injection. Ignore that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A mistake which almost went me mad

[Ian Kelly]

On Thu, Mar 3, 2016 at 11:50 AM, Tim Chase
 wrote:
> I think that relative imports should ameliorate this, as I usually
> hit it when I'm using smtplib which in turn imports "email" (and, in
> 2.x when it found my local email.py would crash and burn). If it used
> a relative import that forced it to find the one in the stdlib, it
> should(?) prevent it from finding my local version first.

Relative imports only work inside packages. You can't use a relative
import to import one top-level module from another.

Besides, the relative import doesn't help to disambiguate in this
case. The absolute path of the stdlib email module is "email". The
absolute path of the module in your CWD is also "email". Why should a
relative import prefer one over the other? So I would think that even
if it worked, it would still just end up importing the first one it
finds on your sys.path.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A mistake which almost went me mad

[Random832]

On Mon, Mar 7, 2016, at 11:19, Ian Kelly wrote:
> Relative imports only work inside packages. You can't use a relative
> import to import one top-level module from another.
> 
> Besides, the relative import doesn't help to disambiguate in this
> case. The absolute path of the stdlib email module is "email". The
> absolute path of the module in your CWD is also "email". Why should a
> relative import prefer one over the other? So I would think that even
> if it worked, it would still just end up importing the first one it
> finds on your sys.path.

Maybe what we need is a system smarter than sys.path - a clear "import
from stdlib" statement, and a way to designate what directories *contain
stdlib modules*, would be useful.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 15:54, Tony van der Hoff wrote:

I thought I understood this, but apparently not:
Under py3:

1. "import tkinter" imports the whole module into the name space. Any
access to names therein must be prefixed with the module name.
ie top = tkinter.Tk()
But tkinter.messagebox.showwarning()  errors with "module has no
attribute 'messagebox'"

2. "from tkinter import *" loads the name space from the module into the
program name space. No need to prefix the module name onto the attribute
name. Pollutes the name space, but makes typing easier.
ie top = Tk()
But messagebox.showwarning() still errors.

3. in either of the above cases, if I add "from tkinter import
messagebox, the attribute resolves correctly.

I imagined that the "*" form implied "load the lot". Evidently, my
understanding is lacking. Will somebody please put me straight, or give
me a reference to some definitive documentation?



As a slight aside, and as you've all ready had answers, you might like 
to read the Bryan Oakley response to this 
http://stackoverflow.com/questions/17466561/best-way-to-structure-a-tkinter-application


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Xiang Zhang]

Hi,

I know I can get the attribute name in some way, but since I just want the 
attribute name when an AttributeError caused by it raised, I really don't want 
to inspect the string or introduce one more layer over getattr. I hope I can 
get the attribute which causes the exception from the AttributeError raised 
directly. But it seems I can't.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Tim Golden]

On 07/03/2016 16:25, Xiang Zhang wrote:
> Hi,
> 
> I know I can get the attribute name in some way, but since I just
> want the attribute name when an AttributeError caused by it raised, I
> really don't want to inspect the string or introduce one more layer
> over getattr. I hope I can get the attribute which causes the
> exception from the AttributeError raised directly. But it seems I
> can't.
> 

As things stand, you can't. But if you were to chime in on this issue:

  https://bugs.python.org/issue18156

then you might bring discussion there back to life again and see some
forward movement. There seemed to be a consensus (and involving some
active developers) so it's possible that new interest might revive interest.

TJG
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A mistake which almost went me mad

[Chris Angelico]

On Tue, Mar 8, 2016 at 3:19 AM, Ian Kelly  wrote:
> On Thu, Mar 3, 2016 at 11:50 AM, Tim Chase
>  wrote:
>> I think that relative imports should ameliorate this, as I usually
>> hit it when I'm using smtplib which in turn imports "email" (and, in
>> 2.x when it found my local email.py would crash and burn). If it used
>> a relative import that forced it to find the one in the stdlib, it
>> should(?) prevent it from finding my local version first.
>
> Relative imports only work inside packages. You can't use a relative
> import to import one top-level module from another.
>
> Besides, the relative import doesn't help to disambiguate in this
> case. The absolute path of the stdlib email module is "email". The
> absolute path of the module in your CWD is also "email". Why should a
> relative import prefer one over the other? So I would think that even
> if it worked, it would still just end up importing the first one it
> finds on your sys.path.

So the solution is to treat the current directory as a pseudo-package.
It'd be a backward-incompatible change, so it would need to be
explicitly invoked. Something like:

python3 -p somefile.py

which would pretend to create an __init__.py in the current directory,
change to the parent, and "from dirname import somefile". I mentioned
this one time before, but it got kinda missed; I think I'll reraise it
on python-ideas. It's a protection on par with not having the current
directory in $PATH.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Question

[Ben Morales]

I am trying to download Python but I have windows 10 and I do not see a 64
bit download for my operating system. Do you have a 64 bit for windows?
Sincerely,

*Ben Morales*
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Ian Kelly]

On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:
> I am trying to download Python but I have windows 10 and I do not see a 64
> bit download for my operating system. Do you have a 64 bit for windows?

What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.

The other question is are you sure that 64-bit Python is what you
want? If your Python is 64-bit then I believe that any extension
modules you use need to be compiled 64-bit as well. On a 64-bit
Windows system you can run either 32-bit or 64-bit Python, and AFAIK
it's more common to use 32-bit Python.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Xiang Zhang]

On Tuesday, March 8, 2016 at 12:38:31 AM UTC+8, Tim Golden wrote:
> On 07/03/2016 16:25, Xiang Zhang wrote:
> > Hi,
> > 
> > I know I can get the attribute name in some way, but since I just
> > want the attribute name when an AttributeError caused by it raised, I
> > really don't want to inspect the string or introduce one more layer
> > over getattr. I hope I can get the attribute which causes the
> > exception from the AttributeError raised directly. But it seems I
> > can't.
> > 
> 
> As things stand, you can't. But if you were to chime in on this issue:
> 
>   https://bugs.python.org/issue18156
> 
> then you might bring discussion there back to life again and see some
> forward movement. There seemed to be a consensus (and involving some
> active developers) so it's possible that new interest might revive interest.
> 
> TJG

Thanks for mentioning the issue. I'll ping it to see if there is still anyone 
interested in it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Tony van der Hoff]

Thanks to all who replied to my cry for help; I understand it better 
now. But:


On 07/03/16 16:08, Chris Angelico wrote:


The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.


Well, I found the tkinter documentation to be sparse in the extreme 
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it 
certainly doesn't go into that sort of detail.


However, more generally, how am I supposed to know that a module is part 
of a package, and needs a "magic" stanza to get a module loaded?


Cheers,
--
Tony van der Hoff| mailto:t...@vanderhoff.org
Buckinghamshire, England |
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Mark Lawrence]

On 07/03/2016 16:57, Ian Kelly wrote:

On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:

I am trying to download Python but I have windows 10 and I do not see a 64
bit download for my operating system. Do you have a 64 bit for windows?


What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.

The other question is are you sure that 64-bit Python is what you
want? If your Python is 64-bit then I believe that any extension
modules you use need to be compiled 64-bit as well. On a 64-bit
Windows system you can run either 32-bit or 64-bit Python, and AFAIK
it's more common to use 32-bit Python.



I've been running 64 bit Python on Windows for years with no problems. 
Why use 32 bit?  I certainly don't understand why you'd need to.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to get which attribute causes the AttributeError except inspecting strings?

[Mark Lawrence]

On 07/03/2016 16:38, Tim Golden wrote:

On 07/03/2016 16:25, Xiang Zhang wrote:

Hi,

I know I can get the attribute name in some way, but since I just
want the attribute name when an AttributeError caused by it raised, I
really don't want to inspect the string or introduce one more layer
over getattr. I hope I can get the attribute which causes the
exception from the AttributeError raised directly. But it seems I
can't.



As things stand, you can't. But if you were to chime in on this issue:

   https://bugs.python.org/issue18156

then you might bring discussion there back to life again and see some
forward movement. There seemed to be a consensus (and involving some
active developers) so it's possible that new interest might revive interest.

TJG



Thanks, you saved me searching for the link that I knew existed :)

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 17:23, Tony van der Hoff wrote:

Thanks to all who replied to my cry for help; I understand it better
now. But:

On 07/03/16 16:08, Chris Angelico wrote:


The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.


Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.

However, more generally, how am I supposed to know that a module is part
of a package, and needs a "magic" stanza to get a module loaded?

Cheers,


It all depends on whether or not there is an __init__.py file in the 
directory, see e.g. http://pythoncentral.io/how-to-create-a-python-package/


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Chris Angelico]

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff  wrote:
> Thanks to all who replied to my cry for help; I understand it better now.
> But:
>
> On 07/03/16 16:08, Chris Angelico wrote:
>>
>>
>> The documentation should tell you what you need to import to make
>> something work. In this case, I would guess that "import
>> tkinter.messagebox" or "from tkinter import messagebox" would be the
>> recommended way to use this module.
>
>
> Well, I found the tkinter documentation to be sparse in the extreme
> (https://docs.python.org/release/3.1.3/library/tkinter.html), and it
> certainly doesn't go into that sort of detail.

You're looking at an ancient version of the docs. Here's the newest docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Ian Kelly]

On Mon, Mar 7, 2016 at 10:23 AM, Tony van der Hoff  wrote:
> However, more generally, how am I supposed to know that a module is part of
> a package, and needs a "magic" stanza to get a module loaded?

If the import path of the module has a dot in it, then it's part of a package.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Ian Kelly]

On Mon, Mar 7, 2016 at 10:25 AM, Mark Lawrence  wrote:
> On 07/03/2016 16:57, Ian Kelly wrote:
>>
>> On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales 
>> wrote:
>>>
>>> I am trying to download Python but I have windows 10 and I do not see a
>>> 64
>>> bit download for my operating system. Do you have a 64 bit for windows?
>>
>>
>> What page are you looking at?
>> https://www.python.org/downloads/release/python-351/ has downloads for
>> both Windows x86 and Windows x86-64.
>>
>> The other question is are you sure that 64-bit Python is what you
>> want? If your Python is 64-bit then I believe that any extension
>> modules you use need to be compiled 64-bit as well. On a 64-bit
>> Windows system you can run either 32-bit or 64-bit Python, and AFAIK
>> it's more common to use 32-bit Python.
>>
>
> I've been running 64 bit Python on Windows for years with no problems. Why
> use 32 bit?  I certainly don't understand why you'd need to.

It seems to be easier to find 32-bit binaries for libraries. For
example, the official Windows build of pygame is only 32-bit. If
64-bit only works for you though, then by all means use it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, Ian Kelly  wrote:
> On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:
>> I am trying to download Python but I have windows 10 and I do not see a 64
>> bit download for my operating system. Do you have a 64 bit for windows?
>
> What page are you looking at?
> https://www.python.org/downloads/release/python-351/ has downloads for
> both Windows x86 and Windows x86-64.

It only appears to have downloads for 32-bit, or 64-bit AMD processors,
not 64-bit Intel processors.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, Ian Kelly  wrote:
> On Mon, Mar 7, 2016 at 10:25 AM, Mark Lawrence  
> wrote:
>> I've been running 64 bit Python on Windows for years with no problems. Why
>> use 32 bit?  I certainly don't understand why you'd need to.
>
> It seems to be easier to find 32-bit binaries for libraries. For
> example, the official Windows build of pygame is only 32-bit.

There doesn't appear to be an official Windows build of pygame at all,
if you want anything more recent than Python 3.1.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Mark Lawrence]

On 07/03/2016 17:42, Ian Kelly wrote:

On Mon, Mar 7, 2016 at 10:25 AM, Mark Lawrence  wrote:

On 07/03/2016 16:57, Ian Kelly wrote:


On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales 
wrote:


I am trying to download Python but I have windows 10 and I do not see a
64
bit download for my operating system. Do you have a 64 bit for windows?



What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.

The other question is are you sure that 64-bit Python is what you
want? If your Python is 64-bit then I believe that any extension
modules you use need to be compiled 64-bit as well. On a 64-bit
Windows system you can run either 32-bit or 64-bit Python, and AFAIK
it's more common to use 32-bit Python.



I've been running 64 bit Python on Windows for years with no problems. Why
use 32 bit?  I certainly don't understand why you'd need to.


It seems to be easier to find 32-bit binaries for libraries. For
example, the official Windows build of pygame is only 32-bit. If
64-bit only works for you though, then by all means use it.



As http://www.pygame.org/download.shtml hasn't been updated since August 
6th 2009 and only goes up to Python 3.2 I'd stick with 
http://www.lfd.uci.edu/~gohlke/pythonlibs/#pygame which gives a 64 bit 
3.5 download.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[mm0fmf]

On 07/03/2016 18:09, Jon Ribbens wrote:

On 2016-03-07, Ian Kelly  wrote:

On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:

I am trying to download Python but I have windows 10 and I do not see a 64
bit download for my operating system. Do you have a 64 bit for windows?


What page are you looking at?
https://www.python.org/downloads/release/python-351/ has downloads for
both Windows x86 and Windows x86-64.


It only appears to have downloads for 32-bit, or 64-bit AMD processors,
not 64-bit Intel processors.



You didn't read the bit that says

"The binaries for AMD64 will also work on processors that implement the 
Intel 64 architecture. (Also known as the "x64" architecture, and 
formerly known as both "EM64T" and "x86-64".) "




--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Chris Warrick]

On 7 March 2016 at 19:09, Jon Ribbens  wrote:
> On 2016-03-07, Ian Kelly  wrote:
>> On Mon, Mar 7, 2016 at 9:39 AM, Ben Morales  wrote:
>>> I am trying to download Python but I have windows 10 and I do not see a 64
>>> bit download for my operating system. Do you have a 64 bit for windows?
>>
>> What page are you looking at?
>> https://www.python.org/downloads/release/python-351/ has downloads for
>> both Windows x86 and Windows x86-64.
>
> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
> not 64-bit Intel processors.
> --
> https://mail.python.org/mailman/listinfo/python-list

Modern Intel processors use the amd64 (aka x86_64) architecture.
Intel’s Itanium architecture (IA-64) never really took off, and was
not supported by the consumer versions of Windows (other than XP x64).

(Not to mention 32-bit processors are sometimes called i386…i686,
where the i stands for Intel, and those processors were also
manufactured by AMD and others)

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Random832]

On Mon, Mar 7, 2016, at 13:09, Jon Ribbens wrote:
> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
> not 64-bit Intel processors.

Current 64-bit processors produced by Intel use the "AMD64"
architecture, not the Intel IA-64 (Itanium) architecture.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 15:31, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 12:25 AM, BartC  wrote:



(The Python version of that program is here:
http://pastebin.com/cHx3UhQb. It should work with any Python.)


Actually, it won't work with any Python - not if it gets a broken
file. Your abortjpeg() function doesn't work :)


What does it do when it doesn't work? I just get 'can't load file' then 
it stops if there's no such file. With an extra message if there's a 
format error. (I'm not sure what 'raise' does, but it doesn't complain 
about it. I think I realised I didn't know what came next and left it.)



But what you have there is a messy mixture of old-style classes used
as if they were C structs, array.array() as if it were a C array, and
utterly uncommented code that looks like it was ported from, again, C.
Benchmarking Py2 vs Py3 on that kind of code is like trying to figure
out whether it's easier to drag a 747 by your teeth or your toes.
Yeah, you might get a result, but it's a useless one.


I disagree. The program does its job perfectly (you will have to take it 
further to verify the results, such as writing out the .ppm file and 
viewing the contents).


But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for 
a smaller file). /This is in line with other observations./


> You're not going to prove anything useful about Python - any version
> of Python - by using it badly. Try implementing JPEG decoding in a
> more Pythonic way - or, better still, use pillow and don't write that
> kind of code yourself at all.

Before I wrote this, I looked at three other existing Python decoders, 
all more Pythonic (one had been ported from C++ I think and was crammed 
full of classes).


One worked so poorly that it was dismissed. The other two were full of 
bugs and didn't fully support the 3 main colour formats, but when they 
did work, were 3-6 times slower than my version IIRC.


They also only worked with Python 2 (so presumably would have been even 
slower on 3!) It also meant I couldn't test PyPy on them. I tried psyco 
but it made little difference. And actually the main purpose was to have 
something substantial to try PyPy on.


So I don't know who's using Python more badly: me or those other authors.

(I'm quite pleased with my version: smaller, faster, works on all the 
Pythons, supports all 3 colour formats and no decoding bugs that I'm 
aware of, and it's the first Python program I've written that does 
something useful.)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Jon Ribbens]

On 2016-03-07, mm0fmf  wrote:
> On 07/03/2016 18:09, Jon Ribbens wrote:
>> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
>> not 64-bit Intel processors.
>
> You didn't read the bit that says
>
> "The binaries for AMD64 will also work on processors that implement the 
> Intel 64 architecture. (Also known as the "x64" architecture, and 
> formerly known as both "EM64T" and "x86-64".) "

You are quite correct that I did not read the whole of the
ridiculously-long and badly-organised download page.

I would strongly suggest that, at the very least, the 'Description'
for the x86-64 files should say "for all 64-bit PCs (except Itanium*)"
with the '*' linking to a footnote that explains how to tell if you
have one of those.

I must say that Python on Windows was a very poor experience indeed,
"virtualenv" does not work and "venv" refuses to create the 'activate'
shell script so does not work either (and pygame doesn't work, but
that's presumably not Python's fault).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 07/03/2016 11:02, BartC wrote:


PyPy I think is only compatible with Python 3 code (there are a few
other issues too).



Quoting from http://doc.pypy.org/en/latest/release-0.7.0.html "The 
interpreter and object model implementations shipped with the 0.7 
version can run on their own and implement the core language features of 
Python as of CPython 2.4.".


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 17:38, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff  wrote:

Thanks to all who replied to my cry for help; I understand it better now.
But:

On 07/03/16 16:08, Chris Angelico wrote:



The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.



Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.


You're looking at an ancient version of the docs. Here's the newest docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA



As I happen to be playing with tkinter I was searching literally minutes 
ago for the way to put a grid onto a notebook tab.  Regrettably the 
notebook grid is conspicious by its absence from the docs :(


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 5:34 AM, BartC  wrote:
> On 07/03/2016 15:31, Chris Angelico wrote:
>>
>> On Tue, Mar 8, 2016 at 12:25 AM, BartC  wrote:
>
>
>>> (The Python version of that program is here:
>>> http://pastebin.com/cHx3UhQb. It should work with any Python.)
>>
>>
>> Actually, it won't work with any Python - not if it gets a broken
>> file. Your abortjpeg() function doesn't work :)
>
>
> What does it do when it doesn't work? I just get 'can't load file' then it
> stops if there's no such file. With an extra message if there's a format
> error. (I'm not sure what 'raise' does, but it doesn't complain about it. I
> think I realised I didn't know what came next and left it.)

Here's your code:

def abortjpeg(mess):
print ("Error:",mess)
raise

Here's my Python:

$ python3
Python 3.6.0a0 (default:bbfbde6ee9d0, Feb 19 2016, 12:43:06)
[GCC 5.3.1 20160121] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> raise
Traceback (most recent call last):
  File "", line 1, in 
RuntimeError: No active exception to reraise

The only try/except in your code (an except block is the only way
you'll have an "active exception") is this abomination:

def getfile(file):
try:
data=open(file,"rb").read()
except:
return 0
...

>> But what you have there is a messy mixture of old-style classes used
>> as if they were C structs, array.array() as if it were a C array, and
>> utterly uncommented code that looks like it was ported from, again, C.
>> Benchmarking Py2 vs Py3 on that kind of code is like trying to figure
>> out whether it's easier to drag a 747 by your teeth or your toes.
>> Yeah, you might get a result, but it's a useless one.
>
>
> I disagree. The program does its job perfectly (you will have to take it
> further to verify the results, such as writing out the .ppm file and viewing
> the contents).
>
> But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for a
> smaller file). /This is in line with other observations./

What's your meaning of "perfectly"? You're implementing things in a
very C way, and then showing that two different Python interpreters
have different performance. You haven't actually shown how a
properly-written program would perform.

>> You're not going to prove anything useful about Python - any version
>> of Python - by using it badly. Try implementing JPEG decoding in a
>> more Pythonic way - or, better still, use pillow and don't write that
>> kind of code yourself at all.
>
> Before I wrote this, I looked at three other existing Python decoders, all
> more Pythonic (one had been ported from C++ I think and was crammed full of
> classes).
>
> One worked so poorly that it was dismissed. The other two were full of bugs
> and didn't fully support the 3 main colour formats, but when they did work,
> were 3-6 times slower than my version IIRC.
>
> They also only worked with Python 2 (so presumably would have been even
> slower on 3!) It also meant I couldn't test PyPy on them. I tried psyco but
> it made little difference. And actually the main purpose was to have
> something substantial to try PyPy on.
>
> So I don't know who's using Python more badly: me or those other authors.

Did you try the 'pillow' library?

https://pypi.python.org/pypi/Pillow

It supports 2.6/2.7 and 3.2+, and would be most people's "one obvious
way" to do things. At very least, it should get a mention in your
performance comparisons.

> (I'm quite pleased with my version: smaller, faster, works on all the
> Pythons, supports all 3 colour formats and no decoding bugs that I'm aware
> of, and it's the first Python program I've written that does something
> useful.)

"all the Pythons" meaning what, exactly? And, no decoding bugs? Maybe,
but I wouldn't bet my life on it. Unless your code is a pure port of
someone else's (and unless that someone else could guarantee that the
original C code was bug free), I wouldn't trust code that I can't
completely analyze in one sitting. Virtually all code has bugs in it;
the only code that doesn't is code that's so trivially simple that you
can completely understand it in one "headspace".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Ian Kelly]

On Mon, Mar 7, 2016 at 11:51 AM, Jon Ribbens
 wrote:
> I must say that Python on Windows was a very poor experience indeed,
> "virtualenv" does not work and "venv" refuses to create the 'activate'
> shell script so does not work either

I've used both of these on Windows (although not recently) and had no
trouble with them. I never had a problem with venv not creating the
activate.bat file.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Chris Angelico]

On Tue, Mar 8, 2016 at 5:51 AM, Jon Ribbens
 wrote:
> I must say that Python on Windows was a very poor experience indeed,
> "virtualenv" does not work and "venv" refuses to create the 'activate'
> shell script so does not work either (and pygame doesn't work, but
> that's presumably not Python's fault).

To be quite frank, Python on Windows has had a *lot* of problems, and
most of them have been the fault of Windows. Starting with Python 3.5,
there've been some big packaging changes that ought to make Windows
Python a bit easier to use; however, there've been some teething
troubles, and there are still some ways in which setting up a
properly-working Python is a pain.

So you have a few choices:

1) Stick with the vanilla Python on the vanilla Windows. It's not
horrendous, but there will be rough edges. Report those rough edges,
and hopefully they can be fixed in time for 3.6 (or even 3.5.2).

2) Use a different Python distribution, eg Anaconda or ActiveState.
Some of them cost money; when you pay money for open source software,
what you're getting is a level of support, which will mean you can
complain to them when something doesn't work, and insist that they fix
it. This may or may not give you better results than option 1.

3) Use standard Python, and ditch Windows. This is what I do. :)

4) Keep using Windows, but do your Python work in a virtual machine.
Either full-on virtualization software like VirtualBox or VMWare, or
something that gives you a web browser interface to a VM hosted in the
cloud (eg Cloud 9 or Nitrous).

Personally, I don't see any reason to run "real Windows" on any of my
systems. Windows programs get run under Wine or VirtualBox, never
directly on the hardware. But if your needs are different, there are
plenty of ways to improve your Windows+Python setup; just be patient,
and accept that you might have a bit of extra work to do.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[Andrew Farrell]

I'm going to echo Chris Angelo's suggestion #2 to use a python
distribution. This page  has the links
to download Anaconda. It is free and if you need to download libraries
which require compiled external code like numpy you can just run `conda
install numpy`.

On Mon, Mar 7, 2016 at 1:47 PM, Chris Angelico  wrote:

> On Tue, Mar 8, 2016 at 5:51 AM, Jon Ribbens
>  wrote:
> > I must say that Python on Windows was a very poor experience indeed,
> > "virtualenv" does not work and "venv" refuses to create the 'activate'
> > shell script so does not work either (and pygame doesn't work, but
> > that's presumably not Python's fault).
>
> To be quite frank, Python on Windows has had a *lot* of problems, and
> most of them have been the fault of Windows. Starting with Python 3.5,
> there've been some big packaging changes that ought to make Windows
> Python a bit easier to use; however, there've been some teething
> troubles, and there are still some ways in which setting up a
> properly-working Python is a pain.
>
> So you have a few choices:
>
> 1) Stick with the vanilla Python on the vanilla Windows. It's not
> horrendous, but there will be rough edges. Report those rough edges,
> and hopefully they can be fixed in time for 3.6 (or even 3.5.2).
>
> 2) Use a different Python distribution, eg Anaconda or ActiveState.
> Some of them cost money; when you pay money for open source software,
> what you're getting is a level of support, which will mean you can
> complain to them when something doesn't work, and insist that they fix
> it. This may or may not give you better results than option 1.
>
> 3) Use standard Python, and ditch Windows. This is what I do. :)
>
> 4) Keep using Windows, but do your Python work in a virtual machine.
> Either full-on virtualization software like VirtualBox or VMWare, or
> something that gives you a web browser interface to a VM hosted in the
> cloud (eg Cloud 9 or Nitrous).
>
> Personally, I don't see any reason to run "real Windows" on any of my
> systems. Windows programs get run under Wine or VirtualBox, never
> directly on the hardware. But if your needs are different, there are
> plenty of ways to improve your Windows+Python setup; just be patient,
> and accept that you might have a bit of extra work to do.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[MRAB]

On 2016-03-07 19:08, Mark Lawrence wrote:

On 07/03/2016 17:38, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff  wrote:

Thanks to all who replied to my cry for help; I understand it better now.
But:

On 07/03/16 16:08, Chris Angelico wrote:



The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.



Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.


You're looking at an ancient version of the docs. Here's the newest docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA



As I happen to be playing with tkinter I was searching literally minutes
ago for the way to put a grid onto a notebook tab.  Regrettably the
notebook grid is conspicious by its absence from the docs :(


It's in the tkinter.ttk docs:

https://docs.python.org/3/library/tkinter.ttk.html

--
https://mail.python.org/mailman/listinfo/python-list

fnmatch() vs. glob.glob()

[Jinghui Niu]

Hi, I've been studying python 3 modules. I'm a bit confused about the possibly 
overlap between fnmatch() and glob(), they seem to achieve the same goals 
exactly. Why duplicate?
Or maybe something I've missed?

Thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 19:10, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 5:34 AM, BartC  wrote:



def abortjpeg(mess):
 print ("Error:",mess)
 raise

Here's my Python:



RuntimeError: No active exception to reraise


OK I'll have a look. (Or maybe just change 'raise' to 'exit(0)'. It just 
needs to crash out at this point, but should be a bit more graceful than 
what you had.)



I disagree. The program does its job perfectly (you will have to take it
further to verify the results, such as writing out the .ppm file and viewing
the contents).

But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for a
smaller file). /This is in line with other observations./


What's your meaning of "perfectly"? You're implementing things in a
very C way, and then showing that two different Python interpreters
have different performance.


Two interpreters executing exactly the same code and exactly the same 
algorithm. And for a real task which deliberately doesn't just delegate 
to high-level features (otherwise you're just comparing libraries).


What can be more perfect for comparing two implementations?


You haven't actually shown how a
properly-written program would perform.


If I could find one that worked, I would have used that! (But I was more 
interested in 3 vs. PyPy than 2 vs. 3.)



Did you try the 'pillow' library?

https://pypi.python.org/pypi/Pillow

It supports 2.6/2.7 and 3.2+, and would be most people's "one obvious
way" to do things. At very least, it should get a mention in your
performance comparisons.


I don't understand. This is an external library that appears to be 
written in C. How does that help me compare the performance of two 
implementations of Python?


One could be ten times slower than the other (in executing its native 
bytecode), but if it spends 99% of its time an a C image library, how do 
you measure that?



(I'm quite pleased with my version: smaller, faster, works on all the
Pythons, supports all 3 colour formats and no decoding bugs that I'm aware
of, and it's the first Python program I've written that does something
useful.)


"all the Pythons" meaning what, exactly?


I means versions 2 and 3 of the language, tested on CPython versions 
2.5, 2.7, 3.1 and 3.4, as well as PyPy. The other decoders I tried were 
for 2.x.



And, no decoding bugs? Maybe,
but I wouldn't bet my life on it. Unless your code is a pure port of
someone else's (and unless that someone else could guarantee that the
original C code was bug free), I wouldn't trust code that I can't
completely analyze in one sitting. Virtually all code has bugs in it;
the only code that doesn't is code that's so trivially simple that you
can completely understand it in one "headspace".


Let's put it this way: I don't have to hand a jpeg file that it can't 
decode. When one turns up then I will fix that. (I can't say for sure 
there aren't any subtle chroma or quality problems, but there's nothing 
obvious.) I can't say that for the decoders I looked at.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Question

[justin walters]

Unfortunately, it's difficult for the core devs to know every hardware and
os combination there is. Maybe you could submit a bug report?
On Mar 7, 2016 10:56 AM, "Jon Ribbens"  wrote:

> On 2016-03-07, mm0fmf  wrote:
> > On 07/03/2016 18:09, Jon Ribbens wrote:
> >> It only appears to have downloads for 32-bit, or 64-bit AMD processors,
> >> not 64-bit Intel processors.
> >
> > You didn't read the bit that says
> >
> > "The binaries for AMD64 will also work on processors that implement the
> > Intel 64 architecture. (Also known as the "x64" architecture, and
> > formerly known as both "EM64T" and "x86-64".) "
>
> You are quite correct that I did not read the whole of the
> ridiculously-long and badly-organised download page.
>
> I would strongly suggest that, at the very least, the 'Description'
> for the x86-64 files should say "for all 64-bit PCs (except Itanium*)"
> with the '*' linking to a footnote that explains how to tell if you
> have one of those.
>
> I must say that Python on Windows was a very poor experience indeed,
> "virtualenv" does not work and "venv" refuses to create the 'activate'
> shell script so does not work either (and pygame doesn't work, but
> that's presumably not Python's fault).
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 7:19 AM, BartC  wrote:
>>> I disagree. The program does its job perfectly (you will have to take it
>>> further to verify the results, such as writing out the .ppm file and
>>> viewing
>>> the contents).
>>>
>>> But Py3 is slower doing this task than Py2 by 10 or 20% (or even 30% for
>>> a
>>> smaller file). /This is in line with other observations./
>>
>>
>> What's your meaning of "perfectly"? You're implementing things in a
>> very C way, and then showing that two different Python interpreters
>> have different performance.
>
>
> Two interpreters executing exactly the same code and exactly the same
> algorithm. And for a real task which deliberately doesn't just delegate to
> high-level features (otherwise you're just comparing libraries).
>
> What can be more perfect for comparing two implementations?

def valueOf(digit):
return ord(digit) - ord('0')

class Float:
def __init__(self, string):
self.value = 0
self.scale = 0
have_dot = 0
for i in range(len(string)):
if string[i] == '.': have_dot = 1
else:
self.value = self.value * 10 + valueOf(string[i])
if have_dot: self.scale += 1

rosuav@sikorsky:~$ python2 -m timeit -s 'from fp import Float'
'Float("1234.567")'
100 loops, best of 3: 1.84 usec per loop
rosuav@sikorsky:~$ python3 -m timeit -s 'from fp import Float'
'Float("1234.567")'
10 loops, best of 3: 2.76 usec per loop

Look! Creating a floating-point value is faster under Python 2 than
Python 3. What could be more perfect?

This is like a microbenchmark in that it doesn't tell you anything
about real-world usage. Your example also has the problem that it does
file I/O, which adds a ton of noise to your numbers. When you
reimplement a C-designed algorithm in Python, it's often NOT going to
be the best use of the language; and if you're comparing two poor uses
of a language, what do you prove by showing that one interpreter is
faster than another?

>> Did you try the 'pillow' library?
>>
>> https://pypi.python.org/pypi/Pillow
>>
>> It supports 2.6/2.7 and 3.2+, and would be most people's "one obvious
>> way" to do things. At very least, it should get a mention in your
>> performance comparisons.
>
>
> I don't understand. This is an external library that appears to be written
> in C. How does that help me compare the performance of two implementations
> of Python?

I'm not sure that it is written in C, but the point is that this is a
much better way to compare code. Use the language well, not badly.

> One could be ten times slower than the other (in executing its native
> bytecode), but if it spends 99% of its time an a C image library, how do you
> measure that?

You write *real world* code and then profile that. You get actual real
programs that you actually really use, and you run those through
timing harnesses. (Or throughput harnesses, which are pretty much the
same thing. With a web application, "performance" doesn't so much mean
"how long it takes to run", but "how many requests per second the
system can handle". Same thing, opposite way of measuring it.)

>>> (I'm quite pleased with my version: smaller, faster, works on all the
>>> Pythons, supports all 3 colour formats and no decoding bugs that I'm
>>> aware
>>> of, and it's the first Python program I've written that does something
>>> useful.)
>>
>>
>> "all the Pythons" meaning what, exactly?
>
>
> I means versions 2 and 3 of the language, tested on CPython versions 2.5,
> 2.7, 3.1 and 3.4, as well as PyPy. The other decoders I tried were for 2.x.

CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
going to differ in performance. And that's without even looking at
what core devs would refer to as "other Pythons", which would include
IronPython, Jython, PyPy (well, you got that, but you're treating it
as an afterthought), MicroPython, Brython, wpython, Nuitka,
Cython. these are *other Pythons*. What you're looking at is
closer to *all the versions of CPython*, but not even that, since
there are so many different ways that they can be built, and the
performance of array.array() is far from stable. It's not a core
language feature; you're using it because it's the most obvious
translation of your C algorithm, not because it's the best way to use
Python. So I say again, your measurement has little to no significance
to real-world code.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Ben Finney]

Jinghui Niu  writes:

> Hi, I've been studying python 3 modules. I'm a bit confused about the
> possibly overlap between fnmatch() and glob(), they seem to achieve
> the same goals exactly. Why duplicate?

From the module documentation:

Note that unlike fnmatch.fnmatch(), glob treats filenames beginning
with a dot (.) as special cases.

https://docs.python.org/2/library/glob.html>

So the goals are different: ‘glob.glob’ has the goal of matching closer
to the Unix meaning of glob patterns, where filenames starting with a
“.” character are conventionally treated as “hidden by default”.

-- 
 \  “We are stuck with technology when what we really want is just |
  `\ stuff that works.” —Douglas Adams |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Mark Lawrence]

On 07/03/2016 20:07, MRAB wrote:

On 2016-03-07 19:08, Mark Lawrence wrote:

On 07/03/2016 17:38, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 4:23 AM, Tony van der Hoff
 wrote:

Thanks to all who replied to my cry for help; I understand it better
now.
But:

On 07/03/16 16:08, Chris Angelico wrote:



The documentation should tell you what you need to import to make
something work. In this case, I would guess that "import
tkinter.messagebox" or "from tkinter import messagebox" would be the
recommended way to use this module.



Well, I found the tkinter documentation to be sparse in the extreme
(https://docs.python.org/release/3.1.3/library/tkinter.html), and it
certainly doesn't go into that sort of detail.


You're looking at an ancient version of the docs. Here's the newest
docs:

https://docs.python.org/3/library/tkinter.html

Up the top, it says to start with "import tkinter" or "from tkinter
import *", and then it lists some *other modules* (including
"tkinter.messagebox"). Obviously with the "turtle" module, you have to
import that separately (it's completely outside the tkinter
hierarchy); the same is true of the others.

Incidentally, this message is visible in the 3.1.3 docs that you
linked to, too. But I still recommend reading the current docs (unless
you're actually running your code on 3.1.3, in which case you really
REALLY should upgrade).

ChrisA



As I happen to be playing with tkinter I was searching literally minutes
ago for the way to put a grid onto a notebook tab.  Regrettably the
notebook grid is conspicious by its absence from the docs :(


It's in the tkinter.ttk docs:

https://docs.python.org/3/library/tkinter.ttk.html



https://docs.python.org/3/library/tkinter.ttk.html#tkinter.ttk.Notebook 
lists add, forget, hide, identify, index, insert, select, tab, tabs and

enable_traversal, but no grid, possibly others.  yet:-

>>> import tkinter.ttk
>>> help(tkinter.ttk.Notebook.grid)
Help on function grid_configure in module tkinter:

grid_configure(self, cnf={}, **kw)
Position a widget in the parent widget in a grid. Use as options:
column=number - use cell identified with given column (starting with 0)
columnspan=number - this widget will span several columns
in=master - use master to contain this widget
in_=master - see 'in' option description
ipadx=amount - add internal padding in x direction
ipady=amount - add internal padding in y direction
padx=amount - add padding in x direction
pady=amount - add padding in y direction
row=number - use cell identified with given row (starting with 0)
rowspan=number - this widget will span several rows
sticky=NSEW - if cell is larger on which sides will this
  widget stick to the cell boundary

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Struggeling with collections

[Roel Schroeven]

Faling Dutchman schreef op 2016-03-07 09:24:

class Item:
def __init__(self, id, productId, quantity, pageCount, files, option, 
metadata):
self.id = id
self.productId = productId
self.quantity = quantity
self.pageCount = pageCount
self.files = files
self.option = option
self.metadata = metadata

itm = Item(1,None,1,1,'asdf',{'asdf': 3, 'ads': 55},None)
print(itm)

it prints: <__main__.Item object at 0x02EBF3B0>


I'm not 100% sure exactly what you need, but namedtuple might be what 
you're looking for:


>>> import collections
>>> Item = collections.namedtuple('Item', 'id productId quantity 
pageCount files option metadata')

>>> itm = Item(1, None, 1, 1, 'asdf', {'asdf': 3, 'ads': 55}, None)
>>> print(itm)
Item(id=1, productId=None, quantity=1, pageCount=1, files='asdf', 
option={'ads': 55, 'asdf': 3}, metadata=None)


See 
https://docs.python.org/3/library/collections.html?highlight=namedtuple#collections.namedtuple



--
The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom.
  -- Isaac Asimov

Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: importing

[Terry Reedy]

On 3/7/2016 12:23 PM, Tony van der Hoff wrote:


However, more generally, how am I supposed to know that a module is part
of a package, and needs a "magic" stanza to get a module loaded?


The doc for a package usually mentions the submodules it contains, as in
https://docs.python.org/3/library/tkinter.html#tkinter-modules

The doc for a submodule has the package name in its title line.

25.4. tkinter.scrolledtext — Scrolled Text Widget
8.4. collections.abc — Abstract Base Classes for Containers

And yes, tkinter docs need considerable work.  That is why there are 
several external resources list in the intro.  I mostly use #3 Tkinter 
reference (not perfect) supplements by #2, #4, and #5.  Not mentioned 
there are the docstrings (also not perfect), accessible via interactive 
help.


>>> import tkinter as tk
>>> help(tk.Button.configure)
Help on function configure in module tkinter:

configure(self, cnf=None, **kw)
Configure resources of a widget.

The values for resources are specified as keyword
arguments. To get an overview about
the allowed keyword arguments call the method keys.

--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Terry Reedy]

On 3/7/2016 6:54 AM, Andrew Jaffe wrote:


Dumb question, and this probably isn't the place for it, but the three
Pythons being tested are:

  64-bit CPython on Li... latest in branch '2.7'
  64-bit CPython on Li... latest in branch '3.5'
  64-bit CPython on Li... latest


> I understand that the first two are the released 2.7 and 3.5
> versions; is the third the most recent 3.x build?

I am rather sure that none are released versions and that all three 
refer to the current tip ('latest') of each branch in the repository, 
with third being the default branch.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 20:47, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 7:19 AM, BartC  wrote:



What can be more perfect for comparing two implementations?



rosuav@sikorsky:~$ python2 -m timeit -s 'from fp import Float'
'Float("1234.567")'
100 loops, best of 3: 1.84 usec per loop
rosuav@sikorsky:~$ python3 -m timeit -s 'from fp import Float'
'Float("1234.567")'
10 loops, best of 3: 2.76 usec per loop

Look! Creating a floating-point value is faster under Python 2 than
Python 3. What could be more perfect?



This is like a microbenchmark in that it doesn't tell you anything
about real-world usage.


Microbenchmarks have their uses, when you are concentrating on a 
particular aspect of a language. But I tried your code, calling Float a 
million times, and 2.7 took 8.3 seconds; 3.4 took 10.5 seconds. But that 
is meaningless according to you.


(3.1 took 8.5 seconds. What happened between 3.1 and 3.4 to cause such a 
slow-down? Do you think it might be a good idea for /someone/ at least 
to take pay some attention to that, before it grinds to a halt 
completely by version 4.0?)


(I tried near-equivalent code on my byte-coded language. It took 0.7 
seconds. Then I tried a fairer test, as I use a bit of ASM to help the 
interpreter along: I created a C source version of the interpreter, 
compiled it with Tiny C, which generates some of the worst code around, 
and ran it again on the byte-code. It took 3 seconds; still faster than 
either of the Pythons!)


Anyway, is reading a jpeg not real world usage? It's a task normally 
done with a compiled language, but can also be a good test for an 
interpreted one.



You write *real world* code and then profile that. You get actual real
programs that you actually really use, and you run those through
timing harnesses.


Sorry, but that is useless. It tells you that slow, interpreted 
languages can be viable with certain kinds of usage. Everyone knows that 
(I've been creating applications which balance compiled code and 
interpreted code since about the mid-80s).


If you want an interpreted language to take on wider range of tasks, 
then you need to make it faster, and that means measuring how efficient 
it is at executing algorithms itself, not measuring how long some 
library in a different language takes to execute.



CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
going to differ in performance. And that's without even looking at
what core devs would refer to as "other Pythons", which would include
IronPython, Jython, PyPy (well, you got that, but you're treating it
as an afterthought), MicroPython, Brython, wpython, Nuitka,
Cython. these are *other Pythons*.


What are you suggesting here? That all these Pythons are going to be 
faster or slower than each other? I would guess that most of them are 
going to be roughly the same, other than PyPy. If there was a fast 
version, then I would have heard about it!



the
performance of array.array() is far from stable. It's not a core
language feature; you're using it because it's the most obvious
translation of your C algorithm,


I'm using it because this kind of file reading in Python is a mess. If I 
do a read, will I get a string, a byte sequence object, a byte-array, or 
array-array, or what?


Are you saying there array.array will not work on some implementations? 
In that case it's more of a mess than I thought!



not because it's the best way to use
Python. So I say again, your measurement has little to no significance
to real-world code.


Real world use of Python appears to be as a scripting language and for 
glue code, with most of the real work done in libraries implemented in 
native code. I acknowledge that. And I understand that the 10-20% 
difference between Py2 and Py3 will largely disappear for these kinds of 
uses.


But I also said I am interested in using the languages to directly 
implement programs and algorithms not just for scripting. Then you need 
to be able to measure what the core language is capable of.



--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Pythonic love

[Fillmore]

learning Python from Perl here. Want to do things as Pythonicly as possible.

I am reading a TSV, but need to skip the first 5 lines. The following 
works, but wonder if there's a more pythonc way to do things. Thanks


ctr = 0
with open(prfile,mode="rt",encoding='utf-8') as pfile:
for line in pfile:
ctr += 1

if ctr < 5:
continue

allVals = line.strip().split("\t")
print(allVals)
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Rob Gaddi]

Fillmore wrote:

>
> learning Python from Perl here. Want to do things as Pythonicly as possible.
>
> I am reading a TSV, but need to skip the first 5 lines. The following 
> works, but wonder if there's a more pythonc way to do things. Thanks
>
> ctr = 0
> with open(prfile,mode="rt",encoding='utf-8') as pfile:
>  for line in pfile:
>  ctr += 1
>
>  if ctr < 5:
>  continue
>
>  allVals = line.strip().split("\t")
>  print(allVals)

for lineno, line in enumerate(pfile, start=1):

will save you maintaining your own counter.  But other than that, nah,
that's pretty much the approach you're looking for.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[sohcahtoa82]

On Monday, March 7, 2016 at 2:51:50 PM UTC-8, Fillmore wrote:
> learning Python from Perl here. Want to do things as Pythonicly as possible.
> 
> I am reading a TSV, but need to skip the first 5 lines. The following 
> works, but wonder if there's a more pythonc way to do things. Thanks
> 
> ctr = 0
> with open(prfile,mode="rt",encoding='utf-8') as pfile:
>  for line in pfile:
>  ctr += 1
> 
>  if ctr < 5:
>  continue
> 
>  allVals = line.strip().split("\t")
>  print(allVals)

I'd read all the lines at once and then just slice the list.

with open(prfile, mode="rt", encoding="utf-8") as pfile:
lines = pfile.readlines()[5:]
for line in lines:
allVals = line.strip().split("\t")
print(allVals)

Obviously, this will only work well if your file is a reasonable size, as it 
will store the entire thing in memory at once.

On a side note, your "with open..." line uses inconsistent quoting.  You have 
"" on one string, but '' on another.
-- 
https://mail.python.org/mailman/listinfo/python-list

breaking out of outer loops

[Fillmore]

I must be missing something simple because I can't find a way to break 
out of a nested loop in Python.


Is there a way to label loops?

For the record, here's a Perl script of mine I'm trying to port...there 
may be 'malformed' lines in a TSV file I'm parsing that are better 
discarded than fixed.


my $ctr = 0;
OUTER:
while($line = ) {

$ctr++;
if ($ctr < 5) {next;}

my @allVals  = split /\t/,$line;

my $newline;
foreach my $i (0..$#allVals) {

if ($i == 0) {
if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}

$newline =  $allVals[0];
}

if (defined $headers{$i}) {

#if column not a number, skip line
if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}

$newline .= "\t".$allVals[$i+1];
}
}
print $newline."\n";

}
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Ian Kelly]

On Mon, Mar 7, 2016 at 3:51 PM, Fillmore  wrote:
>
> learning Python from Perl here. Want to do things as Pythonicly as possible.
>
> I am reading a TSV, but need to skip the first 5 lines. The following works,
> but wonder if there's a more pythonc way to do things. Thanks

I'd probably use itertools.islice.

from itertools import islice
for line in isiice(pfile, 5, None):
allVals = line.strip().split("\t")
print(allVals)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Fillmore]

On 3/7/2016 6:03 PM, sohcahto...@gmail.com wrote:


On a side note, your "with open..." line uses inconsistent quoting.

> You have "" on one string, but '' on another.

Thanks. I'll make sure I flog myself three times later tonight...



--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Ian Kelly]

On Mon, Mar 7, 2016 at 4:09 PM, Fillmore  wrote:
>
> I must be missing something simple because I can't find a way to break out
> of a nested loop in Python.
>
> Is there a way to label loops?

No, you can't break out of nested loops, apart from structuring your
code such that return does what you want.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pythonic love

[Mark Lawrence]

On 07/03/2016 22:51, Fillmore wrote:


learning Python from Perl here. Want to do things as Pythonicly as
possible.

I am reading a TSV, but need to skip the first 5 lines. The following
works, but wonder if there's a more pythonc way to do things. Thanks

ctr = 0
with open(prfile,mode="rt",encoding='utf-8') as pfile:
 for line in pfile:
 ctr += 1

 if ctr < 5:
 continue

 allVals = line.strip().split("\t")
 print(allVals)


Something like this, completely untested.

with open(prfile,mode="rt",encoding='utf-8') as pfile:
lineIter = iter(pfile)
for _ in range(5):
next(lineIter)
for line in lineIter:
 allVals = line.strip().split("\t")
 print(allVals)

I'd also recommend that you use the csv module 
https://docs.python.org/3/library/csv.html which can also cope with tsv 
files, it's far more reliable than relying on a simple call to split().


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 6:17 PM, Ian Kelly wrote:

On Mon, Mar 7, 2016 at 4:09 PM, Fillmore  wrote:


I must be missing something simple because I can't find a way to break out
of a nested loop in Python.

Is there a way to label loops?


No, you can't break out of nested loops,


wow...this is a bit of a WTF moment to me :(


apart from structuring your
code such that return does what you want.



Can you elaborate? apologies, but I'm new to python and trying to find 
my way out of perl


--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Rob Gaddi]

Fillmore wrote:

>
> I must be missing something simple because I can't find a way to break 
> out of a nested loop in Python.
>
> Is there a way to label loops?
>
> For the record, here's a Perl script of mine I'm trying to port...there 
> may be 'malformed' lines in a TSV file I'm parsing that are better 
> discarded than fixed.
>
> my $ctr = 0;
> OUTER:
> while($line = ) {
>
>  $ctr++;
>  if ($ctr < 5) {next;}
>
>  my @allVals  = split /\t/,$line;
>
>  my $newline;
>  foreach my $i (0..$#allVals) {
>
>   if ($i == 0) {
>   if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}
>
>   $newline =  $allVals[0];
>   }
>
>   if (defined $headers{$i}) {
>
>   #if column not a number, skip line
>   if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}
>
>   $newline .= "\t".$allVals[$i+1];
>   }
>  }
>  print $newline."\n";
>
> }

You're used to Perl, you're used to exceptions being A Thing.  This is
Python, and exceptions are just another means of flow control.

class MalformedLineError(Exception): pass

for line in file:
try:
for part in line.split('\t'):
if thispartisbadforsomereason:
raise MalformedLineError()
otherwisewedothestuff
except MalformedLineError:
pass

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 9:39 AM, BartC  wrote:
>
> What are you suggesting here? That all these Pythons are going to be faster
> or slower than each other? I would guess that most of them are going to be
> roughly the same, other than PyPy. If there was a fast version, then I would
> have heard about it!

Try them, then you'll know. They all have different strengths and weaknesses.

>> the
>> performance of array.array() is far from stable. It's not a core
>> language feature; you're using it because it's the most obvious
>> translation of your C algorithm,
>
> I'm using it because this kind of file reading in Python is a mess. If I do
> a read, will I get a string, a byte sequence object, a byte-array, or
> array-array, or what?

Uhh you'll get either a text string or a byte string, depending on
whether you opened the file as text or binary. Where's the mess?

> Are you saying there array.array will not work on some implementations? In
> that case it's more of a mess than I thought!

No, it'll always be there (at least, I'm pretty sure it will). I said
that its *performance* is all over the place. Try coding in the
simplest possible way first, and only turn to array.array afterward.

> Real world use of Python appears to be as a scripting language and for glue
> code, with most of the real work done in libraries implemented in native
> code. I acknowledge that. And I understand that the 10-20% difference
> between Py2 and Py3 will largely disappear for these kinds of uses.

Real world use of Python includes writing full-scale applications in
it... not sure whether the myriad web apps written with
Flask/Django/etc count as "most of the real work done in native
libraries". But what does "real work" mean? When you write a data
processing program in Python, you could spend most of your time doing
integer arithmetic using the native 'int' type, or you could spend
most of your time using numpy. One of those is using a third-party
library written in Fortran; the other is using the language facilities
directly. But in CPython, the latter means you're using C. Where's the
difference? All your most important code - all the "real work" in
terms of algorithms etc - is in Python. The underlying facilities are
completely generic.

> But I also said I am interested in using the languages to directly implement
> programs and algorithms not just for scripting. Then you need to be able to
> measure what the core language is capable of.

Yeah, because the value of human legs is proven by walking from one
city to the next and seeing how long it takes. You're still using the
language inefficiently, and then proving that one inefficient use is
less inefficient than another. Congratulations.

I'm done.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Kaynor]

On Mon, Mar 7, 2016 at 3:21 PM, Fillmore 
wrote:

> On 3/7/2016 6:17 PM, Ian Kelly wrote:
>
>> On Mon, Mar 7, 2016 at 4:09 PM, Fillmore 
>> wrote:
>>
>>>
>>> I must be missing something simple because I can't find a way to break
>>> out
>>> of a nested loop in Python.
>>>
>>> Is there a way to label loops?
>>>
>>
>> No, you can't break out of nested loops,
>>
>
> wow...this is a bit of a WTF moment to me :(
>
> apart from structuring your
>> code such that return does what you want.
>>
>>
> Can you elaborate? apologies, but I'm new to python and trying to find my
> way out of perl


The normal way of breaking out of some set of loops would be to structure
your code so that you have a function, and when you want to break out of
some of multiple loops, you can just call return.


As a very rough example:

def processFile(file):
for line in file:
section = line.split()
for section in line:
if sectionCorrupt:
return # Stops processing the entire file, by exiting the
function, but keeps processing the list of files.

for file in files:
processFile(file)




Alternatively, as you mention corrupted lines, perhaps raising an exception
would be the best approach.

And the same rough example, using an exception without a function:

for file in files:
try:
for line in file:
section = line.split()
for section in line:
if sectionCorrupt:
raise Exception('Section corrupt') # You probably want
more details here, for possible manual repair. You could also have a custom
exception class for more control.
except Exception as e:
print('Failed to process the file {} with error {}.'.format(file,
str(e))) # Probably should also print the entire traceback to aid in
repairing errors, especially if they are due to a bug in the code, but this
is the rough idea.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 6:29 PM, Rob Gaddi wrote:


You're used to Perl, you're used to exceptions being A Thing.  This is
Python, and exceptions are just another means of flow control.

class MalformedLineError(Exception): pass

for line in file:
 try:
 for part in line.split('\t'):
 if thispartisbadforsomereason:
 raise MalformedLineError()
 otherwisewedothestuff
 except MalformedLineError:
 pass



I am sure you are right, but adapting this thing here into something 
that is a fix to my problem seems like abusing my 'system 2' (for those 
who read a certain book by a guy called Daniel Kanheman :)


--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Angelico]

On Tue, Mar 8, 2016 at 10:09 AM, Fillmore  wrote:
> For the record, here's a Perl script of mine I'm trying to port...there may
> be 'malformed' lines in a TSV file I'm parsing that are better discarded
> than fixed.
>
> my $ctr = 0;
> OUTER:
> while($line = ) {
>
> $ctr++;
> if ($ctr < 5) {next;}
>
> my @allVals  = split /\t/,$line;
>
> my $newline;
> foreach my $i (0..$#allVals) {
>
> if ($i == 0) {
> if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}
>
> $newline =  $allVals[0];
> }
>
> if (defined $headers{$i}) {
>
> #if column not a number, skip line
> if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}
>
> $newline .= "\t".$allVals[$i+1];
> }
> }
> print $newline."\n";
>
> }

I'm not too fluent in Perl, but my understanding here is that "next
OUTER" is equivalent to Python's 'continue' statement, right? So your
flow control is roughly this:

for _ in range(5): skip header row
for line in file:
for cell in split(line):
if bad_cell_value:
break # Skip out of the inner loop now
else:
print_stuff # We didn't break

Does that look right? The key here is that a 'for' loop has an 'else'
clause, which happens if there's no 'break' in the main loop. I think
that's what you're after, here; you get to break out of "the loop and
a little bit more", so to speak. The outer loop is actually immaterial
here.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Angelico]

On Tue, Mar 8, 2016 at 10:42 AM, Chris Kaynor  wrote:
> And the same rough example, using an exception without a function:
>
> for file in files:
> try:
> for line in file:
> section = line.split()
> for section in line:
> if sectionCorrupt:
> raise Exception('Section corrupt') # You probably want
> more details here, for possible manual repair. You could also have a custom
> exception class for more control.
> except Exception as e:
> print('Failed to process the file {} with error {}.'.format(file,
> str(e))) # Probably should also print the entire traceback to aid in
> repairing errors, especially if they are due to a bug in the code, but this
> is the rough idea.

Yes, although I would more strongly suggest the custom exception
class. In fact, the way I'd word it is: Never raise Exception, always
a subclass. Otherwise it's too easy to accidentally catch something
elsewhere in the code (a ValueError or TypeError or even
AttributeError).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 6:09 PM, Fillmore wrote:


I must be missing something simple because I can't find a way to break
out of a nested loop in Python.


Thanks to everyone who has tried to help so far. I suspect this may be a 
case where I just need to get my head around a new paradigm




--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Ben Finney]

BartC  writes:

> On 07/03/2016 20:47, Chris Angelico wrote:
> > Look! Creating a floating-point value is faster under Python 2 than
> > Python 3. What could be more perfect?
>
> > This is like a microbenchmark in that it doesn't tell you anything
> > about real-world usage.
>
> Microbenchmarks have their uses, when you are concentrating on a
> particular aspect of a language.

The “micro” is not referring to the focus of the operation; that's a
good thing to do and is a way to produce useful, salient results. Choose
a specific aspect and design a test case which focusses as narrowly as
possible on that aspect. Then you can gather data to make meaningful
statements about that aspect in isolation from others.

What is “micro” about the benchmark you showed is that it doesn't gather
enough data to be useful. The test should not just do one statement; it
should do some relevant complete unit of the algorithm. Focussed, but
interesting: a huge amount of data, or a complex data conversion, or
something else that qualifies as “real”.

> Anyway, is reading a jpeg not real world usage? It's a task normally
> done with a compiled language, but can also be a good test for an
> interpreted one.

Try testing on a large battery of highly varied JPEG images, and getting
the aggregate results from that. Or something that is more interesting
(i.e. relevant to the real usage) than one image.

> If you want an interpreted language to take on wider range of tasks,
> then you need to make it faster, and that means measuring how
> efficient it is at executing algorithms itself, not measuring how long
> some library in a different language takes to execute.

That's quite true. The trick is to avoid being *so* focussed that you
are mostly measuring a corner case, instead of a relevant use case.

> > CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
> > going to differ in performance. And that's without even looking at
> > what core devs would refer to as "other Pythons", which would
> > include IronPython, Jython, PyPy (well, you got that, but you're
> > treating it as an afterthought), MicroPython, Brython, wpython,
> > Nuitka, Cython. these are *other Pythons*.
>
> What are you suggesting here? That all these Pythons are going to be
> faster or slower than each other?

At specific, highly-focussed operations? Of course. That is going to be
true almost by definition: the implementations are different, so if your
test cases are extremely focussed they will be much more likely to find
variations in the implementations — differences that are not relevant to
which version of the Python source you're coming from.

> I would guess that most of them are going to be roughly the same,
> other than PyPy.

That's the kind of guess which should be subjected to testing. We humans
are *very* bad at estimating those kinds of behaviour in implementation.
This is especially true because we humans also demand baroque,
highly-complex implementations in order to get our speed improvements.

And those complexities of implementation will be a major factor
thwarting our intuitions about how those implementations will perform.
If you think you know how an implementation will perform, the data
indicates you should not trust that instinct.

> If there was a fast version, then I would have heard about it!

The mistake, I think, is in assuming a “fast version” means anything.

Different implementations will be faster at some things, slower at other
things, where “things” can be a very lumpy and convoluted landscape of
complex use cases. Test those assumptions with realistic benchmarks run
under each implementation.

-- 
 \  “When [science] permits us to see the far side of some new |
  `\  horizon, we remember those who prepared the way – seeing for |
_o__)  them also.” —Carl Sagan, _Cosmos_, 1980 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Chris Angelico]

On Tue, Mar 8, 2016 at 11:02 AM, Fillmore  wrote:
> On 3/7/2016 6:09 PM, Fillmore wrote:
>>
>>
>> I must be missing something simple because I can't find a way to break
>> out of a nested loop in Python.
>
>
> Thanks to everyone who has tried to help so far. I suspect this may be a
> case where I just need to get my head around a new paradigm

Yep, which is why we're offering a variety of new paradigms. Because
it's ever so much easier to get your head around three than one!

We are SO helpful, guys. So helpful. :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

What will I get when reading from a file? (was: Pyhon 2.x or 3.x, which is faster?)

[Ben Finney]

BartC  writes:

> I'm using [a third-party library function] because this kind of file
> reading in Python is a mess. If I do a read, will I get a string, a
> byte sequence object, a byte-array, or array-array, or what?

That's a strange statement; there is virtually no mystery in what you'll
get from a Python 3 file object, because you're eessentially forced to
declare explicitly what kind of results you want from it.

In fact, this no-mystery is fairly special to Python. Most other
languages will guess aspects of the file and won't force the programmer
to be explicit; if anything, newcomers are frustrated at how *little*
guesswork (i.e. how much explicit declaration) is involved in opening a
Python 3 file.

If you want bytes, you open the file in that mode. If you want text, you
must instead open it with that mode and tell it what text codec to use.

And so on. Where are you seeing mysterious results?

-- 
 \  “Any intelligent fool can make things bigger and more complex… |
  `\It takes a touch of genius – and a lot of courage – to move in |
_o__)the opposite direction.” —Albert Einstein |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Fillmore]

On 3/7/2016 7:08 PM, Chris Angelico wrote:



Yep, which is why we're offering a variety of new paradigms. Because
it's ever so much easier to get your head around three than one!

We are SO helpful, guys. So helpful. :)


not too dissimilarly from human languages, speaking a foreign language 
is more often than not a matter of learning to think differently...


--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Joel Goldstick]

On Mon, Mar 7, 2016 at 7:13 PM, Fillmore 
wrote:

> On 3/7/2016 7:08 PM, Chris Angelico wrote:
>
>
>> Yep, which is why we're offering a variety of new paradigms. Because
>> it's ever so much easier to get your head around three than one!
>>
>> We are SO helpful, guys. So helpful. :)
>>
>
> not too dissimilarly from human languages, speaking a foreign language is
> more often than not a matter of learning to think differently...
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

I found this on break and continue.  I don't know perl, but if what you
want to do is quit continuing in a loop and go to the next iteration, then
continue will do that.
http://www.tutorialspoint.com/python/python_loop_control.htm

-- 
Joel Goldstick
http://joelgoldstick.com/ 
http://cc-baseballstats.info/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 07/03/2016 23:40, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 9:39 AM, BartC  wrote:



I'm using it because this kind of file reading in Python is a mess. If I do
a read, will I get a string, a byte sequence object, a byte-array, or
array-array, or what?


Uhh you'll get either a text string or a byte string, depending on
whether you opened the file as text or binary. Where's the mess?


(Is a byte string the same as a byte array? Is a byte array the same as 
an array.array? If I remove this line from my code, where 'data' has 
just been read from a file:


   data=array.array('B',data)

then it still works - Python 3. But not on Python 2. If I do .read on a 
binary file I get a byte string in Python 3, but a string in Python 2. 
That sort of mess.


And how do I write that deceptively simple header on the output file 
without array.array because I tried all sorts:


   f = open(file+".ppm", "wb")
   s="P6\n%d %d\n255\n" % (hdr.width, hdr.height)
   sbytes=array.array('B',list(map(ord,s)))
   f.write(sbytes)

Note this file mixes text and binary mode - blame the ppm format - but 
is mostly binary.)



But I also said I am interested in using the languages to directly implement
programs and algorithms not just for scripting. Then you need to be able to
measure what the core language is capable of.


Yeah, because the value of human legs is proven by walking from one
city to the next and seeing how long it takes. You're still using the
language inefficiently, and then proving that one inefficient use is
less inefficient than another. Congratulations.


So whoever created the speed comparison site linked to at the start of 
the thread is wasting their time?


Obviously some people are interested in this stuff, even if you're not.

The slow speed of Python (and a number of other scripting languages) is 
well known. But obviously that is not any sort of issue, and these 
people working flat out on tracing JIT compilers are all wasting their 
time too!


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: breaking out of outer loops

[Mark Lawrence]

On 07/03/2016 23:09, Fillmore wrote:


I must be missing something simple because I can't find a way to break
out of a nested loop in Python.

Is there a way to label loops?

For the record, here's a Perl script of mine I'm trying to port...there
may be 'malformed' lines in a TSV file I'm parsing that are better
discarded than fixed.

my $ctr = 0;
OUTER:
while($line = ) {

 $ctr++;
 if ($ctr < 5) {next;}

 my @allVals  = split /\t/,$line;

 my $newline;
 foreach my $i (0..$#allVals) {

 if ($i == 0) {
 if ($allVals[0] =~ /[^[:print:]]/) {next OUTER;}

 $newline =  $allVals[0];
 }

 if (defined $headers{$i}) {

 #if column not a number, skip line
 if ($allVals[$i+1] !~ /^\d+$/) {next OUTER;}

 $newline .= "\t".$allVals[$i+1];
 }
 }
 print $newline."\n";

}


I suggest you read and digest the various responses here 
http://stackoverflow.com/questions/189645/how-to-break-out-of-multiple-loops-in-python, 
some of which are similar if not identical to answers you've all ready 
been given.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 08/03/2016 00:22, BartC wrote:



The slow speed of Python (and a number of other scripting languages) is
well known. But obviously that is not any sort of issue, and these
people working flat out on tracing JIT compilers are all wasting their
time too!



The slow speed of Python hasn't stopped it gaining a huge following in 
just about every possible area of computing.  Who cares about speed when 
you're waiting for the file, database or network to respond?  Sure, some 
people want things faster, but I suspect that 99% couldn't care less.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Chris Angelico]

On Tue, Mar 8, 2016 at 11:22 AM, BartC  wrote:
>
> (Is a byte string the same as a byte array? Is a byte array the same as an
> array.array? If I remove this line from my code, where 'data' has just been
> read from a file:
>
>data=array.array('B',data)
>
> then it still works - Python 3. But not on Python 2. If I do .read on a
> binary file I get a byte string in Python 3, but a string in Python 2. That
> sort of mess.

The default string in Py2 *is* a byte string.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[MRAB]

On 2016-03-08 00:22, BartC wrote:

On 07/03/2016 23:40, Chris Angelico wrote:

On Tue, Mar 8, 2016 at 9:39 AM, BartC  wrote:



I'm using it because this kind of file reading in Python is a mess. If I do
a read, will I get a string, a byte sequence object, a byte-array, or
array-array, or what?


Uhh you'll get either a text string or a byte string, depending on
whether you opened the file as text or binary. Where's the mess?


(Is a byte string the same as a byte array? Is a byte array the same as
an array.array? If I remove this line from my code, where 'data' has
just been read from a file:

 data=array.array('B',data)

then it still works - Python 3. But not on Python 2. If I do .read on a
binary file I get a byte string in Python 3, but a string in Python 2.
That sort of mess.

In Python 2, an unmarked string literal _is_ a bytestring; in Python 3, 
an unmarked string literal is Unicode.





And how do I write that deceptively simple header on the output file
without array.array because I tried all sorts:

 f = open(file+".ppm", "wb")
 s="P6\n%d %d\n255\n" % (hdr.width, hdr.height)
 sbytes=array.array('B',list(map(ord,s)))
 f.write(sbytes)


You don't need to use array.array:

sbytes = bytes(map(ord, s))

In Python 3.5, you can use '%' with a bytestring:

s = b"P6\n%d %d\n255\n" % (hdr.width, hdr.height)

[snip]

--
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Jinghui Niu]

On Monday, March 7, 2016 at 1:37:45 PM UTC-8, Ben Finney wrote:
> Jinghui Niu  writes:
> 
> > Hi, I've been studying python 3 modules. I'm a bit confused about the
> > possibly overlap between fnmatch() and glob(), they seem to achieve
> > the same goals exactly. Why duplicate?
> 
> >From the module documentation:
> 
> Note that unlike fnmatch.fnmatch(), glob treats filenames beginning
> with a dot (.) as special cases.
> 
> https://docs.python.org/2/library/glob.html>
> 
> So the goals are different: 'glob.glob' has the goal of matching closer
> to the Unix meaning of glob patterns, where filenames starting with a
> "." character are conventionally treated as "hidden by default".
> 
> -- 
>  \  "We are stuck with technology when what we really want is just |
>   `\ stuff that works." --Douglas Adams |
> _o__)  |
> Ben Finney

Thank you for your reply. So if for a beginner learner who doesn't care so much 
about Unix-compliant, which one is a better choice for learning purpose?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Ben Finney]

Jinghui Niu  writes:

> Thank you for your reply. So if for a beginner learner who doesn't
> care so much about Unix-compliant, which one is a better choice for
> learning purpose?

Learn to care about the difference :-) They are both useful and both do
different things.

-- 
 \ “I must say that I find television very educational. The minute |
  `\   somebody turns it on, I go to the library and read a book.” |
_o__)—Groucho Marx |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: fnmatch() vs. glob.glob()

[Jinghui Niu]

On Monday, March 7, 2016 at 5:00:57 PM UTC-8, Ben Finney wrote:
> Jinghui Niu  writes:
> 
> > Thank you for your reply. So if for a beginner learner who doesn't
> > care so much about Unix-compliant, which one is a better choice for
> > learning purpose?
> 
> Learn to care about the difference :-) They are both useful and both do
> different things.
> 
> -- 
>  \ "I must say that I find television very educational. The minute |
>   `\   somebody turns it on, I go to the library and read a book." |
> _o__)--Groucho Marx |
> Ben Finney

May I take this opportunity to ask a bold question for a beginner: if I want to 
mimic Sublime Text's fuzzy search for a file search in given directories, which 
module of these two would be more suitable? Could you shed some insights? 
Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[BartC]

On 08/03/2016 00:05, Ben Finney wrote:

BartC  writes:


On 07/03/2016 20:47, Chris Angelico wrote:

Look! Creating a floating-point value is faster under Python 2 than
Python 3. What could be more perfect?



This is like a microbenchmark in that it doesn't tell you anything
about real-world usage.


Microbenchmarks have their uses, when you are concentrating on a
particular aspect of a language.


The “micro” is not referring to the focus of the operation; that's a
good thing to do and is a way to produce useful, salient results. Choose
a specific aspect and design a test case which focusses as narrowly as
possible on that aspect. Then you can gather data to make meaningful
statements about that aspect in isolation from others.

What is “micro” about the benchmark you showed is that it doesn't gather
enough data to be useful. The test should not just do one statement; it
should do some relevant complete unit of the algorithm. Focussed, but
interesting: a huge amount of data, or a complex data conversion, or
something else that qualifies as “real”.


What's the purpose of the measurement? In my case, I'm not just 
measuring, but also implementing: either developing a program (the 
decoder for example) and trying to get it fast, and/or using it to tweak 
some interpreter or other I'm working on. Then I might well want to 
focus on particular single byte-code or some narrow aspect of the program.



Anyway, is reading a jpeg not real world usage? It's a task normally
done with a compiled language, but can also be a good test for an
interpreted one.


Try testing on a large battery of highly varied JPEG images, and getting
the aggregate results from that. Or something that is more interesting
(i.e. relevant to the real usage) than one image.


If your efforts manage to double the speed of reading file A, then 
probably the reading file B is also going to be improved! In practice 
you use a variety of files, but one at a time will do.



If there was a fast version, then I would have heard about it!


The mistake, I think, is in assuming a “fast version” means anything.


Yes of course it does. As does 'being slow'. Take another microbenchmark:

def whiletest():
|   i=0
|   while i<=1:
|   |   i+=1

whiletest()

Python 2.7:  8.4 seconds
Python 3.1: 12.5 seconds
Python 3.4: 18.0 seconds

Even if you don't care about speed, you must admit that there appears to 
be something peculiar going on here: why would 3.4 take more than twice 
as long as 2.7? What do they keep doing to 3.x to cripple it on each new 
version?


(For comparison, my interpreter managed 0.4s, PyPy 0.3s (it's hot on 
loops) and part-optimised C might be 0.07s (fully-optimised would be 0s).)


So you might say that a 'fast' version wouldn't give silly timings like 
the above.


Trying something like the silly loop above on a new language can be 
quite revealing.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pyhon 2.x or 3.x, which is faster?

[Mark Lawrence]

On 08/03/2016 01:00, BartC wrote:


If your efforts manage to double the speed of reading file A, then
probably the reading file B is also going to be improved! In practice
you use a variety of files, but one at a time will do.



What is the difference in your timing when you first read the file, and 
then read it a second time when it's been cached by the OS?  In other 
words, you are probably measuring more of the response time of the disk 
than the code that does the reading, hence making your figures useless.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list