Re: [Python-Dev] cpython: remove duplicate -I

[Ned Deily]

In article ,
 benjamin.peterson  wrote:

> http://hg.python.org/cpython/rev/0d0ca6f95d9d
> changeset:   70879:0d0ca6f95d9d
> user:Benjamin Peterson 
> date:Sun Jun 19 17:17:30 2011 -0500
> summary:
>   remove duplicate -I
> 
> files:
>   Makefile.pre.in |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Makefile.pre.in b/Makefile.pre.in
> --- a/Makefile.pre.in
> +++ b/Makefile.pre.in
> @@ -73,7 +73,7 @@
>  # Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py 
>  to
>  # be able to build extension modules using the directories specified in the
>  # environment variables
> -PY_CPPFLAGS= -I. -IInclude -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) 
> $(CPPFLAGS)
> +PY_CPPFLAGS= -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
>  PY_LDFLAGS=  $(CONFIGURE_LDFLAGS) $(LDFLAGS)
>  NO_AS_NEEDED=@NO_AS_NEEDED@
>  LDLAST=  @LDLAST@

I think someone has already mentioned that this change breaks builds 
with a source directory outside the build directory and that breaks my 
OS X installer builds.   Please revert or make the following change 
which works for me (but not tested elsewhere):

-PY_CPPFLAGS=   -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
+PY_CPPFLAGS=   -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snow Leopard buildbot failing again...

[Antoine Pitrou]

On Mon, 27 Jun 2011 10:01:13 PDT
Bill Janssen  wrote:
> I see that parc-snowleopard-1 went down again.  I've done a software
> update, rebooted, and installed the latest buildslave, 0.8.4.  I can
> ping dinsdale.python.org successfully from the machine.  However, when I
> start the buildslave, I get this:

Well, doesn't the slave work correctly now? Or is there a problem?



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Fred Drake]

On Tue, Jun 28, 2011 at 2:33 AM, Nick Coghlan  wrote:
> The two terms I've got out of this thread are "callable attributes"
> (instance/static/class methods, etc) and "data attributes" (everything
> else). Both seem reasonable to me, creating two largely disjoint sets
> that together cover all the different kinds of attribute you're likely
> to encounter.

But "callable attributes" aren't the same thing as methods; most are methods,
but not all.  Sometimes, they're data used by the object.  The fact that
data attributes can be callable is irrelevant.


  -Fred

-- 
Fred L. Drake, Jr.    
"Give me the luxuries of life and I will willingly do without the necessities."
   --Frank Lloyd Wright
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 11:44, Fred Drake wrote:

On Tue, Jun 28, 2011 at 2:33 AM, Nick Coghlan  wrote:

The two terms I've got out of this thread are "callable attributes"
(instance/static/class methods, etc) and "data attributes" (everything
else). Both seem reasonable to me, creating two largely disjoint sets
that together cover all the different kinds of attribute you're likely
to encounter.

But "callable attributes" aren't the same thing as methods; most are methods,
but not all.  Sometimes, they're data used by the object.  The fact that
data attributes can be callable is irrelevant.


Added to which there are other descriptors, notably property, that are 
not directly callable but are not provided as normal "data attributes" 
(although the access syntax is the same). Properties are much closer to 
methods as they are implemented on the class and fetched via the 
descriptor protocol. Instead of "data attributes" I prefer the term 
"instance attributes" although that doesn't include "class attributes" 
(or more precisely it doesn't cover "class attributes that aren't 
descriptors").


The problem with "data attributes" is that it doesn't mean *anything*, 
which I suppose is useful for invented terminology, but it means it 
doesn't convey anything precise to those who haven't heard the term 
before. If it becomes widely used then that changes I guess. I'd still 
normally just use "attributes" though...


All the best,

Michael Foord


   -Fred




--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Fred Drake]

On Tue, Jun 28, 2011 at 6:54 AM, Michael Foord
 wrote:
> Added to which there are other descriptors, notably property, that are not
> directly callable but are not provided as normal "data attributes" (although
> the access syntax is the same). Properties are much closer to methods as
> they are implemented on the class and fetched via the descriptor protocol.
> Instead of "data attributes" I prefer the term "instance attributes"
> although that doesn't include "class attributes" (or more precisely it
> doesn't cover "class attributes that aren't descriptors").

Given the availability of __getattr__ and __getattribute__, I consider
properties an implementation detail for some attributes.  The fact that
Python code is called on access is only marginally interesting.

> The problem with "data attributes" is that it doesn't mean *anything*, which
> I suppose is useful for invented terminology, but it means it doesn't convey
> anything precise to those who haven't heard the term before. If it becomes
> widely used then that changes I guess. I'd still normally just use
> "attributes" though...

I'd read "data attributes" the same as "non-method attributes".  For readers,
calling them "attributes" is typically sufficient.  It's rare to need to
distinguish them from methods.


  -Fred

-- 
Fred L. Drake, Jr.    
"Give me the luxuries of life and I will willingly do without the necessities."
   --Frank Lloyd Wright
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 12:04, Fred Drake wrote:

On Tue, Jun 28, 2011 at 6:54 AM, Michael Foord
  wrote:

Added to which there are other descriptors, notably property, that are not
directly callable but are not provided as normal "data attributes" (although
the access syntax is the same). Properties are much closer to methods as
they are implemented on the class and fetched via the descriptor protocol.
Instead of "data attributes" I prefer the term "instance attributes"
although that doesn't include "class attributes" (or more precisely it
doesn't cover "class attributes that aren't descriptors").

Given the availability of __getattr__ and __getattribute__, I consider
properties an implementation detail for some attributes.  The fact that
Python code is called on access is only marginally interesting.



Well, they're *usually* implemented as methods and backed by a real 
instance attribute. Usually (but not always) it makes more sense (IME) 
to group them with methods. The fact that they're *accessed* as an 
attribute is the uninteresting detail.


__getattr__ and __getattribute__ are interesting - they allow you to use 
attribute access syntax for things that *aren't* attributes.


I appreciate the fact that the Python data-model means methods are just 
object attributes, but they're not *instance attributes* and sometimes 
you need to make a distinction. (And yes it is helpful if the standard 
terminology leads people into a better understanding of the Python data 
model, but that still doesn't change the need on occasion for 
terminology that doesn't need to be explained whenever it is used.)


Given that how even methods are to be described depends on the context 
(if you're fetching bound methods as objects then it makes perfect sense 
to just talk about them as attributes) it doesn't seem an area amenable 
to one-size-fits-all terminology.



The problem with "data attributes" is that it doesn't mean *anything*, which
I suppose is useful for invented terminology, but it means it doesn't convey
anything precise to those who haven't heard the term before. If it becomes
widely used then that changes I guess. I'd still normally just use
"attributes" though...

I'd read "data attributes" the same as "non-method attributes".  For readers,
calling them "attributes" is typically sufficient.  It's rare to need to
distinguish them from methods.


Yeah, this is all a grand bikeshed. I'm not sure I would understand 
"data attributes" unless it was clear from the context. I would wonder 
what qualifies something as "data".


It is an interesting question what terminology we should use in the 
documentation if we need to distinguish them, but I think that is still 
wandering away from the original question that was posed.


All the best,

Michael



   -Fred




--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[R. David Murray]

On Tue, 28 Jun 2011 11:54:39 +0100, Michael Foord  
wrote:
> On 28/06/2011 11:44, Fred Drake wrote:
> > But "callable attributes" aren't the same thing as methods; most are 
> > methods,
> > but not all.  Sometimes, they're data used by the object.  The fact that
> > data attributes can be callable is irrelevant.
> 
> Added to which there are other descriptors, notably property, that are 
> not directly callable but are not provided as normal "data attributes" 
> (although the access syntax is the same). Properties are much closer to 
> methods as they are implemented on the class and fetched via the 
> descriptor protocol. Instead of "data attributes" I prefer the term 
> "instance attributes" although that doesn't include "class attributes" 
> (or more precisely it doesn't cover "class attributes that aren't 
> descriptors").

Also, instances can have methods as instance attributes.

Trying to use 'instance attributes' for non-method attributes is a bad
idea, I think.

Given that there is no one thing that covers all non-method
attributes, I suspect 'non-method attributes' is as good as
we're going to manage.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 12:51, R. David Murray wrote:

On Tue, 28 Jun 2011 11:54:39 +0100, Michael Foord  
wrote:

On 28/06/2011 11:44, Fred Drake wrote:

But "callable attributes" aren't the same thing as methods; most are methods,
but not all.  Sometimes, they're data used by the object.  The fact that
data attributes can be callable is irrelevant.

Added to which there are other descriptors, notably property, that are
not directly callable but are not provided as normal "data attributes"
(although the access syntax is the same). Properties are much closer to
methods as they are implemented on the class and fetched via the
descriptor protocol. Instead of "data attributes" I prefer the term
"instance attributes" although that doesn't include "class attributes"
(or more precisely it doesn't cover "class attributes that aren't
descriptors").

Also, instances can have methods as instance attributes.

Trying to use 'instance attributes' for non-method attributes is a bad
idea, I think.


I would use instance attributes for members that are held in the 
instance dict (or have specific slots). As this can't be a normal 
"method" (it could be any object including a callable one - but won't be 
a standard method descriptor) it seems uncontroversial. (Or more to the 
point it seems to be *precise* in its meaning in the context of the 
Python data model.)


What do you mean by "instances can have methods as instance attributes"? 
Once you attach a bound method directly to an instance it becomes a 
slightly different beast I think. (On top of which that is pretty rare 
behaviour.)



Given that there is no one thing that covers all non-method
attributes, I suspect 'non-method attributes' is as good as
we're going to manage.


Hehe, yeah - that's not bad... It's hard to see how it could be 
misunderstood too.


Michael


--
R. David Murray   http://www.bitdance.com



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Nick Coghlan]

On Tue, Jun 28, 2011 at 8:54 PM, Michael Foord
 wrote:
> The problem with "data attributes" is that it doesn't mean *anything*, which
> I suppose is useful for invented terminology, but it means it doesn't convey
> anything precise to those who haven't heard the term before. If it becomes
> widely used then that changes I guess. I'd still normally just use
> "attributes" though...

In this context, precision is an illusion. There is *no* precise
terminology, not only because Python blurs the boundaries by design in
many respects (callable or not, method or not, attribute or not, etc),
but also because *people* use the same words to mean slightly
different things.

The best we can hope for is to encourage the right way of thinking
about the situation, and in that regard you have the complete set of
attributes accessible via an object (i.e. via __getattribute__), some
of which are callables (and may or may not act like instance methods)
and the remainder of which are data attributes (some of which may
incidentally be callable, even if they aren't used that way). How a
*particular* attribute is classified is not an inherent property of
the attribute, but also an artifact of the way it is used by the
application.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/28/2011 06:44 AM, Fred Drake wrote:
> On Tue, Jun 28, 2011 at 2:33 AM, Nick Coghlan  wrote:
>> The two terms I've got out of this thread are "callable attributes"
>> (instance/static/class methods, etc) and "data attributes" (everything
>> else). Both seem reasonable to me, creating two largely disjoint sets
>> that together cover all the different kinds of attribute you're likely
>> to encounter.
> 
> But "callable attributes" aren't the same thing as methods; most are methods,
> but not all.  Sometimes, they're data used by the object.  The fact that
> data attributes can be callable is irrelevant.

Isn't it fuzzy / incorrect to refer to any callable attribute as a
method until it have been extracted via the dot operator / getattr, and
therefore bound via descriptor semantics?  In this sense, 'staticmethod'
doesn't create a "method" at all -- it just defeats the default creation
of a method-yielding descriptor.


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4JzHcACgkQ+gerLs4ltQ6I5ACZAXlBsZkrzQowKYBmJE4NQb4V
F14AnRtkWByqwpRATan4OOTMgPqwyjxH
=hru9
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 13:31, Nick Coghlan wrote:

On Tue, Jun 28, 2011 at 8:54 PM, Michael Foord
  wrote:

The problem with "data attributes" is that it doesn't mean *anything*, which
I suppose is useful for invented terminology, but it means it doesn't convey
anything precise to those who haven't heard the term before. If it becomes
widely used then that changes I guess. I'd still normally just use
"attributes" though...

In this context, precision is an illusion. There is *no* precise
terminology,
"instance members" is precise, (whether or not an attribute is fetched 
"from the instance" is one thing that is clear in the python data model 
- there are just lots of other ways of fetching attributes too). 
"instance members" just doesn't cover all the cases where you may want 
to group attributes together though.


However, providing "__getattr__" and "__getattribute__" is not the 
common case and even properties are *usually* backed by a real instance 
attribute. So "instance attributes" as opposed to other types of 
attributes is *often* a useful distinction.


I don't think "data attributes" is clear or precise. Is a property a 
data attribute (well it depends how it is implemented and what it does), 
is a staticmethod a data attribute (no - but then Tres' question - it 
isn't a normal method either so if you define data attribute to mean 
"all non method attributes" then its potentially an open question).


All the best,

Michael Foord


not only because Python blurs the boundaries by design in
many respects (callable or not, method or not, attribute or not, etc),
but also because *people* use the same words to mean slightly
different things.

The best we can hope for is to encourage the right way of thinking
about the situation, and in that regard you have the complete set of
attributes accessible via an object (i.e. via __getattribute__), some
of which are callables (and may or may not act like instance methods)
and the remainder of which are data attributes (some of which may
incidentally be callable, even if they aren't used that way). How a
*particular* attribute is classified is not an inherent property of
the attribute, but also an artifact of the way it is used by the
application.

Cheers,
Nick.




--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Steven D'Aprano]

Michael Foord wrote:

What do you mean by "instances can have methods as instance attributes"? 
Once you attach a bound method directly to an instance it becomes a 
slightly different beast I think. (On top of which that is pretty rare 
behaviour.)


>>> class C:
... def method(self, x):
... return x+1
...
>>> c = C()
>>> c.method = types.MethodType(lambda self, x: x+101, c)
>>> c.method(1)
102

I don't know how rare it is, but it's a useful trick for customising the 
behaviour of instances.



As I see it, there are three dichotomies we sometimes need to make:


(1) Instance attributes vs class (shared) attributes.

Broadly speaking, whether the attribute is in instance.__dict__ or 
type(instance).__dict__.


(2) Computed vs non-computed attributes.

Attributes which are computed by __getattr__ or via the descriptor 
protocol (which includes properties) are all computed attributes; 
everything else is non-computed.


(3) Method attributes (methods) vs non-method/data attributes.

Broadly speaking, methods are callable, non-method (data) attributes are 
not.



The three are orthogonal: e.g. a staticmethod is a method by virtue of 
being callable, computed by virtue of being generated by a descriptor, 
and a class attribute by virtue of existing in the type __dict__ rather 
than the instance __dict__.


Strictly speaking, (3) is not truly a dichotomy, since functions and 
methods are first class-objects in Python. E.g. one may store a function 
as an attribute with the intention of using it as data rather than as a 
method. But that's a moderately obscure corner case, and in my opinion 
it's not worth obscuring the practical distinction between "methods are 
things you call, data are not" for the sake of it. Leave the 
functions-as-data case for a footnote.




--
Steven

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 13:56, Michael Foord wrote:

On 28/06/2011 13:31, Nick Coghlan wrote:

On Tue, Jun 28, 2011 at 8:54 PM, Michael Foord
  wrote:
The problem with "data attributes" is that it doesn't mean 
*anything*, which
I suppose is useful for invented terminology, but it means it 
doesn't convey
anything precise to those who haven't heard the term before. If it 
becomes

widely used then that changes I guess. I'd still normally just use
"attributes" though...

In this context, precision is an illusion. There is *no* precise
terminology,
"instance members" is precise, (whether or not an attribute is fetched 
"from the instance" is one thing that is clear in the python data 
model - there are just lots of other ways of fetching attributes too). 
"instance members" just doesn't cover all the cases where you may want 
to group attributes together though.


However, providing "__getattr__" and "__getattribute__" is not the 
common case and even properties are *usually* backed by a real 
instance attribute. So "instance attributes" as opposed to other types 
of attributes is *often* a useful distinction.


I don't think "data attributes" is clear or precise. 


Although to be fair "data attributes" mirrors the terminology for 
descriptors where we have "data descriptors" (which have both __set__ 
and __get__) and "non data descriptors" (__get__ only). Unfortunately 
there are also non-non-data decriptors too (only __set__ and / or 
__delete__) so that terminology is at least slightly confusing / 
imprecise [1]... There was a genuine Python "bug" caused by that 
confusion at one point (unfortunately I forget what it was).


All the best,

Michael

[1] From: http://users.rcn.com/python/download/Descriptor.htm

If an object defines both __get__ and __set__, it is considered a 
data descriptor. Descriptors that only define __get__ are called 
non-data descriptors (they are typically used for methods but other uses 
are possible).



Is a property a data attribute (well it depends how it is implemented 
and what it does), is a staticmethod a data attribute (no - but then 
Tres' question - it isn't a normal method either so if you define data 
attribute to mean "all non method attributes" then its potentially an 
open question).


All the best,

Michael Foord


not only because Python blurs the boundaries by design in
many respects (callable or not, method or not, attribute or not, etc),
but also because *people* use the same words to mean slightly
different things.

The best we can hope for is to encourage the right way of thinking
about the situation, and in that regard you have the complete set of
attributes accessible via an object (i.e. via __getattribute__), some
of which are callables (and may or may not act like instance methods)
and the remainder of which are data attributes (some of which may
incidentally be callable, even if they aren't used that way). How a
*particular* attribute is classified is not an inherent property of
the attribute, but also an artifact of the way it is used by the
application.

Cheers,
Nick.







--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 14:20, Steven D'Aprano wrote:

Michael Foord wrote:

What do you mean by "instances can have methods as instance 
attributes"? Once you attach a bound method directly to an instance 
it becomes a slightly different beast I think. (On top of which that 
is pretty rare behaviour.)


>>> class C:
... def method(self, x):
... return x+1
...
>>> c = C()
>>> c.method = types.MethodType(lambda self, x: x+101, c)
>>> c.method(1)
102

I don't know how rare it is, but it's a useful trick for customising 
the behaviour of instances.



Right - that method is an instance attribute.



As I see it, there are three dichotomies we sometimes need to make:


(1) Instance attributes vs class (shared) attributes.

Broadly speaking, whether the attribute is in instance.__dict__ or 
type(instance).__dict__.


(2) Computed vs non-computed attributes.

Attributes which are computed by __getattr__ or via the descriptor 
protocol (which includes properties) are all computed attributes; 
everything else is non-computed.

Technically also via __getattribute__ when overridden.



(3) Method attributes (methods) vs non-method/data attributes.

Broadly speaking, methods are callable, non-method (data) attributes 
are not.



The three are orthogonal: e.g. a staticmethod is a method by virtue of 
being callable, computed by virtue of being generated by a descriptor, 
and a class attribute by virtue of existing in the type __dict__ 
rather than the instance __dict__.


Strictly speaking, (3) is not truly a dichotomy, since functions and 
methods are first class-objects in Python. E.g. one may store a 
function as an attribute with the intention of using it as data rather 
than as a method. But that's a moderately obscure corner case, and in 
my opinion it's not worth obscuring the practical distinction between 
"methods are things you call, data are not" for the sake of it. Leave 
the functions-as-data case for a footnote.



Yep, useful summary.

Michael








--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Nick Coghlan]

On Tue, Jun 28, 2011 at 10:56 PM, Michael Foord
 wrote:
> I don't think "data attributes" is clear or precise. Is a property a data
> attribute (well it depends how it is implemented and what it does), is a
> staticmethod a data attribute (no - but then Tres' question - it isn't a
> normal method either so if you define data attribute to mean "all non method
> attributes" then its potentially an open question).

"callable attributes" and "data attributes" are purely about the API
exposed by the object in question. If we're going for object model
neutral terminology, that's the only view that makes sense. They're
descriptions about how attributes are intended to be *used* that are
completely silent on the question of how they're *implemented*. So
staticmethod would fall into the first group, while property would
fall into the latter.

>From an implementation point of view, you carve up the world
differently, so it makes sense to worry about class attributes,
instance attributes, dynamic attributes, etc. (the class vs instance
distinction can also matter to some degree from the usage point of
view, since it affects the scope of any mutable attributes, and the
static vs dynamic distinction can also matter, especially for
introspection purposes).

This goes back to the original point about all of this being highly
context dependent - how you carve up the set of all attributes is
going to change based on what you're trying to explain (e.g. the
distinction between enforced 'data' descriptors, descriptors that
allow shadowing in the instance dict, class attributes that aren't
descriptors at all, instance attributes and dynamic attributes
retrieved via __getattr__ is another way of dividing them)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Nick Coghlan]

On Tue, Jun 28, 2011 at 11:27 PM, Michael Foord
 wrote:
> Technically also via __getattribute__ when overridden.

Since object.__getattribute__ is the hook that implements the entire
attribute lookup protocol, *all* attributes are technically retrieved
via __getattribute__ (which is why overriding it correctly can be such
a PITA). That's also the hook type() overrides to make class attribute
lookup differ from ordinary instance lookup.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] EuroPython Language Summit report

[Michael Foord]

On 24/06/2011 15:55, R. David Murray wrote:

On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson  wrote:

EuroPython 2011 Language Summit
===

[...]

 Unicode character classes is a particular need).  [Subtopic:  what needs
 to be done to get the new regex module into Python?  Should it replace
 the existing module?  What about backwards compatibility issues?]

I'm pretty sure regex has backward compatibility as a goal for just this
reason (so it can replace the current module).


The new regex library has some great improvements:

http://bugs.python.org/issue2636

It also has users and committed maintainers, so I hope we can bring it 
into 3.3. It wasn't easy to tell from skimming the change notes that 
Unicode character classes are amongst the new features. Is that the case?


Michael



PEP 0380: Syntax for delegating to a subgenerator.

PEP 3150: Statement local namespaces.

PEP 3152: Cofunctions.

For all three of the above PEPs, there was some feeling that additional syntax
changes to the language are unnecessary and make it harder to learn; where
possible, we should prefer using 3rd partly libraries.

I disagree with this with respect to 380.  Haven't looked at the others.


Python 2.7
--

How long will this be maintained?
Original decision was 5 years.

As I recall, the actual decision was "at *least* 5 years".  It's only
been one so far...was the discussion that five years was too short or
too long?

As the code bases continue to drift apart, and we fix the fixable bugs,
the number of patches going in to 2.7 will decrease over time, so I
don't think the burden of continuing to support it will be too heavy.
Currently about half of the non-feature-request issues (ie: bugs) in
the tracker are tagged 2.7.  I expect to see that percentage continue
to decrease over time.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Victor Stinner]

In Python 2, open() opens the file in binary mode (e.g. file.readline()
returns a byte string). codecs.open() opens the file in binary mode by
default, you have to specify an encoding name to open it in text mode.

In Python 3, open() opens the file in text mode by default. (It only
opens the binary mode if the file mode contains "b".) The problem is
that open() uses the locale encoding if the encoding is not specified,
which is the case *by default*. The locale encoding can be:

 - UTF-8 on Mac OS X, most Linux distributions
 - ISO-8859-1 os some FreeBSD systems
 - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
Western Europe, cp952 in Japan, ...
 - ASCII if the locale is manually set to an empty string or to "C", or
if the environment is empty, or by default on some systems
 - something different depending on the system and user configuration...

If you develop under Mac OS X or Linux, you may have surprises when you
run your program on Windows on the first non-ASCII character. You may
not detect the problem if you only write text in english... until
someone writes the first letter with a diacritic.



As discussed before on this list, I propose to set the default encoding
of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
open() is called without an explicit encoding and if the locale encoding
is not UTF-8. Using the warning, you will quickly notice the potential
problem (using Python 3.2.2 and -Werror) on Windows or by using a
different locale encoding (.e.g using LANG="C").

I expect a lot of warnings from the Python standard library, and as many
in third party modules and applications. So do you think that it is too
late to change that in Python 3.3? One argument for changing it directly
in Python 3.3 is that most users will not notice the change because
their locale encoding is already UTF-8.

An alternative is to:
 - Python 3.2: use the locale encoding but emit a warning if the locale
encoding is not UTF-8
 - Python 3.3: use UTF-8 and emit a warning if the locale encoding is
not UTF-8... or maybe always emit a warning?
 - Python 3.3: use UTF-8 (but don't emit warnings anymore)

I don't think that Windows developer even know that they are writing
files into the ANSI code page. MSDN documentation of
WideCharToMultiByte() warns developer that the ANSI code page is not
portable, even accross Windows computers:

"The ANSI code pages can be different on different computers, or can be
changed for a single computer, leading to data corruption. For the most
consistent results, applications should use Unicode, such as UTF-8 or
UTF-16, instead of a specific code page, unless legacy standards or data
formats prevent the use of Unicode. If using Unicode is not possible,
applications should tag the data stream with the appropriate encoding
name when protocols allow it. HTML and XML files allow tagging, but text
files do not."

It will always be possible to use ANSI code page using
encoding="mbcs" (only work on Windows), or an explicit code page number
(e.g. encoding="cp2152").

--

The two other (rejetected?) options to improve open() are:

- raise an error if the encoding argument is not set: will break most
programs
- emit a warning if the encoding argument is not set

--

Should I convert this email into a PEP, or is it not required?

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[M.-A. Lemburg]

Victor Stinner wrote:
> In Python 2, open() opens the file in binary mode (e.g. file.readline()
> returns a byte string). codecs.open() opens the file in binary mode by
> default, you have to specify an encoding name to open it in text mode.
> 
> In Python 3, open() opens the file in text mode by default. (It only
> opens the binary mode if the file mode contains "b".) The problem is
> that open() uses the locale encoding if the encoding is not specified,
> which is the case *by default*. The locale encoding can be:
> 
>  - UTF-8 on Mac OS X, most Linux distributions
>  - ISO-8859-1 os some FreeBSD systems
>  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> Western Europe, cp952 in Japan, ...
>  - ASCII if the locale is manually set to an empty string or to "C", or
> if the environment is empty, or by default on some systems
>  - something different depending on the system and user configuration...
> 
> If you develop under Mac OS X or Linux, you may have surprises when you
> run your program on Windows on the first non-ASCII character. You may
> not detect the problem if you only write text in english... until
> someone writes the first letter with a diacritic.

How about a more radical change: have open() in Py3 default to
opening the file in binary mode, if no encoding is given (even
if the mode doesn't include 'b') ?

That'll make it compatible to the Py2 world again and avoid
all the encoding guessing.

Making such default encodings depend on the locale has already
failed to work when we first introduced a default encoding in
Py2, so I don't understand why we are repeating the same
mistake again in Py3 (only in a different area).

Note that in Py2, Unix applications often leave out the 'b'
mode, since there's no difference between using it or not.
Only on Windows, you'll see a difference.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 28 2011)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] EuroPython Language Summit report

[Victor Stinner]

Le mardi 28 juin 2011 à 14:41 +0100, Michael Foord a écrit :
> On 24/06/2011 15:55, R. David Murray wrote:
> > On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson  
> > wrote:
> >> EuroPython 2011 Language Summit
> >> ===
> > [...]
> >>  Unicode character classes is a particular need).  [Subtopic:  what 
> >> needs
> >>  to be done to get the new regex module into Python?  Should it replace
> >>  the existing module?  What about backwards compatibility issues?]
> > I'm pretty sure regex has backward compatibility as a goal for just this
> > reason (so it can replace the current module).
> >
> The new regex library has some great improvements:
> 
>  http://bugs.python.org/issue2636

This issue is open since April 2008, has also the longest list of
attached files, and has a very long history. What is the status of the
issue? I see that there is now a third party project on:

   http://code.google.com/p/mrab-regex-hg/

--

There is also the re2 library from Google and especially this project:

   http://pypi.python.org/pypi/re2/

"pyre2 is a Python extension that wraps Google's RE2 regular expression
library.

This version of pyre2 is similar to the one you'd find at facebook's
github repository except that the stated goal of *this version is to be
a drop-in replacement for the re module*.)"

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Antoine Pitrou]

On Tue, 28 Jun 2011 15:43:05 +0200
Victor Stinner  wrote:
>  - ISO-8859-1 os some FreeBSD systems
>  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> Western Europe, cp952 in Japan, ...
>  - ASCII if the locale is manually set to an empty string or to "C", or
> if the environment is empty, or by default on some systems
>  - something different depending on the system and user configuration...

Why would utf-8 be the right thing in these cases?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Terry Reedy]

On 6/28/2011 9:43 AM, Victor Stinner wrote:

In Python 2, open() opens the file in binary mode (e.g. file.readline()
returns a byte string). codecs.open() opens the file in binary mode by
default, you have to specify an encoding name to open it in text mode.

In Python 3, open() opens the file in text mode by default. (It only
opens the binary mode if the file mode contains "b".) The problem is
that open() uses the locale encoding if the encoding is not specified,
which is the case *by default*. The locale encoding can be:

  - UTF-8 on Mac OS X, most Linux distributions
  - ISO-8859-1 os some FreeBSD systems
  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
Western Europe, cp952 in Japan, ...
  - ASCII if the locale is manually set to an empty string or to "C", or
if the environment is empty, or by default on some systems
  - something different depending on the system and user configuration...

If you develop under Mac OS X or Linux, you may have surprises when you
run your program on Windows on the first non-ASCII character. You may
not detect the problem if you only write text in english... until
someone writes the first letter with a diacritic.



As discussed before on this list, I propose to set the default encoding
of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
open() is called without an explicit encoding and if the locale encoding
is not UTF-8. Using the warning, you will quickly notice the potential
problem (using Python 3.2.2 and -Werror) on Windows or by using a
different locale encoding (.e.g using LANG="C").

I expect a lot of warnings from the Python standard library, and as many
in third party modules and applications. So do you think that it is too
late to change that in Python 3.3? One argument for changing it directly
in Python 3.3 is that most users will not notice the change because
their locale encoding is already UTF-8.

An alternative is to:
  - Python 3.2: use the locale encoding but emit a warning if the locale
encoding is not UTF-8
  - Python 3.3: use UTF-8 and emit a warning if the locale encoding is
not UTF-8... or maybe always emit a warning?
  - Python 3.3: use UTF-8 (but don't emit warnings anymore)

I don't think that Windows developer even know that they are writing
files into the ANSI code page. MSDN documentation of
WideCharToMultiByte() warns developer that the ANSI code page is not
portable, even accross Windows computers:

"The ANSI code pages can be different on different computers, or can be
changed for a single computer, leading to data corruption. For the most
consistent results, applications should use Unicode, such as UTF-8 or
UTF-16, instead of a specific code page, unless legacy standards or data
formats prevent the use of Unicode. If using Unicode is not possible,
applications should tag the data stream with the appropriate encoding
name when protocols allow it. HTML and XML files allow tagging, but text
files do not."

It will always be possible to use ANSI code page using
encoding="mbcs" (only work on Windows), or an explicit code page number
(e.g. encoding="cp2152").

--

The two other (rejetected?) options to improve open() are:

- raise an error if the encoding argument is not set: will break most
programs
- emit a warning if the encoding argument is not set

--

Should I convert this email into a PEP, or is it not required?


I think a PEP is needed.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Terry Reedy]

On 6/28/2011 10:02 AM, M.-A. Lemburg wrote:


How about a more radical change: have open() in Py3 default to
opening the file in binary mode, if no encoding is given (even
if the mode doesn't include 'b') ?

That'll make it compatible to the Py2 world again


I disagree. I believe
S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.

> and avoid all the encoding guessing.


Making such default encodings depend on the locale has already
failed to work when we first introduced a default encoding in
Py2, so I don't understand why we are repeating the same
mistake again in Py3 (only in a different area).


I do not remember any proposed change during the Py3 design discussions.


Note that in Py2, Unix applications often leave out the 'b'
mode, since there's no difference between using it or not.


I believe it makes a difference now as to whether one gets str or bytes.


Only on Windows, you'll see a difference.


I believe the only difference now on Windows is the decoding used, not 
the return type.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: remove duplicate -I

[Benjamin Peterson]

2011/6/28 Ned Deily :
> In article ,
>  benjamin.peterson  wrote:
>
>> http://hg.python.org/cpython/rev/0d0ca6f95d9d
>> changeset:   70879:0d0ca6f95d9d
>> user:        Benjamin Peterson 
>> date:        Sun Jun 19 17:17:30 2011 -0500
>> summary:
>>   remove duplicate -I
>>
>> files:
>>   Makefile.pre.in |  2 +-
>>   1 files changed, 1 insertions(+), 1 deletions(-)
>>
>>
>> diff --git a/Makefile.pre.in b/Makefile.pre.in
>> --- a/Makefile.pre.in
>> +++ b/Makefile.pre.in
>> @@ -73,7 +73,7 @@
>>  # Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py
>>  to
>>  # be able to build extension modules using the directories specified in the
>>  # environment variables
>> -PY_CPPFLAGS= -I. -IInclude -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS)
>> $(CPPFLAGS)
>> +PY_CPPFLAGS= -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
>>  PY_LDFLAGS=  $(CONFIGURE_LDFLAGS) $(LDFLAGS)
>>  NO_AS_NEEDED=        @NO_AS_NEEDED@
>>  LDLAST=              @LDLAST@
>
> I think someone has already mentioned that this change breaks builds
> with a source directory outside the build directory and that breaks my
> OS X installer builds.   Please revert or make the following change
> which works for me (but not tested elsewhere):
>
> -PY_CPPFLAGS=   -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
> +PY_CPPFLAGS=   -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)

Committed. Thanks for the review.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Terry Reedy]

On 6/28/2011 10:06 AM, Antoine Pitrou wrote:

On Tue, 28 Jun 2011 15:43:05 +0200
Victor Stinner  wrote:

  - ISO-8859-1 os some FreeBSD systems
  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
Western Europe, cp952 in Japan, ...
  - ASCII if the locale is manually set to an empty string or to "C", or
if the environment is empty, or by default on some systems
  - something different depending on the system and user configuration...


Why would utf-8 be the right thing in these cases?


Because utf-8 is the only way to write out any Python 3 text.
By default, writing and reading an str object should work on all Python 
installations.
And because other apps are (increasingly) using it for exactly the same 
reason.



--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Paul Moore]

On 28 June 2011 14:43, Victor Stinner  wrote:
> As discussed before on this list, I propose to set the default encoding
> of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
> open() is called without an explicit encoding and if the locale encoding
> is not UTF-8. Using the warning, you will quickly notice the potential
> problem (using Python 3.2.2 and -Werror) on Windows or by using a
> different locale encoding (.e.g using LANG="C").

-1. This will make things harder for simple scripts which are not
intended to be cross-platform.

I use Windows, and come from the UK, so 99% of my text files are
ASCII. So the majority of my code will be unaffected. But in the
occasional situation where I use a £ sign, I'll get encoding errors,
where currently things will "just work". And the failures will be data
dependent, and hence intermittent (the worst type of problem). I'll
write a quick script, use it once and it'll be fine, then use it later
on some different data and get an error. :-(

I appreciate that the point here is to make sure that people think a
bit more carefully about encoding issues. But doing so by making
Python less friendly for casual, adhoc script use, seems to me to be a
mistake.

> I don't think that Windows developer even know that they are writing
> files into the ANSI code page. MSDN documentation of
> WideCharToMultiByte() warns developer that the ANSI code page is not
> portable, even accross Windows computers:

Probably true. But for many uses they also don't care. If you're
writing something solely for a one-off job on your own PC, the ANSI
code page is fine, and provides interoperability with other programs
on your PC, which is really what you care about. (UTF-8 without BOM
displays incorrectly in Vim, wordpad, and powershell get-content. MBCS
works fine in all of these. It also displays incorrectly in CMD type,
but in a less familiar form than the incorrect display mbcs produces,
for what that's worth...)

> It will always be possible to use ANSI code page using
> encoding="mbcs" (only work on Windows), or an explicit code page number
> (e.g. encoding="cp2152").

So, in effect, you propose making the default favour writing
multiplatform portable code at the expense of quick and dirty scripts?
My personal view is that this is the wrong choice ("practicality beats
purity") but I guess it's ultimately a question of Python's design
philosophy.

> The two other (rejetected?) options to improve open() are:
>
> - raise an error if the encoding argument is not set: will break most
> programs
> - emit a warning if the encoding argument is not set

IMHO, you missed another option - open() does not need improving, the
current behaviour is better than any of the 3 options noted.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Michael Foord]

On 28/06/2011 15:36, Terry Reedy wrote:

On 6/28/2011 10:02 AM, M.-A. Lemburg wrote:


How about a more radical change: have open() in Py3 default to
opening the file in binary mode, if no encoding is given (even
if the mode doesn't include 'b') ?

That'll make it compatible to the Py2 world again


I disagree. I believe
S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.


Nope, it returns a bytestring in Python 2. Mistakenly treating 
bytestrings as text is one of the things we aimed to correct in the 
transition to Python 3.


Michael



> and avoid all the encoding guessing.


Making such default encodings depend on the locale has already
failed to work when we first introduced a default encoding in
Py2, so I don't understand why we are repeating the same
mistake again in Py3 (only in a different area).


I do not remember any proposed change during the Py3 design discussions.


Note that in Py2, Unix applications often leave out the 'b'
mode, since there's no difference between using it or not.


I believe it makes a difference now as to whether one gets str or bytes.


Only on Windows, you'll see a difference.


I believe the only difference now on Windows is the decoding used, not 
the return type.





--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Antoine Pitrou]

On Tue, 28 Jun 2011 10:41:38 -0400
Terry Reedy  wrote:
> On 6/28/2011 10:06 AM, Antoine Pitrou wrote:
> > On Tue, 28 Jun 2011 15:43:05 +0200
> > Victor Stinner  wrote:
> >>   - ISO-8859-1 os some FreeBSD systems
> >>   - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> >> Western Europe, cp952 in Japan, ...
> >>   - ASCII if the locale is manually set to an empty string or to "C", or
> >> if the environment is empty, or by default on some systems
> >>   - something different depending on the system and user configuration...
> >
> > Why would utf-8 be the right thing in these cases?
> 
> Because utf-8 is the only way to write out any Python 3 text.

Er, no, you also have utf-16, utf-32, utf-7 (and possibly others,
including home-baked encodings).

> By default, writing and reading an str object should work on all Python 
> installations.

But that's only half of the problem. If the text is supposed to be read
or processed by some other program, then writing it in some encoding
that the other program doesn't expect doesn't really help. That's why
we use the locale encoding: because it's a good guess as to what the
system (and its users) expects text to be encoded in.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Steffen Daode Nurpmeso]

@ Paul Moore  wrote (2011-06-28 16:46+0200):
> UTF-8 without BOM displays incorrectly in vim(1)

Stop right now (you're oh so wrong)!  :-)

(By the way: UTF-8 and BOM?
Interesting things i learn on this list.
And i hope in ten years we can laugh about this -> UTF-8
transition all over the place, 'cause it's simply working.)

--
Ciao, Steffen
sdaoden(*)(gmail.com)
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Terry Reedy]

On 6/28/2011 7:51 AM, R. David Murray wrote:


Also, instances can have methods as instance attributes.


Functions that are instance attributes do not act like methods 
(instance.func() does not automagically turn instance in the first arg 
of func) and have never, to my knowledge, been called methods. In Python 
2, they are not wrapped as methods whereas functions attached to classes 
are.


So-called 'staticmethods' are not really methods either, but are class 
function attributes that are just functions and not treated as methods. 
The decorator that negates normal method treatment could/should have 
been called 'non_method'.


Using 'function' is its generic 'callable' sense ...

Method: a class function attribute that in its intended and normal use 
automagically turns the object it is called on into its first arg. 
'Method' is a useful and needed subcategory of class attribute precisely 
because of this behavior.


Instance method: a class function attribute that is (normally) called on 
instances of the class or subclasses.


Class method: a class function attribute that is (normally) called on 
the class or subclasses.


Bound method: a method that has already has the first-arg object bundled 
with it, so that it can be used as a normal (partial or curried) function.


Except for 'classmethod', which was added later, these have been the 
meanings as I understood them since at least Py 1.4, 15 years ago, and 
they are exactly what one needs to know to use Python. Any object can be 
an attribute. However, function attributes of classes normally get 
special 'method' treatment which alters the meaning of syntax.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Eric Snow]

On Tue, Jun 28, 2011 at 5:35 AM, Michael Foord
 wrote:
>
> Well, they're *usually* implemented as methods and backed by a real instance
> attribute. Usually (but not always) it makes more sense (IME) to group them
> with methods. The fact that they're *accessed* as an attribute is the
> uninteresting detail.
>
> __getattr__ and __getattribute__ are interesting - they allow you to use
> attribute access syntax for things that *aren't* attributes.
>
> I appreciate the fact that the Python data-model means methods are just
> object attributes, but they're not *instance attributes* and sometimes you
> need to make a distinction. (And yes it is helpful if the standard
> terminology leads people into a better understanding of the Python data
> model, but that still doesn't change the need on occasion for terminology
> that doesn't need to be explained whenever it is used.)
>

While the distinction between class-specific attributes and
instance-specific is important, I don't think merging it with the
method/data distinction is as helpful.  The valuable information here
is the expectation of how the attributes are used.  In my mind, that
boils down to data (holds static or dymanic state) and methods (does
something on the class or instance).

-eric

> Given that how even methods are to be described depends on the context (if
> you're fetching bound methods as objects then it makes perfect sense to just
> talk about them as attributes) it doesn't seem an area amenable to
> one-size-fits-all terminology.
>
>>> The problem with "data attributes" is that it doesn't mean *anything*,
>>> which
>>> I suppose is useful for invented terminology, but it means it doesn't
>>> convey
>>> anything precise to those who haven't heard the term before. If it
>>> becomes
>>> widely used then that changes I guess. I'd still normally just use
>>> "attributes" though...
>>
>> I'd read "data attributes" the same as "non-method attributes".  For
>> readers,
>> calling them "attributes" is typically sufficient.  It's rare to need to
>> distinguish them from methods.
>
> Yeah, this is all a grand bikeshed. I'm not sure I would understand "data
> attributes" unless it was clear from the context. I would wonder what
> qualifies something as "data".
>
> It is an interesting question what terminology we should use in the
> documentation if we need to distinguish them, but I think that is still
> wandering away from the original question that was posed.
>
> All the best,
>
> Michael
>
>>
>>   -Fred
>>
>
>
> --
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Michael Foord]

On 28/06/2011 16:23, Terry Reedy wrote:

On 6/28/2011 7:51 AM, R. David Murray wrote:


Also, instances can have methods as instance attributes.


Functions that are instance attributes do not act like methods 
(instance.func() does not automagically turn instance in the first arg 
of func) and have never, to my knowledge, been called methods. In 
Python 2, they are not wrapped as methods whereas functions attached 
to classes are.


So-called 'staticmethods' are not really methods either, but are class 
function attributes that are just functions and not treated as 
methods. The decorator that negates normal method treatment 
could/should have been called 'non_method'.


Using 'function' is its generic 'callable' sense ...

Method: a class function attribute that in its intended and normal use 
automagically turns the object it is called on into its first arg. 
'Method' is a useful and needed subcategory of class attribute 
precisely because of this behavior.


Instance method: a class function attribute that is (normally) called 
on instances of the class or subclasses.




So what is the difference between "Instance method" and "Method" above? 
Is it just that "Method" is broader and includes class methods and bound 
methods?


If anyone said "instance method" to me I would assume they meant bound 
method. (A normal method fetched from an instance.)


All the best,

Michael

Class method: a class function attribute that is (normally) called on 
the class or subclasses.


Bound method: a method that has already has the first-arg object 
bundled with it, so that it can be used as a normal (partial or 
curried) function.


Except for 'classmethod', which was added later, these have been the 
meanings as I understood them since at least Py 1.4, 15 years ago, and 
they are exactly what one needs to know to use Python. Any object can 
be an attribute. However, function attributes of classes normally get 
special 'method' treatment which alters the meaning of syntax.





--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Eric Snow]

On Tue, Jun 28, 2011 at 7:20 AM, Steven D'Aprano  wrote:
> Michael Foord wrote:
>
>> What do you mean by "instances can have methods as instance attributes"?
>> Once you attach a bound method directly to an instance it becomes a slightly
>> different beast I think. (On top of which that is pretty rare behaviour.)
>
 class C:
> ...     def method(self, x):
> ...             return x+1
> ...
 c = C()
 c.method = types.MethodType(lambda self, x: x+101, c)
 c.method(1)
> 102
>
> I don't know how rare it is, but it's a useful trick for customising the
> behaviour of instances.
>
>
> As I see it, there are three dichotomies we sometimes need to make:
>
>
> (1) Instance attributes vs class (shared) attributes.
>
> Broadly speaking, whether the attribute is in instance.__dict__ or
> type(instance).__dict__.
>
> (2) Computed vs non-computed attributes.
>
> Attributes which are computed by __getattr__ or via the descriptor protocol
> (which includes properties) are all computed attributes; everything else is
> non-computed.
>
> (3) Method attributes (methods) vs non-method/data attributes.
>
> Broadly speaking, methods are callable, non-method (data) attributes are
> not.
>

For terminology, is it important that data attributes are [usually]
not callable, or is it that being callable is not relevant to their
use as attributes of the class/instance?  The same for "methods".

We have precedent for where the the terminology represents an
expectation rather than a firm constraint (__isabstractmethod__).

>
> The three are orthogonal: e.g. a staticmethod is a method by virtue of being
> callable, computed by virtue of being generated by a descriptor, and a class
> attribute by virtue of existing in the type __dict__ rather than the
> instance __dict__.
>
> Strictly speaking, (3) is not truly a dichotomy, since functions and methods
> are first class-objects in Python. E.g. one may store a function as an
> attribute with the intention of using it as data rather than as a method.
> But that's a moderately obscure corner case, and in my opinion it's not
> worth obscuring the practical distinction between "methods are things you
> call, data are not" for the sake of it. Leave the functions-as-data case for
> a footnote.
>

+1

The "three dichotomies" is a great way to look at it.

-eric

>
>
> --
> Steven
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Paul Moore]

On 28 June 2011 16:06, Steffen Daode Nurpmeso  wrote:
> @ Paul Moore  wrote (2011-06-28 16:46+0200):
>> UTF-8 without BOM displays incorrectly in vim(1)
>
> Stop right now (you're oh so wrong)!  :-)

Sorry. Please add "using the default settings of gvim on Windows". My
context throughout was Windows not Unix. Sorry I didn't make that
clear.

> (By the way: UTF-8 and BOM?

Windows uses it, I believe. My tests specifically used files with no
BOM, just utf8-encoded text. I made this statement to head off people
assuming that UTF8 can be detected in Windows by looking at the first
few bytes.

> Interesting things i learn on this list.

:-)

> And i hope in ten years we can laugh about this -> UTF-8
> transition all over the place, 'cause it's simply working.)

That would be good...

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Terry Reedy]

On 6/28/2011 9:20 AM, Steven D'Aprano wrote:


 >>> class C:
... def method(self, x):
... return x+1
...
 >>> c = C()
 >>> c.method = types.MethodType(lambda self, x: x+101, c)


types.MethodType creates a bound method, not a method. A bound method is 
a partial or curried function, which is to say, a function.


Herw and below, I am ignoring the fact that the example ignores self. My 
comments would be the same for "lambda self,x: return self.value+x"



 >>> c.method(1)


The fact that you make the bound function an attribute of the same 
object to which it is bound is irrelevant to how it *acts* when called. 
It only affect how you *access* it. You could bind it to anything else, 
including another instance or a plain name:


d = C()
d.method = types.MethodType(lambda self, x: x+101, c)
d.method(1)

m = types.MethodType(lambda self, x: x+101, c)
m(1)


102



I don't know how rare it is, but it's a useful trick for customising the
behaviour of instances.


As I see it, there are three dichotomies we sometimes need to make:


(1) Instance attributes vs class (shared) attributes.
(2) Computed vs non-computed attributes.
(3) Method attributes (methods) vs non-method/data attributes.


Nicely put so far...

> The three are orthogonal:

Non-class instance function attributes are not methods in any useful sense.

> a staticmethod is a method by virtue of being callable

A 'staticmethod' is a non-method class function attribute. The name is 
unfortunate. 'Static' mean 'does not get the dynamic method treatment'.



Strictly speaking, (3) is not truly a dichotomy,


I disagree here. A method is usefully defined as a class function 
attribute that gets an automagic first arg when accessed and called 
'normally'. In other words, the dichotomy is needed because being a 
method affects the meaning of syntax.



since functions and
methods are first class-objects in Python. E.g. one may store a function
as an attribute with the intention of using it as data rather than as a
method.


The fact that one can *also* access a method as a function does not 
negate the effect on syntax.



"methods are things you call"


Way too broad, even for attributes. Callable things are methods when 
'a.b(c)' causes a to be the first arg of b.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Toshio Kuratomi]

On Tue, Jun 28, 2011 at 03:46:12PM +0100, Paul Moore wrote:
> On 28 June 2011 14:43, Victor Stinner  wrote:
> > As discussed before on this list, I propose to set the default encoding
> > of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
> > open() is called without an explicit encoding and if the locale encoding
> > is not UTF-8. Using the warning, you will quickly notice the potential
> > problem (using Python 3.2.2 and -Werror) on Windows or by using a
> > different locale encoding (.e.g using LANG="C").
> 
> -1. This will make things harder for simple scripts which are not
> intended to be cross-platform.
> 
> I use Windows, and come from the UK, so 99% of my text files are
> ASCII. So the majority of my code will be unaffected. But in the
> occasional situation where I use a £ sign, I'll get encoding errors,
> where currently things will "just work". And the failures will be data
> dependent, and hence intermittent (the worst type of problem). I'll
> write a quick script, use it once and it'll be fine, then use it later
> on some different data and get an error. :-(
>
I don't think this change would make things "harder".  It will just move
where the pain occurs.  Right now, the failures are intermittent on A)
computers other than the one that you're using. or B) intermittent when run
under a different user than yourself.  Sys admins where I'm at are
constantly writing ad hoc scripts in python that break because you stick
something in a cron job and the locale settings suddenly become "C" and
therefore the script suddenly only deals with ASCII characters.

I don't know that Victor's proposed solution is the best (I personally would
like it a whole lot more than the current guessing but I never develop on
Windows so I can certainly see that your environment can lead to the
opposite assumption :-) but something should change here.  Issuing a warning
like "open used without explicit encoding may lead to errors" if open() is
used without an explicit encoding would help a little (at least, people who
get errors would then have an inkling that the culprit might be an open()
call).  If I read Victor's previous email correctly, though, he said this
was previously rejected.

Another brainstorming solution would be to use different default encodings on
different platforms.  For instance, for writing files, utf-8 on *nix systems
(including macosX) and utf-16 on windows.  For reading files, check for a utf-16
BOM, if not present, operate as utf-8.  That would seem to address your
issue with detection by vim, etc but I'm not sure about getting "£" in your
input stream.  I don't know where your input is coming from and how Windows
equivalent of locale plays into that.

-Toshio


pgp7J0rQuExcz.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Terry Reedy]

On 6/28/2011 10:48 AM, Michael Foord wrote:

On 28/06/2011 15:36, Terry Reedy wrote:



S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.


Nope, it returns a bytestring in Python 2.


Which, in Py2 is a str() object. In both Pythons, .read() in default 
mode returns an object of type str() and 'abc' is an object of type 
str() and so expressions involving undecorated string literals and input 
just work, but would not work if input defaulted to bytes in Py 3. Sorry 
if I was not clear enough.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] svn.python.org confusion

[Ned Deily]

Not surprisingly, people are still attempting to browse and/or checkout 
Python source from svn.python.org.  They could be following out-of-date 
instructions from somewhere or might be expecting the svn repos are 
mirroring the hg ones.  This causes wasted time and frustration for 
users and for us following up on issues.

Could some text be added to the svn browser pages to at least indicate 
that the repo is no longer being updated with a link to the 
hg.python.org browser?   I'm not sure what to do about the repos 
themselves if people attempt to do an svn co.  Perhaps that should just 
be disabled totally for python?

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] svn.python.org confusion

[Michael Foord]

On 28/06/2011 17:37, Ned Deily wrote:

Not surprisingly, people are still attempting to browse and/or checkout
Python source from svn.python.org.  They could be following out-of-date
instructions from somewhere or might be expecting the svn repos are
mirroring the hg ones.  This causes wasted time and frustration for
users and for us following up on issues.

Could some text be added to the svn browser pages to at least indicate
that the repo is no longer being updated with a link to the
hg.python.org browser?   I'm not sure what to do about the repos
themselves if people attempt to do an svn co.  Perhaps that should just
be disabled totally for python?

For what it's worth we've had a couple of emails about this to 
webmas...@python.org (in particular because the viewc svn browser breaks 
on some latin-1 in source files).


All the best,

Michael Foord

--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snow Leopard buildbot failing again...

[Bill Janssen]

Antoine Pitrou  wrote:

> On Mon, 27 Jun 2011 10:01:13 PDT
> Bill Janssen  wrote:
> > I see that parc-snowleopard-1 went down again.  I've done a software
> > update, rebooted, and installed the latest buildslave, 0.8.4.  I can
> > ping dinsdale.python.org successfully from the machine.  However, when I
> > start the buildslave, I get this:
> 
> Well, doesn't the slave work correctly now? Or is there a problem?

It does seem to have connected, but the error message from buildslave is
new.  Let's see if it goes into the zombie state again running 0.8.4.

The Tiger buildbot (an old dual-1GHz G4 running buildbot-slave 0.8.1p1)
is the only one which seems to run reliably :-).

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Michael Foord]

On 28/06/2011 17:34, Terry Reedy wrote:

On 6/28/2011 10:48 AM, Michael Foord wrote:

On 28/06/2011 15:36, Terry Reedy wrote:



S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.


Nope, it returns a bytestring in Python 2.


Which, in Py2 is a str() object. 


Yes, but not a "text string". The equivalent of the Python 2 str in 
Python 3 is bytes. Irrelevant discussion anyway.


In both Pythons, .read() in default mode returns an object of type 
str() and 'abc' is an object of type str() and so expressions 
involving undecorated string literals and input just work, but would 
not work if input defaulted to bytes in Py 3. Sorry if I was not clear 
enough.


Well, I think you're both right. Both semantics break some assumption or 
other.


All the best,

Michael

--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Bill Janssen]

M.-A. Lemburg  wrote:

> How about a more radical change: have open() in Py3 default to
> opening the file in binary mode, if no encoding is given (even
> if the mode doesn't include 'b') ?

+1.

> That'll make it compatible to the Py2 world again and avoid
> all the encoding guessing.

Yep.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Bill Janssen]

Terry Reedy  wrote:

> > Making such default encodings depend on the locale has already
> > failed to work when we first introduced a default encoding in
> > Py2, so I don't understand why we are repeating the same
> > mistake again in Py3 (only in a different area).
> 
> I do not remember any proposed change during the Py3 design discussions.

I certainly proposed it, more than once.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Terry Reedy]

On 6/28/2011 10:46 AM, Paul Moore wrote:


I use Windows, and come from the UK, so 99% of my text files are
ASCII. So the majority of my code will be unaffected. But in the
occasional situation where I use a £ sign, I'll get encoding errors,


I do not understand this. With utf-8 you would never get a string 
encoding error.



where currently things will "just work".


As long as you only use the machine-dependent restricted character set.

> And the failures will be data dependent, and hence intermittent
> (the worst type of problem).

That is the situation now, with platform/machine dependencies added in.
Some people share code with other machines, even locally.


So, in effect, you propose making the default favour writing
multiplatform portable code at the expense of quick and dirty scripts?


Let us frame it another way. Should Python installations be compatible 
with other Python installations, or with the other apps on the same 
machine? Part of the purpose of Python is to cover up platform 
differences, to the extent possible (and perhaps sensible -- there is 
the argument). This was part of the purpose of writing our own io module 
instead of using the compiler stdlib. The evolution of floating point 
math has gone in the same direction. For instance, float now expects 
uniform platform-independent Python-dependent names for infinity and nan 
instead of compiler-dependent names.


As for practicality. Notepad++ on Windows offers ANSI, utf-8 (w,w/o 
BOM), utf-16 (big/little endian). I believe that ODF documents are utf-8 
encoded xml (compressed or not). My original claim for this proposal 
was/is that even Windows apps are moving to uft-8 and that someday 
making that the default for Python everywhere will be the obvious and 
sensible thing.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] EuroPython Language Summit report

[Bill Janssen]

Michael Foord  wrote:

> The new regex library has some great improvements:
> 
> http://bugs.python.org/issue2636
> 
> It also has users and committed maintainers, so I hope we can bring it
> into 3.3. It wasn't easy to tell from skimming the change notes that
> Unicode character classes are amongst the new features. Is that the
> case?

According to http://effbot.org/zone/unicode-objects.htm (from 2004), the
existing re module already supports Unicode character classes, so the
regex module will as well.  But the support has been updated, according
to the change notes.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Michael Foord]

On 28/06/2011 18:06, Terry Reedy wrote:

On 6/28/2011 10:46 AM, Paul Moore wrote:


I use Windows, and come from the UK, so 99% of my text files are
ASCII. So the majority of my code will be unaffected. But in the
occasional situation where I use a £ sign, I'll get encoding errors,


I do not understand this. With utf-8 you would never get a string 
encoding error.




I assumed he meant that files written out as utf-8 by python would then 
be read in using the platform encoding (i.e. not utf-8 on Windows) by 
the other applications he is inter-operating with. The error would not 
be in Python but in those applications.



where currently things will "just work".


As long as you only use the machine-dependent restricted character set.



Which is the situation he is describing. You do go into those details 
below, and which choice is "correct" depends on which trade-off you want 
to make.


For the sake of backwards compatibility we are probably stuck with the 
current trade-off however - unless we deprecate using open(...) without 
an explicit encoding.


All the best,

Michael


> And the failures will be data dependent, and hence intermittent
> (the worst type of problem).

That is the situation now, with platform/machine dependencies added in.
Some people share code with other machines, even locally.


So, in effect, you propose making the default favour writing
multiplatform portable code at the expense of quick and dirty scripts?


Let us frame it another way. Should Python installations be compatible 
with other Python installations, or with the other apps on the same 
machine? Part of the purpose of Python is to cover up platform 
differences, to the extent possible (and perhaps sensible -- there is 
the argument). This was part of the purpose of writing our own io 
module instead of using the compiler stdlib. The evolution of floating 
point math has gone in the same direction. For instance, float now 
expects uniform platform-independent Python-dependent names for 
infinity and nan instead of compiler-dependent names.


As for practicality. Notepad++ on Windows offers ANSI, utf-8 (w,w/o 
BOM), utf-16 (big/little endian). I believe that ODF documents are 
utf-8 encoded xml (compressed or not). My original claim for this 
proposal was/is that even Windows apps are moving to uft-8 and that 
someday making that the default for Python everywhere will be the 
obvious and sensible thing.





--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] EuroPython Language Summit report

[Michael Foord]

On 28/06/2011 18:08, Bill Janssen wrote:

Michael Foord  wrote:


The new regex library has some great improvements:

 http://bugs.python.org/issue2636

It also has users and committed maintainers, so I hope we can bring it
into 3.3. It wasn't easy to tell from skimming the change notes that
Unicode character classes are amongst the new features. Is that the
case?

According to http://effbot.org/zone/unicode-objects.htm (from 2004), the
existing re module already supports Unicode character classes, so the
regex module will as well.  But the support has been updated, according
to the change notes.
Thanks. Support for Unicode character classes was one of the 
improvements needed in the re module reported from the language summit - 
so I wonder if the changes in regex are sufficient.


Michael


Bill



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] EuroPython Language Summit report

[Bill Janssen]

Michael Foord  wrote:

> Thanks. Support for Unicode character classes was one of the
> improvements needed in the re module reported from the language summit
> - 
> so I wonder if the changes in regex are sufficient.

I guess it depends on what you're asking -- what does "support for
Unicode character classes" mean?  Apparently something different from
what it meant in 2004 :-).

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Antoine Pitrou]

On Tue, 28 Jun 2011 13:06:44 -0400
Terry Reedy  wrote:
> 
> As for practicality. Notepad++ on Windows offers ANSI, utf-8 (w,w/o 
> BOM), utf-16 (big/little endian).

Well, that's *one* application. We would need much more data than that.

> I believe that ODF documents are utf-8 
> encoded xml (compressed or not).

XML doesn't matter for this discussion, since it explicitly declares the
encoding. What we are talking about is "raw" text files that don't have
an encoding declaration and for which the data format doesn't specify
any default encoding (which also rules out Python source code, by the
way).

> My original claim for this proposal 
> was/is that even Windows apps are moving to uft-8
> and that someday 
> making that the default for Python everywhere will be the obvious and 
> sensible thing.

True, but that may be 5 or 10 years from now.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Terry Reedy]

On 6/28/2011 11:35 AM, Michael Foord wrote:

On 28/06/2011 16:23, Terry Reedy wrote:



So-called 'staticmethods' are not really methods either, but are class
function attributes that are just functions and not treated as
methods. The decorator that negates normal method treatment
could/should have been called 'non_method'.

Using 'function' is its generic 'callable' sense ...

Method: a class function attribute that in its intended and normal use
automagically turns the object it is called on into its first arg.
'Method' is a useful and needed subcategory of class attribute
precisely because of this behavior.

Instance method: a class function attribute that is (normally) called
on instances of the class or subclasses.


Before 2.2, these were the only methods.


So what is the difference between "Instance method" and "Method" above?
Is it just that "Method" is broader and includes class methods


Since 2.2, yes. The current glossary entry starts

"method A function which is defined inside a class body."

This includes 'staticmethods', but as I have said, I think that is a 
mistake. Static methods are functions without special method treatment. 
A class staticmethod function act the same as any other function.

Also, 'defined inside' is not necessary.

I would change the above to "A function that get bound to a instance or 
class when called as an attribute of the instance or class. Methods are 
usually defined inside a class body."



and bound methods?


The result of accessing an instance or class method via an instance or 
class. Accessing a static method does not create a bound method. Bound 
methods are usually anonymous and ephemeral, being used for one call and 
then deleted.



If anyone said "instance method" to me I would assume they meant bound
method. (A normal method fetched from an instance.)


Instance methods are the 'permanent' class function attributes, not the 
ephemeral object that implements a.b(c). Bound methods would be an 
implementation detail, except that a.b has to evaluate to something and 
saving bound methods is quite handy when calling a method or methods 
repeatedly on the same instance.


Functools.partial is a generalization of bound methods, which were 
Python's first (special-case) implementation of the partial function 
idea. Leaving out implementation details, if b is a function attribute 
of type(a), 'a.b' is an abbreviated way of writing 
'functools.partial(type(a).b,a)' (and yes, I have tested an example of 
this).



Class method: a class function attribute that is (normally) called on
the class or subclasses.

Bound method: a method that has already has the first-arg object
bundled with it, so that it can be used as a normal (partial or
curried) function.


---
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] EuroPython Language Summit report

[Georg Brandl]

On 28.06.2011 16:06, Victor Stinner wrote:
> Le mardi 28 juin 2011 à 14:41 +0100, Michael Foord a écrit :
>> On 24/06/2011 15:55, R. David Murray wrote:
>> > On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson  
>> > wrote:
>> >> EuroPython 2011 Language Summit
>> >> ===
>> > [...]
>> >>  Unicode character classes is a particular need).  [Subtopic:  what 
>> >> needs
>> >>  to be done to get the new regex module into Python?  Should it 
>> >> replace
>> >>  the existing module?  What about backwards compatibility issues?]
>> > I'm pretty sure regex has backward compatibility as a goal for just this
>> > reason (so it can replace the current module).
>> >
>> The new regex library has some great improvements:
>> 
>>  http://bugs.python.org/issue2636
> 
> This issue is open since April 2008, has also the longest list of
> attached files, and has a very long history. What is the status of the
> issue? I see that there is now a third party project on:
> 
>http://code.google.com/p/mrab-regex-hg/

This should be the same module as in the issue (and thankfully, because
code management of such a big project does not belong exclusively in a
tracker issue).  "mrab" stands for Matthew Barnett, who is the author of
regex.

> --
> 
> There is also the re2 library from Google and especially this project:
> 
>http://pypi.python.org/pypi/re2/
> 
> "pyre2 is a Python extension that wraps Google's RE2 regular expression
> library.
> 
> This version of pyre2 is similar to the one you'd find at facebook's
> github repository except that the stated goal of *this version is to be
> a drop-in replacement for the re module*.)"

Well, while it can be called drop-in, it is hardly a good replacement:

"""
That being said, there are features of the re module that this module may never
have. For example, RE2 does not handle lookahead assertions ((?=...)).
"""

It falls back to old re in these cases.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Georg Brandl]

On 28.06.2011 19:06, Terry Reedy wrote:
> On 6/28/2011 10:46 AM, Paul Moore wrote:
> 
>> I use Windows, and come from the UK, so 99% of my text files are
>> ASCII. So the majority of my code will be unaffected. But in the
>> occasional situation where I use a £ sign, I'll get encoding errors,
> 
> I do not understand this. With utf-8 you would never get a string 
> encoding error.

Yes, but you'll get plenty of *decoding* errors.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Ethan Furman]

Michael Foord wrote:

On 28/06/2011 17:34, Terry Reedy wrote:

On 6/28/2011 10:48 AM, Michael Foord wrote:

On 28/06/2011 15:36, Terry Reedy wrote:



S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.


Nope, it returns a bytestring in Python 2.


Which, in Py2 is a str() object. 


Yes, but not a "text string". The equivalent of the Python 2 str in 
Python 3 is bytes. Irrelevant discussion anyway.


Irrelevant to the OP, yes, but a Python 2 string *is not* the same as 
Python 3 bytes.  If you don't believe me fire up your Python 3 shell and 
try b'xyz'[1] == 'y'.


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Ethan Furman]

Ethan Furman wrote:

Michael Foord wrote:

On 28/06/2011 17:34, Terry Reedy wrote:

On 6/28/2011 10:48 AM, Michael Foord wrote:

On 28/06/2011 15:36, Terry Reedy wrote:



S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.


Nope, it returns a bytestring in Python 2.


Which, in Py2 is a str() object. 


Yes, but not a "text string". The equivalent of the Python 2 str in 
Python 3 is bytes. Irrelevant discussion anyway.


Irrelevant to the OP, yes, but a Python 2 string *is not* the same as 
Python 3 bytes.  If you don't believe me fire up your Python 3 shell and 
try b'xyz'[1] == 'y'.


er, make that b'xyz'[1] == b'y'  :(
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue10403 - using 'attributes' instead of members in documentation

[Brett Cannon]

On Tue, Jun 28, 2011 at 06:28, Nick Coghlan  wrote:

> On Tue, Jun 28, 2011 at 10:56 PM, Michael Foord
>  wrote:
> > I don't think "data attributes" is clear or precise. Is a property a data
> > attribute (well it depends how it is implemented and what it does), is a
> > staticmethod a data attribute (no - but then Tres' question - it isn't a
> > normal method either so if you define data attribute to mean "all non
> method
> > attributes" then its potentially an open question).
>
> "callable attributes" and "data attributes" are purely about the API
> exposed by the object in question. If we're going for object model
> neutral terminology, that's the only view that makes sense. They're
> descriptions about how attributes are intended to be *used* that are
> completely silent on the question of how they're *implemented*. So
> staticmethod would fall into the first group, while property would
> fall into the latter.
>

These two terms also have the benefit of being understandable by non Python
programmers. Since we can't rely on people read the glossary or any other
specific doc before reading the stdlib docs we need to make sure that the
terminology that we use can be understood by newbies. Thus I like data and
callable attributes since it makes sense even if you come from another
programming language since you can easily reason it out.

And now I am done adding my graffiti to the shed.

-Brett


>
> >From an implementation point of view, you carve up the world
> differently, so it makes sense to worry about class attributes,
> instance attributes, dynamic attributes, etc. (the class vs instance
> distinction can also matter to some degree from the usage point of
> view, since it affects the scope of any mutable attributes, and the
> static vs dynamic distinction can also matter, especially for
> introspection purposes).
>
> This goes back to the original point about all of this being highly
> context dependent - how you carve up the set of all attributes is
> going to change based on what you're trying to explain (e.g. the
> distinction between enforced 'data' descriptors, descriptors that
> allow shadowing in the instance dict, class attributes that aren't
> descriptors at all, instance attributes and dynamic attributes
> retrieved via __getattr__ is another way of dividing them)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Paul Moore]

On 28 June 2011 18:22, Michael Foord  wrote:
> On 28/06/2011 18:06, Terry Reedy wrote:
>>
>> On 6/28/2011 10:46 AM, Paul Moore wrote:
>>
>>> I use Windows, and come from the UK, so 99% of my text files are
>>> ASCII. So the majority of my code will be unaffected. But in the
>>> occasional situation where I use a £ sign, I'll get encoding errors,
>>
>> I do not understand this. With utf-8 you would never get a string encoding
>> error.
>>
>
> I assumed he meant that files written out as utf-8 by python would then be
> read in using the platform encoding (i.e. not utf-8 on Windows) by the other
> applications he is inter-operating with. The error would not be in Python
> but in those applications.

That is correct. Or files written out (as platform encoding) by other
applications, will later be read in as UTF-8 by Python, and be seen as
incorrect characters, or worse raise decoding errors. (Sorry, in my
original post I said "encoding" where I meant "decoding"...)

I'm not interested in allocating "blame" for the "error". I'm not
convinced that it *is* an error, merely 2 programs with incompatible
assumptions. What I'm saying is that compatibility between various
programs on a single machine can, in some circumstances, be more
important than compatibility between (the same, or different) programs
running on different machines or OSes. And that I, personally, am in
that situation.

>>> where currently things will "just work".
>>
>> As long as you only use the machine-dependent restricted character set.
>>
>
> Which is the situation he is describing. You do go into those details below,
> and which choice is "correct" depends on which trade-off you want to make.
>
> For the sake of backwards compatibility we are probably stuck with the
> current trade-off however - unless we deprecate using open(...) without an
> explicit encoding.

Backward compatibility is another relevant point. But other than that,
it's a design trade-off, agreed. All I'm saying is that I see the
current situation (which is in favour of quick script use and beginner
friendly at the expense of conceptual correctness and forcing the user
to think about his choices) as being preferable (and arguably more
"Pythonic", in the sense that I see it as a case of "practicality
beats purity" - although it's easy to argue that "in the face of
ambiguity..." also applies here :-))

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Victor Stinner]

Le mardi 28 juin 2011 à 16:02 +0200, M.-A. Lemburg a écrit :
> How about a more radical change: have open() in Py3 default to
> opening the file in binary mode, if no encoding is given (even
> if the mode doesn't include 'b') ?

I tried your suggested change: Python doesn't start.

sysconfig uses the implicit locale encoding to read sysconfig.cfg, the
Makefile and pyconfig.h. I think that it is correct to use the locale
encoding for Makefile and pyconfig.h, but maybe not for sysconfig.cfg.

Python require more changes just to run "make". I was able to run "make"
by using encoding='utf-8' in various functions (of distutils and
setup.py). I didn't try the test suite, I expect too many failures.

--

Then I tried my suggestion (use "utf-8" by default): Python starts
correctly, I can build it (run "make") and... the full test suite pass
without any change. (I'm testing on Linux, my locale encoding is UTF-8.)

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Georg Brandl]

On 28.06.2011 14:24, Terry Reedy wrote:

>> As discussed before on this list, I propose to set the default encoding
>> of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
>> open() is called without an explicit encoding and if the locale encoding
>> is not UTF-8. Using the warning, you will quickly notice the potential
>> problem (using Python 3.2.2 and -Werror) on Windows or by using a
>> different locale encoding (.e.g using LANG="C").

[...]

>> Should I convert this email into a PEP, or is it not required?
> 
> I think a PEP is needed.

Absolutely.  And I hope the hypothetical PEP would be rejected in this form.

We need to stop making incompatible changes to Python 3.  We had the chance
and took it to break all kinds of stuff, some of it gratuitous, with 3.0 and
even 3.1.  Now the users need a period of compatibility and stability (just
like the language moratorium provided for one aspect of Python).

Think about porting: Python 3 uptake is not ahead of time (I don't want to
say it's too slow, but it's certainly not too fast.)  For the sake of porters'
sanity, 3.x should not be a moving target.  New features are not so much of
a problem, but incompatibilities like this one certainly are.

At the very least, a change like this needs a transitional strategy, like
it has been used during the 2.x series:

* In 3.3, accept "locale" as the encoding parameter, meaning the locale encoding
* In 3.4, warn if encoding isn't given and the locale encoding isn't UTF-8
* In 3.5, change default encoding to UTF-8

It might be just enough to stress in the documentation that usage of the
encoding parameter is recommended for cross-platform consistency.

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Victor Stinner]

> > I don't think that Windows developer even know that they are writing
> > files into the ANSI code page. MSDN documentation of
> > WideCharToMultiByte() warns developer that the ANSI code page is not
> > portable, even accross Windows computers:
> 
> Probably true. But for many uses they also don't care. If you're
> writing something solely for a one-off job on your own PC, the ANSI
> code page is fine, and provides interoperability with other programs
> on your PC, which is really what you care about. (UTF-8 without BOM
> displays incorrectly in Vim, wordpad, and powershell get-content.

I tried to open a text file encoded to UTF-8 (without BOM) on Windows
Seven.

The default application displays it correctly, it's the well known
builtin notepad program.

gvim is unable to detect the encoding, it reads the file using the ANSI
code page (WTF? UTF-8 is correctly detected on Linux!?). 

Wordpad reads the file using the ANSI code page, it is unable to detect
the UTF-8 encoding.

The "type" command in a MS-Dos shell (cmd.exe) dosen't display the UTF-8
correctly, but a file encoded to ANSI code is also display incorrectly.
I suppose that the problem is that the terminal uses the OEM code page,
not the ANSI code page.

Visual C++ 2008 detects the UTF-8 encoding.

I don't have other applications to test on my Windows Seven. I agree
that UTF-8 is not well supported by "standard" Windows applications. I
would at least expect that Wordpad and gvim are able to detect the UTF-8
encoding.

> MBCS works fine in all of these. It also displays incorrectly in CMD type,
> but in a less familiar form than the incorrect display mbcs produces,
> for what that's worth...)

True, the encoding of a text file encoded to the ANSI code page is
correctly detected by all applications (except "type" in a shell, it
should be the OEM/ANSI code page conflict).

> IMHO, you missed another option - open() does not need improving, the
> current behaviour is better than any of the 3 options noted.

My original need is to detect that my program will behave differently on
Linux and Windows, because open() uses the implicit locale encoding.
Antoine suggested me to monkeypatch __builtins__.open to do that.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/28/2011 12:52 PM, Bill Janssen wrote:
> M.-A. Lemburg  wrote:
> 
>> How about a more radical change: have open() in Py3 default to
>> opening the file in binary mode, if no encoding is given (even
>> if the mode doesn't include 'b') ?
> 
> +1.
> 
>> That'll make it compatible to the Py2 world again and avoid
>> all the encoding guessing.
> 
> Yep.

+1 from me, as well:  "in the face of ambiguity, refuse the temptation
sto guess."


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4KVbwACgkQ+gerLs4ltQ4gPACgoWGjAhmOg9IGQgMht2KsZYn5
mKUAnjcLP6BGCVFSudm1v77ZHere3VHw
=4SPu
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Victor Stinner]

Le mardi 28 juin 2011 à 09:33 -0700, Toshio Kuratomi a écrit :
> Issuing a warning like "open used without explicit encoding may lead
> to errors" if open() is used without an explicit encoding would help
> a little (at least, people who get errors would then have an inkling
> that the culprit might be an open() call).  If I read Victor's previous
> email correctly, though, he said this was previously rejected.

Oh sorry, I used the wrong word. I listed two other possible solutions,
but there were not really rejetected. I just thaugh that changing the
default encoding to UTF-8 was the most well accepted idea.

If I mix different suggestions together: another solution is to emit a
warning if the encoding is not specified (not only if the locale
encoding is different than UTF-8). Using encoding="locale" would make it
quiet. It would be annoying if the warning would be displayed by default
("This will make things harder for simple scripts which are not
intended to be cross-platform." wrote Paul Moore). It only makes sense
if we use the same policy than unclosed files/sockets: hidden by
default, but it can be configured using command line options (-Werror,
yeah!).

> Another brainstorming solution would be to use different default encodings on
> different platforms.  For instance, for writing files, utf-8 on *nix systems
> (including macosX) and utf-16 on windows.

I don't think that UTF-16 is a better choice than UTF-8 on Windows :-(

> For reading files, check for a utf-16 BOM, if not present, operate as utf-8.

Oh oh. I already suggested to read the BOM. See
http://bugs.python.org/issue7651 and read the email thread "Improve
open() to support reading file starting with an unicode BOM"
http://mail.python.org/pipermail/python-dev/2010-January/097102.html

Reading the BOM is a can of worm, everybody expects something different.
I forgot the idea of changing that.

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Terry Reedy]

On 6/28/2011 5:42 PM, Georg Brandl wrote:


At the very least, a change like this needs a transitional strategy, like
it has been used during the 2.x series:

* In 3.3, accept "locale" as the encoding parameter, meaning the locale encoding
* In 3.4, warn if encoding isn't given and the locale encoding isn't UTF-8
* In 3.5, change default encoding to UTF-8


3.5 should be 4-5 years off. I actually would not propose anything 
faster than that.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 380 acceptance (was Re: EuroPython Language Summit report)

[renaud]

Nick Coghlan  gmail.com> writes:

> 
> On Tue, Jun 28, 2011 at 1:09 AM, renaud  gmail.com> wrote:
> > Nick Coghlan  gmail.com> writes:
> >
> >> I hit a snag with this. The real tests of the PEP 380 functionality
> >> aren't currently part of the patch - they're a big set of "golden
> >> output" tests in the zipfile hosted on Greg's site. Those need to be
> >> refactored into proper unittest or doctest based additions to the test
> >> suite and incorporated into the patch before I could commit this with
> >> a clear conscience.
> >
> > let me know if i can help.
> 
> It would be good if you could take a look at Greg's original test
> suite, consider ways of bringing it into the main regression tests and
> then update the patch queue on bitbucket accordingly.
> 
> My preference is for something unittest based, essentially taking the
> "golden output" comparisons and turning them into appropriate
> self.assert* invocations.
> 
> Given the number of tests Greg has, it will probably make more sense
> to do it as a new test subdirectory rather than as a single test file
> (although that depends on how many tests are in each file - if there
> are only a few, or if they overlap a lot, then having them as separate
> test cases within a single file may be a better choice).

ok, i've generated a single test_pep380.py using greg tests wrapped to be
run by unittest.
it's ugly, but it does the job.
some things that may not be desirable:
- line numbers in the tracebacks do rely on the position of the tests in the 
file, so any editing before the last test case will probably screw up 
everything;
- the test touches sys.stdout & sys.stderr

the whole thing is available here:


renaud

ps. i had to edit the test24 (Test parser module) expected output to make it
match the actual output, i guess that the parser module has changed since greg
wrote the tests.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] svn.python.org confusion

[Vinay Sajip]

Ned Deily  acm.org> writes:

> Could some text be added to the svn browser pages to at least indicate 
> that the repo is no longer being updated with a link to the 
> hg.python.org browser?   I'm not sure what to do about the repos 
> themselves if people attempt to do an svn co.  Perhaps that should just 
> be disabled totally for python?

At least some of the stuff in the SVN repo is still needed, AFAICT. I recently
did a build of the Python Windows binary installer, and I understand (from the
buildbot instructions) that the best source of some of the external dependencies
(bz2, Tcl/Tk, openssl etc.) is still the SVN repo.

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Stefan Behnel]

Victor Stinner, 28.06.2011 15:43:

In Python 2, open() opens the file in binary mode (e.g. file.readline()
returns a byte string). codecs.open() opens the file in binary mode by
default, you have to specify an encoding name to open it in text mode.

In Python 3, open() opens the file in text mode by default. (It only
opens the binary mode if the file mode contains "b".) The problem is
that open() uses the locale encoding if the encoding is not specified,
which is the case *by default*. The locale encoding can be:

  - UTF-8 on Mac OS X, most Linux distributions
  - ISO-8859-1 os some FreeBSD systems
  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
Western Europe, cp952 in Japan, ...
  - ASCII if the locale is manually set to an empty string or to "C", or
if the environment is empty, or by default on some systems
  - something different depending on the system and user configuration...

If you develop under Mac OS X or Linux, you may have surprises when you
run your program on Windows on the first non-ASCII character. You may
not detect the problem if you only write text in english... until
someone writes the first letter with a diacritic.


I agree that this is a *very* common source of problems. People write code 
that doesn't care about encodings all over the place, and are then 
surprised when it stops working at some point, either by switching 
environments or by changing the data. I've seen this in virtually all 
projects I've ever come to work in[1]. So, eventually, all of that code was 
either thrown away or got debugged and fixed to use an explicit (and 
usually configurable) encoding.


Consequently, I don't think it's a bad idea to break out of this ever 
recurring development cycle by either requiring an explicit encoding right 
from the start, or by making the default encoding platform independent. The 
opportunity to fix this was very unfortunately missed in Python 3.0.


Personally, I don't buy the argument that it's harder to write quick 
scripts if an explicit encoding is required. Most code that gets written is 
not just quick scripts, and even those tend to live longer than initially 
intended.


Stefan


[1] Admittedly, most of those projects were in Java, where the situation is 
substantially worse than in Python. Java entirely lacks a way to define a 
per-module source encoding, and it even lacks a straight forward way to 
encode/decode a file with an explicit encoding. So, by default, *both* 
input encodings are platform dependent, whereas in Python it's only the 
default file encoding, and properly decoding a file is straight forward there.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] open(): set the default encoding to 'utf-8' in Python 3.3?

[Nick Coghlan]

On Wed, Jun 29, 2011 at 7:42 AM, Georg Brandl  wrote:
> On 28.06.2011 14:24, Terry Reedy wrote:
>> I think a PEP is needed.
>
> Absolutely.  And I hope the hypothetical PEP would be rejected in this form.
>
> We need to stop making incompatible changes to Python 3.  We had the chance
> and took it to break all kinds of stuff, some of it gratuitous, with 3.0 and
> even 3.1.  Now the users need a period of compatibility and stability (just
> like the language moratorium provided for one aspect of Python).

+1 to everything Georg said.

- nothing can change in 3.2
- perhaps provide a way for an application to switch the default
behaviour between 'locale' and 'utf-8' in 3.3
- if this is done, also provide a way to explicitly request the
'locale' behaviour (likely via a locale dependent codec alias)
- maybe start thinking about an actual transition to 'utf-8' as
default in the 3.4/5 time frame

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] svn.python.org confusion

[Barry Warsaw]

On Jun 29, 2011, at 01:05 AM, Vinay Sajip wrote:

>Ned Deily  acm.org> writes:
>
>> Could some text be added to the svn browser pages to at least indicate 
>> that the repo is no longer being updated with a link to the 
>> hg.python.org browser?   I'm not sure what to do about the repos 
>> themselves if people attempt to do an svn co.  Perhaps that should just 
>> be disabled totally for python?
>
>At least some of the stuff in the SVN repo is still needed, AFAICT. I recently
>did a build of the Python Windows binary installer, and I understand (from the
>buildbot instructions) that the best source of some of the external 
>dependencies
>(bz2, Tcl/Tk, openssl etc.) is still the SVN repo.

In fact, it's also not entirely true that svn isn't being updated.  We're
still cutting any 2.5 and 2.6 releases from svn, though we're also porting the
changes into hg.

-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com