[Python-Dev] Updated version of PEP-0397 - Python launcher for Windows.

2011-05-17 Thread Mark Hammond

Hi all,
  I've updated PEP-0397 to try and address some of the comments from 
the last draft.  I've checked the new version into hg, so you can find a 
full diff there, but the key items I've changed are:


* Spelled out the "version qualifier" rules for the shebang lines.
* Spelled out some customization options, both for fine-tuning the 
specific Python version selected and for supporting other Python 
implementations via "custom" commands.

* Indicated the launcher is not supported at all on Win2k or earlier.
* Removed some cruft.

The new version is attached and I welcome all comments, including 
bike-shedding on the environment variable names and INI section/value names.


Note that the reference implementation has not changed - I'll update 
that once there is general agreement on the functionality described in 
the PEP.


Thanks,

Mark
PEP: 397
Title: Python launcher for Windows
Version: $Revision$
Last-Modified: $Date$
Author: Mark Hammond 
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 15-Mar-2011
Post-History: 17-May-2011, 15-Mar-2011

Abstract

This PEP describes a Python launcher for the Windows platform.  A 
Python launcher is a single executable which uses a number of 
heuristics to locate a Python executable and launch it with a
specified command line.


Rationale

Windows provides "file associations" so an executable can be associated
with an extension, allowing for scripts to be executed directly in some
contexts (eg., double-clicking the file in Windows Explorer.)  Until now,
a strategy of "last installed Python wins" has been used and while not
ideal, has generally been workable due to the conservative changes in
Python 2.x releases.  As Python 3.x scripts are often syntactically
incompatible with Python 2.x scripts, a different strategy must be used
to allow files with a '.py' extension to use a different executable based
on the Python version the script targets.  This will be done by borrowing
the existing practices of another operating system - scripts will be able
to nominate the version of Python they need by way of a "shebang" line, as
described below.

Unix-like operating systems (referred to simply as "Unix" in this
PEP) allow scripts to be executed as if they were executable images
by examining the script for a "shebang" line which specifies the
actual executable to be used to run the script.  This is described in
detail in the evecve(2) man page [1] and while user documentation will
be created for this feature, for the purposes of this PEP that man
page describes a valid shebang line.

Additionally, these operating systems provide symbolic-links to
Python executables in well-known directories. For example, many
systems will have a link /usr/bin/python which references a
particular version of Python installed under the operating-system.
These symbolic links allow Python to be executed without regard for
where Python it actually installed on the machine (eg., without
requiring the path where Python is actually installed to be
referenced in the shebang line or in the PATH.)  PEP 394 'The "python"
command on Unix-Like Systems' [2] describes additional conventions
for more fine-grained specification of a particular Python version.

These 2 facilities combined allow for a portable and somewhat 
predictable way of both starting Python interactively and for allowing
Python scripts to execute.  This PEP describes an implementation of a 
launcher which can offer the same benefits for Python on the Windows 
platform and therefore allows the launcher to be the executable
associated with '.py' files to support multiple Python versions
concurrently.

While this PEP offers the ability to use a shebang line which should
work on both Windows and Unix, this is not the primary motivation for
this PEP - the primary motivation is to allow a specific version to be
specified without inventing new syntax or conventions to describe it.

An overview of the launcher.

This PEP outlines the general functionality and key guidelines of a
launcher for Windows.  It is accompanied by a reference implementation
[3], written in Python, which defines the detailed implementation.  Over
time, changes to the implementation may be desired - if the changes 
adhere to the guidelines in this PEP and have been made following 
the standard Python development model this PEP need not change.  
In other words, this PEP makes no attempt to describe in detail every 
feature offered by the launcher but instead to offer guidelines the 
launcher should adhere to.

The actual implementation of the launcher will be written in C and
will not link directly or indirectly with Python, but the Python
based reference implementation should always remain the canonical
description of the functi

[Python-Dev] Success x86 XP-4 2.7 buildbot without any log and should be a failure

2011-05-17 Thread Victor Stinner
Hi,

I broke recently all tests of CJK encodings (#12057) in Python 2.7 (sorry, it 
is now fixed). But the "x86 XP-4 2.7" buildbot is green, I don't understand 
how (the bug was not fixed in the build 894):

http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%202.7/builds/894

This build doesn't contain any log.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] "packaging" merge imminent

2011-05-17 Thread Tarek Ziadé
Hello

I am about to merge packaging in the stdlib, and we will continue our
work there :)

The impact is:

- addition of Lib/packaging
- addition of test/test_packaging.py
- changes in Lib/sysconfig.py
- addition of Lib/sysconfig.cfg

For the last one, I would like to make sure again that everyone is ok
with having a .cfg file added in the Lib/ directory. If not, we need
to discuss how to do this differently.

== purpose of sysconfig.cfg ==

The sysconfig.cfg file is a ini-like file that sysconfig.py reads to
get the installation paths. We currently have these paths harcoded in
the python module.

The next change I have planned is to allow several levels of
configuration, like distutils.cfg does. sysconfig.py will look for a
sysconfig.cfg file in these places:

1. the current working directory -- so can be potentially included in
a project source release
2. the user home  (specific location be defined, maybe in ~/local)
[inherits from the previous one]
3. the global
[inherits from the previous one]

I have decided to make it a .cfg file instead of a .py file for various reasons:

- easier for people to edit, without the danger of ending-up with an
over-engineered python module (that's the problem we have with
setup.py files)
- the override logic is easier to implement and understand: if I want
to change a single path, I add a ini file in my home with this single
path.

If I have no complains, the merge will happen tomorrow of my time

== next moves ==

- make sysconfig.py stop reading Makefile and pyconfig.h, this will be
done by adding a _sysconfig.py file created by the Makefile
- continue our work in packaging for 3.3
- we're planning to merge the doc in Doc/packaging very soon (still
working on it)


Cheers
Tarek

-- 
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Christian Heimes
Am 17.05.2011 17:36, schrieb Tarek Ziadé:
> The next change I have planned is to allow several levels of
> configuration, like distutils.cfg does. sysconfig.py will look for a
> sysconfig.cfg file in these places:
> 
> 1. the current working directory -- so can be potentially included in
> a project source release
> 2. the user home  (specific location be defined, maybe in ~/local)
> [inherits from the previous one]
> 3. the global

You may want to study my site package PEP [1] regarding possible
security implications. I recommend that you ignore the current working
directory and user's home directory under conditions like different
effective user or the -E option.

A good place for a local sysconfig.cfg could be the user's stdlib
directory (e.g. ~/.local/lib/python3.2/sysconfig.cfg).

Christian

[1] http://www.python.org/dev/peps/pep-0370
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Bug in json (the format and the module)

2011-05-17 Thread Dirkjan Ochtman
On Tue, May 17, 2011 at 19:40, Jeremy Dunck  wrote:
> So, to start with, is there a maintainer for the json module, or how
> should I go about discussing implementing this solution?

Your subject states that there is an actual bug in the json module,
but your message fails to mention any actual bug. Is this what you
mean?

Python 2.7.1 (r271:86832, Mar 28 2011, 09:54:04)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> print json.dumps(u'foo\u2028bar')
"foo\u2028bar"

Cheers,

Dirkjan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Ronald Oussoren

On 17 May, 2011, at 17:36, Tarek Ziadé wrote:

> Hello
> 
> I am about to merge packaging in the stdlib, and we will continue our
> work there :)
> 
> The impact is:
> 
> - addition of Lib/packaging
> - addition of test/test_packaging.py
> - changes in Lib/sysconfig.py
> - addition of Lib/sysconfig.cfg
> 
> For the last one, I would like to make sure again that everyone is ok
> with having a .cfg file added in the Lib/ directory. If not, we need
> to discuss how to do this differently.
> 
> == purpose of sysconfig.cfg ==
> 
> The sysconfig.cfg file is a ini-like file that sysconfig.py reads to
> get the installation paths. We currently have these paths harcoded in
> the python module.
> 
> The next change I have planned is to allow several levels of
> configuration, like distutils.cfg does. sysconfig.py will look for a
> sysconfig.cfg file in these places:
> 
> 1. the current working directory -- so can be potentially included in
> a project source release

Does this mean that python behaves differently when there happens to be a 
sysconfig.cfg file in the current working directory? That's a potentional 
security risk.  


> 2. the user home  (specific location be defined, maybe in ~/local)
> [inherits from the previous one]

How hard would it be to disable this behavior for tools like virtualenv and 
py2app?

Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Bug in json (the format and the module)

2011-05-17 Thread Jeremy Dunck
On Tue, May 17, 2011 at 1:21 PM, Dirkjan Ochtman  wrote:
> On Tue, May 17, 2011 at 19:40, Jeremy Dunck  wrote:
>> So, to start with, is there a maintainer for the json module, or how
>> should I go about discussing implementing this solution?
>
> Your subject states that there is an actual bug in the json module,
> but your message fails to mention any actual bug. Is this what you
> mean?
>
> Python 2.7.1 (r271:86832, Mar 28 2011, 09:54:04)
> [GCC 4.4.5] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
 import json
 print json.dumps(u'foo\u2028bar')
> "foo\u2028bar"

Actually, that would be fine, and Bob's right that this is a non-issue
with ensure_ascii=True (the default).  His change upstream seems good
for the ensure_ascii=False case.

To be complete, this is what I meant:

>>> s = '{"JSON":"ro
cks!"}' # this string has a literal U+2028 in it
>>> s
'{"JSON":"ro\xe2\x80\xa8cks!"}'

>>> import json
>>> json.dumps(s) # fine by default
'"{\\"JSON\\":\\"ro\\u2028cks!\\"}"'

>>> json.dumps(s, ensure_ascii=False) # not fine with ensure_ascii=False
'"{\\"JSON\\":\\"ro\xe2\x80\xa8cks!\\"}"'
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] [RELEASED] Python 3.2.1 rc 1

2011-05-17 Thread Georg Brandl
On behalf of the Python development team, I am pleased to announce the
first release candidate of Python 3.2.1.

Python 3.2.1 will the first bugfix release for Python 3.2, fixing over 120
bugs and regressions in Python 3.2.

For an extensive list of changes and features in the 3.2 line, see

http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2.1 visit:

http://www.python.org/download/releases/3.2.1/

This is a testing release: Please consider trying Python 3.2.1 with your code
and reporting any bugs you may notice to:

http://bugs.python.org/


Enjoy!

-- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.2's contributors)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Victor Stinner
Le mardi 17 mai 2011 à 17:36 +0200, Tarek Ziadé a écrit :
> - addition of Lib/packaging
> - addition of test/test_packaging.py
> - changes in Lib/sysconfig.py
> - addition of Lib/sysconfig.cfg

Does setup.py continue to use the "old" distutils module?

I fixed recently some bugs in distutils. Should I also fix them in the
packaging module, or are both modules already "synchronized"?

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Tarek Ziadé
On Tue, May 17, 2011 at 7:21 PM, Ronald Oussoren  wrote:
...
>> 1. the current working directory -- so can be potentially included in
>> a project source release
>
> Does this mean that python behaves differently when there happens to be a 
> sysconfig.cfg file in the current working directory? That's a potentional 
> security risk.

The use case is to have it there at install time so packaging can have
alternative locations if needed.

We could also drop the working dir scanning and have it:

1- passed explicitly to the pysetup script via an option.
2- used only if found in a root of a project at installation time, and only then

>
>> 2. the user home  (specific location be defined, maybe in ~/local)
>> [inherits from the previous one]
>
> How hard would it be to disable this behavior for tools like virtualenv and 
> py2app?

Not hard at all, just an option. And the goal is also to allow
virtualenv to have its own copy, like it does for distutils.cfg

>
> Ronald



-- 
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Tarek Ziadé
On Tue, May 17, 2011 at 10:40 PM, Victor Stinner
 wrote:
> Le mardi 17 mai 2011 à 17:36 +0200, Tarek Ziadé a écrit :
>> - addition of Lib/packaging
>> - addition of test/test_packaging.py
>> - changes in Lib/sysconfig.py
>> - addition of Lib/sysconfig.cfg
>
> Does setup.py continue to use the "old" distutils module?

Yes. The plan is to keep distutils support, so projects with setup.py
still work.

For the new packaging, people will have to provide new sections in setup.cfg.

The pysetup script will detect at installation time if the project has
the required bits in the cfg, and if not will fallback to executing
setup.py


> I fixed recently some bugs in distutils. Should I also fix them in the
> packaging module, or are both modules already "synchronized"?

They need to be backported yes. We did some, but we'll need to double
check distutils timeline to make sure it's synced

>
> Victor
>
>



-- 
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Tarek Ziadé
On Tue, May 17, 2011 at 6:42 PM, Christian Heimes  wrote:
> Am 17.05.2011 17:36, schrieb Tarek Ziadé:
>> The next change I have planned is to allow several levels of
>> configuration, like distutils.cfg does. sysconfig.py will look for a
>> sysconfig.cfg file in these places:
>>
>> 1. the current working directory -- so can be potentially included in
>> a project source release
>> 2. the user home  (specific location be defined, maybe in ~/local)
>> [inherits from the previous one]
>> 3. the global
>
> You may want to study my site package PEP [1] regarding possible
> security implications. I recommend that you ignore the current working
> directory and user's home directory under conditions like different
> effective user or the -E option.

Sounds good, thanks


> A good place for a local sysconfig.cfg could be the user's stdlib
> directory (e.g. ~/.local/lib/python3.2/sysconfig.cfg).

Yes, so, part of the packaging imcoming work will be to relocate all
user .cfg files in the same, python-specific place.

That includes pydistutils.cfg, and pypirc. I remember we did talk
about that a few months ago, and will restart this discussion asap


> Christian
>
> [1] http://www.python.org/dev/peps/pep-0370
>



-- 
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Python 3.x and bytes

2011-05-17 Thread Ethan Furman

The bytes type in Python 3 does not feel very consistent.

For example:

--> some_var = 'abcdef'
--> some_var
'abcdef'
--> some_var[3]
'd'
--> some_other_var = b'abcdef'
--> some_other_var
b'abcdef'
--> some_other_var[3]
100


On the one hand we have the 'bytes are ascii data' type interface, and 
on the other we have the 'bytes are a list of integers between 0 - 256' 
interface.  And trying to use the two is not intuitive:


--> some_other_var[3] == b'd'
False

When I'm parsing a .dbf file and extracting field types from the byte 
stream, I'm not thinking, "okay, 67 is a Character field" -- what I'm 
thinking is, "b'C' is a Character field".


Considering that ord() still works fine, I'm not sure why it was done 
this way.


Is there code out there that is using this "list of int's" interface, or 
is there time to make changes to bytes?


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Benjamin Peterson
2011/5/17 Ethan Furman :
> Considering that ord() still works fine, I'm not sure why it was done this
> way.

I agree that this change was unfortunate and not too useful in practice.

>
> Is there code out there that is using this "list of int's" interface, or is
> there time to make changes to bytes?

I don't doubt there is, and I'm afraid it's far to late to change this.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Raymond Hettinger

On May 17, 2011, at 5:27 PM, Ethan Furman wrote:

> The bytes type in Python 3 does not feel very consistent.
> 
> For example:
> 
> --> some_var = 'abcdef'
> --> some_var
> 'abcdef'
> --> some_var[3]
> 'd'
> --> some_other_var = b'abcdef'
> --> some_other_var
> b'abcdef'
> --> some_other_var[3]
> 100
> 
> 
> On the one hand we have the 'bytes are ascii data' type interface,

This is incidental.  Bytes can and often do contain data with non-ascii encoded 
text,  plain binary data, or structs, or raw data read off a disk, etc.

> and on the other we have the 'bytes are a list of integers between 0 - 256' 
> interface.  And trying to use the two is not intuitive:
> 
> --> some_other_var[3] == b'd'
> False
> 
> When I'm parsing a .dbf file and extracting field types from the byte stream, 
> I'm not thinking, "okay, 67 is a Character field" -- what I'm thinking is, 
> "b'C' is a Character field".
> 
> Considering that ord() still works fine, I'm not sure why it was done this 
> way.
> 
> Is there code out there that is using this "list of int's" interface,

Yes.

> or is there time to make changes to bytes?

No.


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Ned Deily
In article ,
 Tarek Ziadé  wrote:
> On Tue, May 17, 2011 at 10:40 PM, Victor Stinner
>  wrote:
> > Le mardi 17 mai 2011 à 17:36 +0200, Tarek Ziadé a écrit :
> >> - addition of Lib/packaging
> >> - addition of test/test_packaging.py
> >> - changes in Lib/sysconfig.py
> >> - addition of Lib/sysconfig.cfg
> >
> > Does setup.py continue to use the "old" distutils module?
> 
> Yes. The plan is to keep distutils support, so projects with setup.py
> still work.

Just to be clear: what about for the build of the interpreter itself, 
i.e. its setup.py for the standard library extension modules?  Will the 
existing distutils code continue to be used for that?  Or is it being 
replaced by code in packaging?  If so, have Python builds been tested 
yet on the various platforms?

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Tarek Ziadé
On Wed, May 18, 2011 at 1:25 AM, Ned Deily  wrote:
...
> Just to be clear: what about for the build of the interpreter itself,
> i.e. its setup.py for the standard library extension modules?  Will the
> existing distutils code continue to be used for that?  Or is it being
> replaced by code in packaging?  If so, have Python builds been tested
> yet on the various platforms?

It will remain distutils-based for now. Moving it to packaging is not
our top priority.


Cheers
Tarek
-- 
Tarek Ziadé | http://ziade.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "packaging" merge imminent

2011-05-17 Thread Ned Deily
In article ,
 Tarek Ziadé  wrote:
> On Wed, May 18, 2011 at 1:25 AM, Ned Deily  wrote:
> > Just to be clear: what about for the build of the interpreter itself,
> > i.e. its setup.py for the standard library extension modules?  Will the
> > existing distutils code continue to be used for that? [...]
> It will remain distutils-based for now. Moving it to packaging is not
> our top priority.

+1.  Thanks!

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Nick Coghlan
On Wed, May 18, 2011 at 8:27 AM, Ethan Furman  wrote:
> On the one hand we have the 'bytes are ascii data' type interface, and on
> the other we have the 'bytes are a list of integers between 0 - 256'
> interface.

No. Bytes are a list of integers between 0-256. End of story. Using
them to represent text as well was precisely the problem with 2.x
8-bit strings, since the boundaries got blurred.

However, as a matter of practicality, many byte-oriented protocols use
ASCII to make elements of the protocol readable by humans. The
"text-like" elements of the bytes and bytearray types are a concession
to the existence of those protocols. However, that doesn't make them
text - they're still binary data streams. If you want to treat them as
text, convert them to "str" objects first (e.g. that's what
urlib.urlparse does internally in order to operate on bytes and
bytearray instances).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Robert Collins
On Wed, May 18, 2011 at 3:13 PM, Nick Coghlan  wrote:
> On Wed, May 18, 2011 at 8:27 AM, Ethan Furman  wrote:
>> On the one hand we have the 'bytes are ascii data' type interface, and on
>> the other we have the 'bytes are a list of integers between 0 - 256'
>> interface.
>
> No. Bytes are a list of integers between 0-256. End of story. Using
> them to represent text as well was precisely the problem with 2.x
> 8-bit strings, since the boundaries got blurred.
>
> However, as a matter of practicality, many byte-oriented protocols use
> ASCII to make elements of the protocol readable by humans. The
> "text-like" elements of the bytes and bytearray types are a concession
> to the existence of those protocols. However, that doesn't make them
> text - they're still binary data streams. If you want to treat them as
> text, convert them to "str" objects first (e.g. that's what
> urlib.urlparse does internally in order to operate on bytes and
> bytearray instances).

This is a not a useful argument - its an implementation choice in
Python 3, and urlparse converting bytes to 'str' to operate on them is
at best a kludge - you're forcing 5 times the storage (the original
bytes + 4 bytes-per-byte when its decoded into unicode) to work on
something which is defined as a BNF * that uses ascii *.

The Python 2 confusion was deplorable, but it doesn't make the Python
3 situation better: its different, but still very awkward for people
to write code that is correct and fast in.

Its probably too late to change, but please don't try to argue that
its correct: the continued confusion of folk running into this is
evidence that confusion *is happening*. Treat that as evidence and
think about how to fix it going forward.

_Rob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Nick Coghlan
On Wed, May 18, 2011 at 1:23 PM, Robert Collins
 wrote:
> The Python 2 confusion was deplorable, but it doesn't make the Python
> 3 situation better: its different, but still very awkward for people
> to write code that is correct and fast in.

When Python 3 goes wrong, it raises exceptions or executes the wrong
control flow. That's a vast improvement over silently corrupting the
data stream the way that 2.x does.

If it really bothers anyone, they should feel free to implement and
promote their own "ascii" data type on PyPI. If it is explicitly
restricted to 7 bit characters, it may even avoid many of the problems
of silent corruption that the 2.x str had. Speculation on python-dev
isn't going to be convincing here, though: only code in real use will
be effective on that front.

As far as the memory and runtime overhead goes, yes, that's a real
problem (indeed, that overhead is *why* bytes and bytearray have as
many str-like features as they do). PEP 393 is intended to at least
alleviate the memory burden of the Unicode text.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Greg Ewing

Ethan Furman wrote:

On the one hand we have the 'bytes are ascii data' type interface, and 
on the other we have the 'bytes are a list of integers between 0 - 256' 
interface.


I think the weird part is that there exists a literal for
writing a byte array as an ascii string, and furthermore
that it's the *only* kind of literal available for bytes.

Personally I think that the default literal syntax for
bytes, and also the form produced by repr(), should have
been something more neutral, such as hex, with the ascii
form available for use when it makes sense. Currently if
you want to write a bytes literal in hex, you have to
say something like

   some_var = b'\xde\xad\xbe\xef'

which is ugly and unreadable. Much nicer would be

   some_var = x'deadbeef'

As for


--> some_other_var[3] == b'd'


there ought to be a literal for specifying an integer
using an ascii character, so you could say something like

  if some_other_var[3] == c'd':

which would be equivalent to

  if some_other_var[3] == ord(b'd')

but without the overhead of computing the value each time
at run time.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Greg Ewing

Robert Collins wrote:

urlparse converting bytes to 'str' to operate on them is
at best a kludge - you're forcing 5 times the storage (the original
bytes + 4 bytes-per-byte when its decoded into unicode)


That is itself an implementation detail of current Python,
though, due to it only having one internal representation of
unicode.

In principle there could be a form of str that keeps its
data encoded in latin1, in which case constructing it from
a byte string could simply involve storing a pointer to the
original bytes data.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Glenn Linderman

On 5/17/2011 10:39 PM, Greg Ewing wrote:

Personally I think that the default literal syntax for
bytes, and also the form produced by repr(), should have
been something more neutral, such as hex, with the ascii
form available for use when it makes sense.



Much nicer would be

   some_var = x'deadbeef'

As for


--> some_other_var[3] == b'd'


there ought to be a literal for specifying an integer
using an ascii character, so you could say something like

  if some_other_var[3] == c'd':

which would be equivalent to

  if some_other_var[3] == ord(b'd')

but without the overhead of computing the value each time
at run time.


+1

Seems this could be added compatibly?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] how do you find out what version of Python a PEP landed in?

2011-05-17 Thread Chris Withers

Hi All,

A friend of mine is coming over to Python and asked a question I thought 
would have a better answer than it appears to:


How do I know which version of Python a PEP lands in?

I was expecting there to be a note at the bottom of the PEP, 342 in this 
case, but that doesn't appear to be the case.


What is the policy on this? Where should we be looking?

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] how do you find out what version of Python a PEP landed in?

2011-05-17 Thread Amaury Forgeot d'Arc
Hi,

2011/5/18 Chris Withers :
> A friend of mine is coming over to Python and asked a question I thought
> would have a better answer than it appears to:
>
> How do I know which version of Python a PEP lands in?
>
> I was expecting there to be a note at the bottom of the PEP, 342 in this
> case, but that doesn't appear to be the case.
>
> What is the policy on this? Where should we be looking?

Normally PEPs are important enough to be mentioned in the "whatsnew"
document of each release.
Googling for "what's new pep 342" suggests that it was released with Python 2.5.

Now, an "official" way to get this information would probably be better...

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [RELEASED] Python 3.2.1 rc 1

2011-05-17 Thread anatoly techtonik
That's great, but where is the list if changes?
--
anatoly t.



On Tue, May 17, 2011 at 9:50 PM, Georg Brandl  wrote:
> On behalf of the Python development team, I am pleased to announce the
> first release candidate of Python 3.2.1.
>
> Python 3.2.1 will the first bugfix release for Python 3.2, fixing over 120
> bugs and regressions in Python 3.2.
>
> For an extensive list of changes and features in the 3.2 line, see
>
>    http://docs.python.org/3.2/whatsnew/3.2.html
>
> To download Python 3.2.1 visit:
>
>    http://www.python.org/download/releases/3.2.1/
>
> This is a testing release: Please consider trying Python 3.2.1 with your code
> and reporting any bugs you may notice to:
>
>    http://bugs.python.org/
>
>
> Enjoy!
>
> --
> Georg Brandl, Release Manager
> georg at python.org
> (on behalf of the entire python-dev team and 3.2's contributors)
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [RELEASED] Python 3.2.1 rc 1

2011-05-17 Thread Amaury Forgeot d'Arc
Hi,

2011/5/18 anatoly techtonik :
> That's great, but where is the list if changes?

All changes are always listed in the Misc/NEWS file.
A "Change log" link on every download page displays this file.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] how do you find out what version of Python a PEP landed in?

2011-05-17 Thread Martin v. Löwis
> How do I know which version of Python a PEP lands in?

You should look at the Python-Version header of the PEP.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Georg Brandl
On 18.05.2011 07:39, Greg Ewing wrote:
> Ethan Furman wrote:
> 
>> On the one hand we have the 'bytes are ascii data' type interface, and 
>> on the other we have the 'bytes are a list of integers between 0 - 256' 
>> interface.
> 
> I think the weird part is that there exists a literal for
> writing a byte array as an ascii string, and furthermore
> that it's the *only* kind of literal available for bytes.
> 
> Personally I think that the default literal syntax for
> bytes, and also the form produced by repr(), should have
> been something more neutral, such as hex, with the ascii
> form available for use when it makes sense. Currently if
> you want to write a bytes literal in hex, you have to
> say something like
> 
> some_var = b'\xde\xad\xbe\xef'
> 
> which is ugly and unreadable. Much nicer would be
> 
> some_var = x'deadbeef'

We do have

  bytes.fromhex('deadbeef')

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python 3.x and bytes

2011-05-17 Thread Martin v. Löwis
> Is there code out there that is using this "list of int's" interface

Just in case this isn't clear yet: yes, certainly. Any non-trivial piece
of Python 3 code that has been written already (and there is some) will
have run into that issue.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [RELEASED] Python 3.2.1 rc 1

2011-05-17 Thread Martin v. Löwis
>> That's great, but where is the list if changes?
> 
> All changes are always listed in the Misc/NEWS file.
> A "Change log" link on every download page displays this file.

I think it would be good if the release announcement made some
summary statement, though, like "NNN bugs have been fixed, in MMM
modules; see NEWS for details", or some such.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] how do you find out what version of Python a PEP landed in?

2011-05-17 Thread Amaury Forgeot d'Arc
2011/5/18 "Martin v. Löwis" :
>> How do I know which version of Python a PEP lands in?
>
> You should look at the Python-Version header of the PEP.

But some PEPs don't have it: 341, 342, 343, 353...

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] how do you find out what version of Python a PEP landed in?

2011-05-17 Thread Nick Coghlan
On Wed, May 18, 2011 at 4:24 PM, "Martin v. Löwis"  wrote:
>> How do I know which version of Python a PEP lands in?
>
> You should look at the Python-Version header of the PEP.

Which is unfortunately missing from some PEPs (including PEP 342). PEP
344 shows where this information *should* be, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com