Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Nick Coghlan
David Hopwood wrote: > Martin v. Löwis wrote: >> Programs that rely on sys.getfilesystemencoding() being able to >> represent arbitrary file names on Windows might have a bug; >> programs that rely on sys.getfilesystemencoding() being able >> to encode all elements of sys.path do not (at least not

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: > Elements of sys.path can be Unicode strings in Python 2.5, and should be > pathnames supported by the underlying OS. Where is it documented that there > is any further restriction on them? And why should there be any further > restriction on them? It's not documented in th

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Martin v. Löwis wrote: > David Hopwood schrieb: > >>On Windows, file system pathnames can contain arbitrary Unicode characters >>(well, almost). Despite the existence of "ANSI" filesystem APIs, and >>regardless of what 'sys.getfilesystemencoding()' returns, the underlying >>file system encoding fo

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: > On Windows, file system pathnames can contain arbitrary Unicode characters > (well, almost). Despite the existence of "ANSI" filesystem APIs, and > regardless of what 'sys.getfilesystemencoding()' returns, the underlying > file system encoding for NTFS and FAT filesystems i

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Nick Coghlan wrote: > David Hopwood wrote: >> Martin v. Löwis wrote: >>> Nick Coghlan schrieb: >>> So this is taking something that *already works properly on POSIX systems* and making it work on Windows as well. >>> >>> I doubt it does without side effects. For example, an application th

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
Nick Coghlan schrieb: > I think MvL is looking at it from the point of view of consumers of the list > of strings in sys.path, such as PEP 302 importer and loader objects, and > tools > like module_finder. Currently, the list of values in sys.path is limited to: That, and all kinds of inspectio

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Nick Coghlan
David Hopwood wrote: > Martin v. Löwis wrote: >> Nick Coghlan schrieb: >> >>> So this is taking something that *already works properly on POSIX >>> systems* and making it work on Windows as well. >> I doubt it does without side effects. For example, an application that >> would go through sys.path,

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
David Hopwood schrieb: >> I doubt it does without side effects. For example, an application that >> would go through sys.path, and encode everything with >> sys.getfilesystemencoding() currently works, but will break if the patch >> is applied and non-mbcs strings are put on sys.path. > > Huh? It

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread David Hopwood
Martin v. Löwis wrote: > Nick Coghlan schrieb: > >>So this is taking something that *already works properly on POSIX >>systems* and making it work on Windows as well. > > I doubt it does without side effects. For example, an application that > would go through sys.path, and encode everything with

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Steve Holden
Martin v. Löwis wrote: > Nick Coghlan schrieb: > >>So this is taking something that *already works properly on POSIX >>systems* and making it work on Windows as well. > > > I doubt it does without side effects. For example, an application that > would go through sys.path, and encode everything w

Re: [Python-Dev] Unicode Imports

2006-09-09 Thread Martin v. Löwis
Nick Coghlan schrieb: > So this is taking something that *already works properly on POSIX > systems* and making it work on Windows as well. I doubt it does without side effects. For example, an application that would go through sys.path, and encode everything with sys.getfilesystemencoding() curre

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Nick Coghlan
Martin v. Löwis wrote: > Steve Holden schrieb: >> Or simply that this inability isn't currently >> described in a bug report on Sourceforge? > > No: sys.path is specified (originally) as containing a list of byte > strings; it was extended to also support path importers (or whatever > that PEP ca

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Giovanni Bajo schrieb: > +1, but I would love to see a more formal definition of what a "bugfix" is, > which would reduce the ambiguous cases, and thus reduce the number of times > the > release managers are called to pronounce. > > Other projects, for instance, describe point releases as "open f

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: > I don't regard this as the provision of a new feature but as the removal > of an unnecessary restriction (which I would prefer to call a bug). You got the definition of "bug" wrong. Primarily, a bug is a deviation from the specification. Extending the domain of an argument

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Nick Coghlan schrieb: >> But it *is* a desirable, albeit new, feature, so I'm surprised that you >> don't appear to perceive it as such for a downstream release. > > And unlike 2.2's True/False problem, it is an *environmental* feature, rather > than a programmatic one. Not sure what you mean b

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: >>> I agree it's a relatively large patch for a release candidate but if >>> prudence suggests deferring it, it should be a *definite* for 2.5.1 and >>> subsequent releases. >> >> Possibly. I remain unconvinced. >> > > But it *is* a desirable, albeit new, feature, so I'm sur

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Martin v. Löwis
Steve Holden schrieb: >> As this can't be considered a bugfix (that I can see), I'd be against it >> being >> checked into 2.5. >> > Are you suggesting that Python's inability to correctly handle Unicode > path elements isn't a bug? Not sure whether Anthony suggests it, but I do. > Or simply

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread M.-A. Lemburg
Kristján V. Jónsson wrote: > Hello All. > I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) > which allows unicode paths in sys.path and uses the unicode file api on > windows. > This is tried and tested on 2.5, and backported to 2.3 and is currently > running on client

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Raymond Hettinger
Giovanni Bajo wrote: > >+1, but I would love to see a more formal definition of what a "bugfix" is, >which would reduce the ambiguous cases, and thus reduce the number of times the >release managers are called to pronounce. > > Sorry, that is just a pipe-dream. To some degree, all bug-fixes are

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Giovanni Bajo
Guido van Rossum <[EMAIL PROTECTED]> wrote: > IMO it's the call of the release managers. Board members ought to > trust the release managers and not apply undue pressure. +1, but I would love to see a more formal definition of what a "bugfix" is, which would reduce the ambiguous cases, and thus

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread skip
Guido> IMO it's the call of the release managers. Board members ought to Guido> trust the release managers and not apply undue pressure. Indeed. Let's not go whacking people with boards. The Perl people would just laugh at us... Skip ___ Pyth

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Guido van Rossum
On 9/8/06, Steve Holden <[EMAIL PROTECTED]> wrote: > Anthony Baxter wrote: > > On Friday 08 September 2006 19:19, Steve Holden wrote: > > > >>But it *is* a desirable, albeit new, feature, so I'm surprised that you > >>don't appear to perceive it as such for a downstream release. > > > > > > Point r

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: > On Friday 08 September 2006 19:19, Steve Holden wrote: > >>But it *is* a desirable, albeit new, feature, so I'm surprised that you >>don't appear to perceive it as such for a downstream release. > > > Point releases (2.x.1 and suchlike) are absolutely not for new features

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Anthony Baxter
On Friday 08 September 2006 19:19, Steve Holden wrote: > But it *is* a desirable, albeit new, feature, so I'm surprised that you > don't appear to perceive it as such for a downstream release. Point releases (2.x.1 and suchlike) are absolutely not for new features. They're for bugfixes, only. It'

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Nick Coghlan
Steve Holden wrote: > Anthony Baxter wrote: >> On Friday 08 September 2006 18:24, Steve Holden wrote: >>> I agree it's a relatively large patch for a release candidate but if >>> prudence suggests deferring it, it should be a *definite* for 2.5.1 and >>> subsequent releases. >> >> Possibly. I remai

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: > On Friday 08 September 2006 18:24, Steve Holden wrote: > >>>As this can't be considered a bugfix (that I can see), I'd be against it >>>being checked into 2.5. >> >>Are you suggesting that Python's inability to correctly handle Unicode >>path elements isn't a bug? Or simply

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Anthony Baxter
On Friday 08 September 2006 18:24, Steve Holden wrote: > > As this can't be considered a bugfix (that I can see), I'd be against it > > being checked into 2.5. > > Are you suggesting that Python's inability to correctly handle Unicode > path elements isn't a bug? Or simply that this inability isn't

Re: [Python-Dev] Unicode Imports

2006-09-08 Thread Steve Holden
Anthony Baxter wrote: > On Friday 08 September 2006 02:56, Kristján V. Jónsson wrote: > >>Hello All. >>I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) >>which allows unicode paths in sys.path and uses the unicode file api on >>windows. This is tried and tested on 2.5, a

Re: [Python-Dev] Unicode Imports

2006-09-07 Thread Anthony Baxter
On Friday 08 September 2006 02:56, Kristján V. Jónsson wrote: > Hello All. > I just added patch 1552880 to sourceforge. It is a patch for 2.6 (and 2.5) > which allows unicode paths in sys.path and uses the unicode file api on > windows. This is tried and tested on 2.5, and backported to 2.3 and is

Re: [Python-Dev] unicode imports

2006-06-20 Thread Martin v. Löwis
Thomas Heller wrote: >> Is that code available somewhere still? Does it still work? > > Available as patch 1093253, I have not tried if it stil works I see. It's quite a huge change, that's probably why nobody found the time to review it, yet. > To be really useful, wide char versions of other t

Re: [Python-Dev] unicode imports

2006-06-20 Thread Thomas Heller
Martin v. Löwis schrieb: > Thomas Heller wrote: >> It should be noted that I once started to convert the import machinery >> to be fully unicode aware. As far as I can tell, a *lot* has to be changed >> to make this work. > > Is that code available somewhere still? Does it still work? Available

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Thomas Heller wrote: > It should be noted that I once started to convert the import machinery > to be fully unicode aware. As far as I can tell, a *lot* has to be changed > to make this work. Is that code available somewhere still? Does it still work? > I started with refactoring Python/import.c

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Kristján V. Jónsson wrote: > Wouldn´t it be possible then to emulate the unix way? Simply encode > any unicode paths to utf-8, process them as normal, and then decode > them just prior to the actual windows io call? That won't work. People also put path names from the ANSI code page onto sys.path

Re: [Python-Dev] unicode imports

2006-06-19 Thread Martin v. Löwis
Kristján V. Jónsson wrote: > I don't have specific information on the machines. We didn´t try > very hard to get things to work with 2.3 since we simply assumed it > would work automatically when we upgraded to a more mature 2.4. I > could try to get more info, but it would be 2.3 specific. Have

Re: [Python-Dev] unicode imports

2006-06-19 Thread Guido van Rossum
On 6/16/06, Kristján V. Jónsson <[EMAIL PROTECTED]> wrote: > Although python has had full unicode support for filenames for a long time > on selected platforms (e.g. Windows), there is one glaring deficiency: It > cannot import from paths containing unicode. I´ve tried creating folders > with chi

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
something I will have to patch in for our application. Cheers, Kristján -Original Message- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 19. júní 2006 13:46 To: Kristján V. Jónsson Cc: "Martin v. Löwis"; Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jón

Re: [Python-Dev] unicode imports

2006-06-19 Thread Nick Coghlan
Kristján V. Jónsson wrote: > Funny that no other platforms could benefit from a unicode import path. > Does that mean that windows will reign supreme? Please explain. As near as I can tell, other platforms use encoded strings with the normal (byte-based) posix file API, so the Python interpreter

Re: [Python-Dev] unicode imports

2006-06-19 Thread M.-A. Lemburg
Thomas Heller wrote: > It should be noted that I once started to convert the import machinery > to be fully unicode aware. As far as I can tell, a *lot* has to be changed > to make this work. > > I started with refactoring Python/import.c, but nobody responded to the > question > whether such a

Re: [Python-Dev] unicode imports

2006-06-19 Thread Thomas Heller
It should be noted that I once started to convert the import machinery to be fully unicode aware. As far as I can tell, a *lot* has to be changed to make this work. I started with refactoring Python/import.c, but nobody responded to the question whether such a refactoring patch would be accepted

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson wrote: > The standard install path in chinese distributions can be with a > non-ANSI path, and installing an embedded python application there > will break it. I very much doubt this.

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
trary directory. Kristján -Original Message- From: Neil Hodgson [mailto:[EMAIL PROTECTED] Sent: 17. júní 2006 04:53 To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján V. Jónsson: > Although python has had full unicode support for filenames for

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
#x27;C:\\PyDev25\\lib', 'C:\\PyDev25\\lib\\plat-win', 'C:\\PyDev25\\lib\\lib-tk', 'C:\\PyDev25\\PCbuild8 ', 'C:\\PyDev25', 'C:\\PyDev25\\lib\\site-packages'] Default encoding: ascii Input encoding: cp850 Output encodings: cp850 cp850 -Ori

Re: [Python-Dev] unicode imports

2006-06-19 Thread Kristján V . Jónsson
of source files, but I cannot write "kristján = 1". But that's for a future PEP. Kristján -Original Message- From: Nick Coghlan [mailto:[EMAIL PROTECTED] Sent: 16. júní 2006 15:30 To: Kristján V. Jónsson Cc: Python Dev Subject: Re: [Python-Dev] unicode imports Kristján

Re: [Python-Dev] unicode imports

2006-06-17 Thread Ronald Oussoren
On 17-jun-2006, at 6:44, Nick Coghlan wrote: > Bob Ippolito wrote: >> There's a similar issue in that if sys.prefix contains a colon, >> Python >> is also busted: >> http://python.org/sf/1507224 >> >> Of course, that's not a Windows issue, but it is everywhere else. The >> offending code in tha

Re: [Python-Dev] unicode imports

2006-06-17 Thread Martin v. Löwis
Neil Hodgson wrote: >It should be unusual for a Chinese installation to use an install > path that can not be represented in MBCS. Try encoding the install > directory into MBCS before adding it to sys.path. Indeed. Unfortunately, people apparently install an English version (because they can

Re: [Python-Dev] unicode imports

2006-06-17 Thread Martin v. Löwis
Kristján V. Jónsson wrote: > The standard install path in chinese distributions can be with a > non-ANSI path, and installing an embedded python application there will > break it. I very much doubt this. On a Chinese system, the Program Files folder likely has a non-*ASCII* name, but it will have

Re: [Python-Dev] unicode imports

2006-06-16 Thread Bob Ippolito
On Jun 16, 2006, at 9:44 PM, Nick Coghlan wrote: > Bob Ippolito wrote: >> There's a similar issue in that if sys.prefix contains a colon, >> Python is also busted: >> http://python.org/sf/1507224 >> Of course, that's not a Windows issue, but it is everywhere else. >> The offending code in tha

Re: [Python-Dev] unicode imports

2006-06-16 Thread Neil Hodgson
Kristján V. Jónsson: > Although python has had full unicode support for filenames for a long time > on selected platforms (e.g. Windows), there is one glaring deficiency: It > cannot import from paths containing unicode. I´ve tried creating folders > with chinese characters and adding them to pa

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Bob Ippolito wrote: > There's a similar issue in that if sys.prefix contains a colon, Python > is also busted: > http://python.org/sf/1507224 > > Of course, that's not a Windows issue, but it is everywhere else. The > offending code in that case is Modules/getpath.c, Since it has to do with the

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Phillip J. Eby wrote: > Actually, you would want to put it in sys.path_hooks, and then instances > would be placed in path_importer_cache automatically. If you are adding > it to the path_hooks after the fact, you should simply clear the > path_importer_cache. Simply poking stuff into the path

Re: [Python-Dev] unicode imports

2006-06-16 Thread Bob Ippolito
On Jun 16, 2006, at 9:02 AM, Phillip J. Eby wrote: > At 01:29 AM 6/17/2006 +1000, Nick Coghlan wrote: >> Kristján V. Jónsson wrote: >>> A cursory glance at import.c shows that the import mechanism is >>> fairly >>> complicated, and riddled with "char *path" thingies, and manual >>> string >>>

Re: [Python-Dev] unicode imports

2006-06-16 Thread Phillip J. Eby
At 01:29 AM 6/17/2006 +1000, Nick Coghlan wrote: >Kristján V. Jónsson wrote: > > A cursory glance at import.c shows that the import mechanism is fairly > > complicated, and riddled with "char *path" thingies, and manual string > > arithmetic. Do you have any suggestions on a clean way to unicodify

Re: [Python-Dev] unicode imports

2006-06-16 Thread Nick Coghlan
Kristján V. Jónsson wrote: > A cursory glance at import.c shows that the import mechanism is fairly > complicated, and riddled with "char *path" thingies, and manual string > arithmetic. Do you have any suggestions on a clean way to unicodify the > import mechanism? Can you install a PEP 302 p