Re: [Python-Dev] Import and unicode: part two

2011-01-27 Thread Martin v. Löwis
>When switching to a UTF-8 locale, they can also change the file > names of their modules to be encoded in UTF-8. It would be fairly easy > to write a script that identifies non-ASCII file names in a directory > and offers to transcode their names from their current encoding to > UTF-8. In fac

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Glenn Linderman
On 1/26/2011 4:47 PM, Toshio Kuratomi wrote: There's one further case that I am worried about that has no real "transfer". Since people here seem to think that unicode module names are the future (for instance, the comments about redefining the C locale to include utf-8 and the comments about ar

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Neil Hodgson
Toshio Kuratomi: > When they update their OS to a version that has > utf-8 python module names, they will find that they have to make a choice. > They can either change their locale settings to a utf-8 encoding and have > the system installed modules work or they can leave their encoding on their

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Toshio Kuratomi
On Wed, Jan 26, 2011 at 11:12:02AM +0100, "Martin v. Löwis" wrote: > Am 26.01.2011 10:40, schrieb Victor Stinner: > > Le lundi 24 janvier 2011 à 19:26 -0800, Toshio Kuratomi a écrit : > >> Why not locale: > >> * Relying on locale is simply not portable. (...) > >> * Mixing of modules from different

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Martin v. Löwis
> If NFSv3 doesn't reencode filenames for each client and the clients > don't reencode filenames, all clients have to use the same locale > encoding than the server. Otherwise, I don't see how it can work. In practice, users accept that they get mojibake - their editors can still open the files, a

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread James Y Knight
On Jan 26, 2011, at 11:47 AM, Victor Stinner wrote: > Not exactly. Gtk+ uses the glib library, and to encode/decode filenames, > the glib library uses: > > - UTF-8 on Windows > - G_FILENAME_ENCODING environment variable if set (comma-separated list > of encodings) > - UTF-8 if G_BROKEN_FILENAMES

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Victor Stinner
Le mercredi 26 janvier 2011 à 08:24 -0500, James Y Knight a écrit : > On Jan 26, 2011, at 4:40 AM, Victor Stinner wrote: > > During > > Python 3.2 development, we tried to be able to use a filesystem encoding > > different than the locale encoding (PYTHONFSENCODING environment > > variable): but it

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread James Y Knight
On Jan 26, 2011, at 4:40 AM, Victor Stinner wrote: > During > Python 3.2 development, we tried to be able to use a filesystem encoding > different than the locale encoding (PYTHONFSENCODING environment > variable): but it doesn't work simply because Python is not alone in the > OS. Except Python, a

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Victor Stinner
Le mercredi 26 janvier 2011 à 11:12 +0100, "Martin v. Löwis" a écrit : > There are cases where there is no real "transfer", in the sense in which > you are using the word. For example, with NFS, you can access the very > same file simultaneously on two systems, with no file name conversion > (unles

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Oleg Broytman
On Wed, Jan 26, 2011 at 11:12:02AM +0100, "Martin v. L??wis" wrote: > There are cases where there is no real "transfer", in the sense in which > you are using the word. For example, with NFS, you can access the very > same file simultaneously on two systems, with no file name conversion > (unless y

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Martin v. Löwis
Am 26.01.2011 10:40, schrieb Victor Stinner: > Le lundi 24 janvier 2011 à 19:26 -0800, Toshio Kuratomi a écrit : >> Why not locale: >> * Relying on locale is simply not portable. (...) >> * Mixing of modules from different locales won't work. (...) > > I don't understand what you are talking about

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Victor Stinner
Le lundi 24 janvier 2011 à 19:26 -0800, Toshio Kuratomi a écrit : > Why not locale: > * Relying on locale is simply not portable. (...) > * Mixing of modules from different locales won't work. (...) I don't understand what you are talking about. When you import a module, the module name becomes a

Re: [Python-Dev] Import and unicode: part two

2011-01-26 Thread Stephen J. Turnbull
Toshio Kuratomi writes: > Sure ... but with these systems, neither read-modules-as-locale or > read-modules-as-utf-8 are a good solution to work, correct? Good solution, no, but I believe that read-modules-as-locale *should* work to a great extent. AFAIK Python 3 reads Python programs as str (

Re: [Python-Dev] Import and unicode: part two

2011-01-25 Thread Toshio Kuratomi
On Wed, Jan 26, 2011 at 11:24:54AM +0900, Stephen J. Turnbull wrote: > Toshio Kuratomi writes: > > > On Linux there's no defined encoding that will work; file names are just > > bytes to the Linux kernel so based on people's argument that the convention > > is and should be that filenames are u

Re: [Python-Dev] Import and unicode: part two

2011-01-25 Thread Stephen J. Turnbull
Toshio Kuratomi writes: > On Linux there's no defined encoding that will work; file names are just > bytes to the Linux kernel so based on people's argument that the convention > is and should be that filenames are utf-8 and anything else is > a misconfigured system -- python should mandate th

Re: [Python-Dev] Import and unicode: part two

2011-01-25 Thread Toshio Kuratomi
On Tue, Jan 25, 2011 at 10:22:41AM +0100, Xavier Morel wrote: > On 2011-01-25, at 04:26 , Toshio Kuratomi wrote: > > > > * If you can pick a set of encodings that are valid (utf-8 for Linux and > > MacOS > > HFS+ uses UTF-16 in NFD (actually in an Apple-specific variant of NFD). Right > here yo

Re: [Python-Dev] Import and unicode: part two

2011-01-25 Thread exarkun
On 09:22 am, catch-...@masklinn.net wrote: On 2011-01-25, at 04:26 , Toshio Kuratomi wrote: * If you can pick a set of encodings that are valid (utf-8 for Linux and MacOS HFS+ uses UTF-16 in NFD (actually in an Apple-specific variant of NFD). Right here you've already broken Python module

Re: [Python-Dev] Import and unicode: part two

2011-01-25 Thread Xavier Morel
On 2011-01-25, at 04:26 , Toshio Kuratomi wrote: > > * If you can pick a set of encodings that are valid (utf-8 for Linux and > MacOS HFS+ uses UTF-16 in NFD (actually in an Apple-specific variant of NFD). Right here you've already broken Python modules on OSX. And as far as I know, Linux soft

Re: [Python-Dev] Import and unicode: part two

2011-01-25 Thread Stephen J. Turnbull
As Nick points out, nobody really seems to think this is an argument against your patch. I'm going to bow out of this thread after this post, as I'm clearly out of my technical depth. Victor Stinner writes: > Le lundi 24 janvier 2011 11:35:22, Stephen J. Turnbull a écrit : > > ... VFAT-formatt

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Toshio Kuratomi
On Thu, Jan 20, 2011 at 03:27:08PM -0500, Glyph Lefkowitz wrote: > > On Jan 20, 2011, at 11:46 AM, Guido van Rossum wrote: > Same here. *Most* code will never be shared, or will only be shared > between users in the same community. When it goes wrong it's also a > learning opportunity.

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Martin v. Löwis
Am 24.01.2011 16:39, schrieb Victor Stinner: > Le lundi 24 janvier 2011 11:35:22, Stephen J. Turnbull a écrit : >> ... VFAT-formatted file systems and Shift JIS file names ... > > I missed something: VFAT stores filenames as unicode (whereas FAT only > supports byte filenames). Well, VFAT stores

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Oleg Broytman
On Mon, Jan 24, 2011 at 04:39:39PM +0100, Victor Stinner wrote: > I missed something: VFAT stores filenames as unicode (whereas FAT only > supports byte filenames). Well, VFAT stores filenames twice: as a 8+3 byte > strings and as a 255 unicode (UTF-16-LE) string (UTF-16-LE). > > On which OS do

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Victor Stinner
Le lundi 24 janvier 2011 16:39:39, Victor Stinner a écrit : > Le lundi 24 janvier 2011 11:35:22, Stephen J. Turnbull a écrit : > > ... VFAT-formatted file systems and Shift JIS file names ... > > I missed something: VFAT stores filenames as unicode (whereas FAT only > supports byte filenames). Wel

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Victor Stinner
Le lundi 24 janvier 2011 11:35:22, Stephen J. Turnbull a écrit : > ... VFAT-formatted file systems and Shift JIS file names ... I missed something: VFAT stores filenames as unicode (whereas FAT only supports byte filenames). Well, VFAT stores filenames twice: as a 8+3 byte strings and as a 255 u

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Nick Coghlan
On Mon, Jan 24, 2011 at 8:35 PM, Stephen J. Turnbull wrote: > First of all, these aren't just phones; these are all kinds of gadgets > (the example I gave was a camera).  They're not as smart as an Android > or iPhone-like device, and I don't know what OS they use. We're getting a little far afie

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Stephen J. Turnbull
"Martin v. Löwis" writes: > It's one thing how the file systems are formatted, but another thing > how they are presented to APIs. For example, the phones using Windows CE > would have to convert the file names to Unicode in the OS kernel. > > So: for these phones - do you know how they pres

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Martin v. Löwis
> > Really? I would have thought that cell phones have long been the > > platforms most supportive of Unicode. > > I would think so too, except in Japan. > > However, my previous phones exposed file systems with names encoded in > Shift JIS to USB and IR browsers, though. (My current one uses

Re: [Python-Dev] Import and unicode: part two

2011-01-24 Thread Stephen J. Turnbull
Guido van Rossum writes: > Really? I would have thought that cell phones have long been the > platforms most supportive of Unicode. I would think so too, except in Japan. However, my previous phones exposed file systems with names encoded in Shift JIS to USB and IR browsers, though. (My curre

Re: [Python-Dev] Import and unicode: part two

2011-01-23 Thread Guido van Rossum
On Sun, Jan 23, 2011 at 6:33 PM, Stephen J. Turnbull wrote: > "Martin v. Löwis" writes: >  > Actually, as long people only involve Windows, or only involve Mac, >  > it will all work just fine. It's only when they use non-Mac Unix >  > (such as Linux), or try to move files across systems using sub

Re: [Python-Dev] Import and unicode: part two

2011-01-23 Thread Stephen J. Turnbull
"Martin v. Löwis" writes: > Actually, as long people only involve Windows, or only involve Mac, > it will all work just fine. It's only when they use non-Mac Unix > (such as Linux), or try to move files across systems using sub-prime > technology (such as your typical Windows zip utility) they

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Martin v. Löwis
>> I don't think anybody is *encouraging* it. The argument is for >> *permitting* it, partly for consistency with other identifiers, and >> partly because of Python's usual "consenting adults" standard for >> permitting "dangerous" practices. > > I'm sorry, I was not clear. I was afraid that sayi

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Atsuo Ishimoto
On Fri, Jan 21, 2011 at 5:45 PM, Stephen J. Turnbull wrote: > Nick Coghlan writes: >  > On Fri, Jan 21, 2011 at 3:44 PM, Atsuo Ishimoto > wrote: > >  > > I don't want Python to encourage people to use non-ascii module names. > > I don't think anybody is *encouraging* it.  The argument is for > *

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Antoine Pitrou
On Thu, 20 Jan 2011 22:25:17 -0500 James Y Knight wrote: > > On Jan 20, 2011, at 3:55 PM, Antoine Pitrou wrote: > > > On Thu, 20 Jan 2011 15:27:08 -0500 > > Glyph Lefkowitz wrote: > >> > >> To support the latter, could we just make sure that zipimport has a > >> consistent, > >> non-locale-or

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Stephen J. Turnbull
Atsuo Ishimoto writes: > Java, a leading language of IT industry, have already support > non-ASCII class files for years. But I've never seen such files in > production in Japan, and didn't improve situation until now. So why wouldn't Python work the same way? The rest of the world can use no

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Martin v. Löwis
> I don't want Python to encourage people to use non-ascii module names. I don't think the feature is open for debate anymore. PEP 3131 has been accepted (after *long* debates), and I'll pronounce that supporting non-ASCII module names is a direct consequence of having it accepted. Of course, the

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Stephen J. Turnbull
Nick Coghlan writes: > On Fri, Jan 21, 2011 at 3:44 PM, Atsuo Ishimoto wrote: > > I don't want Python to encourage people to use non-ascii module names. I don't think anybody is *encouraging* it. The argument is for *permitting* it, partly for consistency with other identifiers, and partly be

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Atsuo Ishimoto
On Fri, Jan 21, 2011 at 2:59 PM, Nick Coghlan wrote: > > These all sound like good reasons to continue to *advise* against > using non-ASCII module names. But aside from that, they sound exactly > like a lot of the arguments we heard when Py3k started enforcing the > bytes/text distinction more ri

Re: [Python-Dev] Import and unicode: part two

2011-01-21 Thread Atsuo Ishimoto
On Fri, Jan 21, 2011 at 1:46 AM, Guido van Rossum wrote: > On Thu, Jan 20, 2011 at 5:16 AM, Nick Coghlan wrote: >> On Thu, Jan 20, 2011 at 10:08 PM, Simon Cross >> wrote: >>> I'm changing my vote on this to a +1 for two reasons: >>> >>> * Initially I thought this wasn't supported by Python at al

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Nick Coghlan
On Fri, Jan 21, 2011 at 4:44 PM, Atsuo Ishimoto wrote: > On Fri, Jan 21, 2011 at 2:59 PM, Nick Coghlan wrote: >> >> These all sound like good reasons to continue to *advise* against >> using non-ASCII module names. But aside from that, they sound exactly >> like a lot of the arguments we heard wh

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Terry Reedy
On 1/20/2011 12:44 PM, Toshio Kuratomi wrote: The problem occurs in that the code that one of the parties develops (either the students or the professors) is developed on one of those OS's and then used on the other OS. The problem that I reported and hope will be fixed is that private code w

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Nick Coghlan
On Fri, Jan 21, 2011 at 3:44 PM, Atsuo Ishimoto wrote: > I don't want Python to encourage people to use non-ascii module names. > Today, seeing UnicodeEncodingError is one of popular reasons for > newbies to abandon learning Python in Japan. Non-ascii module name is > an another source of confusio

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread James Y Knight
On Jan 20, 2011, at 3:55 PM, Antoine Pitrou wrote: > On Thu, 20 Jan 2011 15:27:08 -0500 > Glyph Lefkowitz wrote: >> >> To support the latter, could we just make sure that zipimport has a >> consistent, >> non-locale-or-operating-system-dependent interpretation of encoding? > > It already has,

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Nick Coghlan
On Fri, Jan 21, 2011 at 5:27 AM, Toshio Kuratomi wrote: > I think that both ideas are inferior to mandating that every python module > filename is ascii.  From what I'm getting from Victor's posts is that he, at > least, considers the portability problems to be ignorable because dealing > with amb

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Georg Brandl
Am 20.01.2011 12:51, schrieb Victor Stinner: > You only give theorical arguments Read Anathem lately? ;) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/ma

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Glenn Linderman
On 1/20/2011 12:27 PM, Glyph Lefkowitz wrote: To support the latter, could we just make sure that zipimport has a consistent, non-locale-or-operating-system-dependent interpretation of encoding? That way a distributed egg would be importable from a zipfile regardless of how screwed up the distri

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Antoine Pitrou
On Thu, 20 Jan 2011 15:27:08 -0500 Glyph Lefkowitz wrote: > > To support the latter, could we just make sure that zipimport has a > consistent, > non-locale-or-operating-system-dependent interpretation of encoding? It already has, but it's dependent on a flag in the zip file itself (actually, o

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Neil Hodgson
Toshio Kuratomi: > My examples that you're replying to involve two "properly > configured" OS's.  The Linux workstations are configured with a UTF-8 > locale.  The Windows OS's use wide character unicode.  The problem occurs in > that the code that one of the parties develops (either the students

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Glyph Lefkowitz
On Jan 20, 2011, at 11:46 AM, Guido van Rossum wrote: > On Thu, Jan 20, 2011 at 5:16 AM, Nick Coghlan wrote: >> On Thu, Jan 20, 2011 at 10:08 PM, Simon Cross >> wrote: >>> I'm changing my vote on this to a +1 for two reasons: >>> >>> * Initially I thought this wasn't supported by Python at all

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Toshio Kuratomi
On Thu, Jan 20, 2011 at 01:43:03PM -0500, Alexander Belopolsky wrote: > On Thu, Jan 20, 2011 at 12:44 PM, Toshio Kuratomi wrote: > > .. My examples that you're replying to involve two "properly > > configured" OS's.  The Linux workstations are configured with a UTF-8 > > locale.  The Windows OS's

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Alexander Belopolsky
On Thu, Jan 20, 2011 at 12:44 PM, Toshio Kuratomi wrote: > .. My examples that you're replying to involve two "properly > configured" OS's.  The Linux workstations are configured with a UTF-8 > locale.  The Windows OS's use wide character unicode.  The problem occurs in > that the code that one of

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Alexander Belopolsky
On Thu, Jan 20, 2011 at 11:45 AM, Andy Teijelo wrote: .. > but if the code said: > > import café > > then Python would look, in any platform, for a file named: > > café.py  or  café.py  or something nicer. > > Something along the lines of xmlcharrefreplace. > Just an idea. Curiously, something li

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Toshio Kuratomi
On Thu, Jan 20, 2011 at 12:51:29PM +0100, Victor Stinner wrote: > Le mercredi 19 janvier 2011 à 20:39 -0800, Toshio Kuratomi a écrit : > > Teaching students to write non-portable code (relying on filesystem encoding > > where your solution is, don't upload to pypi anything that has non-ascii > > fi

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Andy Teijelo
(Hi, I'm writing from an address different to the one I'm subscribed with to the list because I don't have reverse dns in my mail server and mail.python.org rejects my messages. I hope that's not much trouble) Maybe Python should always use an ASCII encodable filename for modules: a translatio

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Guido van Rossum
On Thu, Jan 20, 2011 at 5:16 AM, Nick Coghlan wrote: > On Thu, Jan 20, 2011 at 10:08 PM, Simon Cross > wrote: >> I'm changing my vote on this to a +1 for two reasons: >> >> * Initially I thought this wasn't supported by Python at all but I see >> that currently it is supported but that support is

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Nick Coghlan
On Thu, Jan 20, 2011 at 10:08 PM, Simon Cross wrote: > I'm changing my vote on this to a +1 for two reasons: > > * Initially I thought this wasn't supported by Python at all but I see > that currently it is supported but that support is broken (or at least > limited to UTF-8 filesystem encodings).

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Simon Cross
On Wed, Jan 19, 2011 at 5:01 PM, Simon Cross wrote: > On Wed, Jan 19, 2011 at 2:34 PM, Victor Stinner > wrote: >>  (a) Python 3 doesn't support non-ASCII module names > > -0: I'm vaguely against this being supported because I'd rather not > have to deal with what happens when the guess regarding

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread Victor Stinner
Le mercredi 19 janvier 2011 à 20:39 -0800, Toshio Kuratomi a écrit : > Teaching students to write non-portable code (relying on filesystem encoding > where your solution is, don't upload to pypi anything that has non-ascii > filenames) seems like the exact opposite of how you'd want to shape a youn

Re: [Python-Dev] Import and unicode: part two

2011-01-20 Thread James Y Knight
On Jan 19, 2011, at 11:39 PM, Toshio Kuratomi wrote: > Tangent: This is not true about Linux. UTF-8 is a matter of the > interpretation of the filesystem bytes that the user specifies by setting > their system locale. Setting system locale to ASCII for use in system-wide > scripts, is quite commo

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Glenn Linderman
On 1/19/2011 11:20 PM, Toshio Kuratomi wrote: On Wed, Jan 19, 2011 at 09:02:17PM -0800, Glenn Linderman wrote: On 1/19/2011 8:39 PM, Toshio Kuratomi wrote: use this:: import cafe as café When you do things this way you do not have to translate between unknown encodings

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Toshio Kuratomi
On Wed, Jan 19, 2011 at 09:02:17PM -0800, Glenn Linderman wrote: > On 1/19/2011 8:39 PM, Toshio Kuratomi wrote: > > use this:: > >import cafe as café > > When you do things this way you do not have to translate between unknown > encodings into unicode. Everything is within p

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Thu, Jan 20, 2011 at 12:11 AM, Glyph Lefkowitz wrote: .. >> But for local code, having to think up an ASCII name for a module rather >> than use the obvious native-language name, is just brain-burden when >> creating the code. > > Is it really?  You already had to type 'import', presumably if y

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Glyph Lefkowitz
On Jan 20, 2011, at 12:19 AM, Glenn Linderman wrote: > Now if the stuff after m_ was the hex UTF-8 of "café", that could get > interesting :) (As it happens, it's the hex digest of the MD5 of the UTF-8 of café... ;-))___ Python-Dev mailing list Pytho

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Glenn Linderman
On 1/19/2011 9:11 PM, Glyph Lefkowitz wrote: On Jan 20, 2011, at 12:02 AM, Glenn Linderman wrote: But for local code, having to think up an ASCII name for a module rather than use the obvious native-language name, is just brain-burden when creating the code. Is it really? You already had t

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Wed, Jan 19, 2011 at 11:39 PM, Toshio Kuratomi wrote: .. > Teaching students to write non-portable code (relying on filesystem encoding > where your solution is, don't upload to pypi anything that has non-ascii > filenames) seems like the exact opposite of how you'd want to shape a young > stud

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Glyph Lefkowitz
On Jan 20, 2011, at 12:02 AM, Glenn Linderman wrote: > But for local code, having to think up an ASCII name for a module rather than > use the obvious native-language name, is just brain-burden when creating the > code. Is it really? You already had to type 'import', presumably if you can thi

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Glenn Linderman
On 1/19/2011 8:39 PM, Toshio Kuratomi wrote: use this:: import cafe as café When you do things this way you do not have to translate between unknown encodings into unicode. Everything is within python source where you have a defined encoding. This is a great way of converting non-portable

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Wed, Jan 19, 2011 at 9:07 PM, Toshio Kuratomi wrote: .. > Do you have a solution to the problem?  I haven't looked at your patch so > perhaps you have an ingenous method of translating from the unicode > representation of the module in the import statement to the bytes in > arbitrary encodings

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Toshio Kuratomi
On Thu, Jan 20, 2011 at 03:51:05AM +0100, Victor Stinner wrote: > For a lesson at school, it is nice to write examples in the > mother language, instead of using "raw" english with ASCII identifiers > and filenames. Then use this:: import cafe as café When you do things this way you do not hav

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Victor Stinner
Le mercredi 19 janvier 2011 à 18:07 -0800, Toshio Kuratomi a écrit : > Saying that multiple encodings on a single system is a misconfiguration > every time it comes up does not make it true. Yes, each filesystem can have its own encoding. For example, this is supported by Linux. Python doesn't sup

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Toshio Kuratomi
On Thu, Jan 20, 2011 at 01:26:01AM +0100, Victor Stinner wrote: > Le mercredi 19 janvier 2011 à 15:44 -0800, Toshio Kuratomi a écrit : > > Additionally, many unix filesystem don't specify a filesystem encoding for > > filenames; they deal in legal and illegal bytes which could lead to > > troubles

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Toshio Kuratomi
On Wed, Jan 19, 2011 at 07:11:52PM -0500, James Y Knight wrote: > On Jan 19, 2011, at 6:44 PM, Toshio Kuratomi wrote: > > This problem of which encoding to use is a problem that can be > > seen on UNIX systems even now. Try this: > > > > echo 'print("hi")' > café.py > > convmv -f utf-8 -t latin

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread James Y Knight
On Jan 19, 2011, at 6:44 PM, Toshio Kuratomi wrote: > This problem of which encoding to use is a problem that can be > seen on UNIX systems even now. Try this: > > echo 'print("hi")' > café.py > convmv -f utf-8 -t latin1 café.py > python3 -c 'import café' > > ASCII seems very sensible to me w

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Victor Stinner
Le mercredi 19 janvier 2011 à 15:44 -0800, Toshio Kuratomi a écrit : > Additionally, many unix filesystem don't specify a filesystem encoding for > filenames; they deal in legal and illegal bytes which could lead to > troubles. This problem of which encoding to use is a problem that can be > seen

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Terry Reedy
On 1/19/2011 6:05 PM, Alexander Belopolsky wrote: On Wed, Jan 19, 2011 at 5:47 PM, Brett Cannon wrote: .. Indeed. Last time I looked, we still had cProfile in stdlib. Yes, but that is because no one got around to hiding cProfile behind profile before we released Python 3.0. I would still lik

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Terry Reedy
On 1/19/2011 6:44 PM, Toshio Kuratomi wrote: I believe we now have the situation that a package that works on *nix could fail on Windows, whereas I believe that patch would *improve* portability. I'm not so sure about this Forget that claim if it is not true. The patch will certainly imp

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Toshio Kuratomi
On Wed, Jan 19, 2011 at 04:40:24PM -0500, Terry Reedy wrote: > On 1/19/2011 4:05 PM, Simon Cross wrote: > > >I have no problem with non-ASCII module identifiers being valid > >syntax. It's a question of whether attempting to translate a non-ASCII > > If the names are the same, ie, produced with t

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Wed, Jan 19, 2011 at 5:47 PM, Brett Cannon wrote: .. >> Indeed.  Last time I looked, we still had cProfile in stdlib. > > Yes, but that is because no one got around to hiding cProfile behind > profile before we released Python 3.0. I would still like to see it > (slowly) go away from being dire

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Brett Cannon
On Wed, Jan 19, 2011 at 14:23, Alexander Belopolsky wrote: > On Wed, Jan 19, 2011 at 4:40 PM, Terry Reedy wrote: > .. >>> For similar reasons we tend to avoid capital letters in module names. >> >> That is a stdlib style guide followed by many, but intentionally not >> enforced. > > Indeed.  Last

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Wed, Jan 19, 2011 at 4:40 PM, Terry Reedy wrote: .. >> For similar reasons we tend to avoid capital letters in module names. > > That is a stdlib style guide followed by many, but intentionally not > enforced. Indeed. Last time I looked, we still had cProfile in stdlib. __

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Victor Stinner
Le mercredi 19 janvier 2011 à 12:19 -0800, Glenn Linderman a écrit : > Since Python allows non-ASCII variable names, I think it should allow > non-ASCII module names also, on any platform that supports the > appropriate characters in the filesystem. > > Since some platforms already accept them,

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Terry Reedy
On 1/19/2011 4:05 PM, Simon Cross wrote: I have no problem with non-ASCII module identifiers being valid syntax. It's a question of whether attempting to translate a non-ASCII If the names are the same, ie, produced with the same sequence of keystrokes in the save-as box and importing box, th

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Georg Brandl
Am 19.01.2011 21:32, schrieb Terry Reedy: > On 1/19/2011 7:34 AM, Victor Stinner wrote: >> Hi, >> >> I patched Python 3.2 to support modules with non-ASCII paths (*). It >> works well on all operating systems. But the task is not completly >> done: >> >> (a) Python 3 doesn't support non-ASCII modul

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Simon Cross
On Wed, Jan 19, 2011 at 10:32 PM, Terry Reedy wrote: > I am a little shocked at the so-far tepid response to (a), so let me > defend and explain my claim that it is a bug. > > In the simplest case (from 6.11. The import statement and  2.3. Identifiers > and keywords) > > import_stmt ::= "import" m

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Victor Stinner
Le mercredi 19 janvier 2011 à 13:38 -0500, Alexander Belopolsky a écrit : > PEP 3131 does not distinguish between different types of identifiers, > so I think it assumes that non-ascii module names should be supported. My opinion is that we should suport non-ASCII module names and unencodable path

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Terry Reedy
On 1/19/2011 7:34 AM, Victor Stinner wrote: Hi, I patched Python 3.2 to support modules with non-ASCII paths (*). It works well on all operating systems. But the task is not completly done: (a) Python 3 doesn't support non-ASCII module names (b) Python 3 doesn't support unencodable characters i

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Glenn Linderman
On 1/19/2011 11:31 AM, Victor Stinner wrote: If we decide to reject non-ASCII module names, it should be done on any operating systems, not only on Windows. Since Python allows non-ASCII variable names, I think it should allow non-ASCII module names also, on any platform that supports the app

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Victor Stinner
Le mercredi 19 janvier 2011 à 10:42 -0800, Brett Cannon a écrit : > > I am not sure what exactly is not supported. On my OSX system: > > Victor said this is a Windows-specific issue. Autoquote: "(a) (...) doesn't work with a locale encoding different than UTF-8" Hum, it's not exactly the locale

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Wed, Jan 19, 2011 at 1:42 PM, Brett Cannon wrote: .. >> I am not sure what exactly is not supported.  On my OSX system: > > Victor said this is a Windows-specific issue. I missed that part. In this case, I change my vote to +0 to reflect my lack of knowledge or exposure to Windows-only issues

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Brett Cannon
On Wed, Jan 19, 2011 at 10:38, Alexander Belopolsky wrote: > On Wed, Jan 19, 2011 at 1:23 PM, Brett Cannon wrote: > ..  (a) Python 3 doesn't support non-ASCII module names > .. >> -0 from me (unless the Unicode variable naming PEP says otherwise). >> > > I am not sure what exactly is not sup

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Alexander Belopolsky
On Wed, Jan 19, 2011 at 1:23 PM, Brett Cannon wrote: .. >>>  (a) Python 3 doesn't support non-ASCII module names .. > -0 from me (unless the Unicode variable naming PEP says otherwise). > I am not sure what exactly is not supported. On my OSX system: $ ./python.exe Python 3.2b2+ .. >>> import

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Brett Cannon
On Wed, Jan 19, 2011 at 07:01, Simon Cross wrote: > On Wed, Jan 19, 2011 at 2:34 PM, Victor Stinner > wrote: >>  (a) Python 3 doesn't support non-ASCII module names > > -0: I'm vaguely against this being supported because I'd rather not > have to deal with what happens when the guess regarding th

Re: [Python-Dev] Import and unicode: part two

2011-01-19 Thread Simon Cross
On Wed, Jan 19, 2011 at 2:34 PM, Victor Stinner wrote: >  (a) Python 3 doesn't support non-ASCII module names -0: I'm vaguely against this being supported because I'd rather not have to deal with what happens when the guess regarding the filesystem encoding is wrong. On the other hand, a general

[Python-Dev] Import and unicode: part two

2011-01-19 Thread Victor Stinner
Hi, I patched Python 3.2 to support modules with non-ASCII paths (*). It works well on all operating systems. But the task is not completly done: (a) Python 3 doesn't support non-ASCII module names (b) Python 3 doesn't support unencodable characters in the module path I would like to know if