Re: [Python-Dev] PEP 529: Change Windows filesystem encoding to UTF-8

2016-09-03 Thread Adam Bartoš
t's relevant is that "sys.getfilesystemencoding" is a > bit of a misnomer, as it's also used to determine the assumed encoding > of command line arguments and environment variables. > > Regarding sys.argv, AFAIK Unicode arguments work well on Python 3. Even non-BMP

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

2016-09-03 Thread Adam Bartoš
rocess-stdinreader-py> ). > Code that assumes a particular encoding for the standard streams other than > ASCII will likely break. Note that for example in IDLE there are sys.std* stream objects that don't have buffer attribute. I would argue that it is incorrect to suppose tha

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

2016-09-03 Thread Adam Bartoš
eadline` won't be needed in future if http://bugs.python.org/issue17620 gets resolved so the default hook on Windows just reads from sys.stdin? This would also reduce code duplicity and all the Read/WriteConsoleW stuff would be gathered together in one special class. Regards, Adam Bartoš _

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

2016-09-03 Thread Adam Bartoš
es. > > If 0 bytes are returned, and size was not 0, this indicates end of > file. If the object is in non-blocking mode and no bytes are > available, None is returned. > """ > > You're not allowed to return 0 bytes if the requested size was not 0, > and y

Re: [Python-Dev] Python semantic: Is it ok to replace not x == y with x != y? (no)

2015-12-15 Thread Adam Bartoš
Hello, the comparisons >=, <=, >, < cannot be optimized this way. Not every order is a total order. For example, sets a = {1, 2} and b = {2, 3} are incomparable (in the sense that both a >= b and a <= b is False), and it is no pathology. Re

Re: [Python-Dev] Improving the reading part of REPL

2015-11-20 Thread Adam Bartoš
Another issue with the current implementation is http://bugs.python.org/issue24829. Even if I fix my Python environment by win_unicode_console so >>> "α" really results in "α" rather than "?", the feature vanishes when I try to redirect stdout. On Thu, Nov 1

[Python-Dev] Improving the reading part of REPL

2015-11-19 Thread Adam Bartoš
t and read a line from sys.stdin. On Linux, the default implementation would call GNU readline if it is available and sys.stdin and sys.stdout are standard TTYs (the check present in the current implementation of the input function), and it would call stdio_readline otherwise. Regards, Adam B

[Python-Dev] How to call PyOS_Readline from ctypes?

2015-08-04 Thread Adam Bartoš
hon 3? In Python 2, one could do PyFile_AsFile(py_object(sys.stdin)). I'm asking here since it is quite specific Python implementation related question. Actually, no one answered me on python-list ( https://mail.python.org/pipermail/python-list/2015-July/694633.html). Regards, A

Re: [Python-Dev] Unicode literals in Python 2.7

2015-05-10 Thread Adam Bartoš
Glenn Linderman wrote: > Is this going to get released in 3.5, I hope? Python 3 is pretty > limited without some solution for Unicode on the console... probably the > biggest deficiency I have found in Python 3, since its introduction. It > has great Unicode support for files and processing, which

Re: [Python-Dev] Unicode literals in Python 2.7

2015-05-09 Thread Adam Bartoš
, "Martin v. Löwis" wrote: > Am 02.05.15 um 21:57 schrieb Adam Bartoš: > > Even if sys.stdin contained a file-like object with proper encoding > > attribute, it wouldn't work since sys.stdin has to be instance of > 'file'>. So the question is, whether i

Re: [Python-Dev] Unicode literals in Python 2.7

2015-05-02 Thread Adam Bartoš
I think I have found out where the problem is. In fact, the encoding of the interactive input is determined by sys.stdin.encoding, but only in the case that it is a file object (see https://hg.python.org/cpython/file/d356e68de236/Parser/tokenizer.c#l890 and the implementation of tok_stdin_decode).

Re: [Python-Dev] Unicode literals in Python 2.7

2015-05-01 Thread Adam Bartoš
On Fri, May 1, 2015 at 6:14 AM, Stephen J. Turnbull wrote: > Adam Bartoš writes: > > > Unfortunately, it doesn't work. With PYTHONIOENCODING=utf-8, the > > sys.std* streams are created with utf-8 encoding (which doesn't > > help on Windows since the

Re: [Python-Dev] Unicode literals in Python 2.7

2015-04-30 Thread Adam Bartoš
L works differently than eval/exec on raw_input. It seems that the only option is to bypass the REPL by a custom REPL (e.g. based on code.InteractiveConsole). However, wrapping up the execution of a script, so that the custom REPL is invoked at the right place, is complicated. > >>>

Re: [Python-Dev] Unicode literals in Python 2.7

2015-04-29 Thread Adam Bartoš
ssion is *not* always in UTF8. It probably > depends on the keyboard mapping of your terminal emulator. I imagine in > Windows it's the current code page. > > On Wed, Apr 29, 2015 at 9:19 AM, Adam Bartoš wrote: > >> Yes, that works for eval. But I want it for code entere

Re: [Python-Dev] Unicode literals in Python 2.7

2015-04-29 Thread Adam Bartoš
ce\xb1'. Because of how eval works, I believe that it would work correctly if the PyCF_SOURCE_IS_UTF8 was set, but it is not. That is why I'm asking if there is a way to set it. Also, my naive thought is that it should be always set in the case of interactive session. On Wed, Apr 29, 20

Re: [Python-Dev] Unicode literals in Python 2.7

2015-04-29 Thread Adam Bartoš
#x27;\xce\xb1'. I understand that in the second case eval has no idea how are the given bytes encoded. But the first case is actually implemented by encoding to utf-8 and setting PyCF_SOURCE_IS_UTF8. That's why I'm talking about the flag. Regards, Drekin On Wed, Apr 29, 2015 at 9:25

[Python-Dev] Unicode literals in Python 2.7

2015-04-28 Thread Adam Bartoš
Hello, is it possible to somehow tell Python 2.7 to compile a code entered in the interactive session with the flag PyCF_SOURCE_IS_UTF8 set? I'm considering adding support for Python 2 in my package ( https://github.com/Drekin/win-unicode-console) and I have run into the fact that when u"α" is ent