STINNER Victor added the comment:
Commited to 3.2 as r84182.
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/
STINNER Victor added the comment:
No, thank *you*, Antoine :-)
--
___
Python tracker
<http://bugs.python.org/issue9433>
___
___
Python-bugs-list mailin
STINNER Victor added the comment:
#8207 was the same issue and r79426 (for trunk and r79426 in py3k) was supposed
to fix it.
--
___
Python tracker
<http://bugs.python.org/issue8
STINNER Victor added the comment:
dmalcolm asked if it would be possible to display the Python backtrace on
Py_FatalError(). I don't know: Py_FatalError() is usually called when Python
internals are broken. But well, segfaults do also usually occurs when something
is b
New submission from STINNER Victor :
With python3 -bb: {'key': 'value'}[b'key'] raises a BytesWarning, but {'key':
'value'}[b'missing_key'] doesn't. The warning is unexpected here because it's
an implicit comparaison (I mea
STINNER Victor added the comment:
I found this problem while running test_os with python -bb: there is an error
on os.get_exec_path() because this function checks if b'PATH' key exists in the
input dictionary. Extract of the function:
def get_exec_path(env=None):
if e
STINNER Victor added the comment:
Oh. flox told me that there are other cases raising BytesWarning:
'abc' in {b'abc'}
'abc' in (b'xxx',)
'abc' in [b'xxx']
I suppose that the behaviour is different (only fail with same value / fai
STINNER Victor added the comment:
New version of the patch: remove also _Py_SetFileSystemEncoding().
--
Added file:
http://bugs.python.org/file18576/remove_sys_setfilesystemencoding-2.patch
___
Python tracker
<http://bugs.python.org/issue9
Changes by STINNER Victor :
Removed file:
http://bugs.python.org/file18565/remove_sys_setfilesystemencoding.patch
___
Python tracker
<http://bugs.python.org/issue9
STINNER Victor added the comment:
Oh, I realized that PYTHONFSENCODING is ignored on Windows and Mac OS X. r84201
and r84202 fix test_sys, and r84203 fixes the documentation and Python usage
(hide PYTHONFSENCODING variable in Python help on Windows and Mac OS X).
We might allow to override
New submission from STINNER Victor :
mbcs codec functions are surrounded by:
#if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
(especially in unicodeobject.c and _codecsmodule.c)
or
#ifdef MS_WIN32
(in unicodeobject.h)
or
#if defined(MS_WINDOWS) && !defined(_
STINNER Victor added the comment:
> since the modules were successfully imported, surely it means that
> their filenames where correctly computed and encoded? So why is the
> __filename__ attribute wrong?
Python starts with 'utf-8' encoding. If the new encoding is "sma
STINNER Victor added the comment:
> > Oh, I realized that PYTHONFSENCODING is ignored on Windows and Mac OS X.
> > r84201 and r84202 fix test_sys, and r84203 fixes the documentation and
> > Python usage (hide PYTHONFSENCODING variable in Python help on Windows
> > and
STINNER Victor added the comment:
Le jeudi 19 août 2010 22:40:53, vous avez écrit :
> Just please make sure that on other platforms such as BSD, Solaris,
> AIX, etc. that don't have this special Python support
> the env vars are honored.
I added much more tests on the files
STINNER Victor added the comment:
This issue may be related to #9561.
--
___
Python tracker
<http://bugs.python.org/issue6011>
___
___
Python-bugs-list mailin
STINNER Victor added the comment:
Change _io.TextIOWrapper() heuristic to choose the encoding is a bad idea.
--
___
Python tracker
<http://bugs.python.org/issue6
STINNER Victor added the comment:
I'm working on a fix for test_sys failure. test_os should not fail anymore.
--
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
I know test_warnings failure and I will try to fix it soon.
--
___
Python tracker
<http://bugs.python.org/issue8589>
___
___
STINNER Victor added the comment:
See also #1552880.
--
___
Python tracker
<http://bugs.python.org/issue9425>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> I think #9425 super*s*edes this. Am I correct?
#8611 or #9425, as you want. Anyway, I'm working on this topic and I will try
to fix it before Python 3.2 release.
--
nosy: +haypo
___
Python tracke
STINNER Victor added the comment:
r84308 should fix the last problems on Mac OS X, FreeBSD and Solaris.
The last failure on test_sys is on Windows with test_undecodable_code
(TypeError: Type str doesn't support the buffer API), which is unrelated.
Reopen the issue if you see new fai
STINNER Victor added the comment:
> Think of e.g. embedded Python interpreters or py2exe-style applications
> running on Linux or other systems that don't use Unicode APIs
> for FS-interaction or have fixed FS-encodings.
What is the problem here? Python does guess the filesyste
STINNER Victor added the comment:
About the patch: it should patch "Filenames and unicode" section of
Doc/whatsnew/3.2.rst (to explain that sys.setfilesystemencoding() is replaced
by the PYTHONFSENCODING env var).
--
___
Python trac
STINNER Victor added the comment:
> test_sys is still failing on my system where LC_CTYPE
> only is set to utf-8
Oh yes, test_sys fails if LC_ALL or LC_CTYPE is a locale using a different
encoding than ascii (eg. LC_ALL=fr_FR.utf8). Fixed by r84314.
--
status: open -&g
STINNER Victor added the comment:
> r84012 patchs zipimporter_init() to use the new PyUnicode_FSDecoder()
> and use Py_UNICODE* (unicode) strings instead of char* (byte) strings.
oops, it's r84013 (not r84012)
--
___
Python tra
STINNER Victor added the comment:
pitrou> The FTP class is not supposed to guess in which charset
pitrou> your data should be encoded.
pitrou> (the "encoding" argument on the FTP class is meant
pitrou> for protocol commands (such as file names), not for file
pitrou&g
Changes by STINNER Victor :
Added file: http://bugs.python.org/file18671/Py_UNICODE_strcat.patch
___
Python tracker
<http://bugs.python.org/issue9425>
___
___
Python-bug
STINNER Victor added the comment:
Py_UNICODE_strcat.patch: create Py_UNICODE_strcat() function.
Py_UNICODE_strdup.patch: create Py_UNICODE_strdup() function.
--
Added file: http://bugs.python.org/file18672/Py_UNICODE_strdup.patch
___
Python tracker
STINNER Victor added the comment:
The problem is not specific to Py_CompileString(): all functions based
(indirectly) on PyParser_ASTFromString() and PyParser_ASTFromFile() expect
filenames encoded in utf-8 with the strict error handler.
If we choose to use something else than utf-8 in
Changes by STINNER Victor :
--
components: +Unicode -None
versions: +Python 3.2
___
Python tracker
<http://bugs.python.org/issue9713>
___
___
Python-bugs-list m
STINNER Victor added the comment:
utf-8 codec (in strict mode) rejects surrogates in python3, and so you doesn't
support undecodable filenames (filenames decoded using surrogateescape error
handler which produces surrogate characters). It may be possible if you use
surrogateescape every
STINNER Victor added the comment:
Ok to remove it from Python 3.2. I don't think that it is necessary to update
Python 2.7 code/doc.
--
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
> According to the Unicode standard the high and low surrogate halves used
> by UTF-16 (...)
Yes, but in Python, U+DC80..D+DCFF range is used to store undecodable bytes.
Eg. 'abc\xff'.decode('ascii', 'surrogateescape')
New submission from STINNER Victor :
Many C functions have bytes argument (char* type) but the encoding is not
documented. If would not be a problem if the encoding was always the same, but
it is not. Examples:
- format of PyUnicode_FromFormat() should be encoded as ISO-8859-1
- filename of
STINNER Victor added the comment:
r84429 creates Py_UNICODE_strcat() (change with the patch: return the right
value).
r84430 creates PyUnicode_strdup() (change with the patch: rename the function
from Py_UNICODE_strdup() to PyUnicode_strdup() and mangle the function name
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file18672/Py_UNICODE_strdup.patch
___
Python tracker
<http://bugs.python.org/issue9425>
___
___
Python-bug
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file18671/Py_UNICODE_strcat.patch
___
Python tracker
<http://bugs.python.org/issue9425>
___
___
Python-bug
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue7077>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
<< I found this crash while playing with proxies (thanks haypo).
http://code.activestate.com/recipes/496741-object-proxying/ >>
My question was: why does isinstance(Proxy('abc'), str) works (give True),
whereas re.match('abc'
STINNER Victor added the comment:
>>> class Spam(object):
... def __getattribute__(self, name):
... if name == '__class__':
... return str
... raise AttributeError
...
>>> spam = Spam('spam')
>>> isinstance(s
STINNER Victor added the comment:
I am able to reproduce the crash with z > 4:
# (magic, type (rle, bpp), dim, x, y, z)
open('image', 'wb').write(struct.pack('>hh', 0732, 1, 1, 1, 1, 10))
rgbimg.longimagedata('image')
--
But not
New submission from STINNER Victor :
I'm trying to document the encoding of all bytes argument of the C API: see
#9738. I tried to understand which encoding is used by PyUnicode_FromFormat*()
(and PyErr_Format() which calls PyUnicode_FromFormatV()). It looks like
ISO-8859-1
STINNER Victor added the comment:
About PyErr_Format() and PyUnicode_FromFormat*() encoding: it's not exactly
ISO-8859-1... there is a bug => issue #9769.
--
___
Python tracker
<http://bugs.python.or
STINNER Victor added the comment:
Another possibility is to use _Py_char2wchar() + PyUnicode_FromWideChar() /
_Py_wchar2char() + PyUnicode_AsWideChar() to decode / encode filenames. These
functions use the locale encoding. This solution was possible in Python 3.1,
but no more in Python 3.2
STINNER Victor added the comment:
> In such environments you cannot expect the user to configure the
> system properly (i.e. set an environment variable).
Why would it be different for embeded python?
> Instead, the application has to provide an educated guess
> to the Python in
STINNER Victor added the comment:
PyUnicode_Check(op) checks op->ob_type->tp_flags & Py_TPFLAGS_UNICODE_SUBCLASS.
--
___
Python tracker
<http://bugs.python.
STINNER Victor added the comment:
I have different questions:
- Should we trust PyObject_IsInstance() or PyUnicode_Check() (because they
give different results)?
- Should PyObject_IsInstance() and PyUnicode_Check() give the same result?
- Should we fix the segfault?
To fix the segfault, I
STINNER Victor added the comment:
Oh, I didn't see that the issue was specific to Python2. I updated the issue's
title. If I understood correctly, the issue is also specific to Windows.
Do you know if your patch changes the public API? (break the compatibility)
--
FYI abo
STINNER Victor added the comment:
> Do we really want to support this kind of configuration?
There is also a problem is the directory name is b'py3k\xe9': at startup (utf-8
encoding), the name is decoded to 'py3k\udce9'. When the locale encoding is set
to iso-885
STINNER Victor added the comment:
> PyUnicode_FromFormat("%s", text) expects a utf-8 buffer.
Really? I don't see how "*s++ = *f;" (where s is Py_UNICODE* and f is char*)
can decode utf-8. It looks more like ISO-8859-1.
> Very recently (r84472, r84485), some
STINNER Victor added the comment:
About "embedded Python interpreters or py2exe-style applications": do you mean
that the application calls a C function to set the encoding before starting the
interpreter? Or you mean the Python function, sys.setfilesystemencoding()?
I would like
STINNER Victor added the comment:
"keep the C function"
Hum, currently, Python3 only has a *private* function called
_Py_SetFileSystemEncoding() which can only be called after _Py_InitializeEx()
(because it relies on the codecs API). If you consider that there is a real use
case,
STINNER Victor added the comment:
I commited my patch (with a new test, iso-8859-1:replace) to 2.7: r84621. I
will no backport to 2.6 because this branch now only accept security fixes.
--
resolution: -> fixed
status: open -> closed
___
STINNER Victor added the comment:
> My remark is that utf-8 tend to be applied to all kind of files;
> if someone once decide that non-ascii chars are allowed in (some)
> string constants, they will be stored in utf-8.
In this case, it will be better to raise an error on non-a
STINNER Victor added the comment:
For unicode, ascii(x) is implemented as repr(x).encode('ascii',
'backslashreplace').decode('ascii').
repr(x) is "'" + x + "'" for printable characters (eg. U+1D121), and "'U+%08x'&q
STINNER Victor added the comment:
> >>> s = "'\0\"\n\r\t abcd\x85é\U00012fff\U0001D121xxx\uD800."
> (...)
> (I think I've included everything:
> - normal chars
> - control chars
> - one-byte non-ASCII
> - two-byte non-ASCII (and lone sur
STINNER Victor added the comment:
#6543 changed the encoding of the filename argument of
PyRun_SimpleFileExFlags() (and all functions based on PyRun_SimpleFileExFlags)
and c_filename attribute of the compiler (private) structure in Python 3.1.3:
use utf-8 in strict mode instead of filesystem
STINNER Victor added the comment:
#6543 changed the encoding of the filename argument of
PyRun_SimpleFileExFlags() (and all functions based on PyRun_SimpleFileExFlags)
and c_filename attribute of the compiler (private) structure in Python 3.1.3:
use utf-8 in strict mode instead of filesystem
STINNER Victor added the comment:
See also #9713 (Py_CompileString fails on non decode-able paths) and #9738
(Document the encoding of functions bytes arguments of the C API).
--
___
Python tracker
<http://bugs.python.org/issue8
STINNER Victor added the comment:
Do you think that it is a Python bug? You should first try to report a bug on
eGenenix bug tracker: http://www.egenix.com/services/support/
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue9
STINNER Victor added the comment:
> WARNING: The filename '@test_464_tmp-共有される' CAN be encoded
> by (...) cp932
We should find character not encodable in any Windows code page, but accepted
as filenames.
> characters like "\u2661" or "\u2668" (..
New submission from STINNER Victor :
In Python 3.2, mbcs encoding (default filesystem encoding on Windows) is now
strict: raise an error on unencodable/undecodable characters/bytes. But
os.listdir(b'.') encodes unencodable bytes as b'?'.
Example:
>>> os.mkdir
STINNER Victor added the comment:
I found this bug while trying to find an unencodable filename for #9819
(TESTFN_UNDECODABLE).
Anyway, the bytes API should be avoided on Windows since Windows native
filename type is unicode.
--
___
Python
STINNER Victor added the comment:
See also #9820.
--
___
Python tracker
<http://bugs.python.org/issue9819>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from STINNER Victor :
It would be nice to support PEP 383 (surrogateescape) on Windows, but the mbcs
codec doesn't support it for performance reason. The Windows functions to
encode/decode MBCS don't give the index of the unencodable/undecodable
character/byte. For en
STINNER Victor added the comment:
> os.listdir(b'listdir') should raise an error (and not ignore
> the filename or replaces unencodable characters by b'?').
To avoid the error, a solution is to support the PEP 383 on Windows (for the
mbcs encoding). I opened a separ
STINNER Victor added the comment:
> "dir" command cannot print filename correctly, though.
Who cares? We just have to be able to create a file with a name containing non
encodable characters, list the directory, and then remove this evil file.
--
With r84666, Python uses &
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file18823/find_unencode_filename.py
___
Python tracker
<http://bugs.python.org/issue9819>
___
___
Pytho
STINNER Victor added the comment:
Oh wait. PEP 383 is a solution to store undecodable bytes in an unicode string,
but for mbcs I'm trying to get the opposite: store unicode in bytes and this is
not possible (at least with PEP 383).
Example with Python 3.1:
>>> print("abcŁ
STINNER Victor added the comment:
> With r84666, Python uses "-\u5171\u6709\u3055\u308c\u308b"
> suffix for TESTFN_UNENCODABLE.
Backported to 3.1 as r84668. I don't want to patch Python 2.x (its unicode
support is lower and the code is too different than Python3) and
STINNER Victor added the comment:
Patch:
- Remove the bytes version of listdir(): reuse the unicode version but
converts the filename to bytes using PyUnicode_EncodeFSDefault() if the
directory name is not unicode
- use Py_XDECREF(d) instead of Py_DECREF(d) at the end (because d=NULL on
STINNER Victor added the comment:
Close this issue: PEP 383 is specific to filesystem using bytes, it is useless
on Windows (the problem on Windows is on encoding, not on decoding).
--
resolution: -> invalid
status: open -> closed
___
STINNER Victor added the comment:
@amaury: Do you agree to reject non-ascii bytes?
TODO: document format encoding in Doc/c-api/*.rst.
--
___
Python tracker
<http://bugs.python.org/issue9
STINNER Victor added the comment:
I didn't proposed to add a new parameter to Py_InitializeEx() (which means
create a new function to not break the API), I just wrote that
_Py_SetFileSystemEncoding() doesn't work for your use case.
> If you embed Python into another applic
STINNER Victor added the comment:
> If you still consider that the change on .data as a bug,
> I think that the fix is to remove .data (mark it as
> protected: environ.data => environ._data).
r84690 marks os.environ.data as protected. Close this issue again.
--
STINNER Victor added the comment:
Fixed by r84692.
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue9402>
___
__
STINNER Victor added the comment:
I don't know how to fix this issue, and I don't know if it can be fixed. As the
issue is very unlikely, I prefer to close it.
--
resolution: -> wont fix
status: open -> closed
___
Pytho
STINNER Victor added the comment:
Well, it was trivial to workaround this bug in my application (convert host to
bytes using explicit host = str(host)). Python3 doesn't have this issue and
Python 2.7 is released, I prefer to close this bug as wont fix.
--
resolution: -> fixe
Changes by STINNER Victor :
--
resolution: fixed -> wont fix
___
Python tracker
<http://bugs.python.org/issue7093>
___
___
Python-bugs-list mailing list
Un
STINNER Victor added the comment:
Well, it's not a bug, just a gcc warning. We don't need this patch.
--
resolution: -> wont fix
status: open -> closed
___
Python tracker
<http://bugs.p
STINNER Victor added the comment:
Does anyone agree with me?
--
___
Python tracker
<http://bugs.python.org/issue9408>
___
___
Python-bugs-list mailing list
Unsub
STINNER Victor added the comment:
It should be fixed by r84694.
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue8589>
___
___
Python-
STINNER Victor added the comment:
It is fixed in 2.7 with the backport of the Python3's io library (r73394).
--
resolution: accepted -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python
STINNER Victor added the comment:
New patch:
- add encoding option to TextFile constructor
- parse_makefile() uses the heuristic from text_file.diff
Note: sys.getfilesystemencoding() is always set in Python 3.2 (but it may be
None in Python 2.x and Python < 3.2).
--
Added f
STINNER Victor added the comment:
I attached a patch to #6011 to set the encoding to read the Makefile.
--
___
Python tracker
<http://bugs.python.org/issue9
STINNER Victor added the comment:
Fixed in r84696+r84697: confstr-minimal.diff +
PyUnicode_DecodeFSDefaultAndSize().
--
___
Python tracker
<http://bugs.python.org/issue9
Changes by STINNER Victor :
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue9579>
___
___
Python-bugs-list
STINNER Victor added the comment:
Fixed in r84696+r84697: confstr-minimal.diff from #9579 +
PyUnicode_DecodeFSDefaultAndSize().
Thanks for the patch, sorry for the delay.
--
resolution: -> duplicate
status: open -> closed
___
Python t
Changes by STINNER Victor :
--
resolution: duplicate -> fixed
___
Python tracker
<http://bugs.python.org/issue9580>
___
___
Python-bugs-list mailing list
Un
STINNER Victor added the comment:
test_pep277.patch removes the usage of os.path.supports_unicode_filenames from
test_pep277: the test still pass on Debian Sid (Linux). Can someone test the
patch on Mac OS X, FreeBSD and Solaris (and maybe other POSIX/UNIX OSes)?
About Windows
STINNER Victor added the comment:
Oops, forget test_pep277.patch: I misunderstood r81149 (new way to detect if
the filesystem supports unicode or not). test_pep277 fails with my patch on
Linux with LC_CTYPE=C.
--
___
Python tracker
<h
STINNER Victor added the comment:
r84701 fixes supports_unicode_filenames's definition in Python 3.2 (and r84702
in Python 3.1): os.listdir(str) now always return unicode filenames (including
non-ascii characters).
--
___
Python tracker
STINNER Victor added the comment:
> Maybe os.path.supports_unicode_filenames should be deprecated.
> The doc currently says:
> "True if arbitrary Unicode strings can be used as file names
> (within limitations imposed by the file system), and if os.listdir()
> returns U
STINNER Victor added the comment:
$ ldd $(/usr/bin/python3.1 -c 'import readline; print(readline.__file__)')|grep
curses
libncurses.so.5 => /lib/libncurses.so.5 (0xb7537000)
$ ldd /lib/libreadline.so.6|grep curses
libncurses.so.5 => /lib/libncurses.so.5 (0xb76a6
STINNER Victor added the comment:
Fixed by r84704 in Python 3.2.
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/
STINNER Victor added the comment:
> How about TESTFN_UNICODE (test_unicode_file) issue?
File "e:\python-dev\py3k\lib\test\test_unicode_file.py", line 12, in
TESTFN_UNICODE.encode(TESTFN_ENCODING)
UnicodeEncodeError: 'mbcs' codec can't encode character
Changes by STINNER Victor :
Added file: http://bugs.python.org/file18845/unicode_file.patch
___
Python tracker
<http://bugs.python.org/issue9819>
___
___
Python-bug
STINNER Victor added the comment:
> Thank you, your patch works.
Ok, patch commited to 3.2 as r84710. Thanks for your feedback.
--
___
Python tracker
<http://bugs.python.org/iss
STINNER Victor added the comment:
> Still happens with r84709 on PPC Tiger 3.x
It's not the same error, PYTHONWARNINGS is decoded from the wrong encoding:
locale encodind instead of utf-8. r84731 should fix this bug (at least, it
restores the encoding used because my last commit
STINNER Victor added the comment:
Fixed by r84730, thanks for the issue.
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/
2201 - 2300 of 35284 matches
Mail list logo
