Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue2128>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
Here are some results of my test of unicode2.py. I'm testing py3k on Windows
XP, OEM: cp850, ANSI: cp1252.
Raster fonts
With a fresh console, unicode2.py displays "?". input() accepts
characters encodable to th
STINNER Victor added the comment:
> ..., because right now, I'm including instructions for the use to
> (1) choose Lucida or Consolas font if they can't figure out
> any other font that gets rid of the square boxes
> (2) chcp 65001
> (3) set PYTHONIOENCODING=
STINNER Victor added the comment:
Le vendredi 14 janvier 2011 à 19:11 +, R. David Murray a écrit :
> Victor: we normally leave the patch file that was committed attached
> to the issue for future reference.
Sorry, but there were too much files. I was trying to figure out if
th
STINNER Victor added the comment:
> PS D:\jm> chcp 65001
> Page de codes active : 65001
Please don't do that: it is useless (it doesn't help to display or read more
unicode characters) and it breaks Windows console: see issue #1602 (especially
msg120414 and msg126303), a
STINNER Victor added the comment:
If the "crash" is the following message, this issue is a duplicate of #6058.
---
PS D:\jm> c:\python31\python.exe
Fatal Python error: Py_Initialize: can't initialize sys standard
streams
LookupError: unknown encoding: cp65001
---
It is n
STINNER Victor added the comment:
See also #8846: Andre Wobst wrote other tests.
--
nosy: +wobsta
___
Python tracker
<http://bugs.python.org/issue10911>
___
___
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10923>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue4681>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by STINNER Victor :
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue10920>
___
___
Python-bugs-list mailing list
Unsubscri
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue9257>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
Here is a work-in-progress patch: issue3080-3.patch. The patch is HUGE and
written for Python 3.3.
$ diffstat issue3080-3.patch
Doc/c-api/module.rst | 24
Include/import.h | 73 +
Include/moduleobject.h |2
Include/pycapsule.h|4
STINNER Victor added the comment:
This patch changes more lines of code than my previous crazy unicode patch
(msg103663, issue #8242 #8611 #9425), but it changes less files.
--
___
Python tracker
<http://bugs.python.org/issue3
STINNER Victor added the comment:
Oh, msg103663 was not the final patch. A more recent version of my patch for
#8611 / #9425 is http://codereview.appspot.com/1874048:
Doc/library/sys.rst |6
Include/Python.h|4
Include/fileobject.h| 20
Include
Changes by STINNER Victor :
--
title: Cannot use nonascii utf8 in names of files imported from -> Python 3
doesn't support non-ASCII module names with a locale encoding different than
UTF-8
___
Python tracker
<http://bugs.python.org
New submission from STINNER Victor :
To analyze #9880, I installed gcc-4.6. It looks like this new gcc version emits
new warnings. Here is a report of py3k warnings generated with "gcc-4.6 -O3
-Wall -Wextra -Wstrict-prototypes -Wno-missing-field-initializers
-Wno-unused-parameter" o
Changes by STINNER Victor :
--
components: +Build
versions: +Python 3.3
___
Python tracker
<http://bugs.python.org/issue10951>
___
___
Python-bugs-list mailin
New submission from STINNER Victor :
The Python 3 parser normalizes all identifiers using NFKC (as described in the
PEP 3131). Examples:
- U+00B5 (µ: Micro sign) is normalized to U+03BC (μ: Greek small letter mu)
- U+FB03 (ffi: Latin small ligature ffi) is normalized to 'ffi'
The
STINNER Victor added the comment:
"µTorrent.py" filename example comes from #10754.
This issue is unrelated to the Python parser or the import machinery: it is a
surprising behaviour of the MBCS codec which replaces unencodable characters to
a similar glyph. I changed the MBCS in
STINNER Victor added the comment:
New problem: if the parser doesn't normalize module names on import, it does
still normalize module names on other instructions.
Example: "import \xB5Torrent; del \xB5Torrent" raises an error on del because
the parser normalized del identif
STINNER Victor added the comment:
See also #3080 (which is not directly related).
--
___
Python tracker
<http://bugs.python.org/issue10952>
___
___
Python-bug
STINNER Victor added the comment:
> This won't be a problem if you make
> "import \xB5Torrent"
> behave as (...)
> "\u03BCTorrent = __import__('\xB5Torrent')"
"import name" is compiled to "IMPORT_NAME(name); STORE_NAME(n
Changes by STINNER Victor :
--
title: Python 2.7 hangs on Unicode+threading -> Deadlock because of the import
lock when loading the utf8 codec
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
Dummy question: can't we raise a RuntimeError on a deadlock?
--
___
Python tracker
<http://bugs.python.org/issue10923>
___
___
STINNER Victor added the comment:
> b) what if the file system implementation mangles file names.
>
> I'd use the same approach as with case-insensitive lookups: verify
> that the file we read is really the one we want.
Only Mac OS X and the HFS+ filesystem normalize filena
STINNER Victor added the comment:
> > can't we raise a RuntimeError on a deadlock?
(I mean: deadlock on the import lock)
> Deadlock detection is difficult, and probably impossible if the
> involved locks don't use the same underlying mechanism
If it is impossible to d
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10451>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> There is also issue c) what if the filesystem encoding can only
> represent a compatibility character, say U+00B5, but not its NFKC
> equivalent, U+03BC?
It is the same problem than not being able to write U+03BC with a keyboard: in
this setup, do
STINNER Victor added the comment:
> Victor, could you please create a Reitveld review for this?
Yes, but not yet. I have first to cleanup the patch.
--
___
Python tracker
<http://bugs.python.org/iss
STINNER Victor added the comment:
> If the import lock is changed to raise an exception,
> heaps of multi-threaded software will be broken.
You are right. It has done so for 12 years already (10011), so it's a bit late
to do anything about it. And backward compatibility is very im
STINNER Victor added the comment:
> for 12 years already (10011)
Oops, it's r10011 (to get a nice URL on the commit).
--
___
Python tracker
<http://bugs.python.org
STINNER Victor added the comment:
See also issue #9260 for a possible improvment.
--
___
Python tracker
<http://bugs.python.org/issue10923>
___
___
Python-bug
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue9260>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> Use "U" format to parse a module name, and "%R" to format a module name
> (to escape surrogates characters and add quotes, instead of
> "... '%.200s' ...").
See also #8754: repr() is better than str() for o
STINNER Victor added the comment:
It should be a regression introduced by #8611 or #9425.
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10
Changes by STINNER Victor :
--
nosy: +georg.brandl
priority: normal -> release blocker
___
Python tracker
<http://bugs.python.org/issue10955>
___
___
Python-
STINNER Victor added the comment:
zipimport decodes filenames of the archive from cp437 or UTF-8 (depending on a
flag in each file entry). Python has a builtin UTF-8 codec, but no cp437
builtin codec. You should try to add encodings/cp437.py to your python3.2/
directory, or to build a ZIP
STINNER Victor added the comment:
Restore priority to normal: this is a workaround, and a better fix cannot be
done before 3.2 final.
--
priority: release blocker -> normal
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
The regression was introduced in r85690: use the correct encoding to decode the
filename from the ZIP file. Attached patch fixes the bootstrap issue.
--
keywords: +patch
Added file: http://bugs.python.org/file20465/issue10955.patch
STINNER Victor added the comment:
Le jeudi 20 janvier 2011 à 18:15 +, Amaury Forgeot d'Arc a écrit :
> But if the zip file contains the stdlib *and* some other custom
> modules with cp437 names, the whole operation will fail; it can be the
> case with py2exe applicatio
STINNER Victor added the comment:
> A packaging mechanism that prepares code developed on a Latin-1
> filesystem for distribution, would have to NFKC-normalize
> filenames before encoding them using UTF-8.
It causes portability issues: if you copy a non-ASCII module on a new
host, th
STINNER Victor added the comment:
Version 4 of the patch.
--
Added file: http://bugs.python.org/file20474/issue3080-4.patch
___
Python tracker
<http://bugs.python.org/issue3
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file20448/issue3080-3.patch
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bug
STINNER Victor added the comment:
Same patch (version 4) generated by svn.
--
Added file: http://bugs.python.org/file20475/issue3080-4-svn.patch
___
Python tracker
<http://bugs.python.org/issue3
STINNER Victor added the comment:
You can review the patch with Rietveld:
http://codereview.appspot.com/3972045
--
___
Python tracker
<http://bugs.python.org/issue3
STINNER Victor added the comment:
Oops, there is a dummy typo in imp_init_builtin() that makes test_importlib to
crash (which proves that importlib has a good coverage :-)): replace "s:" by
"U:" in if (!PyArg_ParseTuple(args, &qu
STINNER Victor added the comment:
test_reprlib fails on Windows, because '\' in quoted '\\' in the filename on
repr(module). Workaround:
***
index b0dc4d7..e476941 100644
--- a/Lib/test/test_reprlib.py
+++ b/Lib/test/test_reprlib.py
@@ -234,7 +234,7 @@ class LongReprTe
STINNER Victor added the comment:
test_runpy fails on Windows on make_legacy_pyc() (of test.support), I don't
know why.
--
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
> except for the segfault in test_importlib.
Yes, as reported in my previous comment :-) Let's update the patch for
practical reasons. But I don't want to touch
http://codereview.appspot.com/1874048 (based on patch version 4).
--
Add
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file20474/issue3080-4.patch
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bug
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file20475/issue3080-4-svn.patch
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bug
STINNER Victor added the comment:
> No, your change is in the read_directory() function,
> which reads the whole archive the first time it's used.
Oh, I though that read_directory() only reads files one by one.
--
___
Python tra
STINNER Victor added the comment:
Ronald Oussoren and Amaury Forgeot d'Arc: do you think that it is an acceptable
limitation to only accept ASCII filenames in python32.zip? (not in all ZIP
files, just in the file loaded at startup)
All possible solutions:
a) Only accept ASCII filenam
New submission from STINNER Victor :
"./python Lib/test/regrtest.py test_zipimport_support" pass, but not with the
-R option.
$ ./python Lib/test/regrtest.py -R 3:3: test_zipimport_support
[1/1] test_zipimport_support
beginning 6 repetitions
123456
test test_zipimport_supp
STINNER Victor added the comment:
Oh, py2app is implemented in Python and use the zipfile module. So if we can
control how the filename is encoded, we can fix py2app to workaround this
limitation :-)
7zip and WinRAR uses the same algorithm than ZipFile._encodeFilename(): try
cp437 or use
New submission from STINNER Victor :
ZipInfo._encodeFilename() tries cp437 encoding or use UTF-8. It is not possible
to decide the encoding.
To workaround #10955 (bootstrap issue with python32.zip), it would be nice to
be able to create a ZIP file using only UTF-8 filenames.
Attached patch
STINNER Victor added the comment:
Oh, this patch fixes also a bug: ZipFile._RealGetContents() doesn't keep the
unicode flag, so open a ZIP file and then write it somewhere else may change
the unicode flag if unicode flag was set but the filename is also encodable to
UTF-8 (eg. ASCII fil
STINNER Victor added the comment:
#10972 has a patch for zipfile to set the filename encoding if a ZipInfo object
(to force the encoding to UTF-8).
--
___
Python tracker
<http://bugs.python.org/issue10
STINNER Victor added the comment:
7zip and WinRAR uses the same algorithm than ZipFile._encodeFilename(): try
cp437 or use UTF-8. Eg. if a filename contains ∞ (U+221E), it is encoded to
UTF-8.
WinZIP encodes all filenames to cp437: ∞ (U+221E) is replaced by 8 (U+0038), ☺
(U+263A) is
Changes by STINNER Victor :
--
title: zipfile: add unicode option to the choose filename encoding -> zipfile:
add "unicode" option to the force the filename encoding to UTF-8
___
Python tracker
<http://bugs.pytho
STINNER Victor added the comment:
On Linux, the "zip" command line program (InfoZIP zip program) only sets the
unicode flag if it is able to set the locale to "en_US.UTF-8". It can do
better: check if the locale encoding is UTF-8, and only "en_US.UTF-8" locale
STINNER Victor added the comment:
> What kind of problem are you trying to solve?
Support non-ASCII filenames in python32.zip (#10955): at bootstrap, Python 3.2
can only use UTF-8 codec (not cp437).
But I suppose also that forcing the encoding to UTF-8 gives a better Unicode
support (w
STINNER Victor added the comment:
Patch version 2: display a more useful error message:
$ python
Fatal Python error: Py_Initialize: Unable to get the locale encoding
NotImplementedError: bootstrap issue: python32.zip contains non-ASCII filenames
without the unicode flag
Aborted
Instead of
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10973>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> I cannot avoid haveing non-ascii filenames when a python package
> contains data files that have such names.
Are "data files" Python modules (.py files)? Or can it be anything?
--
___
Pytho
STINNER Victor added the comment:
> The question is, rather, why you need an external flag for that.
Because I don't want to change the default encoding. I am not sure that all
applications support UTF-8 encodings.
But if you control your environment, force UTF-8 encoding should impr
STINNER Victor added the comment:
No problem on Linux (Debian Sid): I tried "ŁñØ=1" in IDLE interpreter (written
using the compose key).
It looks like the bug is specific to Mac OS X and comes from Tk directly:
http://sourceforge.net/tracker/index.php?func=detail&aid=2907388&
STINNER Victor added the comment:
> I cannot start IDLE by double-clicking its icon in the Finder.
You may open a new issue for this proble.
--
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
> but I'm going to have trouble diagnosing things that don't fail
> on my development machine.
On Windows, try any character not encodable into your ANSI code page (eg. Ł
with cp1252) in the module path and non-ASCII characters in the module
STINNER Victor added the comment:
I tried issue3080-5.patch. The whole test suite pass on Windows. It pass also
on Linux with "-Wd -Werror -R 3:3:" (except #10971 which is unrelated to this
issue).
I should maybe add some unit tests for non-ASCII module paths and non-ASCII
mo
STINNER Victor added the comment:
My patch for #3080 uses repr() to format module name in all error messages.
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue8
STINNER Victor added the comment:
In #10972, I propose to add an option for the filename encoding to UTF-8. But I
would like to force UTF-8 to create a ZIP file, it doesn't concern the
decompression of a ZIP file.
Proposal of a specification to fix both issues at the same
STINNER Victor added the comment:
georg.brandl> Patch #2 looks innocent enough to me,
georg.brandl> and is clearly an improvement.
Ok, issue fixed by r88140 (+r88141):
Issue #10955: zipimport uses ASCII encoding instead of cp497 to decode
filenames, at bootstrap, if the codec registry
STINNER Victor added the comment:
Extract of PEP : << Note also that strings passed to start_response() as a
status or as response headers must follow RFC 2616 with respect to encoding.
That is, they must either be ISO-8859-1 characters, or use RFC 2047 MIME
encoding. >>
STINNER Victor added the comment:
RFC 5987 (Character Set and Language Encoding for Hypertext Transfer Protocol
(HTTP) Header Field Parameters), August 2010:
http://greenbytes.de/tech/webdav/rfc5987.html#language.specification.in.encoded.words
<< 3.3 Language Specification in Encoded
New submission from STINNER Victor :
ssl.SSLContext(True).load_verify_locations(None, True) does segfault.
Py_DECREF(cafile_bytes) in Modules/_ssl.c:1686 should be replaced by
Py_XDECREF(cafile_bytes).
--
components: Library (Lib)
messages: 126901
nosy: haypo, pitrou
priority: normal
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue9124>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from STINNER Victor :
Attached script, bug.py, ends with:
$ python3.2 source2.py
deque.remove(x): x not in deque
python: ./Modules/_collectionsmodule.c:536: deque_count: Assertion
`leftblock->rightlink != ((void *)0)' failed.
Abandon (core dumped)
--
co
New submission from STINNER Victor :
Attached script (bug2.py) ends with:
$ ~/prog/SVN/py3k/python bug2.py
python: ./Modules/_threadmodule.c:399: rlock_acquire_restore: Assertion
`self
Changes by STINNER Victor :
--
versions: +Python 2.7
___
Python tracker
<http://bugs.python.org/issue11004>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
deque.patch: I'm unable to say if the patch is correct or not, but it is always
a good thing to remove asser(...) :-)
--
___
Python tracker
<http://bugs.python.org/is
Changes by STINNER Victor :
--
nosy: -haypo
___
Python tracker
<http://bugs.python.org/issue10973>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue11010>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
mailbox.patch:
- open files in binary mode not as text
- parse as bytes not as Unicode
- replace email.generator.Generator() by email.generator.BytesGenerator()
- use .message_from_bytes() instead of .message_from_str()
- use .message_from_binary_file
STINNER Victor added the comment:
While working on this issue, I found and fixed two bugs in the email binary
parser: r88196 and r88197.
--
___
Python tracker
<http://bugs.python.org/issue9
STINNER Victor added the comment:
> I reverted r88197 because it was incorrect and caused an email test
> to fail. Once I come up with a test for it I'll fix it correctly.
test_mailbox is a good (indirect) test suite for this change. The problem of
r88197 is that it replaces msg.
STINNER Victor added the comment:
All test_email and test_mailbox pass with
mailbox.patch+BytesGenerator_handle_text.patch on Windows except one test:
==
ERROR: test_set_item (test.test_mailbox.TestBabyl
STINNER Victor added the comment:
(Patch without patch from #9124)
--
keywords: +patch
nosy: +haypo
Added file: http://bugs.python.org/file20535/BytesGenerator_handle_text.patch
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
The new test fails without the fix, and it pass with the fix. The fix itself
looks correct because I wrote a similar patch :-) Go on.
--
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
pitrou> There's a missing conversion in mailbox.patch.
pitrou> Running with -bb shows the issue.
pitrou> Here is an updated patch.
Good catch: test_mailbox now pass on Windows.
--
Some remarks on mailbox2.patch.
get_string() returns a b
STINNER Victor added the comment:
> - Using locale.setlocale(..., ...)
> - Re-open causes same error, I/O layer codec has not been changed!
Yes, this is the expected behaviour with the current code.
TextIOWrapper uses indirectly locale.getpreferredencoding() to choose your file
encodi
STINNER Victor added the comment:
> This bug may be based on same problem as Issue 6203.
Nope, both issues are different. Here you want that TextIOWrapper reads your
current locale, and not your environment variables. Issue #6203 asks why
LC_CTYPE is not C by default, but the user loc
STINNER Victor added the comment:
> To add a little bit more analysis: posix.device_encoding requires that
> the LC_CTYPE is set. Setting it just in this function would not be
> possible, as setlocale is not thread-safe.
open() does indirectly (locale.getpreferredencoding()) change
STINNER Victor added the comment:
> Something is very wrong with our code too. I have dumped the text
> that's cousing the "freeze" and run it using the test case scripts.
> It worked slow, but worked.
I retried test_MIMEText.tar.bz2 on FreeBSD 8.0 with 640 MB of memor
STINNER Victor added the comment:
It looks like there is nothing interesting to do here, so I close the issue
(which is not a bug :-)).
--
resolution: -> invalid
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
Set version to 3.3, I think that it is too late to change such critical code in
Python 3.2.
--
versions: +Python 3.3 -Python 3.1, Python 3.2
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
> upon program startup, init LibC environment: setlocale(LC_ALL, "");
Python 3 does something like that: Py_InitializeEx() calls setlocale(LC_CTYPE,
""). But I (and others) consider that as a bug (see #6203 discussion): Python
shou
New submission from STINNER Victor :
$ ./python
Python 3.2rc1+ (unknown, Jan 18 2011, 00:55:20)
>>> import _string
>>> _string.formatter_field_name_split(1)
python: Objects/stringlib/string_format.h:1347: formatter_field_name_split:
Assertion `((PyObject*)(self))-&g
STINNER Victor added the comment:
Here is a patch including a test.
The test pass on regrtest with -R 3:3: (no reference leak).
--
keywords: +patch
Added file: http://bugs.python.org/file20563/ssl.patch
___
Python tracker
<http://bugs.python.
STINNER Victor added the comment:
Should it be fixed in Python 3.2 or not? (load_verify_locations was introduced
with SSLContext in Python 3.2)
--
priority: high -> release blocker
___
Python tracker
<http://bugs.python.org/issu
3501 - 3600 of 35284 matches
Mail list logo
