[ python-Bugs-1606092 ] csv module broken for unicode
Bugs item #1606092, was opened at 2006-11-30 16:46 Message generated for change (Comment added) made by jettlogic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1606092&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Unicode Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: JettLogic (jettlogic) Assigned to: Nobody/Anonymous (nobody) Summary: csv module broken for unicode Initial Comment: The csv module does not accept data to write/read as anything other than ascii-or-utf-8 str, and the do-it-yourself example in the Python 2.5 Manual to write in another encoding is extremely clunky: 1) convert unicode to utf-8 2) use csv on utf-8 with cStringIO output 3) convert utf-8 to unicode 4) convert unicode to target encoding (may be utf-8...) So clunky as to be a bug - csv clearly can't handle unicode at all. The module functions are in dire need of either accepting unicode objects (letting the output stream worry about the encoding, like codecs.StreamWriter), or at the very least accepting data directly in a target encoding instead of roundabout utf-8. To read another encoding is a bit less onerous than writing: 1) wrap file to return utf-8 2) use csv, getting utf-8 output 3) convert utf-8 to unicode object Anyone willing to fix the csv module? -- >Comment By: JettLogic (jettlogic) Date: 2006-12-05 13:53 Message: Logged In: YES user_id=1345991 Originator: YES Anyone know why it uses a C extension? The C code apparently appends fields to a writable byte buffer (so patching for unicode is impossible), reallocated as it grows. How much efficiency is gained by doing that, with its many lines of logic overhead, versus careful use of python strings? For montanaro, the UnicodeWriter with three coding conversions and a StringIO shows there is however much efficiency to be lost. Perhaps lemburg's suggestion of a pure-python re-implementation of _csv is the way to go. It does not look like a fun task, after adding in back-compatibility, benchmarks and tests, and I couldn't commit to it just yet. Are C->Py patches typically accepted? (assume quality code and comparable benchmarks) I'll have to leave it at that. If you leave this open, someone might take it up at some point. -- Comment By: Skip Montanaro (montanaro) Date: 2006-12-03 20:22 Message: Logged In: YES user_id=44345 Originator: NO I must admit I don't understand the criticism of the UnicodeReader and UnicodeWriter example classes in the module documentation. Sure, their implementations jump through some hoops, but that's so you don't have to. If you use them as written I believe their API's should be about the same as the csv.reader and csv.writer classes with the added improvement that the reader returns Unicode and the writer accepts Unicode. If your desire is to read and write Unicode why do you care that those objects are encoded using utf-8 in the file? Like Martin asked, are you willing to come up with better examples? Better yet, are you willing to provide a patch for the underlying extension module so it handles Unicode? Hint: I'm fairly certain that if it was trivial it would have been done by now. -- Comment By: M.-A. Lemburg (lemburg) Date: 2006-12-03 14:54 Message: Logged In: YES user_id=38388 Originator: NO It should be easy to provide a wrapper class which implements the above in plain Python. However, if noone volunteers to write such code, it's not going to happen. I've found that the builtin csv module is not flexible enough to deal with the often broken CSV data you typically find in practice, so perhaps adding a pure Python implementation which works with Unicode might prove to be a better approach. Unassigning the report, since I don't have time for this. -- Comment By: Martin v. Löwis (loewis) Date: 2006-12-03 12:34 Message: Logged In: YES user_id=21627 Originator: NO Are you willing to fix it? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1606092&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1603424 ] subprocess.py (py2.5) wrongly claims py2.2 compatibility
Bugs item #1603424, was opened at 2006-11-26 20:07 Message generated for change (Comment added) made by racarr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1603424&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tim Wegener (twegener) Assigned to: Nobody/Anonymous (nobody) Summary: subprocess.py (py2.5) wrongly claims py2.2 compatibility Initial Comment: >From the comments in subprocess.py (py2.5): # This module should remain compatible with Python 2.2, see PEP 291. However, using it from Python 2.2 gives: NameError: global name 'set' is not defined (set built-in used on line 1005) The subprocess.py in py2.4 was 2.2 compatible. Either the compatibility comment should be removed/amended or compatibility fixed. -- Comment By: Robert Carr (racarr) Date: 2006-12-05 10:10 Message: Logged In: YES user_id=1649655 Originator: NO Index: subprocess.py === --- subprocess.py (revision 52918) +++ subprocess.py (working copy) @@ -1004,8 +1004,8 @@ # Close pipe fds. Make sure we don't close the same # fd more than once, or standard fds. -for fd in set((p2cread, c2pwrite, errwrite))-set((0,1,2)): -if fd: os.close(fd) +for fd in (p2cread,c2pwrite,errwrite): +if fd not in (0,1,2): os.close(fd) # Close all other fds, if asked for if close_fds: Fixed? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1603424&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1593751 ] poor urllib error handling
Bugs item #1593751, was opened at 2006-11-09 16:04
Message generated for change (Comment added) made by racarr
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1593751&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Nobody/Anonymous (nobody)
Summary: poor urllib error handling
Initial Comment:
I set up a simple server that returns an empty response.
>>> from socket import *
>>> s = socket()
>>> s.bind(("", ))
>>> while 1: x, c = s.accept(); print c; x.recv(1000);
x.close()
...
Pointing urllib at this gives a traceback:
Python 2.6a0 (trunk:52099M, Oct 3 2006, 09:59:17)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import urllib
>>> urllib.urlopen('http://localhost:/')
Traceback (most recent call last):
File "", line 1, in
File "/home/guido/p/Lib/urllib.py", line 82, in urlopen
return opener.open(url)
File "/home/guido/p/Lib/urllib.py", line 190, in open
return getattr(self, name)(url)
File "/home/guido/p/Lib/urllib.py", line 334, in
open_http
return self.http_error(url, fp, errcode, errmsg,
headers)
File "/home/guido/p/Lib/urllib.py", line 351, in
http_error
return self.http_error_default(url, fp, errcode,
errmsg, headers)
File "/home/guido/p/Lib/urllib.py", line 608, in
http_error_default
return addinfourl(fp, headers, "http:" + url)
File "/home/guido/p/Lib/urllib.py", line 951, in __init__
addbase.__init__(self, fp)
File "/home/guido/p/Lib/urllib.py", line 898, in __init__
self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'
>>>
I can repeat this with 2.2.3 and 2.4.3 as well (don't
have 2.3 around for testing).
The direct cause of the problem is that h.getfile() on
line 329 of urllib.py (in head of trunk) returns None.
--
Comment By: Robert Carr (racarr)
Date: 2006-12-05 10:26
Message:
Logged In: YES
user_id=1649655
Originator: NO
Fix?
Index: urllib.py
===
--- urllib.py (revision 52918)
+++ urllib.py (working copy)
@@ -895,8 +895,10 @@
def __init__(self, fp):
self.fp = fp
-self.read = self.fp.read
-self.readline = self.fp.readline
+ try:
+ self.read = self.fp.read
+ self.readline = self.fp.readline
+ except: print "File handler is none"
if hasattr(self.fp, "readlines"): self.readlines =
self.fp.readlines
if hasattr(self.fp, "fileno"):
self.fileno = self.fp.fileno
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1593751&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1566280 ] Logging problem on Windows XP
Bugs item #1566280, was opened at 2006-09-27 04:49 Message generated for change (Comment added) made by eloff You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1566280&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Pavel Krupets (pavel_krupets) Assigned to: Nobody/Anonymous (nobody) Summary: Logging problem on Windows XP Initial Comment: Traceback (most recent call last): File "C:\Python\Lib\logging\handlers.py", line 73, in emit if self.shouldRollover(record): File "C:\Python\Lib\logging\handlers.py", line 147, in shouldRollover self.stream.seek(0, 2) #due to non-posix-compliant Windows feature ValueError: I/O operation on closed file not sure why this file is closed. -- Comment By: Daniel Eloff (eloff) Date: 2006-12-05 19:05 Message: Logged In: YES user_id=730918 Originator: NO I have this problem, I'm googling this and finding lots of people having the same problem. I'm running python 2.5 on windows XP and using the rotating file handler. I've disabled the logger in my application so I can continue development. -- Comment By: Pavel Krupets (pavel_krupets) Date: 2006-09-29 06:52 Message: Logged In: YES user_id=1007725 to start application please use: src/py/run.bat to get closed handler error (if you manage to start it) please open your web browser and try to visit: http://localhost:8080 You can change http settings in src/conf/development/robot.conf sorry code is quite raw just started. :) -- Comment By: Martin v. Löwis (loewis) Date: 2006-09-27 20:29 Message: Logged In: YES user_id=21627 Can you provide a test case for either problem? -- Comment By: Pavel Krupets (pavel_krupets) Date: 2006-09-27 05:01 Message: Logged In: YES user_id=1007725 And I think python crashes on Windows if I try to use logger from several threads. Unhandled exception at 0x7c901010 in python.exe: 0xC005: Access violation reading location 0x0034. > ntdll.dll!7c901010() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] msvcr71.dll!7c34f639() msvcr71.dll!7c36b3b1() python25.dll!1e06c6c0() python25.dll!1e08dc97() python25.dll!1e03ac12() python25.dll!1e03c735() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e03db7d() python25.dll!1e0715df() python25.dll!1e0268ec() python25.dll!1e040a04() python25.dll!1e039a8c() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e0622d3() python25.dll!1e062660() python25.dll!1e061028() python25.dll!1e0db1bd() python25.dll!1e062676() python25.dll!1e03e8c1() python25.dll!1e041475() python25.dll!1e0414c3() python25.dll!1e094093() python25.dll!1e062676() python25.dll!1e0268ec() python25.dll!1e03987a() python25.dll!1e033edc() python25.dll!1e08dc97() python25.dll!1e03ac12() python25.dll!1e03cc5f() python25.dll!1e07041e() python25.dll!1e070385() python25.dll!1e03db7d() python25.dll!1e039a8c() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e07041e() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f(
[ python-Bugs-1566280 ] Logging problem on Windows XP
Bugs item #1566280, was opened at 2006-09-27 13:49 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1566280&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None >Priority: 7 Private: No Submitted By: Pavel Krupets (pavel_krupets) >Assigned to: Martin v. Löwis (loewis) Summary: Logging problem on Windows XP Initial Comment: Traceback (most recent call last): File "C:\Python\Lib\logging\handlers.py", line 73, in emit if self.shouldRollover(record): File "C:\Python\Lib\logging\handlers.py", line 147, in shouldRollover self.stream.seek(0, 2) #due to non-posix-compliant Windows feature ValueError: I/O operation on closed file not sure why this file is closed. -- Comment By: Daniel Eloff (eloff) Date: 2006-12-06 04:05 Message: Logged In: YES user_id=730918 Originator: NO I have this problem, I'm googling this and finding lots of people having the same problem. I'm running python 2.5 on windows XP and using the rotating file handler. I've disabled the logger in my application so I can continue development. -- Comment By: Pavel Krupets (pavel_krupets) Date: 2006-09-29 15:52 Message: Logged In: YES user_id=1007725 to start application please use: src/py/run.bat to get closed handler error (if you manage to start it) please open your web browser and try to visit: http://localhost:8080 You can change http settings in src/conf/development/robot.conf sorry code is quite raw just started. :) -- Comment By: Martin v. Löwis (loewis) Date: 2006-09-28 05:29 Message: Logged In: YES user_id=21627 Can you provide a test case for either problem? -- Comment By: Pavel Krupets (pavel_krupets) Date: 2006-09-27 14:01 Message: Logged In: YES user_id=1007725 And I think python crashes on Windows if I try to use logger from several threads. Unhandled exception at 0x7c901010 in python.exe: 0xC005: Access violation reading location 0x0034. > ntdll.dll!7c901010() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] msvcr71.dll!7c34f639() msvcr71.dll!7c36b3b1() python25.dll!1e06c6c0() python25.dll!1e08dc97() python25.dll!1e03ac12() python25.dll!1e03c735() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e03db7d() python25.dll!1e0715df() python25.dll!1e0268ec() python25.dll!1e040a04() python25.dll!1e039a8c() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e0622d3() python25.dll!1e062660() python25.dll!1e061028() python25.dll!1e0db1bd() python25.dll!1e062676() python25.dll!1e03e8c1() python25.dll!1e041475() python25.dll!1e0414c3() python25.dll!1e094093() python25.dll!1e062676() python25.dll!1e0268ec() python25.dll!1e03987a() python25.dll!1e033edc() python25.dll!1e08dc97() python25.dll!1e03ac12() python25.dll!1e03cc5f() python25.dll!1e07041e() python25.dll!1e070385() python25.dll!1e03db7d() python25.dll!1e039a8c() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e07041e() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03
[ python-Bugs-1566280 ] Logging problem on Windows XP
Bugs item #1566280, was opened at 2006-09-27 13:49 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1566280&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 7 Private: No Submitted By: Pavel Krupets (pavel_krupets) Assigned to: Martin v. Löwis (loewis) Summary: Logging problem on Windows XP Initial Comment: Traceback (most recent call last): File "C:\Python\Lib\logging\handlers.py", line 73, in emit if self.shouldRollover(record): File "C:\Python\Lib\logging\handlers.py", line 147, in shouldRollover self.stream.seek(0, 2) #due to non-posix-compliant Windows feature ValueError: I/O operation on closed file not sure why this file is closed. -- >Comment By: Martin v. Löwis (loewis) Date: 2006-12-06 08:29 Message: Logged In: YES user_id=21627 Originator: NO Ok, so tsample.zip is a test case for the original problem, right? I can reproduce the problem on Linux also. I can't make it crash (on Linux); what do have to do to make it crash? If I access localhost:8080, I get log messages saying 2006-12-06 07:21:06,999 INFO servlet::__init__:1091 code 404, message File not found eloff: this report actually reports two problems (the I/O error, and the crash). Which of these are you having and have found lots of people having? As for the traceback problem: this is due to the main thread terminating, and therefore the logging atexit handler getting invoked, which closes the file. Only then is the threading atexit handler invoked, which waits until all non-daemon threads terminate. As a work-around, add httpServer.join() at the end of your script. I'll attach a patch that fixes this problem in the Python library. File Added: threading.diff -- Comment By: Daniel Eloff (eloff) Date: 2006-12-06 04:05 Message: Logged In: YES user_id=730918 Originator: NO I have this problem, I'm googling this and finding lots of people having the same problem. I'm running python 2.5 on windows XP and using the rotating file handler. I've disabled the logger in my application so I can continue development. -- Comment By: Pavel Krupets (pavel_krupets) Date: 2006-09-29 15:52 Message: Logged In: YES user_id=1007725 to start application please use: src/py/run.bat to get closed handler error (if you manage to start it) please open your web browser and try to visit: http://localhost:8080 You can change http settings in src/conf/development/robot.conf sorry code is quite raw just started. :) -- Comment By: Martin v. Löwis (loewis) Date: 2006-09-28 05:29 Message: Logged In: YES user_id=21627 Can you provide a test case for either problem? -- Comment By: Pavel Krupets (pavel_krupets) Date: 2006-09-27 14:01 Message: Logged In: YES user_id=1007725 And I think python crashes on Windows if I try to use logger from several threads. Unhandled exception at 0x7c901010 in python.exe: 0xC005: Access violation reading location 0x0034. > ntdll.dll!7c901010() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] msvcr71.dll!7c34f639() msvcr71.dll!7c36b3b1() python25.dll!1e06c6c0() python25.dll!1e08dc97() python25.dll!1e03ac12() python25.dll!1e03c735() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e04026b() python25.dll!1e039a2e() python25.dll!1e03ac82() python25.dll!1e03cc5f() python25.dll!1e03db7d() python25.dll!1e0715df() python25.dll!1e0268ec() python25.dll!1e040a04() python25.dll!1e039a8c() python25.dll!1e03ac82() pyth
