[issue1765140] logging: delay_fh option and configuration kwargs

[Vinay Sajip]

Vinay Sajip added the comment:

I'm not sure I agree with "If fileConfig has no delay_fh option,
fileConfig() would create the FileHandlers with delay_fh as False".
fileConfig will create the FileHandlers using the args= section entry to
pass to the constructor. Hence, by specifying True for this parameter,
you get the FileHandlers to delay opening the files. In fact you get
more control - you can specify True for some FileHandlers and False for
others.

The relevant code is in config.py in _install_handlers:

...
klass = eval(klass, vars(logging))
args = cp.get(sectname, "args")
args = eval(args, vars(logging))
h = apply(klass, args)
...

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[christen]

New submission from christen:

September 11, 2007 I downloaded py 3.k

The good news :
Under Windows, Python 3k properly reads files larger than 4 Go (in
contrast to python 2.5 that skips some lines, see below)

The bad news : py 3k is very slow compared to py 2.5; see the results below
the code is 
it reads a 4.9 Go file of 81,017,719 lines (a genbank entry of bacterial
sequences)

###
import time 
print (time.localtime())
fichin=open(r'D:\pythons\16s\total_gb_161_16S.gb')
t0= time.localtime()
print (t0)
i=0

for li in fichin:
i+=1
if i%100==0: 
print (i,time.localtime())

fichin.close()
print ()
print (i)
print (time.localtime())
#


I got the following results (Windows XP 64) on the same machine, using
either py 3k or py 2.5
As soon as my BSD and Linux machines are done with calculations, I will
try that on them.
Best
Richard Christen


python 3k

(2007, 9, 10, 13, 53, 36, 0, 253, 1)
(2007, 9, 10, 13, 53, 36, 0, 253, 1)
100 (2007, 9, 10, 13, 53, 49, 0, 253, 1)
200 (2007, 9, 10, 13, 54, 3, 0, 253, 1)
300 (2007, 9, 10, 13, 54, 18, 0, 253, 1)
400 (2007, 9, 10, 13, 54, 32, 0, 253, 1)
500 (2007, 9, 10, 13, 54, 47, 0, 253, 1)

7700 (2007, 9, 10, 14, 14, 55, 0, 253, 1)
7800 (2007, 9, 10, 14, 15, 9, 0, 253, 1)
7900 (2007, 9, 10, 14, 15, 22, 0, 253, 1)
8000 (2007, 9, 10, 14, 15, 36, 0, 253, 1)
8100 (2007, 9, 10, 14, 15, 49, 0, 253, 1)

81017719#this is the proper number of lines 
(2007, 9, 10, 14, 15, 50, 0, 253, 1)


Python 2.5

(2007, 9, 10, 14, 18, 33, 0, 253, 1)
(2007, 9, 10, 14, 18, 33, 0, 253, 1)
(100, (2007, 9, 10, 14, 18, 34, 0, 253, 1))
(200, (2007, 9, 10, 14, 18, 34, 0, 253, 1))
(300, (2007, 9, 10, 14, 18, 35, 0, 253, 1))
(400, (2007, 9, 10, 14, 18, 35, 0, 253, 1))
(500, (2007, 9, 10, 14, 18, 36, 0, 253, 1))
...
(7700, (2007, 9, 10, 14, 19, 10, 0, 253, 1))
(7800, (2007, 9, 10, 14, 19, 11, 0, 253, 1))
(7900, (2007, 9, 10, 14, 19, 11, 0, 253, 1))
(8000, (2007, 9, 10, 14, 19, 12, 0, 253, 1))
(8100, (2007, 9, 10, 14, 19, 12, 0, 253, 1))
()
81014962  #python 2.5 missed some lines 
(2007, 9, 10, 14, 19, 12, 0, 253, 1)

--
components: Tests
messages: 55777
nosy: [EMAIL PROTECTED]
severity: normal
status: open
title: reading large files
type: behavior
versions: Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[Martin v. Löwis]

Martin v. Löwis added the comment:

If you would like to help resolving the issue with the missing lines,
please submit a separate report for that. It is very difficult to track
unrelated bugs in a single tracker issue. It would help if you could
determine which lines are missing, e.g. by writing out all lines and
then comparing the two files.

If you want to compute runtimes, it is better to not convert them to
local time. Instead, use the pattern

start = time.time()
...
  print time.time()-start # seconds since the program started

--
nosy: +loewis

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[christen]

christen added the comment:

Hi Martin

I could certainly do that, but how you get my huge files ? 5 Go of data 
is quite big...

> If you want to compute runtimes, it is better to not convert them to
> local time. Instead, use the pattern
>
> start = time.time()
> ...
>   print time.time()-start # seconds since the program started
>   

OK I'll do that next time

Richard

__
Tracker <[EMAIL PROTECTED]>

__begin:vcard
fn:Richard Christen
n:Christen;Richard
org;quoted-printable:CNRS UMR 6543  & Universit=C3=A9 de Nice;Laboratoire de Biologie Virtuelle
adr:Parc Valrose;;Centre de Biochimie;Nice;;06108;France
email;internet:[EMAIL PROTECTED]
title;quoted-printable:Champion de saut en =C3=A9paisseur
tel;work:33- 492 076 947
url:http://bioinfo.unice.fr
version:2.1
end:vcard

___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1765140] logging: delay_fh option and configuration kwargs

[Chris Leary]

Chris Leary added the comment:

Ah, I see what the miscommunication is now.

Yes, implementing the feature as you suggested would give the user the
ability to specify the delayed handle-opening /within/ the configuration
file. What I had intended was a way to load/inspect a file configuration
without it having any handle-opening side affects, even if it were a
totally unknown configuration file.

If you want to assume a good trust relationship with the contents of a
file configuration that you're loading (as in, you trust that it won't
clobber any files that you still want), then that works well. I just
assumed it would be useful to open a file configuration in an
"untrusted" way (via delay_fh) in order to inspect/modify it. It seems
to me to put more power in the hands of the loader of the configuration,
rather than leaving the loader subject to the potentially "bad" contents
of the configuration file.

If you think I'm being too paranoid or feel that this is too obscure a
use case, feel free to implement it how you see fit! :)

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[Martin v. Löwis]

Martin v. Löwis added the comment:

> I could certainly do that, but how you get my huge files ? 5 Go of data 
> is quite big...

[not sure what "that" is] I did not mean to suggest that you attach such
a large file. Instead, just report that as a separate bug report, and be
prepared to answer follow-up questions.

Regards,
Martin

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[Stefan Sonnenberg-Carstens]

Stefan Sonnenberg-Carstens added the comment:

Perhaps this is an issue of line separation ?
Could you provide the output of wc -l on a *NIX box ?
And, could you try with this code:

import sys
print(sys.version_info)
import time 
print (time.localtime())
fichin=open(r'D:\pythons\16s\total_gb_161_16S.gb')
start = time.time()
for i,li in enumerate(fichin):
if i%100==0 and i>0: 
print (i,start-time.time())
fichin.close()
print(i)
print(start-time.time())

Thx

--
nosy: +pythonmeister

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[Stefan Sonnenberg-Carstens]

Stefan Sonnenberg-Carstens added the comment:

Sorry, this way:

import sys
print(sys.version_info)
import time 
print (time.strftime('%Y-%m-%d %H:%M:%S'))
fichin=open(r'D:\pythons\16s\total_gb_161_16S.gb')
start = time.time()
for i,li in enumerate(fichin):
if i%100==0 and i>0: 
print (i,time.time()-start)
fichin.close()
print(i)
print(time.time()-start)

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1141] reading large files

[christen]

christen added the comment:

Hi Stefan

Calculations are underway
both read and write do not work well with p3k

you can try the code below on your own machine :
fichout.write(str(i)+' '*59+'\n')  #generates a big file
fichout.write(str(i)+'\n')   #generate file <4Go

the big file is not read properly with python 2.5  (the small one is)
the big file is long to write and to read with python 3.k

I send you the results as soon it is done under 3k (very very slow indeed)

best
r

import sys
print(sys.version_info)
import time
print (time.strftime('%Y-%m-%d %H:%M:%S'))
liste=[]
start = time.time()
fichout=open('test.txt','w')
for i in xrange(85014961):
if i%500==0 and i>0:
print (i,time.time()-start)
fichout.write(str(i)+' '*59+'\n')
fichout.close()
print ('total lines written ',i)
print (i,time.time()-start)
print ('*'*50)
fichin=open('test.txt')
start3 = time.time()
for i,li in enumerate(fichin):
if i%500==0 and i>0:
print (i,time.time()-start3)
fichin.close()
print ('total lines read ',i)
print(time.time()-start)

__
Tracker <[EMAIL PROTECTED]>

__begin:vcard
fn:Richard Christen
n:Christen;Richard
org;quoted-printable:CNRS UMR 6543  & Universit=C3=A9 de Nice;Laboratoire de Biologie Virtuelle
adr:Parc Valrose;;Centre de Biochimie;Nice;;06108;France
email;internet:[EMAIL PROTECTED]
title;quoted-printable:Champion de saut en =C3=A9paisseur
tel;work:33- 492 076 947
url:http://bioinfo.unice.fr
version:2.1
end:vcard

___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5

[christen]

New submission from christen:

Error in reading >4Go files under windows

try this:

import sys
print(sys.version_info)
import time
print (time.strftime('%Y-%m-%d %H:%M:%S'))
liste=[]
start = time.time()
fichout=open('test.txt','w')
for i in xrange(85014961):
if i%500==0 and i>0:
print (i,time.time()-start)
fichout.write(str(i)+' '*59+'\n')
fichout.close()
print ('total lines written ',i)
print (i,time.time()-start)
print ('*'*50)
fichin=open('test.txt')
start3 = time.time()
for i,li in enumerate(fichin):
if i%500==0 and i>0:
print (i,time.time()-start3)
fichin.close()
print ('total lines read ',i)
print(time.time()-start)

it generates a >4Go file,not all lines are read !!
example:
('total lines written ', 85014960)
('total lines read ', 85014950)
10 lines are missing

if you replace by
fichout.write(str(i)+' '*59+'\n')

file is now under 4Go, is properly read
Used both a 32 and 64 Windows XP machines

seems to work with Linux and BSD (did not tried this example but had no
pb with my home made big files)
Pb : many examples of >4Go files for the human genome and other
biological applications. Almost sure that people are doing mistakes,
because it took me a while before discovering that...
Note : does not happen with py 3k :-)

--
components: Windows
messages: 55785
nosy: [EMAIL PROTECTED]
severity: urgent
status: open
title: code sample showing errors reading large files with py 2.5
type: behavior
versions: Python 2.5

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5

[christen]

christen added the comment:

made an error in copy paste

if you replace by
fichout.write(str(i)+' '*59+'\n')

should be 
if you replace by
fichout.write(str(i)+'\n')
of course :-(

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1125] bytes.split shold have same interface as str.split, or different name

[Guido van Rossum]

Guido van Rossum added the comment:

Committed revision 58093.

--
resolution:  -> accepted
status: open -> closed

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1139] PyFile_Encoding should be PyFile_SetEncoding

[Guido van Rossum]

Changes by Guido van Rossum:


--
assignee:  -> georg.brandl
nosy: +georg.brandl

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Guido van Rossum]

Guido van Rossum added the comment:

I agree.  I wonder if it should return Unicode as soon as *any* of the
arguments are unicode???

--
nosy: +gvanrossum

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Beda Kosata]

Beda Kosata added the comment:

I would certainly expect it to return unicode when either the "modified"
string or the replacement are unicode. I don't think that the type of
the replaced string should influence the type of the result.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Guido van Rossum]

Guido van Rossum added the comment:

Actually, it already implements the best possible rules, *except* for
the special case of an empty 3rd argument.  (When there are no
substitutions, it normally returns the input unchanged; but somehow an
empty input is handled with a shortcut even before that point.  It ought
to be a simlpe fix.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue783188] support for server side transactions in _ssl

[Bill Janssen]

Bill Janssen added the comment:

I think the example in Lib/test/test_ssl.py will have to suffice.

It would be nice to have an asyncore example server, too.


Tracker <[EMAIL PROTECTED]>


___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1774736] Binding fails

[Ali Gholami Rudi]

Ali Gholami Rudi added the comment:

The same as issue1028.  Fixed in r57450.

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1734346] patch for bug 1170311 "zipfile UnicodeDecodeError"

[Martin v. Löwis]

Changes by Martin v. Löwis:


--
assignee:  -> loewis
severity: normal -> major

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Guido van Rossum]

Guido van Rossum added the comment:

Here's a patch.

--
assignee:  -> gvanrossum

__
Tracker <[EMAIL PROTECTED]>

__Index: Lib/test/test_re.py
===
--- Lib/test/test_re.py	(revision 57629)
+++ Lib/test/test_re.py	(working copy)
@@ -83,6 +83,18 @@
 self.assertEqual(re.sub('\r\n', '\n', 'abc\r\ndef\r\n'),
  'abc\ndef\n')
 
+def test_bug_1140(self):
+# re.sub(x, y, u'') should return u'', not ''.
+# as well, re.sub(x, y, '') should return '', not u''.
+for x in 'x', u'x':
+for y in 'y', u'y':
+z = re.sub(x, y, u'')
+self.assertEqual(z, u'')
+self.assertEqual(type(z), unicode)
+z = re.sub(x, y, '')
+self.assertEqual(z, '')
+self.assertEqual(type(z), str)
+
 def test_sub_template_numeric_escape(self):
 # bug 776311 and friends
 self.assertEqual(re.sub('x', r'\0', 'x'), '\0')
Index: Modules/_sre.c
===
--- Modules/_sre.c	(revision 57629)
+++ Modules/_sre.c	(working copy)
@@ -1979,7 +1979,7 @@
 #endif
 
 static PyObject*
-join_list(PyObject* list, PyObject* pattern)
+join_list(PyObject* list, PyObject* string)
 {
 /* join list elements */
 
@@ -1993,7 +1993,7 @@
 switch (PyList_GET_SIZE(list)) {
 case 0:
 Py_DECREF(list);
-return PySequence_GetSlice(pattern, 0, 0);
+return PySequence_GetSlice(string, 0, 0);
 case 1:
 result = PyList_GET_ITEM(list, 0);
 Py_INCREF(result);
@@ -2004,7 +2004,7 @@
 /* two or more elements: slice out a suitable separator from the
first member, and use that to join the entire list */
 
-joiner = PySequence_GetSlice(pattern, 0, 0);
+joiner = PySequence_GetSlice(string, 0, 0);
 if (!joiner)
 return NULL;
 
@@ -2443,7 +2443,7 @@
 Py_DECREF(filter);
 
 /* convert list to single string (also removes list) */
-item = join_list(list, self->pattern);
+item = join_list(list, string);
 
 if (!item)
 return NULL;
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1456280] Traceback error when compiling Regex

[Brett Cannon]

Changes by Brett Cannon:


--
superseder:  -> re incompatibility in sre
title: Traceback error when compiling Regex  -> Traceback error when compiling 
Regex

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5

[Stefan Sonnenberg-Carstens]

Stefan Sonnenberg-Carstens added the comment:

Error confirmed for this python:
Python 3.0a1 (py3k, Sep 10 2007, 22:45:51)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2


See this:
[EMAIL PROTECTED]:~$ python2.4 large_io.py
(2, 4, 4, 'final', 0)
2007-09-10 21:41:52
(500, 14.321661949157715)
(1000, 30.311280965805054)
(1500, 45.24985408782959)
(2000, 59.537726879119873)
(2500, 74.075110912322998)
(3000, 87.76087498664856)
(3500, 104.54858303070068)
(4000, 121.84645009040833)
(4500, 137.88236308097839)
(5000, 155.42996501922607)
(5500, 171.81011009216309)
(6000, 188.44834208488464)
(6500, 204.46978211402893)
(7000, 218.81346702575684)
(7500, 232.86778998374939)
(8000, 246.6789391040802)
(8500, 260.89796900749207)
('total lines written ', 85014960)
(85014960, 260.94281101226807)
**
(500, 14.598887920379639)
(1000, 29.428265810012817)
(1500, 44.457981824874878)
(2000, 60.351485967636108)
(2500, 79.3228759765625)
(3000, 94.667810916900635)
(3500, 110.35149884223938)
(4000, 126.19746398925781)
(4500, 141.83787989616394)
(5000, 157.46236801147461)
(5500, 173.10227298736572)
(6000, 188.19510197639465)
(6500, 197.369295835495)
(7000, 206.41998481750488)
(7500, 215.53365993499756)
(8000, 224.55904102325439)
(8500, 233.75891900062561)
('total lines read ', 85014960)
494.727725029
[EMAIL PROTECTED]:~$ python3.0 large_io.py
(3, 0, 0, 'alpha', 1)
2007-09-10 21:50:53
500 194.725461006



Tasks: 144 total,   3 running, 141 sleeping,   0 stopped,   0 zombie
Cpu(s): 50.2%us,  1.3%sy,  0.0%ni, 48.3%id,  0.0%wa,  0.2%hi,  0.0%si, 
0.0%st
Mem:   1026804k total,   846416k used,   180388k free, 7952k buffers
Swap:  1028152k total,66576k used,   961576k free,   679032k cached

PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
28778 stefan25   0  7800 3552 1596 R  100  0.3   6:01.48 python3.0

--
nosy: +pythonmeister

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue214033] re incompatibility in sre

[Brett Cannon]

Changes by Brett Cannon:


--
resolution: accepted -> 
type:  -> behavior
versions: +Python 2.6


Tracker <[EMAIL PROTECTED]>


___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5/3.0

[Stefan Sonnenberg-Carstens]

Changes by Stefan Sonnenberg-Carstens:


--
components: +Interpreter Core
title: code sample showing errors reading large files with py 2.5 -> code 
sample showing errors reading large files with py 2.5/3.0
versions: +Python 3.0

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1096] Deeply recursive repr segfault

[Brett Cannon]

Changes by Brett Cannon:


--
priority:  -> urgent

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1774736] Binding fails

[Martin v. Löwis]

Changes by Martin v. Löwis:


--
resolution:  -> duplicate
status: open -> closed
superseder:  -> Tkinter binding involving Control-spacebar raises unicode error

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue783188] support for server side transactions in _ssl

[Martin v. Löwis]

Martin v. Löwis added the comment:

Marking as out-of-date

--
resolution:  -> out of date
status: open -> closed
superseder:  -> server-side ssl support


Tracker <[EMAIL PROTECTED]>


___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1096] Deeply recursive repr segfault

[Brett Cannon]

Brett Cannon added the comment:

Fixed in rev. 58096.

--
assignee:  -> brett.cannon
nosy: +brett.cannon
resolution:  -> fixed
status: open -> closed
versions: +Python 2.6

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Guido van Rossum]

Guido van Rossum added the comment:

Here's a better patch that also fixes a few related issues.

__
Tracker <[EMAIL PROTECTED]>

__Index: Lib/test/test_re.py
===
--- Lib/test/test_re.py	(revision 57629)
+++ Lib/test/test_re.py	(working copy)
@@ -83,6 +83,31 @@
 self.assertEqual(re.sub('\r\n', '\n', 'abc\r\ndef\r\n'),
  'abc\ndef\n')
 
+def test_bug_1140(self):
+# re.sub(x, y, u'') should return u'', not '', and
+# re.sub(x, y, '') should return '', not u''.
+# Also:
+# re.sub(x, y, unicode(x)) should return unicode(y), and
+# re.sub(x, y, str(x)) should return
+# str(y) if isinstance(y, str) else unicode(y).
+for x in 'x', u'x':
+for y in 'y', u'y':
+z = re.sub(x, y, u'')
+self.assertEqual(z, u'')
+self.assertEqual(type(z), unicode)
+#
+z = re.sub(x, y, '')
+self.assertEqual(z, '')
+self.assertEqual(type(z), str)
+#
+z = re.sub(x, y, unicode(x))
+self.assertEqual(z, y)
+self.assertEqual(type(z), unicode)
+#
+z = re.sub(x, y, str(x))
+self.assertEqual(z, y)
+self.assertEqual(type(z), type(y))
+
 def test_sub_template_numeric_escape(self):
 # bug 776311 and friends
 self.assertEqual(re.sub('x', r'\0', 'x'), '\0')
Index: Modules/_sre.c
===
--- Modules/_sre.c	(revision 57629)
+++ Modules/_sre.c	(working copy)
@@ -1979,7 +1979,7 @@
 #endif
 
 static PyObject*
-join_list(PyObject* list, PyObject* pattern)
+join_list(PyObject* list, PyObject* string)
 {
 /* join list elements */
 
@@ -1990,24 +1990,15 @@
 #endif
 PyObject* result;
 
-switch (PyList_GET_SIZE(list)) {
-case 0:
-Py_DECREF(list);
-return PySequence_GetSlice(pattern, 0, 0);
-case 1:
-result = PyList_GET_ITEM(list, 0);
-Py_INCREF(result);
-Py_DECREF(list);
-return result;
-}
-
-/* two or more elements: slice out a suitable separator from the
-   first member, and use that to join the entire list */
-
-joiner = PySequence_GetSlice(pattern, 0, 0);
+joiner = PySequence_GetSlice(string, 0, 0);
 if (!joiner)
 return NULL;
 
+if (PyList_GET_SIZE(list) == 0) {
+Py_DECREF(list);
+return joiner;
+}
+
 #if PY_VERSION_HEX >= 0x0106
 function = PyObject_GetAttrString(joiner, "join");
 if (!function) {
@@ -2443,7 +2434,7 @@
 Py_DECREF(filter);
 
 /* convert list to single string (also removes list) */
-item = join_list(list, self->pattern);
+item = join_list(list, string);
 
 if (!item)
 return NULL;
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Guido van Rossum]

Guido van Rossum added the comment:

Fredrik, thoughts?

--
assignee: gvanrossum -> effbot
nosy: +effbot

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1583946] SSL "issuer" and "server" names cannot be parsed

[Bill Janssen]

Bill Janssen added the comment:

Fixed in rev 58097.

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Fredrik Lundh]

Fredrik Lundh added the comment:

Looks good to me.  I still subscribe to the idea that
robust code should accept 8-bit *ASCII* strings any-
where it accepts Unicode (especially when the 8-bit
string is empty), but that's me.

Feel free to check this in (or assign back to you if
you don't have the time).

--
assignee: effbot -> gvanrossum
resolution:  -> accepted

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1583946] SSL "issuer" and "server" names cannot be parsed

[Bill Janssen]

Changes by Bill Janssen:


--
resolution:  -> fixed
status: open -> closed

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5/3.0

[Guido van Rossum]

Guido van Rossum added the comment:

PythonMeister, what do you mean, "confirmed"? Your read loop ends printing 

('total lines read ', 85014960)

which is the expected output.  (It's one less than the number of lines
written due to a bug in the program -- it prints the 0-based ordinal of
the last line written rather than the total number of lines written,
which is one more. But the bug is the same in the input and output loop.
 Richard's output from the read loop was

('total lines read ', 85014950)

i.e. 10 less than written.

I wonder if the bug is simply a matter of a failure to flush on Windows?
 I can't reproduce it on Linux (Ubuntu dapper).

Richard, can you somehow view the end of the file to see what its last
lines actually are?  It should end like this:

85014951
85014952
85014953
85014954
85014955
85014956
85014957
85014958
85014959
85014960

--
nosy: +gvanrossum

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1065] ssl.py shouldn't change class names from 2.6 to 3.x

[Bill Janssen]

Bill Janssen added the comment:

Fixed in rev 58097.

--
resolution:  -> fixed
status: open -> closed

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Fredrik Lundh]

Fredrik Lundh added the comment:

(is there a way to just add a comment in the new tracker, btw, or is
everything a "change note", even if nothing has changed?)

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Fredrik Lundh]

Fredrik Lundh added the comment:

Well, I spent a minute hunting around for a "comment" field or an "add
comment" button.  Guess this is a "you only need to learn this once"
thing...

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Guido van Rossum]

Guido van Rossum added the comment:

Thanks, Fredrik.
Fixed in 2.6.
Committed revision 58098.
Someone else could backport to 2.5.
Shouldn't be merged into 3.0.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1140] re.sub returns str when processing empty unicode string

[Fredrik Lundh]

Changes by Fredrik Lundh:


__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1123] split(None, maxsplit) does not strip whitespace correctly

[Fredrik Lundh]

Fredrik Lundh added the comment:

Looks like a *documentation* bug to me; at the implementation level,
None just means "no empty parts, treat runs of whitespace as separators".

--
nosy: +effbot

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1123] split(None, maxsplit) does not strip whitespace correctly

[Nir Soffer]

Nir Soffer added the comment:

I did not look into the source, but obviously there is striping of 
leading and trailing whitespace. 

When you specify a separator you get:
>>> '  '.split(' ')
['', '', '']

>>> '  a  b  '.split('  ')
['', 'a', 'b', '']

So one would expect to get this without striping:
>>> '  a  b  '.split()
['', 'a', 'b', '']

But you get this:
>>> '  a  b  '.split()
['a', 'b']

So the documentation is correct.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1026] Backport ABC to 2.6

[Guido van Rossum]

Guido van Rossum added the comment:

Committed revision 58099.
(I had to backport test_typechecks.py myself, and fix one issue in abc.py.)

Are you going to backport _abcoll.py and its tests?

--
assignee:  -> gvanrossum
nosy: +gvanrossum
resolution:  -> accepted
status: open -> closed

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1123] split(None, maxsplit) does not strip whitespace correctly

[Fredrik Lundh]

Fredrik Lundh added the comment:

But wasn't your complaint that the implementation didn't match the
documentation?

As I said, the *implementation* treats "runs of whitespace" as
separators, except for whitespace at the beginning or end (or in other
words, it never returns empty strings).  That matches the documentation,
except for the "first" in "first, whitespace characters are stripped
from both ends".   As far as I can tell, the documentation has never
matched the implementation here.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5/3.0

[Stefan Sonnenberg-Carstens]

Stefan Sonnenberg-Carstens added the comment:

I can confirm that under Linux (Linux nx6310 2.6.22-1-mepis-smp #1 SMP
PREEMPT Wed Sep 5 22:23:08 EDT 2007 i686 GNU/Linux, SimplyMepis 7.0b3)
1. using Python 3.0a1 is _very_ slow
2. it eats all your cpu (see my post)
I did not take the time to wait for the program to finish with 3.0a1,
as my patience is limited. I don't think it would silently drop lines,
as the windows version.

To see if flushing matters, I'll try this later:

import sys
print(sys.version_info)
import time
print (time.strftime('%Y-%m-%d %H:%M:%S'))
liste=[]
start = time.time()
fichout=open('test.txt','w')
for i in xrange(85014961):
if i%500==0 and i>0:
print (i,time.time()-start)
fichout.write(str(i)+' '*59+'\n')
fishout.flush()
fichout.close()
print ('total lines written ',i)
print (i,time.time()-start)
print ('*'*50)
fichin=open('test.txt')
start3 = time.time()
for i,li in enumerate(fichin):
if i%500==0 and i>0:
print (i,time.time()-start3)
fichin.close()
print ('total lines read ',i)
print(time.time()-start)


I've seen a case lately on Windows XP SP2 with Python 2.3, where a
college of mine wrote some files he read from a zip file to disk.
Before the close() he also had to flush() the written files
explicitly, otherwise he was not able to rename them afterwards.
His first approach was time.sleep(30), which was not an option.
I'll come back, if I ran the code under Windows.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1143] Updated to latest ElementTree in 2.6

[Fredrik Lundh]

New submission from Fredrik Lundh:

The xml.etree package should be updated to ElementTree 1.3/cElementTree
1.0.6 (or later).

--
assignee: effbot
components: XML
messages: 55811
nosy: effbot
priority: normal
severity: minor
status: open
title: Updated to latest ElementTree in 2.6
type: rfe
versions: Python 2.6

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1602189] Suggest a textlist() method for ElementTree

[Fredrik Lundh]

Fredrik Lundh added the comment:

ElementTree 1.3 provides a variant of this (tentatively called "itertext").

--
resolution:  -> accepted
superseder:  -> Updated to latest ElementTree in 2.6

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1143] Update to latest ElementTree in Python 2.6

[Fredrik Lundh]

Changes by Fredrik Lundh:


--
title: Updated to latest ElementTree in 2.6 -> Update to latest ElementTree in 
Python 2.6

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1602189] Suggest a textlist() method for ElementTree

[Fredrik Lundh]

Changes by Fredrik Lundh:


--
status: open -> closed

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1142] code sample showing errors reading large files with py 2.5/3.0

[christen]

christen added the comment:

Hi Guido

It is not the end of the file that is not read (see also below)

I found about that about one year ago when I was parsing very large 
files resulting from "blast" on the human genome
My parser chock after 4 Go, well before the end of the file : one line 
was missing and my acc=li[x:y] end up with an error, because acc was 
never filled...
This was kind of strange because this had not happened before with my 
Linux box.

I opened the file (which I had created myself) with a editor that could 
show hexa code : the proper line was there and allright.
If I remember well, I modified my code to see better what was going on : 
in fact the missing line had been concateneted to the previous line 
despite the proper existence of the end of line (hexa code was ok). see 
also below

I forgot about that because nobody replied to my mails, and I thought it 
was possibly related with windows 32 . I moved to a windows 64 recently 
(windows has the best driver for SQL databases) and forgot about the bug 
until I again ran into it. I then decided to try python 3k, it reads 
 >4Go file with no trouble but is so so slow, both in reading and 
writing files.
The following code produces either <4Go or >4Go files depending upon 
which fichout.write is commented
They both have the same line numbers, but the >4Go does not read 
completely under windows (32 or 64)
I have no such pb on Linux or BSD (Mac).

python 3k on windows read both files ok, but is very very slow (change 
xrange to range , I guess it is preposterous to advice you about that :-).

best
Richard

import sys
print(sys.version_info)
import time
print (time.strftime('%Y-%m-%d %H:%M:%S'))
liste=[]
start = time.time()
fichout=open('test.txt','w')
for i in xrange(85014961):
if i%500==0 and i>0:
print (i,time.time()-start)
fichout.write(str(i)+' '*59+'\n')  #big file
#fichout.write(str(i)+'\n')#small file, same number of lines

fishout.flush()
fichout.close()
print ('total lines written ',i)
print (i,time.time()-start)
print ('*'*50)
fichin=open('test.txt')
start3 = time.time()
for i,li in enumerate(fichin):
if i%500==0 and i>0:
print (i,time.time()-start3)
fichin.close()
print ('total lines read ',i)
print(time.time()-start)

> Richard, can you somehow view the end of the file to see what its last
> lines actually are?  It should end like this:
>
> 85014951
> 85014952
> 85014953
> 85014954
> 85014955
> 85014956
> 85014957
> 85014958
> 85014959
> 85014960
>
>   

using a text editor reads:
85014944  
85014945  
85014946  
85014947  
85014948  
85014949  
85014950  
85014951  
85014952  
85014953  
85014954  
85014955  
85014956  
85014957  
85014958  
85014959  
85014960  

windows py 2.5, with
if i>85014940:
print i, li.strip()

prints :
(2, 5, 0, 'final', 0)
2007-09-11 07:58:47
(500, 2.6720001697540283)
(1000, 5.375)
(1500, 8.032648498535)
(2000, 10.70368664551)
(2500, 13.375)
(3000, 16.047000169754028)
(3500, 18.70368664551)
(4000, 21.36133514404)
(4500, 24.03264849854)
(5000, 26.68763760376)
(5500, 29.36133514404)
(6000, 32.03264849854)
(6500, 34.70368664551)
(7000, 37.40764849854)
(7500, 40.094000101089478)
(8000, 42.797000169754028)
(8500, 45.485000133514404)
85014941 85014951  
85014942 85014952  
85014943 85014953  
85014944 85014954  
85014945 85014955  
85014946 85014956  
85014947 85014957  
85014948 85014958  
85014949 85014959 

[issue1745722] please add wsgi to SimpleXMLRPCServer

[Fredrik Lundh]

Fredrik Lundh added the comment:

A proper patch, including tests (if possible) and documentation, would
be nice.

(also note that SimpleXMLRPCServer was written by Brian Quinlan.)

--
assignee: effbot -> 

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1690840] xmlrpclib methods submit call on __str__, __repr__

[Fredrik Lundh]

Fredrik Lundh added the comment:

I'm trying to think of a reason for actually providing __repr__ over
RPC, but I cannot find any.  Not quite as sure about __str__, though; I
suggest adding a __repr__ method, but leaving the rest as is.

--
assignee: effbot -> collinwinter

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue814253] Grouprefs in lookbehind assertions

[Fredrik Lundh]

Changes by Fredrik Lundh:


--
type:  -> behavior
versions: +Python 2.4, Python 2.5


Tracker <[EMAIL PROTECTED]>


___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1026] Backport ABC to 2.6

[Benjamin Aranguren]

Benjamin Aranguren added the comment:

Yes.  I plan to work on it this weekend.  I hope that's okay.

On 9/10/07, Guido van Rossum <[EMAIL PROTECTED]> wrote:
>
> Guido van Rossum added the comment:
>
> Committed revision 58099.
> (I had to backport test_typechecks.py myself, and fix one issue in abc.py.)
>
> Are you going to backport _abcoll.py and its tests?
>
> --
> assignee:  -> gvanrossum
> nosy: +gvanrossum
> resolution:  -> accepted
> status: open -> closed
>
> __
> Tracker <[EMAIL PROTECTED]>
> 
> __
>

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1690840] xmlrpclib methods submit call on __str__, __repr__

[Greg Hazel]

Greg Hazel added the comment:

How about making ServerProxy a new-style class?

_
Tracker <[EMAIL PROTECTED]>

_
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com