[issue7185] csv reader utf-8 BOM error

[Istvan Szirtes]

Istvan Szirtes  added the comment:

Hi Everyone,

I have tried the "utf-8-sig" and it does not work in this case or 
rather I think not the csv module is wrong. The seek() does not work 
correctly in the csv file or object.

With "utf-8-sig" the file is opend correctly and the first row does not 
include the BOM problem. It is great. 
I am sorry I have not known this until now. (I am not a python expert 
yet :))

However, I have gote some misstake like this 'AFTE\ufeffVALUE".WAV' 
during my running script.

"AFTER" is a valid string in the given csv file but the BOM follows it.
This happens after when I seek up to "0" some times in the csv file.
And the string "aftevalue" LEAVE_HIGHWAY-E" is produced which is wrong.

My sollution is that I convert the csv object into a list after the 
file openeing:

InDistancesFile = codecs.open( Root, 'r', encoding='utf-8' )
txt = InDistancesFile.read()[1:] # to leave the BOM
lines = txt.splitlines()[1:] # to leave the first row which is 
a header
InDistancesObj = list(csv.reader( lines )) # convert the csv 
reader object into a simple list

Many thanks for your help,
Istvan

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1087418] long int bitwise ops speedup (patch included)

[Mark Dickinson]

Mark Dickinson  added the comment:

[Greg]
> It would seem to me that a more suitable implementation would be to
> store numbers as 32*k-bit 2's complement integers; I've used this in 
> C++ libraries. Compared to the rad-15 sign/magnitude approach, it may
> seem trickier to do carries but in practice it's not that big a deal.

Do you know of any publicly available code that takes this approach, 
while remaining simple and portable?

For 32-bit limbs (on a 32-bit platform, say, with no C99 support and no 
uint64_t available), how *do* you deal with carries?  You can write 
portable C89 that does the correct thing, but you then have to cross 
your fingers and hope that the compiler can turn it into sensible 
assembler, and it often can't (I'll post an example of this that I ran 
into just yesterday in a second).  And even if your compiler can 
optimize this effectively, what about all the other compilers out there?  
The alternative is to use inline assembler for selected platforms;  at 
that point, you lose simplicity.

The sign-magnitude versus two's complement is an orthogonal issue, I 
think.  I'm not convinced of the value of two's complement.  Certainly 
two's complement would be more convenient for the bitwise operations 
under discussion, and probably also works well for addition and 
subtraction.  But it's less convenient for negation, multiplication, 
division, power, conversion to and from human-readable strings, etc.  
It's worth noting that GMP uses sign-magnitude, so I can't believe there 
could be much performance gain in moving to two's complement.  
(Actually, I'm not sure I've seen any arbitrary-precision arithmetic 
package that uses two's complement.  Has anyone here?)

Here's the example promised earlier:  yesterday I wanted to add two 128-
bit unsigned integers a and b, each represented as a pair of type 
uint64_t integers, on a 64-bit machine.  In portable C99, the code looks 
something like:

#include 

/* *sumhigh:*sumlow = ahigh:alow + bhigh:blow */

void
add_128(uint64_t *sumhigh, uint64_t *sumlow,
   uint64_t ahigh, uint64_t alow,
   uint64_t bhigh, uint64_t blow)
{
  alow += blow;
  ahigh += bhigh + (alow < blow);
  *sumlow = alow;
  *sumhigh = ahigh;
}

Ideally, the compiler would manage to optimize this to a simple 'addq, 
adcq' pair of instructions.  But gcc-4.4 -O3, on OS X 10.6/x86_64 gives 
me:

_add_128:
LFB0:
leaq(%r9,%rcx), %rcx
xorl%eax, %eax
leaq(%r8,%rdx), %rdx
pushq   %rbp
LCFI0:
cmpq%r9, %rcx
movq%rcx, (%rsi)
setb%al
movq%rsp, %rbp
LCFI1:
addq%rax, %rdx
movq%rdx, (%rdi)
leave
ret

(Here it looks like alow and blow are in r9 and rcx, ahigh and bhigh are 
in r8 and rdx, and rsi and rdi are sumlow and sumhigh.)

How do you write the C code in such a way that gcc produces the right 
result?  I eventually gave up and just put the assembler in directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1087418] long int bitwise ops speedup (patch included)

[Mark Dickinson]

Mark Dickinson  added the comment:

Here's the inline assembly version, for comparison:

#define SUM2_BIN64(sumhigh, sumlow, ahigh, alow, bhigh, blow)\
  __asm__ ("addq\t%5, %1\n\t"\
   "adcq\t%3, %0"\
   : "=r" (sumhigh), "=&r" (sumlow)  \
   : "0" ((uint64_t)(ahigh)), "rm" ((uint64_t)(bhigh)),  \
 "%1" ((uint64_t)(alow)), "rm" ((uint64_t)(blow))\
   : "cc")

void
add_128_asm(uint64_t *sumhigh, uint64_t *sumlow,
   uint64_t ahigh, uint64_t alow,
   uint64_t bhigh, uint64_t blow)
{
  SUM2_BIN64(ahigh, alow, ahigh, alow, bhigh, blow);
  *sumlow = alow;
  *sumhigh = ahigh;
}

And the generated output (again gcc-4.4 with -O3):

_add_128_asm:
LFB1:
pushq   %rbp
LCFI2:
# 29 "test.c" 1
addq%r9, %rcx
adcq%r8, %rdx
# 0 "" 2
movq%rsp, %rbp
LCFI3:
movq%rcx, (%rsi)
movq%rdx, (%rdi)
leave
ret

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7197] test_multiprocessing crashes under Windows when run in verbose mode

[Antoine Pitrou]

New submission from Antoine Pitrou :

When run under Windows in verbose mode ("python -m test.regrtest -v
test_multiprocessing"), most tests in test_multiprocessing fail with a
recursion limit error.

The explanation is that most tests are written in the following manner:

class _TestArray(BaseTestCase):
[...]
def test_array(self, raw=False):
[...]
p = self.Process(target=self.f, args=(arr,))

Running a Process under Windows involved pickling it to send it to the
child process. This also pickles the `target` function, which here is a
method of an unittest instance. This consequently pickles the unittest
instance, which has a reference to the unittest runner, which has a
reference to a unittest.runner._WritelnDecorator instance.

The infinite recursion occurs when unpickling the _WritelnDecorator
instance, because the `stream` attribute is not restored when calling
__new__, and the __getattr__ method then recurses when trying to return
`getattr(self.stream,attr)`.

I see two possible resolutions:
- make unittest.runner._WritelnDecorator properly (un)pickleable 
- change all tests in test_multiprocessing to avoid pickling instances
of unittest.TestCase

The former is simpler and probably more future-proof than the latter.

(NB: in non-verbose mode, test.support uses a custom test runner which
doesn't involve the _WritelnDecorator)


Appendix: here is a traceback example (noticed on the newgil branch but
it really happens on stock trunk and py3k):

test_notify (test.test_multiprocessing.WithProcessesTestCondition) ...
Traceback
 (most recent call last):
  File "", line 1, in 
  File "Z:\py3k\newgil\lib\multiprocessing\forking.py", line 339, in main
self = load(from_parent)
  File "Z:\py3k\newgil\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
return getattr(self.stream,attr)
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
return getattr(self.stream,attr)
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
return getattr(self.stream,attr)
[... snipped ...]
RuntimeError: maximum recursion depth exceeded while calling a Python
object

--
components: Library (Lib), Tests
messages: 94406
nosy: jnoller, michael.foord, pitrou
priority: high
severity: normal
status: open
title: test_multiprocessing crashes under Windows when run in verbose mode
type: behavior
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7197] test_multiprocessing crashes under Windows when run in verbose mode

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Amusingly, there is a Test_TextTestRunner but it's not run by
test_unittest

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7197] test_multiprocessing crashes under Windows when run in verbose mode

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Here is a patch.

--
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file15192/testrunner.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7191] Odd behaviour with zlib.decompressobj optional parameter "wbits"

[Georg Brandl]

Georg Brandl  added the comment:

The decompressor can't have a window size smaller than the size used to
compress the stream.  Therefore in your case, it can't be smaller than 15.

Still, the docs must be improved, e.g. compressobj() has more parameters
than documented.

--
assignee:  -> georg.brandl
components: +Documentation -Library (Lib)
nosy: +georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6242] Fix reference leak in io.StringIO

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I just noticed that this hadn't been committed to trunk. I did the
backport myself, but in the future please first commit IO changes to
trunk and then merge to py3k.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3578] 'dictionary changed size' error in test_multiprocessing

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Actually, it's probably a duplicate of #7105.

--
resolution:  -> duplicate
status: open -> closed
superseder:  -> weak dict iterators are fragile because of unpredictable GC runs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7175] unify pydistutils.cfg and distutils.cfg and use .local

[Tarek Ziadé]

Tarek Ziadé  added the comment:

The subversion system looks pretty nice !

So here's a modified proposal that covers the python version as well.

= per user configuration files = 

 ~/.pythonMAJOR.MINOR

Examples:

~/.python2.7
~/.python3.1

On Windows, ~ will be replaced by APPDATA if founded in the environment.
 
== global configuration files ==


On Linux/Mac:

/etc/pythonMAJOR.MINOR

examples:

/etc/python2.7/
/etc/python3.1/

On Windows:

\All users\Application Data\PythonMAJORMINOR

(\All users\Application Data is specified by the Windows Registry)

examples:

\All users\Application Data\Python26
\All users\Application Data\Python31

== standard naming for configuration files ==

The files are all named "NAME.cfg", in all platforms.

== what gets in per-user configuration directory ==

- pypi.cfg
- distutils.cfg

== what gets in global configuration directory ==

- distutils.cfg

== apis added ==

in site.py:

- getuserconfig() : returns the per-user configuration directory
- getconfig(): returns the global configuration directory

(the names are following site.py naming style)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7060] test_multiprocessing dictionary changed size errors and hang

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Kristján:
> So, can one just clear the __traceback__ member?

Yes, or use `exc_value.with_traceback(None)`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Mark Dickinson]

Mark Dickinson  added the comment:

Some key revision numbers from the py3k short float repr, for reference:

r71663:  include Gay's code, build and configure fixes
r71723:  backout SSE2 detection code added in r71663
r71665:  rewrite of float formatting code to use Gay's code

Backported most of r71663 and r71723 to trunk in:

r75651: Add dtoa.c, dtoa.h, update license.
r75658: configuration changes - detect float endianness,
add functions to get and set x87 control word, and
determine when short float repr can be used.

Significant changes from r71663 not yet included:

* Misc/NEWS update
* Lib/test/formatfloat_testcases.txt needs updating to match py3k.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7066] archive_util.make_archive doesn't restore the cwd if an error is raised

[Tarek Ziadé]

Tarek Ziadé  added the comment:

Done in r75659, r75662 and r75663 (merge in 2.6 maint. waiting for 2.6.4
final tag to be done)

Thanks !

--
status: open -> closed
versions: +Python 2.6, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1087418] long int bitwise ops speedup (patch included)

[Gregory Smith]

Gregory Smith  added the comment:

Antoine, "most uses of it are for small ints (< 2**32 or 2**64), ",
that's my point exactly, the current long type is quite slow for those
cases because of sign-magnitude implementation.

I don't see a problem with sign-magnitude for old long ints, or for GMP,
since in those cases the assumption is that you have a fair percentage
of large #s, this is not valid in Py3.0 where most are small. Most
operations are add and subtract, and in each such operation you need to
look at both signs, and decide if you want to really add or subtract,
and if you are subtracting, you then have to do a magnitude test to see
which way - all of that before you do any actual computation. That's a
lot of overhead for e.g. 'i -= 1'. Especially given that the sign &
length information are rolled together into a single value; you need a
fair number of tests & conditional branches. With a 2's complement
implementation you would test for the most common case where both values
are 1 digit (which would include 0 value) and in that case do an add (or
subtract) followed by a check for overflow, and that's the whole thing.
I'm guessing this is 10% - 25% as much time as in the current
implementation - this is one of those cases where the time for tests and
setup dominate over the actual computation.

Mark, what you've written looks fine to me. It would be a bit faster
with an add-with-carry, but there's no conditional branches in the
generated code, so it's not bad, it's still faster than it would be if
you were extracting carries from the msb of each result and masking them
afterwards. Bear in mind that some RISC processors don't have an
add-with-carry (lacking condition codes altogether) so the method of
comparing the sum to the inputs is the only method available. Given that
in Python the operation is in a loop, I think it's unreasonable to to
expect a compiler to identify an add-with-carry application, but at the
same time it's not necessary. Now that the long int is the *only* int
type, multi-digit ops will be relatively uncommon, and you want to focus
on reducing the overhead before and after the operation.

(And, for speed freaks who want to use long ints to implement large bits
arrays and want fast bit ops - my original motivation for this issue -
it would be possible to use SSE2 vector instructions on specific
processors. That can also be done with the 15-bit implementation, but it
works much better with 32).


The 'algorithmic C' package
(http://www.mentor.com/products/esl/high_level_synthesis/ac_datatypes)
is a C++ template class for arbitrary-length signed/unsigned integers.
It's not suitable for use in Python because of license issues and
because its integers are controlled by templates and are fixed-size at
compile time - but it uses Kx32-bit 2's complement format, and
implements adds, shifts, multiplies, logic ops, and limited division
ops. It works very well, with extremely low overhead on small values -
often the code is exactly the same as if you used int type  - that would
not be possible with a sign/magnitude approach.

As for multiplies and divides - it's certainly possible to proceed
through a multiply entirely in 2's complement, so no overhead is needed
for abs value, or changing sign at the end. For division, it's possible
if the denominator is > 0, and it may be possible if <0 (but probably
not worth the hassle).

I would be happy to do a design doc for this and write some of the inner
loops, but if (a) it's already being done or (b) there's no chance of it
being deployed then it would be a waste of time...
if there was definite interest in it (and reasonable schedule
expectations) I could write a lot more of it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Mark Dickinson]

Mark Dickinson  added the comment:

r75666: Add sys.float_repr_style attribute.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mancausoft]

Mancausoft  added the comment:

This bug is still present on arm. 

Python 2.6.3 

cs-e9302# cat ../prova.py
import  math
print math.atan2(0., -0.)
print (math.copysign(4., -0.), -4.0)
print math.atan2(0., -0.)
print (math.copysign(4., -0.), -4.0)
print math.atan2(0., -0.)

cs-e9302# cat ../prova1.py
import  math
print (math.copysign(4., -0.), -4.0)
print math.atan2(0., -0.)
print (math.copysign(4., -0.), -4.0)
print math.atan2(0., -0.)

cs-e9302# ./python ../prova1.py
(-4.0, -4.0)
-3.14159265359
(-4.0, -4.0)
-3.14159265359
cs-e9302# ./python ../prova.py
0.0
(4.0, -4.0)
0.0
(4.0, -4.0)
0.0




>>> from math import atan2
>>> x = -0.
>>> y = 0.
>>> print atan2(y, -1.)
3.14159265359
>>> exec("from math import atan2; x = -0.; y = 0.; print atan2(y,
-1.)")
-3.14159265359
>>> x = -0.; atan2(0., -1)
-3.1415926535897931
>>> x = 0.; atan2(0., -1)
3.1415926535897931

==
FAIL: testAtan2 (__main__.MathTests)
--
Traceback (most recent call last):
  File "Lib/test/test_math.py", line 131, in testAtan2
self.ftest('atan2(0., -0.)', math.atan2(0., -0.), math.pi)
  File "Lib/test/test_math.py", line 57, in ftest
(name, value, expected))
AssertionError: atan2(0., -0.) returned 0.0, expected
3.1415926535897931

==
FAIL: testCopysign (__main__.MathTests)
--
Traceback (most recent call last):
  File "Lib/test/test_math.py", line 806, in testCopysign
self.assertEqual(math.copysign(4., -0.), -4.0)
AssertionError: 4.0 != -4.0

--

--
nosy: +mancausoft

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mark Dickinson]

Mark Dickinson  added the comment:

Mancausoft:  is this little-endian, OABI?

If so, then I think I know  what the problem is:  the disambiguation code 
in compile.c looks at the first and last bytes of the double to 
distinguish 0.0 and -0.0;  for mixed-endian (aka little-endian, swapped 
words) doubles this will fail.

The solution is to use copysign instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1087418] long int bitwise ops speedup (patch included)

[Martin v . Löwis]

Martin v. Löwis  added the comment:

> I would be happy to do a design doc for this and write some of the inner
> loops, but if (a) it's already being done or (b) there's no chance of it
> being deployed then it would be a waste of time...
> if there was definite interest in it (and reasonable schedule
> expectations) I could write a lot more of it.

I think the only realistic chance in which something may change is that
somebody sits down and does all the work, from beginning to end. I doubt
that a design doc and some inner loops alone will make any difference.

It's not being done (to my knowledge), for the reason alone that it
would be a lot of work. It has a chance to be deployed only if a)
it is significantly faster, for small numbers, b) correct, and c)
only slightly more complicated than the current code.

I notice that such discussion is off-topic for this bug report, which
is about your original patch. All we have to decide here is whether to
accept it (perhaps with modifications), or to reject it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7071] distutils and IronPython compatibility

[Tarek Ziadé]

Tarek Ziadé  added the comment:

Notice that install_lib has a --no-compile option that can be used to
avoid compiling .pyc/.pyo files.

What I am adding now is just a gentle warning in case a compilation is
tried and sys.dont_write_bytecode is True, so the installation may
proceed nevertheless and it doesn't through an error.

For the 2.6 backport, it can be added in 2.6.5 I guess,

--
priority:  -> normal
resolution:  -> accepted
versions: +Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mancausoft]

Mancausoft  added the comment:

Mark Dickinson  scrisse:

> Mancausoft:  is this little-endian, OABI?

Mixed endian

> If so, then I think I know  what the problem is:  the disambiguation
> code in compile.c looks at the first and last bytes of the double to 
> distinguish 0.0 and -0.0;  for mixed-endian (aka little-endian,
> swapped words) doubles this will fail.
> 
> The solution is to use copysign instead.

I try: *p==0 && p[sizeof(double)-1]==0 && p[(sizeof(double)-1)/2]==0;

and now the test_math result is:

Ran 39 tests in 21.323s

OK

It's a safe patch?

Mancausoft

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mark Dickinson]

Mark Dickinson  added the comment:

> I try: *p==0 && p[sizeof(double)-1]==0 && p[(sizeof(double)-1)/2]==0;

Sure, that should work.  It would seem cleaner and safer to use copysign, 
though: that way, things will still work when some other byte layout comes 
along, or when some version of Python starts using 128-bit IEEE 754 
doubles instead of 64-bit, or ...

Reopening:  I've been meaning to fix these checks to use copysign for a 
while now, anyway.

--
priority: high -> normal
resolution: accepted -> 
stage:  -> needs patch
status: closed -> open
type:  -> behavior
versions: +Python 2.6, Python 2.7, Python 3.1, Python 3.2 -Python 2.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7111] core dump when stderr is moved

[Matteo Bertini]

Changes by Matteo Bertini :


--
nosy: +naufraghi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7111] stdout closed

[Matteo Bertini]

Changes by Matteo Bertini :


--
title: core dump when stderr is moved -> stdout closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mark Dickinson]

Mark Dickinson  added the comment:

Here's a patch for floats.  Mancausoft, could you give this a try and let 
me know whether it fixes the issue?

The same fix would also need to be applied for complex numbers.

--
Added file: http://bugs.python.org/file15193/issue1678380.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7111] core dump when stderr is moved

[Matteo Bertini]

Matteo Bertini  added the comment:

sorry, title restored!

--
title: stdout closed -> core dump when stderr is moved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mancausoft]

Mancausoft  added the comment:

Mark Dickinson  scrisse:

> 
> Mark Dickinson  added the comment:
> 
> Here's a patch for floats.  Mancausoft, could you give this a try and
> let me know whether it fixes the issue?

it works.

test_math

Ran 39 tests in 23.561s   

OK

test_float
Ran 19 tests in 275.241s

OK

Mancausoft

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1678380] 0.0 and -0.0 identified, with surprising results

[Mark Dickinson]

Mark Dickinson  added the comment:

Thanks for testing!  I'll apply the fix once the 2.6 branch is unfrozen.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Mark Dickinson]

Mark Dickinson  added the comment:

r75672:  temporarily disable the short float repr while we're putting
 the pieces in place.

When testing, the disablement can be disabled (hah) by defining the 
PY_SHORT_FLOAT_REPR preprocessor symbol, e.g. (on Unix) with

CC='gcc -DPY_SHORT_FLOAT_REPR' ./configure && make

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1488934] file.write + closed pipe = no error

[Matteo Bertini]

Changes by Matteo Bertini :


--
nosy: +naufraghi
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1596] Broken pipes should be handled better in 2.x

[Matteo Bertini]

Changes by Matteo Bertini :


--
nosy: +naufraghi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6986] _json crash on scanner/encoder initialization error

[Antoine Pitrou]

Antoine Pitrou  added the comment:

The tests could go in Lib/json/tests/test_speedups.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Eric Smith]

Eric Smith  added the comment:

I think the next step on my side is to remove _PyOS_double_to_string,
and make all of the internal code call PyOS_double_to_string. The
distinction is that _PyOS_double_to_string gets passed a buffer and
length, but  PyOS_double_to_string returns allocated memory that must be
freed. David Gay's code (_Py_dg_dtoa) returns allocated memory, so
that's the easiest interface to propagate internally.

That's the approach we used in the py3k branch. I'll start work on it.
So Mark's work should be mostly config stuff and hooking up Gay's code
to PyOS_double_to_string. I think it will basically match the py3k version.

The existing _PyOS_double_to_string will become the basis for the
fallback code for use when PY_NO_SHORT_FLOAT_REPR is defined (and it
will then be renamed PyOS_double_to_string and have its signature
changed to match).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Mark Dickinson]

Mark Dickinson  added the comment:

One issue occurs to me: should the backport change the behaviour of the 
round function?

In py3k, round consistently uses round-half-to-even for halfway cases.
In trunk, round semi-consistently uses round-half-away-from-zero (and 
this is documented).  E.g., round(1.25, 1) will give 1.2 in py3k and 
(usually) 1.3 in trunk.

I definitely want to use Gay's code for round in 2.7, since having round 
work sensibly is part of the motivation for the backport in the first 
place.  But this naturally leads to a round-half-to-even version of 
round, since the Python-adapted version of Gay's code isn't capable of 
doing anything other than round-half-to-even at the moment.

Options:

(1) change round in 2.7 to do round-half-to-even.  This is easy,
natural, and means that round will agree with float formatting
(which does round-half-to-even in both py3k and trunk).  But it
may break existing applications.  However:  (a) those applications
would need fixing anyway to work with py3k, and (b) I have little
sympathy for people depending on behaviour of rounding of
*binary* floats for halfway *decimal* cases.  (Decimal is another
matter, of course:  there it's perfectly reasonable to expect
guaranteed rounding behaviour.)

It's more complicated than that, though, since if rounding
becomes round-half-to-even for floats, it should also change
for integers, Fractions, etc.

(2) have round stick with round-half-away-from-zero.  This may be
awkward to implement (though I have some half-formed ideas about
how to make it work), and would lead to round occasionally not
agreeing with float formatting.  For example:

>>> '{0:.1f}'.format(1.25)
'1.2'
>>> round(1.25, 1)
1.3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3488] Provide compress/uncompress functions on the gzip module

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Seems simple enough, I don't see why not.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Eric Smith]

Eric Smith  added the comment:

Adding tim_one as nosy. He'll no doubt have an opinion on rounding. And
hopefully he'll share it!

--
nosy: +tim_one

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1625] bz2.BZ2File doesn't support multiple streams

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I'm not comfortable with the following change (which appears twice in
the patch):

-   BZ2_bzReadClose(&bzerror, self->fp);
+   if (self->fp)
+   BZ2_bzReadClose(&bzerror, self->fp);
break;
case MODE_WRITE:
-   BZ2_bzWriteClose(&bzerror, self->fp,
-0, NULL, NULL);
+   if (self->fp)
+   BZ2_bzWriteClose(&bzerror, self->fp,
+0, NULL, NULL);


If you need to test for the file pointer, perhaps there's a logic flaw
in your patch. Also, it might be dangerous in write mode: could it occur
that the file isn't closed but the problem isn't reported?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1087418] long int bitwise ops speedup (patch included)

[Mark Dickinson]

Mark Dickinson  added the comment:

> Most operations are add and subtract, and in each such operation you
> need to look at both signs, and decide if you want to really add or
> subtract, and if you are subtracting, you then have to do a magnitude
> test to see which way - all of that before you do any actual
> computation. That's a lot of overhead for e.g. 'i -= 1'.

Hmm.  I agree this isn't ideal, and I now see the attraction of
two's complement.  Thanks.

It would be interesting to see timings from such an approach.
Maybe one could just implement the basic operations (+, -, *, /)
to get an idea of whether it's worth considering more seriously.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2504] Add gettext.pgettext() and variants support

[Andrew Ziem]

Changes by Andrew Ziem :


--
nosy: +AndrewZiem

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2746] ElementTree ProcessingInstruction uses character entities in content

[Tom Lynn]

Changes by Tom Lynn :


--
nosy: +tlynn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7111] abort when stderr is moved

[Benjamin Peterson]

Changes by Benjamin Peterson :


--
title: core dump when stderr is moved -> abort when stderr is moved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Eric Smith]

Eric Smith  added the comment:

Another thing to consider is that in py3k we removed all conversions of
converting 'f' to 'g', such as this, from Objects/unicodeobject.c:

if (type == 'f' && fabs(x) >= 1e50)
type = 'g';

Should we also do that as part of this exercise? Or should it be another
issue, or not done at all?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7198] csv.writer

[Bob Cannon]

Changes by Bob Cannon :


--
nosy: zacktu
severity: normal
status: open
title: csv.writer
type: behavior
versions: Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7198] csv.writer

[Bob Cannon]

New submission from Bob Cannon :

I used csv.writer to open a file for writing with comma as separator and
dialect='excel'.  I used writerow to write each row into the file.  When
I execute under linux, each line is terminated by '\r\n'.  When I
execute under windows, each line is terminated by '\r\r\n'.  Thus, under
MS Windows, when I read the csv file, there is a blank line between each
significant line.  I have dropped cvs.writer and now build each line
manually and terminate it with '\n'.  When the line is written in
windows, it is terminated by '\r\n'.  That's what should happen.  

As I see it, writerow with dialect='excel' should only terminate a line
with '\n'.  Windows will automatically place a '\r' in front of the '\n'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7198] csv.writer

[Skip Montanaro]

Skip Montanaro  added the comment:

Your output file should be opened in binary mode.  Sounds like you
opened it in text mode.

--
nosy: +skip.montanaro

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7199] Doc: Logging level order seems inconsistent

[Terry J. Reedy]

New submission from Terry J. Reedy :

LibRef 15.6, Logging module:


Intro start with "The default levels provided are DEBUG, INFO, WARNING,
ERROR and CRITICAL." which seems like the proper ordering. Then

15.6.1.1. Simple examples
"Another useful feature of the logging API is the ability to produce
different messages at different log levels. This allows you to
instrument your code with debug messages, for example, but turning the
log level down so that those debug messages are not written for your
production system. The default levels are CRITICAL, ERROR, WARNING,
INFO, DEBUG and NOTSET.

The logger, handler, and log message call each specify a level. The log
message is only emitted if the handler and logger are configured to emit
messages of that level or lower. "

Ie, the order is reversed and the level specified is the highest level
emitted, which struck me as backwards, versus

15.6.1.2. Loggers
"Logger.setLevel() specifies the lowest-severity log message a logger
will handle, where debug is the lowest built-in severity level and
critical is the highest built-in severity. "

so that it emits the that level or *higher*.
(although notset is the actual lowest built-in.)

Reading further, I see that the builtin levels range for NOTSET=0 to
CRITICAL=50 (15.6.2. Logging Levels), so I think the first quote should
be changed to

"Another useful feature of the logging API is the ability to produce
different messages at different log levels. This allows you to
instrument your code with debug messages, for example, but turning the
log level up so that those debug messages are not written for your
production system. The default levels are NOTSET, DEBUG, INFO, WARNING,
ERROR, and CRITICAL.

The logger, handler, and log message call each specify a level. The log
message is only emitted if the handler and logger are configured to emit
messages of that level or higher. "

I have not checked the rest of the doc for other backwards passages.

--
assignee: vinay.sajip
components: Library (Lib)
messages: 94439
nosy: tjreedy, vinay.sajip
severity: normal
status: open
title: Doc: Logging level order seems inconsistent
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7200] multiprocessing deadlock on Mac OS X when queue collected before process terminates

[Brian Quinlan]

New submission from Brian Quinlan :

This code:

import multiprocessing
import queue

def _process_worker(q):
while True:
try:
something = q.get(block=True, timeout=0.1)
except queue.Empty:
return
else:
pass
# print('Grabbed item from queue:', something)


def _make_some_processes(q):
processes = []
for _ in range(10):
p = multiprocessing.Process(target=_process_worker, args=(q,))
p.start()
processes.append(p)
return processes

#p = []
def _do(i):
print('Run:', i)
q = multiprocessing.Queue()
#p.append(q)
print('Created queue')
for j in range(30):
q.put(i*30+j)
processes = _make_some_processes(q)
print('Created processes')

while not q.empty():
pass
print('Q is empty')

for i in range(100):
_do(i)

Produces this output on Mac OS X (it produces the expected output on
Linux and Windows):

Run: 0
Created queue
Grabbed item from queue: 0
...
Grabbed item from queue: 29
Created processes
Q is empty
Run: 1
Created queue
Grabbed item from queue: 30
...
Grabbed item from queue: 59
Created processes
Q is empty
Run: 2
Created queue
Created processes


Changing the code as follows:

+ p = []
def _do(i):
print('Run:', i)
q = multiprocessing.Queue()
+   p.append(q)
print('Created queue')
for j in range(30):
q.put(i*30+j)
processes = _make_some_processes(q)
print('Created processes')

while not q.empty():
pass
print('Q is empty')

fixes the deadlock. So it looks like if a multiprocessing.Queue is
collected with sub-processes still using it then calling some methods on
other multiprocessing.Queues with deadlock.

--
components: Library (Lib)
messages: 94440
nosy: bquinlan
severity: normal
status: open
title: multiprocessing deadlock on Mac OS X when queue collected before process 
terminates
type: behavior
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7198] csv.writer

[Bob Cannon]

Bob Cannon  added the comment:

Probably so.  I'm sorry to report this as a bug if it's not.  I asked
abut this on a Python group on IRC and got no suggestions.  Thanks for
taking a look.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7198] csv.writer

[Bob Cannon]

Changes by Bob Cannon :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2746] ElementTree ProcessingInstruction uses character entities in content

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Can you include the cElementTree fix and test case in your patch as well?

--
nosy: +pitrou
priority:  -> normal
stage:  -> patch review
versions: +Python 3.2 -Python 2.5, Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2746] ElementTree ProcessingInstruction uses character entities in content

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Oops, sorry, I hadn't read your message about the patch also correcting
cElementTree.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7201] double Endian problem and more on arm

[Mancausoft]

New submission from Mancausoft :

I compile python for arm (on debian etch) but it don't  pass ctype
test:

==
FAIL: test_struct_return_2H
(ctypes.test.test_as_parameter.AsParamPropertyWrapperTestCase)
--  
  
Traceback (most recent call last):  
  
  File
"/mnt/root/stackless-2.6.3/Lib/ctypes/test/test_as_parameter.py", line
171, in test_struct_return_2H
self.failUnlessEqual((s2h.x, s2h.y), (99*2, 88*3))  
   
AssertionError: (99, 88) != (198, 264)  
   

==
FAIL: test_struct_return_2H
(ctypes.test.test_as_parameter.AsParamWrapperTestCase)
--  
  
Traceback (most recent call last):  
  
  File
"/mnt/root/stackless-2.6.3/Lib/ctypes/test/test_as_parameter.py", line
171, in test_struct_return_2H
self.failUnlessEqual((s2h.x, s2h.y), (99*2, 88*3))  
   
AssertionError: (99, 88) != (198, 264)  
   

==
FAIL: test_struct_return_2H
(ctypes.test.test_as_parameter.BasicWrapTestCase)
--   
Traceback (most recent call last):   
  File
"/mnt/root/stackless-2.6.3/Lib/ctypes/test/test_as_parameter.py", line
171, in test_struct_return_2H
self.failUnlessEqual((s2h.x, s2h.y), (99*2, 88*3))
AssertionError: (99, 88) != (198, 264)

==
FAIL: test_endian_double (ctypes.test.test_byteswap.Test)
--
Traceback (most recent call last):
  File "/mnt/root/stackless-2.6.3/Lib/ctypes/test/test_byteswap.py",
line 137, in test_endian_double
self.failUnlessEqual(bin(struct.pack("

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1602] windows console doesn't print utf8 (Py30a2)

[Glenn Linderman]

Glenn Linderman  added the comment:

With Python 3.1.1, the following batch file seems to be necessary to use
UTF-8 successfully from an XP console:

set PYTHONIOENCODING=UTF-8
cmd /u /k chcp 65001
set PYTHONIOENCODING=
exit

the cmd line seems to be necessary because of Windows having
compatibility issues, but it seems that Python should notice the cp65001
and not need the PYTHONIOENCODING stuff.

--
nosy: +v+python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1625] bz2.BZ2File doesn't support multiple streams

[David Bonner]

David Bonner  added the comment:

That was mostly just out of paranoia, since the comments mentioned 
multiple calls to close being legal.  Looking at it again, that particular 
case isn't an issue, since we don't hit that call when the mode is 
MODE_CLOSED.  The testsuite runs happily with those changes reverted.  
Should I upload a new patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7117] Backport py3k float repr to trunk

[Raymond Hettinger]

Raymond Hettinger  added the comment:

+1 on backporting the 'f' and 'g' work also.
We will be well served by getting the two
code bases ins-sync with one another.

Eliminating obscure differences makes it easier
to port code from 2.x to 3.x

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com