[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee:  -> mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Mark Dickinson added the comment:

Trent, many thanks for the report and analysis.  The LOG1P_DROPS_ZERO_SIGN 
hackery was a result of a previous attempt to 'fix' the tests on another 
platform whose log1p didn't preserve zero signs.  But now that this behaviour 
has emerged on OS X too, I think it might be time to fix this in the Python 
core code (i.e., add a workaround for the buggy OS behaviour) instead of trying 
to make the tests pass again.

Attached is a patch that adds a workaround for this on all platforms that have 
a system log1p.  I was originally thinking of adding this workaround only on OS 
X (so as not to unnecessarily punish those platforms that get it right), but 
after running some timings (see below) it seems that the workaround is 
insignificant performance-wise, so we might as well keep things simple and add 
it on all platforms.

Trent, if you have time, please could you try this patch on OS X 10.8 and see 
if it fixes all the test_cmath and test_math failures?




Some timings on OS X 10.6.

Current default branch (pass value directly onto system log1p):

iwasawa:cpython mdickinson$ ./python.exe -m timeit -s 'from math import log1p; 
x = 2.3' 'log1p(x)'
1000 loops, best of 3: 0.0685 usec per loop

With the attached patch (do zero check, then pass value directly onto system 
log1p):

iwasawa:cpython mdickinson$ ./python.exe -m timeit -s 'from math import log1p; 
x = 2.3' 'log1p(x)'
1000 loops, best of 3: 0.0686 usec per loop

--
Added file: http://bugs.python.org/file26884/log1p_sign_workaround.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Mark Dickinson added the comment:

Updated patch that also removes the sysconfig checks from the cmath tests.

--
Added file: http://bugs.python.org/file26885/log1p_sign_workaround.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Changes by Mark Dickinson :


Removed file: http://bugs.python.org/file26884/log1p_sign_workaround.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Changes by Mark Dickinson :


--
components: +Extension Modules
stage:  -> patch review
type:  -> behavior
versions: +Python 2.7, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Mark Dickinson added the comment:

See also http://bugs.python.org/issue9920;  the patch here *should* fix that 
issue, too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Trent Nelson]

Trent Nelson added the comment:

Happy to report your patch does the trick Mark.  test_cmath passes on 10.7 and 
10.8 with it applied.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15588] quopri: encodestring and decodestring handle bytes, not strings

[Petri Lehtinen]

Petri Lehtinen added the comment:

The encode() and decode() functions also expect binary file objects.

Attached a patch that changes documentation and docstrings. Removed the 
mentions of readline(), because read() is called on input instead, if the 
binascii module is available. Also removed the repeated descriptions of 
quotetabs and header in encodestring()'s docs.

--
keywords: +needs review, patch
nosy: +petri.lehtinen
stage:  -> patch review
Added file: http://bugs.python.org/file26886/issue15588.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Martin v . Löwis]

Martin v. Löwis added the comment:

BTW, did anybody file a bug report with Apple? Even to my untrained eye, that 
looks like a clear violation of the POSIX standard:

If x is ±0, or +Inf, x shall be returned.
http://pubs.opengroup.org/onlinepubs/009604599/functions/log1p.html

C seems to make it implementation-defined, unless the implementation defines 
__STDC_IEC_559__, in which case log1p also needs to return ±0 (C99  F.9.3.9, 
C11 F.10.3.9).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15721] PEP 3121, 384 Refactoring applied to tkinter module

[Robin Schreiber]

New submission from Robin Schreiber:

Changes proposed in PEP3121 and PEP384 have now been applied to the tkinter 
module!
When running the test form inside Python.exe (tkinter._test()), the litte 
"test-window" is rendered correctly. However there are still some error 
messages popping up inside the python shell 
(I am running this on OS X 10.8):

>>> tkinter._test()
2012-08-18 12:46:24.775 python.exe[17410:707] speedup (null)
2012-08-18 12:46:24.776 python.exe[17410:707] ERROR: Unable to find method 
[_NSFullScreenTransition 
_startFullScreenTransitionForCGWindow:targetFrame:duration:].
2012-08-18 12:46:24.776 python.exe[17410:707] ERROR: Unable to find method 
[_NSFullScreenTransition 
_startEnterFullScreenTransitionForCGWindow:targetFrame:duration:].
2012-08-18 12:46:24.778 python.exe[17410:707] ERROR: Unable to find method 
[_NSFullScreenTransition 
startExitFullScreenTransitionForCGWindow:targetFrame:duration:].
[87221 refs]
>>> 

I have no Idea how and where these are triggered.

--
components: Extension Modules
files: _tkinter_pep3121-384_v0.patch
keywords: patch
messages: 168504
nosy: Robin.Schreiber
priority: normal
severity: normal
status: open
title: PEP 3121, 384 Refactoring applied to tkinter module
type: resource usage
versions: Python 3.4
Added file: http://bugs.python.org/file26887/_tkinter_pep3121-384_v0.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15721] PEP 3121, 384 Refactoring applied to tkinter module

[Martin v . Löwis]

Martin v. Löwis added the comment:

See

http://www.python.org/download/mac/tcltk/

It may be that using a different build of Tcl/Tk solves that problem.

--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Mark Dickinson added the comment:

> BTW, did anybody file a bug report with Apple?

I'll file one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Roundup Robot]

Roundup Robot added the comment:

New changeset 08418369da7b by Mark Dickinson in branch '3.2':
Issue #15477: Add workaround for log1p(-0.0) on platforms where it's broken.
http://hg.python.org/cpython/rev/08418369da7b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Roundup Robot]

Roundup Robot added the comment:

New changeset 336653319112 by Mark Dickinson in branch 'default':
Issue #15477: Merge fix from 3.2
http://hg.python.org/cpython/rev/336653319112

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15477] test_cmath failures on OS X 10.8

[Mark Dickinson]

Mark Dickinson added the comment:

Apple bug report filed: Bug ID# 12128251.

Closing the issue.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9920] test_cmath on atan fails on AIX

[Mark Dickinson]

Mark Dickinson added the comment:

The fix applied for issue #15477 should also fix this issue.

--
status: open -> closed
versions: +Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15722] PEP 3121, 384 Refactoring applied to decimal module

[Robin Schreiber]

New submission from Robin Schreiber:

Changes proposed in PEP3121 and PEP384 have now been applied to the decimal 
module!

--
components: Regular Expressions
files: _decimal_pep3121-384_v0.patch
keywords: patch
messages: 168511
nosy: Robin.Schreiber, ezio.melotti, mrabarnett, skrah
priority: normal
severity: normal
status: open
title: PEP 3121, 384 Refactoring applied to decimal module
type: resource usage
versions: Python 3.4
Added file: http://bugs.python.org/file26888/_decimal_pep3121-384_v0.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15722] PEP 3121, 384 Refactoring applied to decimal module

[Robin Schreiber]

Changes by Robin Schreiber :


--
components: +Extension Modules -Regular Expressions

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15720] move __import__() out of the default lookup chain

[R. David Murray]

R. David Murray added the comment:

I could be wrong, but it is hard for me to see how we could justify doing this 
before python4, at this point in python3.

Adding a warning would be uncontroversial, though.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14803] Add feature to allow code execution prior to __main__ invocation

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15716] Ability to specify the PYTHONPATH via a command line flag

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15720] move __import__() out of the default lookup chain

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14905] zipimport.c needs to support namespace packages when no 'directory' entry exists

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15645] 2to3 Grammar pickles not created when upgrading to 3.3.0b2

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15718] Possible OverflowError in __len__ method undocumented (when called via len() function)

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15641] Clean up importlib for Python 3.4

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15649] subprocess.Popen.communicate: accept str for input parameter if universal_newlines is False

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15722] PEP 3121, 384 Refactoring applied to decimal module

[Stefan Krah]

Stefan Krah added the comment:

Thank you for the patch and the big amount of work that you are doing on so
many modules!

I'm afraid though that the patch is not acceptable in its current state:

1) The unit tests do not pass. This could be fixed.

2) The patch slows down _decimal by 25% (!).


If 2) cannot be fixed due to the increased amount of state lookups, I'm
firmly -1 on applying pep-3121 changes to _decimal (and any other speed
sensitive module).

--
nosy: +mark.dickinson, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15722] PEP 3121, 384 Refactoring applied to decimal module

[Stefan Krah]

Changes by Stefan Krah :


--
assignee:  -> skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15668] PEP 3121, 384 Refactoring applied to random module

[Stefan Krah]

Stefan Krah added the comment:

I tried to benchmark this patch but I'm getting a segfault:

Python 3.3.0b2+ (default:dc18d73e67a5, Aug 18 2012, 15:37:04) 
[GCC 4.4.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import random
Segmentation fault

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15672] PEP 3121, 384 Refactoring applied to testbuffer module

[Stefan Krah]

Changes by Stefan Krah :


--
assignee:  -> skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8323] buffer objects are picklable but result is not unpicklable

[Florent Xicluna]

Changes by Florent Xicluna :


--
nosy: +flox

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13498] os.makedirs exist_ok documentation is incorrect, as is some of the behavior

[Hynek Schlawack]

Hynek Schlawack added the comment:

Ok, let’s do it here, that’s easier:


.. function:: makedirs(path, mode=0o777, exist_ok=False)

   .. index::
  single: directory; creating
  single: UNC paths; and os.makedirs()

   Recursive directory creation function.  Like :func:`mkdir`, but makes all
   intermediate-level directories needed to contain the leaf directory.

   The default *mode* is ``0o777`` (octal).  On some systems, *mode* is
   ignored.  Where it is used, the current umask value is first masked out.

   If the target directory exists, :exc:`OSError` is raised unless *exist_ok*
   is set to ``True`` and the mode doesn't contradict the designated mode as
   discussed in the previous paragraph.  If the mode doesn't match,
   :exc:`OSError` is raised regardless of the value of *exist_ok*.  If the
   directory cannot be created in other cases, an :exc:`OSError` exception is
   raised too.

   .. note::

  :func:`makedirs` will become confused if the path elements to create
  include :data:`pardir`.

   This function handles UNC paths correctly.

   .. versionadded:: 3.2
  The *exist_ok* parameter.


Python is so much easier than English. :'(

--
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15668] PEP 3121, 384 Refactoring applied to random module

[Mark Dickinson]

Mark Dickinson added the comment:

The Py_tp_bases line below doesn't look right.  I suspect that's what's causing 
the segfault.


+static PyType_Slot Random_Type_slots[] = {
+{Py_tp_getattro, PyObject_GenericGetAttr},
+{Py_tp_doc, random_doc},
+{Py_tp_methods, random_methods},
+{Py_tp_new, random_new},
+{Py_tp_free, PyObject_Free},
+{Py_tp_bases, },
+{0, 0}

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13498] os.makedirs exist_ok documentation is incorrect, as is some of the behavior

[R. David Murray]

R. David Murray added the comment:

This is much better.  But let me try tuning the problem paragraph a bit, since 
I'm a native English speaker:

   If *exists_ok* is ``False`` (the default), an :exc:`OSError` is raised if
   the target directory already exists.  If *exists_ok* is ``True`` an
   :exc:`OSError` is still raised if the umask-masked *mode* is different from
   the existing mode, on systems where the mode is used.  :exc:`OSError` will
   also be raised if the directory creation fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15668] PEP 3121, 384 Refactoring applied to random module

[Mark Dickinson]

Mark Dickinson added the comment:

With the Py_tp_bases line removed, all tests pass for me on a non-debug 64-bit 
build on OS X 10.6.  A quick timing of random.random() showed no 
distinguishable performance impact.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15668] PEP 3121, 384 Refactoring applied to random module

[Stefan Krah]

Stefan Krah added the comment:

Thanks, Mark. With your change applied I can't measure any performance
differences either.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila

[Stefan Mihaila]

Stefan Mihaila added the comment:

Maybe you can set this issue as the superseder of issue9269, because the 
patches there have already been applied here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15719] Sort dict items in urlencode()

[Guido van Rossum]

Guido van Rossum added the comment:

Note, the sort may fail if e.g. bytes and str are mixed in the keys.  So this 
should be caught and ignored.

An alternative would just be to fix the call site to pass in 
sorted(query.items()) instead of sorted(query).

Still, I think that more predictable output would be better.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15721] PEP 3121, 384 Refactoring applied to tkinter module

[Ned Deily]

Ned Deily added the comment:

What version of 10.8 are you seeing those messages with?  And do you see them 
without the patch applied?

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15615] More tests for JSON decoder to test Exceptions

[Roundup Robot]

Roundup Robot added the comment:

New changeset cd9a4883dd02 by Antoine Pitrou in branch '3.2':
Issue #15615: Add some tests for the json module's handling of invalid input 
data.
http://hg.python.org/cpython/rev/cd9a4883dd02

New changeset 01717c3da4fb by Antoine Pitrou in branch 'default':
Issue #15615: Add some tests for the json module's handling of invalid input 
data.
http://hg.python.org/cpython/rev/01717c3da4fb

New changeset b16a5f0d0c87 by Antoine Pitrou in branch '2.7':
Issue #15615: Add some tests for the json module's handling of invalid input 
data.
http://hg.python.org/cpython/rev/b16a5f0d0c87

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15615] More tests for JSON decoder to test Exceptions

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thanks and congratulations for your patch! It's now committed to all three 
branches.

--
resolution:  -> fixed
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15719] Sort dict items in urlencode()

[Antoine Pitrou]

Antoine Pitrou added the comment:

Honestly, this looks much more like an enhancement than a bugfix to me, so I 
think it should only target 3.4.
As for the idea proper, I sounds ok to me, as long as only exact dict instances 
are affected (i.e. OrderedDict and friends keep using their native order).

--
components: +Library (Lib)
nosy: +pitrou
type:  -> enhancement
versions:  -Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15722] PEP 3121, 384 Refactoring applied to decimal module

[Antoine Pitrou]

Antoine Pitrou added the comment:

> 2) The patch slows down _decimal by 25% (!).

Ouch. That's an interesting observation, because I guess other modules could be 
affected as well.

Robin, can you please take care that other modules don't have significant 
performance regressions after your work? I'm not sure what kind of changes it 
involves, or if it's possible at all, but we don't want to slow down Python for 
what is mostly (IMO) a code cleanup.

--
nosy: +loewis, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15653] PEP 3121, 384 refactoring applied to hashopenssl module

[Antoine Pitrou]

Antoine Pitrou added the comment:

> So I'd propose that it is actually the leaf subtype which decrefs
> ob_type. The check whether you are the leaf type is then done by
> checking whether tp_dealloc is the one you are "in" right now.

By leaf, you mean the derived subtype? That sounds a bit quirky (you
need the aforementioned "if"), how about the base heap type instead?

Speaking of which, what does this refactoring bring exactly? The type
declarations are slightly less verbose, but other than that, is there a
point? (using the stable ABI doesn't seem to provide any benefit for
standard extension modules; besides, using PyType_FromSpec is only a
small part of complying with the stable ABI)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Poul-Henning Kamp]

New submission from Poul-Henning Kamp:

When a file is opened in append mode, the operating system guarantees that all 
write(2) system calls atomically appended their payload to the file.

At least on FreeBSD, Python breaks this guarantee, by chopping up large writes 
into multiple write(2) syscalls to the OS.

Try running this program using ktrace/truss/strace or a similar system-call 
tracing facility:

   fo = open("/tmp/_bogus", "ab", 0)
   fo.write(bytearray(1024*1024))
   fo.close()

Instead of one single megabyte write, I see 1024 kilobyte writes.

(BTW: Why only one kilobyte ? That is an incredible pessimisation these days...)

I leave it to the python community to decide if this should be fixed, or merely 
pointed out in documentation (os.write() is a workaround)

--
components: None
messages: 168528
nosy: bsdphk
priority: normal
severity: normal
status: open
title: Python breaks OS' append guarantee on file writes
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Antoine Pitrou]

Antoine Pitrou added the comment:

> When a file is opened in append mode, the operating system guarantees
> that all write(2) system calls atomically appended their payload to the 
> file.

Does it? I don't see such strong guarantees in 
http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html

In any case, Python 2 uses fwrite() not write(), so it may be the explanation. 
Do you observe the same behaviour when using io.open() instead of open()?

(io.open() is the Python 3 IO stack backported to Python 2)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Poul-Henning Kamp]

Poul-Henning Kamp added the comment:

Yes, it does:

If the O_APPEND flag of the file status flags is set, the file offset shall be 
set to the end of the file prior to each write and no intervening file 
modification operation shall occur between changing the file offset and the 
write operation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ah, sorry. I was stupidly looking for "atomic" and only found the pipe-specific 
remarks.
(the other points remain, though :-))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Poul-Henning Kamp]

Poul-Henning Kamp added the comment:

I have not tried io.open(), nor would I suspect most users would realize that 
they needed to do so, in order to get the canonical behaviour from an operation 
called "write" on a file opened in "append" mode.

IMO: If pythons file.write() does not give the guarantee POLA would indicate, 
it's either a bug or a doc-issue, no matter how many workarounds might exist.

But I have neither a clue to the aspirational goals of python, nor to what it 
might take to fix this, so it's entirely your call.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Antoine Pitrou]

Antoine Pitrou added the comment:

> I have not tried io.open(), nor would I suspect most users would
> realize that they needed to do so, in order to get the canonical
> behaviour from an operation called "write" on a file opened in
> "append" mode.

The reason I'm asking is that open() is the same as io.open() in Python
3.x, which is currently the main development line. That said, I can find
the results myself.

Python 2 is in bugfix mode, so it's impossible to rewrite the I/O
routines to use unbuffered I/O instead of C buffered I/O.

> IMO: If pythons file.write() does not give the guarantee POLA would
> indicate, it's either a bug or a doc-issue, no matter how many
> workarounds might exist.

What do you call POLA?

> But I have neither a clue to the aspirational goals of python, nor to
> what it might take to fix this, so it's entirely your call.

Well as I said, Python 2 will be pretty much impossible to fix (we call
fwrite() with the argument, not write()). Python 3 is a different story
since we use our own buffering layer and then C's unbuffered API.

As a sidenote, do you know if writev() has the same guarantee as
write()? POSIX doesn't seem to say so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Poul-Henning Kamp]

Poul-Henning Kamp added the comment:

POLA = Principle Of Least Astonishment

We use that a lot in architectural decision in FreeBSD :-)

As I said: You deal with this as you see fit. If all python2 gets is a doc- or 
errata-notice, that's perfectly fine with me.

I interpret "The writev() function shall be equivalent to write(), except as 
described below." as writev() giving the same atomic append guarantee.

In FreeBSD, write() is implemented using writev() and I expect that is the 
obvious and thus common way it is done.

(You seem to be right with respect to the 1024: That is indeed still the BUFSIZ 
on FreeBSD, I'll work on getting that changed.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1574] Touchpad 2 Finger scroll does not work in IDLE on Mac (But scroll wheel of external mouse does)

[Roger Serwy]

Roger Serwy added the comment:

Tk support on Mac has improved greatly since 2008. Is this still a valid issue?

--
nosy: +serwy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15722] PEP 3121, 384 Refactoring applied to decimal module

[Robin Schreiber]

Robin Schreiber added the comment:

I have removed some redundant modulestate lookups and the testsuite now 
executes the decimal tests as fast as before the patch is applied. (at least 
for me).
May I ask how you tested the decimal performance?

Regarding the failing test:
It appears that the hackcheck() method in typeobject.c is responsible for this 
failure: 

static int
hackcheck(PyObject *self, setattrofunc func, char *what)
{
PyTypeObject *type = Py_TYPE(self);
while (type && type->tp_flags & Py_TPFLAGS_HEAPTYPE)
type = type->tp_base;
/* If type is NULL now, this is a really weird type.
   In the spirit of backwards compatibility (?), just shut up. */
if (type && type->tp_setattro != func) {
PyErr_Format(PyExc_TypeError,
 "can't apply this %s to %s object",
 what,
 type->tp_name);
return 0;
}
return 1;
}

As the context-type is now a heaptype the while-loop will continue to jump to 
the basetype of the conext-type, which is the object-type. There it ultimately 
fails when it compares func (context_setattr) to the tp_setattr of object.
Anyway, I do not understand the purpose of the hackcheck method, so I can not 
propose any kind of solution for this problem.

--
Added file: http://bugs.python.org/file26889/_decimal_pep3121-384_v1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15526] test_startfile crash on Windows 7 AMD64

[Antoine Pitrou]

Antoine Pitrou added the comment:

It crashed again, despite issue15496 being fixed:
http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/496

--
title: regrtest crash on Windows 7 AMD64 -> test_startfile crash on Windows 7 
AMD64

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15653] PEP 3121, 384 refactoring applied to hashopenssl module

[Martin v . Löwis]

Martin v. Löwis added the comment:

> By leaf, you mean the derived subtype? That sounds a bit quirky (you
> need the aforementioned "if"), how about the base heap type instead?

I think this fails (currently) because a subtype defined in Python using
a "class" statement will automatically get subtype_dealloc as its dealloc
function, which will in turn unconditionally decrefs the type (after
calling the tp_dealloc function from its nearest ancestor which doesn't
use subtype_dealloc).

> Speaking of which, what does this refactoring bring exactly? The type
> declarations are slightly less verbose, but other than that, is there a
> point?

It's really about the PEP 3121 changes: modules shouldn't have any
global variables shared across interpreters, so that module cleanup
can work properly. Ultimately, this can lead to gc-based shutdown,
module unloading, and better separation of interpreters.

In addition, it further reduces the differences between "extension
types" and "classes" (which supposedly were removed by dropping
old-style classes, yet some inconsistencies remain where the
interpreter offers some features only to heaptypes).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15653] PEP 3121, 384 refactoring applied to hashopenssl module

[Greg Stein]

Changes by Greg Stein :


--
nosy:  -gstein

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[R. David Murray]

R. David Murray added the comment:

Even if we write in chunks, if we are calling the OS write function and 
O_APPEND is set, wouldn't be satisfying the condition?  Or, rather, the OS 
would be.  That is, I don't really see a guarantee of an *atomic* write in the 
quoted description.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15723] Python breaks OS' append guarantee on file writes

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Even if we write in chunks, if we are calling the OS write function
> and O_APPEND is set, wouldn't be satisfying the condition?  Or,
> rather, the OS would be.  That is, I don't really see a guarantee of
> an *atomic* write in the quoted description.

I'm not sure it's guaranteed to be atomic at the hardware level, but as
AFAIU the updates should be atomic as seen from other processes on the
same machine (i.e. filesystem cache coherency).

As a side-note, I've just tested under Linux with the following script:

with open("foo", "ab") as f:
f.write(b"abcd")
f.write(b"x" * (1024 ** 2))

Results:

- on 2.7, the write buffers get sliced up (the glibc's fwrite() doesn't
care about atomicity):
write(3, "abcd"..., 4096) = 4096
write(3, ""..., 1044480) = 1044480

- on 3.2 and 3.3, our home-grown buffering respects the original
buffers:
write(3, "abcd", 4) = 4
write(3, ""..., 1048576) = 1048576

(but that's purely out of luck, since we didn't design it with that
goal :-))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1574] Touchpad 2 Finger scroll does not work in IDLE on Mac (But scroll wheel of external mouse does)

[Ned Deily]

Ned Deily added the comment:

On current OS X systems (tested on OS X 10.7.4), the 2-finger scrolling gesture 
scrolls IDLE text editor windows with long files, tested with current versions 
of ActiveTcl 8.5 and ActiveTcl 8.4.  So I think the issue can be closed.

--
nosy: +ned.deily
resolution:  -> works for me
stage: needs patch -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15285] test_timeout failure when system on IPv4 10.x.x.x subnet

[Trent Nelson]

Trent Nelson added the comment:

Proposed patch attached.

--
keywords: +patch
nosy: +trent
Added file: http://bugs.python.org/file26890/test_timeout.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15526] test_startfile crash on Windows 7 AMD64

[Jeremy Kloth]

Jeremy Kloth added the comment:

Unfortunately, this is a legitimate failure of the test.  The test
(actually the support code) is attempting to remove a directory that
is the current directory of an active process.  The test has
documented this issue and attempted to work around it by adding a
sleep().

To fix this test there a few options:
1) increase the sleep timeout
2) change the current working directory before calling os.startfile()
(possible that of the Python interpreter itself)
3) code a "wait for child process" function using ctypes and the
Toolhelp API (Process32First/Next)
4) change os.startfile() to use ShellExecuteEx and use the hProcess
handle as the return value and use that with os.waitpid()

#4 is the most accurate, but does introduce an API change (the
introduction of a return value to os.startfile)
#3 is more work but may be helpful for other tests
#2 will definitely work in this case but does violate the testing sandbox
#1 is the least disruptive, but is really just avoiding the root cause

I would like to see #4 but realize that it may be too late for 3.3.
#3 is not bad either as it has impact only on the test suite.  But any
of the above should fix this particular case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15719] Sort dict items in urlencode()

[Gregory P. Smith]

Gregory P. Smith added the comment:

-1 on doing this from me.

While I don't see it hurting anything when "type(query) is dict" I'd much 
rather encourage people to write better tests that do not take the lazy way 
out.  Tests that get by comparing a generated string to a "golden" string are 
soo often wrong because the golden string is not what they actually wanted to 
test. Instead the author meant to test the meaning of the result rather than 
the specific incantation of it that they pasted in their code for an 
assertEqual check.

(This does suggest that some additional unittest methods to make comparing 
common things like this in a safe manner would be useful to people. urls, html 
and xml with attributes, json, etc)

Ex: We've run into many tests that were failing due to hash randomization 
because they compared json strings... rather than suggest that json.dumps sort 
dictionaries before generating output, having the tests use json.loads and 
compare the actual data structure rather than a golden value string is much 
better.

If a change like this is made, it needs to be conditional in 2.7 and 3.2 to 
only sort when hash randomization is enabled to avoid altering the existing 
behavior of urlencode for the default PYTHONHASHSEED=0 case on 2.7 and 3.2.

As far as this being an enhancement rather than a bugfix... I sort of agree 
that this is an enhancement.  And if this is an enhancement, we don't need to 
do it at all: People will already have done the right thing making their code 
work with 3.3's hash randomization by default before long before 3.4 is out.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15719] Sort dict items in urlencode()

[Guido van Rossum]

Guido van Rossum added the comment:

I've come to see more downsides than upsides to this idea.  I'm withdrawing it. 
 Good points, Greg!  (And others.)

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15285] test_timeout failure when system on IPv4 10.x.x.x subnet

[Ned Deily]

Ned Deily added the comment:

The patch looks good to me.  Thanks for setting up the infrastructure to allow 
reproducible testing!

--
stage: needs patch -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8240] ssl.SSLSocket.write may fail on non-blocking sockets

[Ben Darnell]

Ben Darnell added the comment:

Related pypy issue: https://bugs.pypy.org/issue1238

--
nosy: +Ben.Darnell

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com