Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

[Ronald Oussoren]

On 14 Dec, 2012, at 8:27, "Gregory P. Smith"  wrote:

> 
> On Mon, Dec 10, 2012 at 11:16 PM, Antoine Pitrou  wrote:
> On Tue, 11 Dec 2012 03:05:19 +0100 (CET)
> gregory.p.smith  wrote:
> >   Using 'long double' to force this structure to be worst case aligned is no
> > longer required as of Python 2.5+ when the gc_refs changed from an int (4
> > bytes) to a Py_ssize_t (8 bytes) as the minimum size is 16 bytes.
> >
> > The use of a 'long double' triggered a warning by Clang trunk's
> > Undefined-Behavior Sanitizer as on many platforms a long double requires
> > 16-byte alignment but the Python memory allocator only guarantees 8 byte
> > alignment.
> >
> > So our code would allocate and use these structures with technically 
> > improper
> > alignment.  Though it didn't matter since the 'dummy' field is never used.
> > This silences that warning.
> >
> > Spelunking into code history, the double was added in 2001 to force better
> > alignment on some platforms and changed to a long double in 2002 to appease
> > Tru64.  That issue should no loner be present since the upgrade from int to
> > Py_ssize_t where the minimum structure size increased to 16 (unless anyone
> > knows of a platform where ssize_t is 4 bytes?)
> 
> What?? Every 32-bit platform has a 4 bytes ssize_t (and size_t).
> 
> No they don't.
> 
> size_t and ssize_t exist in large part because they are often larger than an 
> int or long on 32bit platforms.  They are 64-bit on Linux regardless of 
> platform (i think there is a way to force a compile in ancient mode that 
> forces them and the APIs being used to be 32-bit size_t variants but nobody 
> does that).

Are you sure about this, what you describe seem to be loff_t (the typedef for 
file offsets), not size_t (the typedef for sizes of memory blocks). The size of 
memory blocks is limited to a 32-bit number on 32-bit systems (for the obvious 
reason).

Size_t is 32-bit on at least some platforms:

$ cat t.c
#include 
#include 

int main(void)
{
printf("sizeof(size_t): %d\n", (int)sizeof(size_t));
return 0;
}

$ cc -o t t.c -arch i386
$ ./t
sizeof(size_t): 4


This session is on an OSX system, but you'll get the same output on a 32-bit 
linux system with default compiler settings (I've tested this on a SLES10 
system).
> 
> 
> > We can probably get rid of the double and this union hack all together 
> > today.
> > That is a slightly more invasive change that can be left for later.
> 
> How do you suggest to get rid of it? Some platforms still have strict
> alignment rules and we must enforce that PyObjects (*) are always
> aligned to the largest possible alignment, since a PyObject-derived
> struct can hold arbitrary C types.
> 
> (*) GC-enabled PyObjects, anyway. Others will be naturally aligned
> thanks to the memory allocator.
> 
> 
> What's more, I think you shouldn't be doing this kind of change in a
> bugfix release. It might break compiled C extensions since you are
> changing some characteristics of object layout (although you would
> probably only break those extensions which access the GC header, which
> is probably not many of them). Resource consumption improvements
> generally go only into the next feature release.
> 
> This isn't a resource consumption improvement.  It is a compilation 
> correctness change with zero impact on the generated code or ABI 
> compatibility before and after.  The structure, as defined, is was flagged as 
> problematic by Clang's undefined behavior sanitizer because it contains a 
> 'long double' which requires 16-byte alignment but Python's own memory 
> allocator was using an 8 byte boundary.
> 
> So changing the definition of the dummy side of the union makes zero 
> difference to already compiled code as it (a) doesn't change the structure's 
> size and (b) all existing implementations already align these on an 8 byte 
> boundary.
> 
> -gps
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP for time zone support.

[Lennart Regebro]

OK, so it's been 12 hours with no further discussion, so I'll make an
attempt to summarize what I think is the consensus changes before
updating the PEP.

1. Python will include a timezone database both in the source
distribution and the Windows installer (although I suspect that binary
packages for Linux distributions may skip this, but that's OK).

2. The timezone module becomes datetime.timezone, meaning datetime.py
is moved to datetime/__init__.py

3. get_timezone() will be just timezone() as no voices was raised to
defend get_timezone()

4. The db parameter in timezone() will be renamed db_path

5. is_dst will default to False

6. The UnknownTimeZoneError exception will be just a ValueError

7. The two errors raised when converting timezones will both inherit
from a base exception.

8. A better name for the timezone data package. "tzdata-override" was
suggested, I prefer "tzdata-update" as it is clearer.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

[Gregory P. Smith]

On Fri, Dec 14, 2012 at 12:02 AM, Ronald Oussoren wrote:

>
> On 14 Dec, 2012, at 8:27, "Gregory P. Smith"  wrote:
>
>
> On Mon, Dec 10, 2012 at 11:16 PM, Antoine Pitrou wrote:
>
>> On Tue, 11 Dec 2012 03:05:19 +0100 (CET)
>> gregory.p.smith  wrote:
>> >   Using 'long double' to force this structure to be worst case aligned
>> is no
>> > longer required as of Python 2.5+ when the gc_refs changed from an int
>> (4
>> > bytes) to a Py_ssize_t (8 bytes) as the minimum size is 16 bytes.
>> >
>> > The use of a 'long double' triggered a warning by Clang trunk's
>> > Undefined-Behavior Sanitizer as on many platforms a long double requires
>> > 16-byte alignment but the Python memory allocator only guarantees 8 byte
>> > alignment.
>> >
>> > So our code would allocate and use these structures with technically
>> improper
>> > alignment.  Though it didn't matter since the 'dummy' field is never
>> used.
>> > This silences that warning.
>> >
>> > Spelunking into code history, the double was added in 2001 to force
>> better
>> > alignment on some platforms and changed to a long double in 2002 to
>> appease
>> > Tru64.  That issue should no loner be present since the upgrade from
>> int to
>> > Py_ssize_t where the minimum structure size increased to 16 (unless
>> anyone
>> > knows of a platform where ssize_t is 4 bytes?)
>>
>> What?? Every 32-bit platform has a 4 bytes ssize_t (and size_t).
>>
>
> No they don't.
>
>
> size_t and ssize_t exist in large part because they are often larger than
> an int or long on 32bit platforms.  They are 64-bit on Linux regardless of
> platform (i think there is a way to force a compile in ancient mode that
> forces them and the APIs being used to be 32-bit size_t variants but nobody
> does that).
>
>
> Are you sure about this, what you describe seem to be loff_t (the typedef
> for file offsets), not size_t (the typedef for sizes of memory blocks). The
> size of memory blocks is limited to a 32-bit number on 32-bit systems (for
> the obvious reason).
>
> Size_t is 32-bit on at least some platforms:
>
> $ cat t.c
> #include 
> #include 
>
> int main(void)
> {
> printf("sizeof(size_t): %d\n", (int)sizeof(size_t));
> return 0;
> }
>
> $ cc -o t t.c -arch i386
> $ ./t
> sizeof(size_t): 4
>
>
> This session is on an OSX system, but you'll get the same output on a
> 32-bit linux system with default compiler settings (I've tested this on a
> SLES10 system).
>

You are correct.  My bad.  size_t vs off_t vs my full brain strikes again.

Regardless it doesn't change the correctness of my change.  Though I'd love
it if someone would figure out the cross platform compiler macro based
struct alignment incantations to get rid of the need for the union with
dummy all together.

It wasn't even clear from the 2002 change description if changing double to
long double over 10 years ago was actually _fixing_ a bug or was done for
no good reason?  Our allocator (obmalloc.c) doesn't allocate memory at many
platform's required 16 byte long double alignment, it uses 8 byte alignment
so the change couldn't have had any impact unless someone compiled
--without-pymalloc using a system allocator that guaranteed a larger value.
 I wouldn't expect that and have seen no indication of that anywhere.

The history of the long double and double additions:

 http://hg.python.org/cpython/rev/7065135f9202   long double
 - http://hg.python.org/cpython/rev/b4f829941f3d   double
  - http://bugs.python.org/issue467145   why double was added, it fixed the
HPUX 11 build.

-gps

>
>
>> > We can probably get rid of the double and this union hack all together
>> today.
>> > That is a slightly more invasive change that can be left for later.
>>
>> How do you suggest to get rid of it? Some platforms still have strict
>> alignment rules and we must enforce that PyObjects (*) are always
>> aligned to the largest possible alignment, since a PyObject-derived
>> struct can hold arbitrary C types.
>>
>> (*) GC-enabled PyObjects, anyway. Others will be naturally aligned
>> thanks to the memory allocator.
>>
>>
>> What's more, I think you shouldn't be doing this kind of change in a
>> bugfix release. It might break compiled C extensions since you are
>> changing some characteristics of object layout (although you would
>> probably only break those extensions which access the GC header, which
>> is probably not many of them). Resource consumption improvements
>> generally go only into the next feature release.
>>
>
> This isn't a resource consumption improvement.  It is a compilation
> correctness change with zero impact on the generated code or ABI
> compatibility before and after.  The structure, as defined, is was flagged
> as problematic by Clang's undefined behavior sanitizer because it contains
> a 'long double' which requires 16-byte alignment but Python's own memory
> allocator was using an 8 byte boundary.
>
> So changing the definition of the dummy side of the union makes zero
> difference to already compiled code as it (a) doesn't

Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

[Mark Dickinson]

On Fri, Dec 14, 2012 at 7:27 AM, Gregory P. Smith  wrote:
> So changing the definition of the dummy side of the union makes zero
> difference to already compiled code as it (a) doesn't change the structure's
> size and (b) all existing implementations already align these on an 8 byte
> boundary.

It looks to me as though the struct size *is* changed, at least on
some platforms.  Before this commit, I get (OS X 10.6, 64-bit
non-debug build):

Python 3.4.0a0 (default:b4c383f31881+, Dec 14 2012, 08:30:39)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class A(object): pass
...
>>> a = A()
>>> import sys
>>> sys.getsizeof(a)
64


After it:

Python 3.4.0a0 (default:76bc92fb90c1+, Dec 14 2012, 08:33:48)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class A(object): pass
...
>>> a = A()
>>> import sys
>>> sys.getsizeof(a)
56

-- 
Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?

[Lennart Regebro]

On Fri, Dec 14, 2012 at 9:49 AM, Antonio Cavallo
 wrote:
> My requirements would quite simple:
>  2. cross compiling

That is *not* a simple requirement.

//Lennart
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

[Gregory P. Smith]

Yes, see the followup.  My comments before were all misinterpreting size_t.

Same result on x86_64 linux. On a 64-bit platform the 24 byte structure now
occupies 24 bytes instead of being padded to 32.  Nice.  On a 32-bit
platform it should remain 16 bytes.

The PyGC_Head union structure is NOT part of the ABI laid out in
http://www.python.org/dev/peps/pep-0384/ and is accurately excluded from
the .h file when Py_LIMITED_API is defined so changing this in 3.4 should
not be a problem.  This structure occupies the space gc tracked PyObject*
pointers.

-gps


On Fri, Dec 14, 2012 at 12:42 AM, Mark Dickinson  wrote:

> On Fri, Dec 14, 2012 at 7:27 AM, Gregory P. Smith  wrote:
> > So changing the definition of the dummy side of the union makes zero
> > difference to already compiled code as it (a) doesn't change the
> structure's
> > size and (b) all existing implementations already align these on an 8
> byte
> > boundary.
>
> It looks to me as though the struct size *is* changed, at least on
> some platforms.  Before this commit, I get (OS X 10.6, 64-bit
> non-debug build):
>
> Python 3.4.0a0 (default:b4c383f31881+, Dec 14 2012, 08:30:39)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> class A(object): pass
> ...
> >>> a = A()
> >>> import sys
> >>> sys.getsizeof(a)
> 64
>
>
> After it:
>
> Python 3.4.0a0 (default:76bc92fb90c1+, Dec 14 2012, 08:33:48)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> class A(object): pass
> ...
> >>> a = A()
> >>> import sys
> >>> sys.getsizeof(a)
> 56
>
> --
> Mark
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial workflow question...

[Antoine Pitrou]

Le Thu, 13 Dec 2012 21:48:23 -0500,
"R. David Murray"  a écrit :
> On Thu, 13 Dec 2012 20:21:24 -0500, Trent Nelson
>  wrote:
> > - Use a completely separate clone to house all the
> > intermediate commits, then generate a diff once the final commit is
> > ready, then apply that diff to the main cpython repo, then push
> > that. This approach is fine, but it seems counter-intuitive to the
> >   whole concept of DVCS.
> 
> Perhaps.  But that's exactly what I did with the email package changes
> for 3.3.
> 
> You seem to have a tension between "all those dirty little commits"
> and "clean history" and the fact that a dvcs is designed to preserve
> all those commits...if you don't want those intermediate commits in
> the official repo, then why is a diff/patch a bad way to achieve
> that?  If you keep your pulls up to date in your feature repo, the
> diff/patch process is simple and smooth.

+1.  We definitely don't want tons of small incremental commits in the
official repo.  "One changeset == one issue" should be the ideal horizon
when committing changes.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Using 'long double' to force this structure to be worst case aligned is no

[Antoine Pitrou]

Le Fri, 14 Dec 2012 01:14:04 -0800,
"Gregory P. Smith"  a écrit :
> Yes, see the followup.  My comments before were all misinterpreting
> size_t.
> 
> Same result on x86_64 linux. On a 64-bit platform the 24 byte
> structure now occupies 24 bytes instead of being padded to 32.
> Nice.  On a 32-bit platform it should remain 16 bytes.

But you are losing the 16-byte alignment that the union was precisely
designed to enforce.

> The PyGC_Head union structure is NOT part of the ABI laid out in
> http://www.python.org/dev/peps/pep-0384/ and is accurately excluded
> from the .h file when Py_LIMITED_API is defined so changing this in
> 3.4 should not be a problem.

Not an ABI problem in 3.4 indeed (except that it might break platforms
with strict alignment requirements).

It should be noted that the GC head isn't part of standard atomic types
(int, float, str...), so the memory gain will not be very noticeable
IMO.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP for time zone support.

[Christian Heimes]

Am 14.12.2012 09:31, schrieb Lennart Regebro:
> 1. Python will include a timezone database both in the source
> distribution and the Windows installer (although I suspect that binary
> packages for Linux distributions may skip this, but that's OK).

You need to specify the details. Where is the database stored and under
which condition is it updated?

Suggestions:

* The zoneinfo database is stored in the new package 'tzdata',
  that's Lib/tzdata in the source dist. The files are kept in
  our hg repository, too.

* A tool chain is provided that compiles the zoneinfos from a Olson
  tar.gz file. (Bonus points for a download + update script). The
  tool chain is included in source dist, e.g. Tools/.

* The db is updated on a regular basis during the development, alpha
  and beta phase by any core dev. Every patch level release shall
  contain the latest version of the db, maybe except for security
  releases.

* It's the release managers responsibility to make sure, all final
  releases contain the current db. This needs to be added to the
  RM's TODO list.


Who is going to create the tzdata_update package, how is it compiled and
how often should it be released?

One other thing, the zoneinfo database should be compatible with zipfile
distributions. The module should be able to load the files from a stdlib
zipfile. The feature is important for freeze, py2exe and py2app.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Draft PEP for time zone support.

[Barry Warsaw]

On Dec 14, 2012, at 12:01 PM, Christian Heimes wrote:

>* It's the release managers responsibility to make sure, all final
>  releases contain the current db. This needs to be added to the
>  RM's TODO list.

That would be PEP 101.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?

[Antonio Cavallo]

Mmm, so the question would be distutils2 or distlib? I think tarek made 
a graph of the different packages systems... seen on reddit some time ago.



My requirements would quite simple:
 1. support DESTDIR approach where a package can be installed in an
intermediate directory before its final destination)
 2. cross compiling
 3. install even if a dependecy package is not installed so decoupling
installation from "configuration"

point 1 is needed for system integrators (eg. people working in rpm builds).

point 2 is entirely mine :)

point 3 is the same philosophical difference between build, install and 
configuration steps: its part of good practices. In short it shouldn't 
replace the system wide dependency manager (in rpm it would be yum/zypp 
and in debian is much more confused, in window.. it doesn't exists as 
well in macos having the approach to pack it up everything in one place).


Funny enough distutils (the old dead horse) does it all except point 2: 
that is my reason to clean up the code. I've just seen py3k distutils 
but it would be worth a back port to py2k.




Thanks








Nick Coghlan wrote:

On Fri, Dec 14, 2012 at 10:10 AM, Antonio Cavallo
mailto:a.cava...@cavallinux.eu>> wrote:

I'll have a look into distutils2, I tough it was (another) dead end.
I every case my target is py2k (2.7.x) and I've no case for
transitioning to py3k (too much risk).


distutils2 started as a copy of distutils, so it's hard to tell the
difference between the parts which have been fixed and the parts which
are still just distutils legacy components (this is why the merge back
was dropped from 3.3 - too many pieces simply weren't ready and simply
would have perpetuated problems inherited from distutils).

distlib (https://distlib.readthedocs.org/en/latest/overview.html) is a
successor project that takes a different view of building up the low
level pieces without inheriting the bad parts of the distutils legacy (a
problem suffered by both setuptools/distribute and distutils2). distlib
also runs natively on both 2.x and 3.x, as the idea is that these
interoperability standards should be well supported in *current* Python
versions, not just those where the stdlib has caught up (i.e. now 3.4 at
the earliest)

The aim is to get to a situation more like that with wsgiref, where the
stdlib defines the foundation and key APIs and data formats needed for
interoperability, while allowing a flourishing ecosystem of
user-oriented tools (like pip, bento, zc.buildout, etc) that still solve
the key problems addressed by setuptools/distribute without the opaque
and hard to extend distutils core that can make the existing tools
impossible to debug when they go wrong.

Cheers,
Nick.

--
Nick Coghlan   | ncogh...@gmail.com    |
Brisbane, Australia

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2012-12-07 - 2012-12-14)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open3826 (+10)
  closed 24631 (+34)
  total  28457 (+44)

Open issues with patches: 1683 


Issues opened (37)
==

#16421: importlib.machinery.ExtensionFileLoader cannot load several mo
http://bugs.python.org/issue16421  reopened by haypo

#16637: py-bt, py-locals, etc. GDB commands fail with output-radix 16
http://bugs.python.org/issue16637  opened by mshroyer

#16638: support multi-line docstring signatures in IDLE calltips
http://bugs.python.org/issue16638  opened by chris.jerdonek

#16640: Less code under lock in sched.scheduler
http://bugs.python.org/issue16640  opened by serhiy.storchaka

#16641: sched.scheduler.enter arguments should not be modifiable
http://bugs.python.org/issue16641  opened by serhiy.storchaka

#16642: Mention new "kwargs" named tuple parameter in sched module
http://bugs.python.org/issue16642  opened by serhiy.storchaka

#16643: Wrong documented default value for timefunc parameter in sched
http://bugs.python.org/issue16643  opened by serhiy.storchaka

#16644: Wrong code in ContextManagerTests.test_invalid_args() in test_
http://bugs.python.org/issue16644  opened by serhiy.storchaka

#16645: Wrong test_extract_hardlink() in test_tarfile.py
http://bugs.python.org/issue16645  opened by serhiy.storchaka

#16646: FTP.makeport() loses socket error details
http://bugs.python.org/issue16646  opened by serhiy.storchaka

#16647: LMTP.connect() loses socket error details
http://bugs.python.org/issue16647  opened by serhiy.storchaka

#16648: stdib should use new exception types from PEP 3151
http://bugs.python.org/issue16648  opened by asvetlov

#16649: Add a PyCF_DISPLAY_EXPRESSION_RESULTS flag
http://bugs.python.org/issue16649  opened by ncoghlan

#16650: Popen._internal_poll() references errno.ECHILD outside of the 
http://bugs.python.org/issue16650  opened by serhiy.storchaka

#16651: Find out what stdlib modules lack a pure Python implementation
http://bugs.python.org/issue16651  opened by brett.cannon

#16652: socket.getfqdn docs are not explicit enough about the algorith
http://bugs.python.org/issue16652  opened by r.david.murray

#16653: reference kept in f_locals prevents the tracing/profiling of a
http://bugs.python.org/issue16653  opened by xdegaye

#16655: IDLE list.append calltips test failures
http://bugs.python.org/issue16655  opened by chris.jerdonek

#16656: os.walk ignores international dirs on Windows
http://bugs.python.org/issue16656  opened by techtonik

#16657: traceback.format_tb incorrect docsting
http://bugs.python.org/issue16657  opened by mgedmin

#16658: Missing "return" in HTTPConnection.send()
http://bugs.python.org/issue16658  opened by amaury.forgeotdarc

#16659: Pure Python implementation of random
http://bugs.python.org/issue16659  opened by serhiy.storchaka

#16661: test_posix.test_getgrouplist fails on some systems - incorrect
http://bugs.python.org/issue16661  opened by gregory.p.smith

#16662: load_tests not invoked in package/__init__.py
http://bugs.python.org/issue16662  opened by rbcollins

#16663: Poor documentation for METH_KEYWORDS
http://bugs.python.org/issue16663  opened by r3m0t

#16664: [PATCH] Test Glob: files starting with .
http://bugs.python.org/issue16664  opened by Sebastian.Kreft

#16665: doc for builtin hex() is poor
http://bugs.python.org/issue16665  opened by rurpy2

#1: docs wrongly imply socket.getaddrinfo takes keyword arguments 
http://bugs.python.org/issue1  opened by Mikel.Ward

#16667: timezone docs need "versionadded: 3.2"
http://bugs.python.org/issue16667  opened by ncoghlan

#16669: Docstrings for namedtuple
http://bugs.python.org/issue16669  opened by serhiy.storchaka

#16670: Point class may be not be a good example for namedtuple
http://bugs.python.org/issue16670  opened by julien.tayon

#16672: improve tracing performances when f_trace is NULL
http://bugs.python.org/issue16672  opened by xdegaye

#16674: Faster getrandbits() for small integers
http://bugs.python.org/issue16674  opened by serhiy.storchaka

#16676: Segfault under Python 3.3 after PyType_GenericNew
http://bugs.python.org/issue16676  opened by tseaver

#16677: Hard to find operator precedence in Lang Ref.
http://bugs.python.org/issue16677  opened by rurpy2

#16678: optparse: parse only known options
http://bugs.python.org/issue16678  opened by techtonik

#16679: Wrong URL path decoding
http://bugs.python.org/issue16679  opened by claudep



Most recent 15 issues with no replies (15)
==

#16677: Hard to find operator precedence in Lang Ref.
http://bugs.python.org/issue16677

#16676: Segfault under Python 3.3 after PyType_GenericNew
http://bugs.python.org/issue16676

#16674: Faster getrandbits() for small integers
http://bugs.python.org/issue16674

#16663: Poor documentation

[Python-Dev] http.client Nagle/delayed-ack optimization

[Ben Leslie]

The http.client HTTPConnection._send_output method has an optimization for
avoiding bad interactions between delayed-ack and the Nagle algorithm:

http://hg.python.org/cpython/file/f32f67d26035/Lib/http/client.py#l884

Unfortunately this interacts rather poorly if the case where the
message_body is a bytes instance and is rather large.

If the message_body is bytes it is appended to the headers, which causes a
copy of the data. When message_body is large this duplication of data can
cause a significant spike in memory usage.

(In my particular case I was uploading a 200MB file to 30 hosts at the same
leading to memory spikes over 6GB.

I've solved this by subclassing and removing the optimization, however I'd
appreciate thoughts on how this could best be solved in the library itself.

Options I have thought of are:

1: Have some size threshold on the copy. A little bit too much magic.
Unclear what the size threshold should be.

2: Provide an explicit argument to turn the optimization on/off. This is
ugly as it would need to be attached up the call chain to the request
method.

3: Provide a property on the HTTPConnection object which enables the
optimization or not. Optionally configured as part of __init__.

4: Add a class level attribute (similar to auto_open, default_port, etc)
which controls the optimization.

Be very interested to get some feedback so I can craft the appropriate
patch.

Thanks,

Benno
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] http.client Nagle/delayed-ack optimization

[Antoine Pitrou]

On Sat, 15 Dec 2012 06:17:19 +1100
Ben Leslie  wrote:
> The http.client HTTPConnection._send_output method has an optimization for
> avoiding bad interactions between delayed-ack and the Nagle algorithm:
> 
> http://hg.python.org/cpython/file/f32f67d26035/Lib/http/client.py#l884
> 
> Unfortunately this interacts rather poorly if the case where the
> message_body is a bytes instance and is rather large.
> 
> If the message_body is bytes it is appended to the headers, which causes a
> copy of the data. When message_body is large this duplication of data can
> cause a significant spike in memory usage.
> 
> (In my particular case I was uploading a 200MB file to 30 hosts at the same
> leading to memory spikes over 6GB.
> 
> I've solved this by subclassing and removing the optimization, however I'd
> appreciate thoughts on how this could best be solved in the library itself.
> 
> Options I have thought of are:
> 
> 1: Have some size threshold on the copy. A little bit too much magic.
> Unclear what the size threshold should be.

I think a hardcoded threshold is the right thing to do. It doesn't
sound very useful to try doing a single send() call when you have a
large chunk of data (say, more than 1 MB).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Distutils] Is is worth disentangling distutils?

[Antonio Cavallo]

It is not that complex... What's ahead is even more complex.




Lennart Regebro wrote:

On Fri, Dec 14, 2012 at 9:49 AM, Antonio Cavallo
  wrote:

My requirements would quite simple:
  2. cross compiling


That is *not* a simple requirement.

//Lennart

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Downloads page: Which version of Python should be listed first?

[Terry Reedy]

On 12/13/2012 4:14 PM, Ross Lagerwall wrote:

On Fri, Dec 14, 2012 at 07:57:52AM +1100, Chris Angelico wrote:

The default version shown on http://docs.python.org/ is now 3.3.0,
which I think is a Good Thing. However, http://python.org/download/
puts 2.7 first, and says:

"""If you don't know which version to use, start with Python 2.7; more
existing third party software is compatible with Python 2 than Python
3 right now."""

Firstly, is this still true? (I wouldn't have a clue.) And secondly,
would this be better worded as "one's better but the other's a good
fall-back"? Something like:

"""Don't know which version to use? Python 3.3 is the recommended
version for new projects, but much existing software is compatible
with Python 2."""



I would say listing 3.3 as the recommended version to use is a good
thing, especially as distros like Ubuntu and Fedora transition to Python
3. It also makes sense, given that the docs default to 3.3.


From the LibreOffice 4.0beta1 release notes
http://wiki.documentfoundation.org/ReleaseNotes/4.0

Python

The bundled Python was upgraded from Python 2.6 to Python 3.3 (Michael 
Stahl)


Python extensions and macros may require some degree of re-work to work 
on the latest Python; see for example Porting to Python 3.


The last is a link to Lennart Regebro's book: http://python3porting.com/

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Downloads page: Which version of Python should be listed first?

[Devin Jeanpierre]

On Thu, Dec 13, 2012 at 9:38 PM, Eric Snow  wrote:
>> """If you don't know which version to use, start with Python 2.7; more
>> existing third party software is compatible with Python 2 than Python
>> 3 right now."""
>>
>> Firstly, is this still true? (I wouldn't have a clue.)
>
> Nope:
>
> http://py3ksupport.appspot.com/
> http://python3wos.appspot.com/ (plone and zope skew the results)

Until those numbers hit 100%, or until projects start dropping support
for Python 2.x, the statement would still be true.

-- Devin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Downloads page: Which version of Python should be listed first?

[Chris Angelico]

On Sat, Dec 15, 2012 at 12:31 PM, Devin Jeanpierre
 wrote:
> On Thu, Dec 13, 2012 at 9:38 PM, Eric Snow  
> wrote:
>>> """If you don't know which version to use, start with Python 2.7; more
>>> existing third party software is compatible with Python 2 than Python
>>> 3 right now."""
>>>
>>> Firstly, is this still true? (I wouldn't have a clue.)
>>
>> Nope:
>>
>> http://py3ksupport.appspot.com/
>> http://python3wos.appspot.com/ (plone and zope skew the results)
>
> Until those numbers hit 100%, or until projects start dropping support
> for Python 2.x, the statement would still be true.

Not necessarily dropping; all you need is for new projects to not
bother supporting 2.x and the statement can become false.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Downloads page: Which version of Python should be listed first?

[Stephen J. Turnbull]

Devin Jeanpierre writes:

 > Until those numbers hit 100%, or until projects start dropping support
 > for Python 2.x, the statement would still be true.

This is simply not true.  Once the numbers hit somewhere in the
neighborhood of 50%, the network effects (the need to "connect" to the
more progressive projects) are going to rule, and "most projects" will
face strong pressure to adapt (which means providing Python 3 support,
*not* "dropping Python 2").  On the other hand, some projects will
never bother because they're completely standalone: you won't ever
reach 100%, and even 90% may not be necessary for the ecology to be
considered "fully adapted to Python 3".

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com