Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.

[Maciej Fijalkowski]

Hi

There was a PSF-sponsored effort to improve the situation with the
https://bitbucket.org/pypy/codespeed2/src being written (thank you
PSF). It's not better enough than codespeed that I would like, but
gives some opportunities.

That said, we have a benchmark machine for benchmarking cpython and I
never deployed nightly benchmarks of cpython for a variety of reasons.

* would be cool to get a small VM to set up the web front

* people told me that py3k is only interesting, so I did not set it up
for py3k because benchmarks are mostly missing

I'm willing to set up a nightly speed.python.org using nightly build
on python 2 and possible python 3 if there is an interest. I need
support from someone maintaining python buildbot to setup builds and a
VM to set up stuff, otherwise I'm good to go

DISCLAIMER: I did facilitate in codespeed rewrite that was not as
successful as I would have hoped. I did not receive any money from the
PSF on that though.

Cheers,
fijal


On Mon, Jun 1, 2015 at 1:14 PM, M.-A. Lemburg  wrote:
> On 01.06.2015 12:44, Armin Rigo wrote:
>> Hi Larry,
>>
>> On 31 May 2015 at 01:20, Larry Hastings  wrote:
>>> p.s. Supporting this patch also helps cut into PyPy's reported performance
>>> lead--that is, if they ever upgrade speed.pypy.org from comparing against
>>> Python *2.7.2*.
>>
>> Right, we should do this upgrade when 2.7.11 is out.
>>
>> There is some irony in your comment which seems to imply "PyPy is
>> cheating by comparing with an old Python 2.7.2": it is inside a thread
>> which started because "we didn't backport performance improvements to
>> 2.7.x so far".
>>
>> Just to convince myself, I just ran a performance comparison.  I ran
>> the same benchmark suite as speed.pypy.org, with 2.7.2 against 2.7.10,
>> both freshly compiled with no "configure" options at all.  The
>> differences are usually in the noise, but range from +5% to... -60%.
>> If anything, this seems to show that CPython should take more care
>> about performance regressions.  If someone is interested:
>>
>> * "raytrace-simple" is 1.19 times slower
>> * "bm_mako" is 1.29 times slower
>> * "spitfire_cstringio" is 1.60 times slower
>> * a number of other benchmarks are around 1.08.
>>
>> The "7.0x faster" number on speed.pypy.org would be significantly
>> *higher* if we upgraded the baseline to 2.7.10 now.
>
> If someone were to volunteer to set up and run speed.python.org,
> I think we could add some additional focus on performance
> regressions. Right now, we don't have any way of reliably
> and reproducibly testing Python performance.
>
> Hint: The PSF would most likely fund such adventures :-)
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jun 01 2015)
 Python Projects, Coaching and Consulting ...  http://www.egenix.com/
 mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
> 
>
> : Try our mxODBC.Connect Python Database Interface for free ! ::
>
>eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>Registered at Amtsgericht Duesseldorf: HRB 46611
>http://www.egenix.com/company/contact/
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Avoiding reference leaks in heap types with custom tp_dealloc

[Petr Viktorin]

On Mon, Jun 1, 2015 at 6:00 PM, Antoine Pitrou  wrote:
[...]
> I think we have been laxist with additions to the stable ABI:
> apparently, they should be conditioned on the API version requested by
> the user.  For example, in pystate.h:
>
> #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x0303
> /* New in 3.3 */
> PyAPI_FUNC(int) PyState_AddModule(PyObject*, struct PyModuleDef*);
> PyAPI_FUNC(int) PyState_RemoveModule(struct PyModuleDef*);
> #endif
>
> (those were added by Martin, so I assume he knew what he was doing :-))
>
> This way, failing to restrict yourself to a given API version fails at
> compile time, not at runtime. However, it's also more work to do so
> when adding stuff, which is why we tend to skimp on it.

I see! I completely missed that memo.
I filed a patch that wraps my 3.5 additions as issue 24365.

I think this should be in the PEP, so people like me can find it. Does
the attached wording look good?
diff --git a/pep-0384.txt b/pep-0384.txt
index f042478..311bae4 100644
--- a/pep-0384.txt
+++ b/pep-0384.txt
@@ -360,6 +360,19 @@ whether their modules conform to the ABI. To avoid users having to
 rewrite their type definitions, a script to convert C source code
 containing type definitions will be provided [3]_.
 
+Considerations for CPython developers
+=
+
+When making additions to the stable ABI, the new definitions should be wrapped
+in a preprocessor conditional block that limits their visibility to
+a minimum stable ABI version. For example::
+
+#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x0303
+/* New in 3.3 */
+PyAPI_FUNC(int) PyState_AddModule(PyObject*, struct PyModuleDef*);
+PyAPI_FUNC(int) PyState_RemoveModule(struct PyModuleDef*);
+#endif
+
 References
 ==
 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.

[Brett Cannon]

On Tue, Jun 2, 2015 at 3:08 PM Maciej Fijalkowski  wrote:

> Hi
>
> There was a PSF-sponsored effort to improve the situation with the
> https://bitbucket.org/pypy/codespeed2/src being written (thank you
> PSF). It's not better enough than codespeed that I would like, but
> gives some opportunities.
>
> That said, we have a benchmark machine for benchmarking cpython and I
> never deployed nightly benchmarks of cpython for a variety of reasons.
>
> * would be cool to get a small VM to set up the web front
>
> * people told me that py3k is only interesting, so I did not set it up
> for py3k because benchmarks are mostly missing
>

Missing how? hg.python.org/benchmarks has plenty of Python 3-compatible
benchmarks to have interesting results.


>
> I'm willing to set up a nightly speed.python.org using nightly build
> on python 2 and possible python 3 if there is an interest. I need
> support from someone maintaining python buildbot to setup builds and a
> VM to set up stuff, otherwise I'm good to go
>

I suspect there is interest.

-Brett


>
> DISCLAIMER: I did facilitate in codespeed rewrite that was not as
> successful as I would have hoped. I did not receive any money from the
> PSF on that though.
>
> Cheers,
> fijal
>
>
> On Mon, Jun 1, 2015 at 1:14 PM, M.-A. Lemburg  wrote:
> > On 01.06.2015 12:44, Armin Rigo wrote:
> >> Hi Larry,
> >>
> >> On 31 May 2015 at 01:20, Larry Hastings  wrote:
> >>> p.s. Supporting this patch also helps cut into PyPy's reported
> performance
> >>> lead--that is, if they ever upgrade speed.pypy.org from comparing
> against
> >>> Python *2.7.2*.
> >>
> >> Right, we should do this upgrade when 2.7.11 is out.
> >>
> >> There is some irony in your comment which seems to imply "PyPy is
> >> cheating by comparing with an old Python 2.7.2": it is inside a thread
> >> which started because "we didn't backport performance improvements to
> >> 2.7.x so far".
> >>
> >> Just to convince myself, I just ran a performance comparison.  I ran
> >> the same benchmark suite as speed.pypy.org, with 2.7.2 against 2.7.10,
> >> both freshly compiled with no "configure" options at all.  The
> >> differences are usually in the noise, but range from +5% to... -60%.
> >> If anything, this seems to show that CPython should take more care
> >> about performance regressions.  If someone is interested:
> >>
> >> * "raytrace-simple" is 1.19 times slower
> >> * "bm_mako" is 1.29 times slower
> >> * "spitfire_cstringio" is 1.60 times slower
> >> * a number of other benchmarks are around 1.08.
> >>
> >> The "7.0x faster" number on speed.pypy.org would be significantly
> >> *higher* if we upgraded the baseline to 2.7.10 now.
> >
> > If someone were to volunteer to set up and run speed.python.org,
> > I think we could add some additional focus on performance
> > regressions. Right now, we don't have any way of reliably
> > and reproducibly testing Python performance.
> >
> > Hint: The PSF would most likely fund such adventures :-)
> >
> > --
> > Marc-Andre Lemburg
> > eGenix.com
> >
> > Professional Python Services directly from the Source  (#1, Jun 01 2015)
>  Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>  mxODBC Plone/Zope Database Adapter ...   http://zope.egenix.com/
>  mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
> > 
> >
> > : Try our mxODBC.Connect Python Database Interface for free ! ::
> >
> >eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
> > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> >Registered at Amtsgericht Duesseldorf: HRB 46611
> >http://www.egenix.com/company/contact/
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fwd: zipimport

[Rose Ames]

At pycon I talked with a few people about bugs.python.org/issue19699.
The consensus seemed to be that zipimport wants a lot of work, possibly
a rewrite.

I'll have some time to work on this over the next couple of months, but
I want to be working on the right thing.  Could the people who were
involved in that conversation remind me of their concerns with
zipimport?  What exactly would the goal of a rewrite be?

Thanks,

Rose





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Avoiding reference leaks in heap types with custom tp_dealloc

[Antoine Pitrou]

On Tue, 2 Jun 2015 21:09:52 +0200
Petr Viktorin  wrote:
> On Mon, Jun 1, 2015 at 6:00 PM, Antoine Pitrou  wrote:
> [...]
> > I think we have been laxist with additions to the stable ABI:
> > apparently, they should be conditioned on the API version requested by
> > the user.  For example, in pystate.h:
> >
> > #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x0303
> > /* New in 3.3 */
> > PyAPI_FUNC(int) PyState_AddModule(PyObject*, struct PyModuleDef*);
> > PyAPI_FUNC(int) PyState_RemoveModule(struct PyModuleDef*);
> > #endif
> >
> > (those were added by Martin, so I assume he knew what he was doing :-))
> >
> > This way, failing to restrict yourself to a given API version fails at
> > compile time, not at runtime. However, it's also more work to do so
> > when adding stuff, which is why we tend to skimp on it.
> 
> I see! I completely missed that memo.
> I filed a patch that wraps my 3.5 additions as issue 24365.
> 
> I think this should be in the PEP, so people like me can find it. Does
> the attached wording look good?

It looks good to me, but Martin should probably validate it. If he
hasn't answered yet in one month, perhaps you can add it anyway :)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: zipimport

[Brett Cannon]

On Tue, Jun 2, 2015 at 5:04 PM Rose Ames  wrote:

> At pycon I talked with a few people about bugs.python.org/issue19699.
> The consensus seemed to be that zipimport wants a lot of work, possibly
> a rewrite.
>
> I'll have some time to work on this over the next couple of months, but
> I want to be working on the right thing.  Could the people who were
> involved in that conversation remind me of their concerns with
> zipimport?  What exactly would the goal of a rewrite be?
>

I believe the participants consisted of Thomas Wouters, Greg Smith, Eric
Snow, Nick Coghlan, and myself. In the end I think the general consensus
for long-term maintenance was to write the minimal amount of code required
that could read zip files and then write all of the import-related code on
top of that. Basically the idea of freezing zipfile seemed messy since it
has a ton of dependencies and baggage that simply isn't needed to replace
zipimport.

I vaguely remember people suggesting writing the minimal zip reading code
in C but I can't remember why since we have I/O access in importlib through
_io and thus it's really just the pulling apart of the zip file to get at
the files to import and thus should be doable in pure Python.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: zipimport

[Eric V. Smith]

On 6/2/2015 5:20 PM, Brett Cannon wrote:
> 
> 
> On Tue, Jun 2, 2015 at 5:04 PM Rose Ames  > wrote:
> 
> At pycon I talked with a few people about bugs.python.org/issue19699
> .
> The consensus seemed to be that zipimport wants a lot of work, possibly
> a rewrite.
> 
> I'll have some time to work on this over the next couple of months, but
> I want to be working on the right thing.  Could the people who were
> involved in that conversation remind me of their concerns with
> zipimport?  What exactly would the goal of a rewrite be?
> 
> 
> I believe the participants consisted of Thomas Wouters, Greg Smith, Eric
> Snow, Nick Coghlan, and myself. In the end I think the general consensus
> for long-term maintenance was to write the minimal amount of code
> required that could read zip files and then write all of the
> import-related code on top of that. Basically the idea of freezing
> zipfile seemed messy since it has a ton of dependencies and baggage that
> simply isn't needed to replace zipimport.

Hey, I was there! This is what I recall as well.

> I vaguely remember people suggesting writing the minimal zip reading
> code in C but I can't remember why since we have I/O access in importlib
> through _io and thus it's really just the pulling apart of the zip file
> to get at the files to import and thus should be doable in pure Python.

I don't think writing it in Python ever came up. I can't think of a
reason why that wouldn't work. Rose: this seems like a good approach to try.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: zipimport

[Rose Ames]

Writing it in Python did come up and was decided against, but I don't 
recall the reasoning.  Could it have been a performance thing?



On 06/02/2015 06:11 PM, Eric V. Smith wrote:

On 6/2/2015 5:20 PM, Brett Cannon wrote:



On Tue, Jun 2, 2015 at 5:04 PM Rose Ames mailto:r...@happyspork.com>> wrote:

 At pycon I talked with a few people about bugs.python.org/issue19699
 .
 The consensus seemed to be that zipimport wants a lot of work, possibly
 a rewrite.

 I'll have some time to work on this over the next couple of months, but
 I want to be working on the right thing.  Could the people who were
 involved in that conversation remind me of their concerns with
 zipimport?  What exactly would the goal of a rewrite be?


I believe the participants consisted of Thomas Wouters, Greg Smith, Eric
Snow, Nick Coghlan, and myself. In the end I think the general consensus
for long-term maintenance was to write the minimal amount of code
required that could read zip files and then write all of the
import-related code on top of that. Basically the idea of freezing
zipfile seemed messy since it has a ton of dependencies and baggage that
simply isn't needed to replace zipimport.


Hey, I was there! This is what I recall as well.


I vaguely remember people suggesting writing the minimal zip reading
code in C but I can't remember why since we have I/O access in importlib
through _io and thus it's really just the pulling apart of the zip file
to get at the files to import and thus should be doable in pure Python.


I don't think writing it in Python ever came up. I can't think of a
reason why that wouldn't work. Rose: this seems like a good approach to try.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/rose%40happyspork.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com