Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Martin Wimpress]

Hi,

On 23/05/17 17:54, Guido van Rossum wrote:

I think the I inevitable conclusion is"thanks, but no thanks."


Can I ask why this the inevitable conclusion? The Python Foundation make 
packages for Windows and macOS, why not snaps for Linux?


On May 23, 2017 8:05 AM, "Martin Wimpress" 
mailto:martin.wimpr...@canonical.com>> 
wrote:


Hi,

On 23/05/17 00:56, Matt Billenstein wrote:

On Tue, May 16, 2017 at 11:31:42AM +0100, Martin Wimpress wrote:

Is there someone here who'd be interested in doing the same
for Python?


Do snaps support Windows and/or MacOS?


You can't install snaps on Windows or macOS. Snaps are currently
targeting various Linux distros, you can create Linux snaps on
Windows using WSL and we're working on improving the tooling for
snap creation on macOS and Windows.

m


-- 
--

Regards, Martin.
___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev

Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org 




--
--
Regards, Martin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

[Victor Stinner]

Hi Ben,

I am not convinced that combining operations will have a significant impact
in term of performance. Mark Shanon implemented that in his HotPy project.

I proposed a RETURN_NONE opcode to combine LOAD_CONST with RETURN_VALUE.
The issue was rejected because I failed to show any speedup.

https://bugs.python.org/issue28800

I would be interested to restart/finish my registervm project to use
register-based bytecode. It allows to implement more optmisations and
reduce the number of instructions. In my experience, less instructions =
faster code.

http://faster-cpython.readthedocs.io/registervm.html

Mark's bytecode uses registers but also a stack.

Victor

Le 24 mai 2017 8:09 PM, "Ben Hoyt"  a écrit :

> Hi folks,
>
> I was looking at some `dis` output today, and I was wondering if anyone
> has investigated optimizing Python (slightly) by adding special-case
> bytecodes for common expressions or statements involving constants?
>
> For example, I (and, based on a quick grep of the stdlib, many others)
> write "x is None" and "x is not None" very often. Or "return True" or
> "return None" or "return 1" and things like that. These all expand into two
> bytecodes, which seems pretty non-optimal (LOAD_CONST + COMPARE_OP or
> LOAD_CONST + RETURN_VALUE). It seems we could get an easy speedup for these
> common cases by adding a peephole optimization and some new opcodes (maybe
> COMPARE_IS_SMALL_CONST and RETURN_SMALL_CONST for these cases).
>
> I'm not proposing to do this yet, as I'd need to benchmark to see how much
> of a gain (if any) it would amount to, but I'm just wondering if there's
> any previous work on this kind of thing. Or, if not, any other thoughts
> before I try it?
>
> Thanks,
> Ben
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Antoine Pitrou]

On Tue, 23 May 2017 23:09:31 -0500
Victor Stinner  wrote:
> Le 23 mai 2017 20:43, "David Wilson"  a écrit :
> In which case, what is to prevent Requests from just depending on
> 
> pyOpenSSL as usual?
> 
> 
> From what I heard, pyOpenSSL development is slowing down, so I'm not sure
> that it's really safe and future-proof (TLS 1.3 anyone?).

So what? Python 2.7 isn't future-proof either...

> I'm still writing 2.7 code every day and would love to see it live a
> little longer, but accepting every feature request seems the wrong way
> to go - and MemoryBIO is a hard sell as a security enhancement, it's new
> functionality.

Agreed with this.

> You are true that they are new features. I disagree on the "accepting every
> feature" part: we are talking about two classes and it's restricted to
> security.

The new TLS API wouldn't significantly improve security.  It's just a
different API.

> I also understood that getting access to system CA allows admins to
> register their company CA and so avoid that users ignore the TLS warning
> (unknown CA).

System admins can add the company CA at the system level in the
system's CA cert store, they have no need for a Python API.  Actually,
they certainly don't want to modify every Python application to add a
company CA.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[Antoine Pitrou]

On Wed, 24 May 2017 23:31:47 +0200
Ivan Levkivskyi  wrote:

> Hi all,
> 
> After collecting suggestions in the previous discussion on python-dev
> https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629
> and playing with implementation, here is an updated version of PEP 544.
> 
> --
> Ivan
> 
> 
> A link for those who don't like reading long e-mails:
> https://www.python.org/dev/peps/pep-0544/
> 
> =
> 
> PEP: 544
> Title: Protocols

Can you give this PEP a more explicit title? "Protocols" sound like
network protocols to me.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

Le 25 mai 2017 1:26 PM, "Antoine Pitrou"  a écrit :

System admins can add the company CA at the system level in the
system's CA cert store, they have no need for a Python API.


If I understood correctly, since the Python ssl module is unable to load
system CAs (at least on Python 2.7) on Windows and macOS, bundled CAs are
needed (like certfi package).

Actually,
they certainly don't want to modify every Python application to add a
company CA.


Yes.

Hum, maybe a short PEP is needed for Python 2.7 to clarify the indirect
advantages of the backport.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Nick Coghlan]

On 25 May 2017 at 21:24, Antoine Pitrou  wrote:
> The new TLS API wouldn't significantly improve security.  It's just a
> different API.

It isn't just a different API. It's an API with *backends for the
native TLS implementations on WIndows and Mac OS X*.

This means that instead of attempting to extract system certs and
inject them into a bundled copy of OpenSSL, it becomes viable to just
use the operating system provided security services and behave the
same was as any other native application (at least as far as secure
network connections are concerned).

Ideally, it would be good to get to a point where we can stop bundling
OpenSSL entirely on Windows and Mac OS X, such that OpenSSL CVEs stop
automatically turning into CPython CVEs, and we can instead delegate
the task of prompt network security stack updates to the OS provider,
the same way we do for non-Apple *nix systems.

Even if it isn't feasible to get Python 2.7 to that happy state by
2020, we may at least be able to get to a point where most routine TLS
connections from Python 2.7 applications are using an OS provided TLS
implementation, so their network security is less dependent on
receiving updates to the Python level components of their stack.

>> I also understood that getting access to system CA allows admins to
>> register their company CA and so avoid that users ignore the TLS warning
>> (unknown CA).
>
> System admins can add the company CA at the system level in the
> system's CA cert store, they have no need for a Python API.  Actually,
> they certainly don't want to modify every Python application to add a
> company CA.

The state of affairs that you're citing as undesirable is pretty much
the way things *currently* work. Most Python applications still won't
see system level certificates on Windows and Mac OS X, and they often
won't see them even on Linux and *BSD systems (unless they're running
directly in the system Python and using the system-provided requests
module (which gets patched to use the system certs rather than the
default Mozilla bundle).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Nick Coghlan]

On 25 May 2017 at 20:32, Martin Wimpress  wrote:
> Hi,
>
> On 23/05/17 17:54, Guido van Rossum wrote:
>>
>> I think the I inevitable conclusion is"thanks, but no thanks."
>
>
> Can I ask why this the inevitable conclusion? The Python Foundation make
> packages for Windows and macOS, why not snaps for Linux?

Mainly because there's no real pay-off to CPython as a project in
lowering barriers to adoption for end users: if someone is running
Linux, they're almost always going to have ready access to completely
usable pre-built Python binaries through their system package manager.

That means that if we were to start publishing our own docker/OCI
images, or our own snaps, or our own FlatPak runtime environment, we'd
be incurring additional ongoing effort without a comparable increase
in the audience we're able to effectively reach.

It also relates to the fact that when it comes to the interminable
packaging format debates in the Linux world, the typical pattern has
been for groups and organisations aiming to promote the use of a
particular package format to use the availability of Python to lower
the barriers to adoption for their particular offering, rather than
the other way around. Some examples:

* Debian takes care of providing deb packages and docker images
* Fedora provides RPMs and docker images
* OpenSUSE provides RPMs (and maybe docker images?)
* the Nix community provide nix packages
* Continuum Analytics provide conda packages
* Heroku take care of providing build packs
* Red Hat provides the RHEL/CentOS and Software Collections RPMs and
docker images
* Docker provide Alpine Linux based docker images
* ActiveState and Enthought provide binaries in a suitable format for
their users
* FreeBSD/NetBSD/OpenBSD also provide their own binaries

With snaps vs docker/OCI and snaps vs FlatPak vs AppImage emerging as
new variations of the longstanding "deb vs RPM vs something else"
arguments, "Here's a source tarball, y'all have fun now" remains the
most sensible publication approach for relatively low level operating
system components like CPython, while the `manylinux` ABI definition
provides "usefully broad" compatibility of pre-built wheel files
across a range of Linux distributions.

Cheers,
Nick.

P.S. Full disclosure: I do work for Red Hat, but I'd still be opposed
to the idea even if the suggestion was to publish our own RPMs or
docker/OCI images

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[Nick Coghlan]

On 25 May 2017 at 21:26, Antoine Pitrou  wrote:
> On Wed, 24 May 2017 23:31:47 +0200
> Ivan Levkivskyi  wrote:
>
>> Hi all,
>>
>> After collecting suggestions in the previous discussion on python-dev
>> https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629
>> and playing with implementation, here is an updated version of PEP 544.
>>
>> --
>> Ivan
>>
>>
>> A link for those who don't like reading long e-mails:
>> https://www.python.org/dev/peps/pep-0544/
>>
>> =
>>
>> PEP: 544
>> Title: Protocols
>
> Can you give this PEP a more explicit title? "Protocols" sound like
> network protocols to me.

Especially given the existing use of the term in an asyncio context:
https://www.python.org/dev/peps/pep-3156/#transports-and-protocols

Given the abstract, I'd suggest "Structural Subtyping" as a suitable
title for the PEP.

That said, I think it's fine to use "protocol" throughout the rest of
the PEP as is currently the case - object protocols and network
protocols are clearly different things, it's just the bare word
"Protocols" appearing as a PEP title in the PEP index with no other
context that's potentially confusing.

I'm +1 on the general idea of the PEP, and only have one question
regarding the specifics. Given:

import typing

class MyContainer:
def __len__(self) -> int:
...
def close(self) -> None:
...

Would that be enough for a static typechecker to consider MyContainer
a structural subtype of both typing.Sized and SupportsClose from the
PEP, even though neither is imported explicitly into the module? I'm
assuming the answer is "Yes", but I didn't see it explicitly stated
anywhere.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

[Ben Hoyt]

Thanks, Victor. That's very helpful. So RETURN_NONE (and probably
RETURN_SMALL_CONST) are not worth it, based on your empirical tests. Your
patch shows how (relatively) straight-forward it is to test out new opcodes.

I'm still optimistic about the value of COMPARE_IS_NONE and
COMPARE_IS_NOT_NONE, though. Mainly because I've done a quick expansion of
LOAD_CONST(None) + COMPARE_OP and it's quite a bit more code and many more
instructions than COMPARE_IS_NONE would be:

LOAD_CONST(None)
COMPARE_OP
PyObject *value = ((PyTupleObject *)(consts))->ob_item[oparg];
value->ob_refcnt++;
*stack_pointer++ = value;
FAST_DISPATCH();
PyObject *right = *--stack_pointer;
PyObject *left = stack_pointer[-1]

// cmp_outcome(), presumably inlined
int r = 0;
switch (compare_oparg) {
case PyCmp_IS:
r = (left == right);
break;
case PyCmp_IS_NOT:
r = (left != right);
break;
case ...
}
PyObject *res = r ? Py_True : Py_False;
res->ob_refcnt++;

if (--(left)->ob_refcnt == 0)
_Py_Dealloc(left);
if (--(right)->ob_refcnt == 0)
_Py_Dealloc(right);
stack_pointer[-1] = res;
if (res == NULL)
goto error;

PREDICT(POP_JUMP_IF_FALSE);
PREDICT(POP_JUMP_IF_TRUE);
DISPATCH();


COMPARE_IS_NONE
PyObject* left = stack_pointer[-1];  // TOP()
PyObject* res = (left == Py_None) ? Py_True : Py_False;
res->ob_refcnt++;
if (--(left)->ob_refcnt == 0)
_Py_Dealloc(left);
stack_pointer[-1] = res;  // SET_TOP(res)
PREDICT(POP_JUMP_IF_FALSE);
PREDICT(POP_JUMP_IF_TRUE);
DISPATCH();

You don't have to get the const arg, there are fewer increfs/decrefs, you
skip a pop, you don't have to test res==NULL (because it's Py_True or
Py_False, which are never NULL), and if there are separate COMPARE_IS_NONE
and COMPARE_IS_NOT_NONE you don't have to switch on the compare arg (though
I'm not sure if that part will be worth it).

For reference, based on a grep, " is None" occurs 2737 times in the CPython
source tree, and " is not None" 2010 times. And I know personally I often
use them in loops as well is at the start of functions (for mutable default
arg handling).

Still, the performance proof will be in the pudding! I might hack these two
opcodes together and test it at some point.

-Ben

On Thu, May 25, 2017 at 6:47 AM, Victor Stinner 
wrote:

> Hi Ben,
>
> I am not convinced that combining operations will have a significant
> impact in term of performance. Mark Shanon implemented that in his HotPy
> project.
>
> I proposed a RETURN_NONE opcode to combine LOAD_CONST with RETURN_VALUE.
> The issue was rejected because I failed to show any speedup.
>
> https://bugs.python.org/issue28800
>
> I would be interested to restart/finish my registervm project to use
> register-based bytecode. It allows to implement more optmisations and
> reduce the number of instructions. In my experience, less instructions =
> faster code.
>
> http://faster-cpython.readthedocs.io/registervm.html
>
> Mark's bytecode uses registers but also a stack.
>
> Victor
>
> Le 24 mai 2017 8:09 PM, "Ben Hoyt"  a écrit :
>
>> Hi folks,
>>
>> I was looking at some `dis` output today, and I was wondering if anyone
>> has investigated optimizing Python (slightly) by adding special-case
>> bytecodes for common expressions or statements involving constants?
>>
>> For example, I (and, based on a quick grep of the stdlib, many others)
>> write "x is None" and "x is not None" very often. Or "return True" or
>> "return None" or "return 1" and things like that. These all expand into two
>> bytecodes, which seems pretty non-optimal (LOAD_CONST + COMPARE_OP or
>> LOAD_CONST + RETURN_VALUE). It seems we could get an easy speedup for these
>> common cases by adding a peephole optimization and some new opcodes (maybe
>> COMPARE_IS_SMALL_CONST and RETURN_SMALL_CONST for these cases).
>>
>> I'm not proposing to do this yet, as I'd need to benchmark to see how
>> much of a gain (if any) it would amount to, but I'm just wondering if
>> there's any previous work on this kind of thing. Or, if not, any other
>> thoughts before I try it?
>>
>> Thanks,
>> Ben
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.
>> stinner%40gmail.com
>>
>>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[Jelle Zijlstra]

2017-05-25 7:19 GMT-07:00 Nick Coghlan :

> On 25 May 2017 at 21:26, Antoine Pitrou  wrote:
> > On Wed, 24 May 2017 23:31:47 +0200
> > Ivan Levkivskyi  wrote:
> >
> >> Hi all,
> >>
> >> After collecting suggestions in the previous discussion on python-dev
> >> https://mail.python.org/pipermail/python-dev/2017-
> March/thread.html#147629
> >> and playing with implementation, here is an updated version of PEP 544.
> >>
> >> --
> >> Ivan
> >>
> >>
> >> A link for those who don't like reading long e-mails:
> >> https://www.python.org/dev/peps/pep-0544/
> >>
> >> =
> >>
> >> PEP: 544
> >> Title: Protocols
> >
> > Can you give this PEP a more explicit title? "Protocols" sound like
> > network protocols to me.
>
> Especially given the existing use of the term in an asyncio context:
> https://www.python.org/dev/peps/pep-3156/#transports-and-protocols
>
> Given the abstract, I'd suggest "Structural Subtyping" as a suitable
> title for the PEP.
>
> That said, I think it's fine to use "protocol" throughout the rest of
> the PEP as is currently the case - object protocols and network
> protocols are clearly different things, it's just the bare word
> "Protocols" appearing as a PEP title in the PEP index with no other
> context that's potentially confusing.
>
> I'm +1 on the general idea of the PEP, and only have one question
> regarding the specifics. Given:
>
> import typing
>
> class MyContainer:
> def __len__(self) -> int:
> ...
> def close(self) -> None:
> ...
>
> Would that be enough for a static typechecker to consider MyContainer
> a structural subtype of both typing.Sized and SupportsClose from the
> PEP, even though neither is imported explicitly into the module? I'm
> assuming the answer is "Yes", but I didn't see it explicitly stated
> anywhere.
>
> Yes, that should be the case. Specifically, if you pass a MyContainer
object to a function whose argument is annotated as typing.Sized or
SupportsClose, a type checker should accept that call.


> Cheers,
> Nick.
>
>
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jelle.zijlstra%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] 2017 Python Language Summit coverage

[Jake Edge]

Hola python-dev,

Thanks to Larry and Barry, I was able to sit in on the Python Language
Summit again this year.  The start of the coverage for that is now
available.

The starting point is the overview article, here:
https://lwn.net/Articles/723251/ (for non-subscribers, this link will
get around the paywall:
https://lwn.net/SubscriberLink/723251/2d2cbb3329dd87f8/ )  It will be
updated as I add more sessions over the next week or so.

LWN subscribers can see the content right away, but one week after they
are published in the weekly edition, they become freely available for
everyone.  Until then, though, feel free to share the SubscriberLinks I
am posting here.

Python 3.6.x, 3.7.0, and beyond: https://lwn.net/Articles/723252/ or
https://lwn.net/SubscriberLink/723252/c5bfea8533558bab/

New CPython workflow issues: https://lwn.net/Articles/723418/ or
https://lwn.net/SubscriberLink/723418/62487f893b324500/

The state of bugs.python.org: https://lwn.net/Articles/723513/ or
https://lwn.net/SubscriberLink/723513/7e4e2e626fc93606/

Progress on the Gilectomy: https://lwn.net/Articles/723514/ or
https://lwn.net/SubscriberLink/723514/70df37b92c746d62/

Hopefully I captured things reasonably well -- if you have corrections
or clarifications (or just comments :) , I would recommend posting them
as comments on the article.

I will post an update next week with the next round (with luck, all of
the rest of them).

enjoy!

jake

-- 
Jake Edge - LWN - j...@lwn.net - http://lwn.net
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

[Cesare Di Mauro]

Hi Ben,

for what you're interested in, you might give a look at WPython 1.0 (
https://code.google.com/archive/p/wpython/downloads ) and 1.1 (
https://code.google.com/archive/p/wpython2/downloads ), but they cover a
lot of optimizations (as you can see from a brief look at the slides):
RETURN_CONST and fusing some opcodes for binary operations are only some of
them.
For this reason, it's also very difficult to micro-benchmark every single
change... :-/

Cheers,
Cesare


Mail
priva di virus. www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2017-05-25 16:28 GMT+02:00 Ben Hoyt :

> Thanks, Victor. That's very helpful. So RETURN_NONE (and probably
> RETURN_SMALL_CONST) are not worth it, based on your empirical tests. Your
> patch shows how (relatively) straight-forward it is to test out new opcodes.
>
> I'm still optimistic about the value of COMPARE_IS_NONE and
> COMPARE_IS_NOT_NONE, though. Mainly because I've done a quick expansion of
> LOAD_CONST(None) + COMPARE_OP and it's quite a bit more code and many more
> instructions than COMPARE_IS_NONE would be:
>
> LOAD_CONST(None)
> COMPARE_OP
> PyObject *value = ((PyTupleObject *)(consts))->ob_item[oparg];
> value->ob_refcnt++;
> *stack_pointer++ = value;
> FAST_DISPATCH();
> PyObject *right = *--stack_pointer;
> PyObject *left = stack_pointer[-1]
>
> // cmp_outcome(), presumably inlined
> int r = 0;
> switch (compare_oparg) {
> case PyCmp_IS:
> r = (left == right);
> break;
> case PyCmp_IS_NOT:
> r = (left != right);
> break;
> case ...
> }
> PyObject *res = r ? Py_True : Py_False;
> res->ob_refcnt++;
>
> if (--(left)->ob_refcnt == 0)
> _Py_Dealloc(left);
> if (--(right)->ob_refcnt == 0)
> _Py_Dealloc(right);
> stack_pointer[-1] = res;
> if (res == NULL)
> goto error;
>
> PREDICT(POP_JUMP_IF_FALSE);
> PREDICT(POP_JUMP_IF_TRUE);
> DISPATCH();
>
>
> COMPARE_IS_NONE
> PyObject* left = stack_pointer[-1];  // TOP()
> PyObject* res = (left == Py_None) ? Py_True : Py_False;
> res->ob_refcnt++;
> if (--(left)->ob_refcnt == 0)
> _Py_Dealloc(left);
> stack_pointer[-1] = res;  // SET_TOP(res)
> PREDICT(POP_JUMP_IF_FALSE);
> PREDICT(POP_JUMP_IF_TRUE);
> DISPATCH();
>
> You don't have to get the const arg, there are fewer increfs/decrefs, you
> skip a pop, you don't have to test res==NULL (because it's Py_True or
> Py_False, which are never NULL), and if there are separate COMPARE_IS_NONE
> and COMPARE_IS_NOT_NONE you don't have to switch on the compare arg (though
> I'm not sure if that part will be worth it).
>
> For reference, based on a grep, " is None" occurs 2737 times in the
> CPython source tree, and " is not None" 2010 times. And I know personally I
> often use them in loops as well is at the start of functions (for mutable
> default arg handling).
>
> Still, the performance proof will be in the pudding! I might hack these
> two opcodes together and test it at some point.
>
> -Ben
>
> On Thu, May 25, 2017 at 6:47 AM, Victor Stinner 
> wrote:
>
>> Hi Ben,
>>
>> I am not convinced that combining operations will have a significant
>> impact in term of performance. Mark Shanon implemented that in his HotPy
>> project.
>>
>> I proposed a RETURN_NONE opcode to combine LOAD_CONST with RETURN_VALUE.
>> The issue was rejected because I failed to show any speedup.
>>
>> https://bugs.python.org/issue28800
>>
>> I would be interested to restart/finish my registervm project to use
>> register-based bytecode. It allows to implement more optmisations and
>> reduce the number of instructions. In my experience, less instructions =
>> faster code.
>>
>> http://faster-cpython.readthedocs.io/registervm.html
>>
>> Mark's bytecode uses registers but also a stack.
>>
>> Victor
>>
>> Le 24 mai 2017 8:09 PM, "Ben Hoyt"  a écrit :
>>
>>> Hi folks,
>>>
>>> I was looking at some `dis` output today, and I was wondering if anyone
>>> has investigated optimizing Python (slightly) by adding special-case
>>> bytecodes for common expressions or statements involving constants?
>>>
>>> For example, I (and, based on a quick grep of the stdlib, many others)
>>> write "x is None" and "x is not None" very often. Or "return True" or
>>> "return None" or "return 1" and things like that. These all expand into two
>>> bytecodes, which seems pretty non-optimal (LOAD_CONST + COMPARE_OP or
>>> LOAD_CONST + RETURN_VALUE). It seems we could get an easy speedup for these
>>> common cases by adding a peephole optimization and some new opcodes (maybe
>>> COMPARE_IS_SMALL_CONST and RETURN_SMALL_CONST for these cases).
>>>
>>> I'm not proposing to do this yet, as I'd need to benchmark to see how
>>> much of a gain (

Re: [Python-Dev] 2017 Python Language Summit coverage

[Eric Snow]

On Thu, May 25, 2017 at 7:03 AM, Jake Edge  wrote:
>
> Hola python-dev,
>
> Thanks to Larry and Barry, I was able to sit in on the Python Language
> Summit again this year.  The start of the coverage for that is now
> available.
>
> [snipped]
>
> Hopefully I captured things reasonably well -- if you have corrections
> or clarifications (or just comments :) , I would recommend posting them
> as comments on the article.
>
> I will post an update next week with the next round (with luck, all of
> the rest of them).

Thanks again, Jake, for taking the time to do this.  I haven't read
through the articles yet, but those from past years have been an
excellent view into the what's going on.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2017 Python Language Summit coverage

[Barry Warsaw]

On May 25, 2017, at 08:03 AM, Jake Edge wrote:

>Thanks to Larry and Barry, I was able to sit in on the Python Language
>Summit again this year.  The start of the coverage for that is now
>available.

Thanks so much for your always excellent reporting Jake.  It's unfortunate
that we can't invite everyone who wants to attend the summit, so your articles
are the best way to improve our transparency, reduce the mystery, and let the
rest of the community see what happens when you lock  a bunch of core
devs in a room for a day.

be-fez-zled-ly y'rs,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[Mark Shannon]

On 24/05/17 14:31, Ivan Levkivskyi wrote:

Hi all,

After collecting suggestions in the previous discussion on python-dev
https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629
and playing with implementation, here is an updated version of PEP 544.

--
Ivan


I really like this PEP in general. I think this brings the type system 
for type-hints closer to Python semantics.


But there are a few points I disagree with.
I don't think Protocol types should be tied to ABCs. It just makes 
things more complex with no obvious benefit.
I also think all references to 'isinstance' and 'issubclass' should be 
removed. Type-hints should not have runtime semantics, beyond those that 
they have as classes. In fact, there is no need for protocol types to be 
classes at all.


Cheers,
Mark.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

[Mark Shannon]

On 25/05/17 03:47, Victor Stinner wrote:

Hi Ben,

I am not convinced that combining operations will have a significant
impact in term of performance. Mark Shanon implemented that in his HotPy
project.


I don't think that I did ;)
HotPy implemented a trace-based optimising interpreter, that preformed 
type specialisation and deferred evaluation of objects, with a "dumb" 
compiler that applied standard optimisations to the resulting traces.
It was technologically similar to PyPy (at the time), but designed for 
ease of engineering,




I proposed a RETURN_NONE opcode to combine LOAD_CONST with RETURN_VALUE.
The issue was rejected because I failed to show any speedup.

https://bugs.python.org/issue28800

I would be interested to restart/finish my registervm project to use
register-based bytecode. It allows to implement more optmisations and
reduce the number of instructions. In my experience, less instructions =
faster code.

http://faster-cpython.readthedocs.io/registervm.html

Mark's bytecode uses registers but also a stack.
The "registers" were used to hold values across calls and the like when 
optimising traces, but I wouldn't use that design if I were to do it again.




Victor

Le 24 mai 2017 8:09 PM, "Ben Hoyt" mailto:benh...@gmail.com>> a écrit :

Hi folks,

I was looking at some `dis` output today, and I was wondering if
anyone has investigated optimizing Python (slightly) by adding
special-case bytecodes for common expressions or statements
involving constants?

For example, I (and, based on a quick grep of the stdlib, many
others) write "x is None" and "x is not None" very often. Or "return
True" or "return None" or "return 1" and things like that. These all
expand into two bytecodes, which seems pretty non-optimal
(LOAD_CONST + COMPARE_OP or LOAD_CONST + RETURN_VALUE). It seems we
could get an easy speedup for these common cases by adding a
peephole optimization and some new opcodes (maybe
COMPARE_IS_SMALL_CONST and RETURN_SMALL_CONST for these cases).

I'm not proposing to do this yet, as I'd need to benchmark to see
how much of a gain (if any) it would amount to, but I'm just
wondering if there's any previous work on this kind of thing. Or, if
not, any other thoughts before I try it?

Thanks,
Ben


___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev

Unsubscribe:

https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2017 Python Language Summit coverage

[Gregory P. Smith]

Thanks Jake!

I'm laughing at not being in the photo. That must've happened soon after I
left the room as I disappeared a tad early.  I'll pretend I'm hiding behind
Brett or Thomas. ;)

-gps


On Thu, May 25, 2017 at 10:34 AM Barry Warsaw  wrote:

> On May 25, 2017, at 08:03 AM, Jake Edge wrote:
>
> >Thanks to Larry and Barry, I was able to sit in on the Python Language
> >Summit again this year.  The start of the coverage for that is now
> >available.
>
> Thanks so much for your always excellent reporting Jake.  It's unfortunate
> that we can't invite everyone who wants to attend the summit, so your
> articles
> are the best way to improve our transparency, reduce the mystery, and let
> the
> rest of the community see what happens when you lock  a bunch of core
> devs in a room for a day.
>
> be-fez-zled-ly y'rs,
> -Barry
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

[Victor Stinner]

After reading Cesare Di Mauro's email, I realized that I was thinking to
WPython in fact...

Victor

Le 25 mai 2017 8:11 PM, "Mark Shannon"  a écrit :

>
>
> On 25/05/17 03:47, Victor Stinner wrote:
>
>> Hi Ben,
>>
>> I am not convinced that combining operations will have a significant
>> impact in term of performance. Mark Shanon implemented that in his HotPy
>> project.
>>
>
> I don't think that I did ;)
> HotPy implemented a trace-based optimising interpreter, that preformed
> type specialisation and deferred evaluation of objects, with a "dumb"
> compiler that applied standard optimisations to the resulting traces.
> It was technologically similar to PyPy (at the time), but designed for
> ease of engineering,
>
>
>> I proposed a RETURN_NONE opcode to combine LOAD_CONST with RETURN_VALUE.
>> The issue was rejected because I failed to show any speedup.
>>
>> https://bugs.python.org/issue28800
>>
>> I would be interested to restart/finish my registervm project to use
>> register-based bytecode. It allows to implement more optmisations and
>> reduce the number of instructions. In my experience, less instructions =
>> faster code.
>>
>> http://faster-cpython.readthedocs.io/registervm.html
>>
>> Mark's bytecode uses registers but also a stack.
>>
> The "registers" were used to hold values across calls and the like when
> optimising traces, but I wouldn't use that design if I were to do it again.
>
>
>> Victor
>>
>> Le 24 mai 2017 8:09 PM, "Ben Hoyt" > > a écrit :
>>
>> Hi folks,
>>
>> I was looking at some `dis` output today, and I was wondering if
>> anyone has investigated optimizing Python (slightly) by adding
>> special-case bytecodes for common expressions or statements
>> involving constants?
>>
>> For example, I (and, based on a quick grep of the stdlib, many
>> others) write "x is None" and "x is not None" very often. Or "return
>> True" or "return None" or "return 1" and things like that. These all
>> expand into two bytecodes, which seems pretty non-optimal
>> (LOAD_CONST + COMPARE_OP or LOAD_CONST + RETURN_VALUE). It seems we
>> could get an easy speedup for these common cases by adding a
>> peephole optimization and some new opcodes (maybe
>> COMPARE_IS_SMALL_CONST and RETURN_SMALL_CONST for these cases).
>>
>> I'm not proposing to do this yet, as I'd need to benchmark to see
>> how much of a gain (if any) it would amount to, but I'm just
>> wondering if there's any previous work on this kind of thing. Or, if
>> not, any other thoughts before I try it?
>>
>> Thanks,
>> Ben
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org 
>> https://mail.python.org/mailman/listinfo/python-dev
>> 
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.
>> stinner%40gmail.com
>> > stinner%40gmail.com>
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%
>> 40hotpy.org
>>
>> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.
> stinner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2017 Python Language Summit coverage

[MRAB]

On 2017-05-25 22:16, Gregory P. Smith wrote:

Thanks Jake!

I'm laughing at not being in the photo. That must've happened soon after 
I left the room as I disappeared a tad early.  I'll pretend I'm hiding 
behind Brett or Thomas. ;)



> -gps
>
Or someone could Photoshop you in. :-)



On Thu, May 25, 2017 at 10:34 AM Barry Warsaw > wrote:


On May 25, 2017, at 08:03 AM, Jake Edge wrote:

 >Thanks to Larry and Barry, I was able to sit in on the Python Language
 >Summit again this year.  The start of the coverage for that is now
 >available.

Thanks so much for your always excellent reporting Jake.  It's
unfortunate
that we can't invite everyone who wants to attend the summit, so
your articles
are the best way to improve our transparency, reduce the mystery,
and let the
rest of the community see what happens when you lock  a bunch
of core
devs in a room for a day.

be-fez-zled-ly y'rs,
-Barry


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com