Re: [Python-Dev] Python possible vulnerabilities in concurrency

[Mark Shannon]

On 16/11/17 04:53, Guido van Rossum wrote:

[snip]



They then go on to explain that sometimes vulnerabilities can be 
exploited, but I object to calling all bugs vulnerabilities -- that's 
just using a scary word to get attention for a sleep-inducing document 
containing such gems as "Use floating-point arithmetic only when 
absolutely needed" (page 230).


Thanks for reading it, so we don't have to :)

As Wes said, cwe.mitre.org is the place to go if you care about this 
stuff, although it can be a bit opaque.
For non-experts, https://www.owasp.org/index.php/Top_10_2013-Top_10 is a 
good starting point to learn about software vulnerabilities,



Cheers,
Mark.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

Ok, I merged my PR adding -X dev: you can now test in Python 3.7 ;-)

> The list of checks can be extended later. For example, we may enable
> the debug mode of asyncio: PYTHONASYNCIODEBUG=1.

I opened https://bugs.python.org/issue32047 to propose enable asyncio
debug mode using the Python developer mode.

What do you think? Is it ok to include asyncio in the global "developer mode"?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Antoine Pitrou]

On Thu, 16 Nov 2017 12:53:24 +0100
Victor Stinner  wrote:

> Ok, I merged my PR adding -X dev: you can now test in Python 3.7 ;-)
> 
> > The list of checks can be extended later. For example, we may enable
> > the debug mode of asyncio: PYTHONASYNCIODEBUG=1.  
> 
> I opened https://bugs.python.org/issue32047 to propose enable asyncio
> debug mode using the Python developer mode.
> 
> What do you think? Is it ok to include asyncio in the global "developer mode"?

I'd rather not.  Those are two orthogonal things.  In particular,
asyncio debug mode is quite expensive.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Antoine Pitrou]

On Thu, 16 Nov 2017 03:57:49 +0100
Victor Stinner  wrote:
> Hi,
> 
> Since Brett and Nick like the idea and nobody complained against it, I
> implemented the -X dev option:
> https://bugs.python.org/issue32043
> (Right now, it's a pull request.)

Could you measure and perhaps document the expected effect on
performance and memory consumption?
(it can be a very rough ballpart estimate)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Ivan Levkivskyi]

On 16 November 2017 at 07:56, Nick Coghlan  wrote:

> On 16 November 2017 at 04:39, Ivan Levkivskyi 
> wrote:
>
>> Nick is exactly right here. Jim, if you want to propose alternative
>> wording, then we could consider it.
>>
>
> Jim also raised an important point that needs clarification at the spec
> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
> do all such methods get passed the *same* orig_bases tuple? Or do they
> receive partially resolved ones, such that bases listed before them have
> already been resolved to their MRO entries by the time they run.
>
>
>
Yes, they all get the same initial bases tuple as an argument. Passing
updated ones will cost a bit more and I don't think it will be needed (in
the worst case a base can resolve another base by calling its
__mro_entries__ manually).
I will clarify this in the PEP.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

2017-11-16 13:09 GMT+01:00 Antoine Pitrou :
>> What do you think? Is it ok to include asyncio in the global "developer 
>> mode"?
>
> I'd rather not.  Those are two orthogonal things.  In particular,
> asyncio debug mode is quite expensive.

Is it really an issue? When you develop an application, the
performance of the application shouldn't be an issue no? From my point
of view, it's the purpose of the opt-in developer mode: enable
"expensive" checks at runtime. But you are right that the cost of the
checks should be evaluated.

About asyncio debug mode, if it's too expensive to be used to develop
an application, maybe there is an issue with additional checks? Should
we remove some of them to be able to use asyncio debug mode in more
cases?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Antoine Pitrou]

Le 16/11/2017 à 13:38, Victor Stinner a écrit :
> 2017-11-16 13:09 GMT+01:00 Antoine Pitrou :
>>> What do you think? Is it ok to include asyncio in the global "developer 
>>> mode"?
>>
>> I'd rather not.  Those are two orthogonal things.  In particular,
>> asyncio debug mode is quite expensive.
> 
> Is it really an issue? When you develop an application, the
> performance of the application shouldn't be an issue no?

When you develop an application, you can run functional tests which have
timing requirements (or simply be too annoying to run if runtimes are
multiplied by 2 or more).  In that case it is good to enable "cheap"
debug checks (those that have less than a 20% cost) while leaving the
expensive ones disabled.

> About asyncio debug mode, if it's too expensive to be used to develop
> an application, maybe there is an issue with additional checks? Should
> we remove some of them to be able to use asyncio debug mode in more
> cases?

Well, I'm sure some people like them, otherwise they wouldn't have been
added to the codebase in the first place :-)  For example, knowing where
a Future was created can make debug logs much more informative.

(see https://bugs.python.org/issue31970)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

2017-11-16 13:11 GMT+01:00 Antoine Pitrou :
> Could you measure and perhaps document the expected effect on
> performance and memory consumption?
> (it can be a very rough ballpart estimate)

Currently "python3 -X dev script.py" behaves as "PYTHONMALLOC=debug
python3 -W default -X faulthandler script.py".

faulthandler has a negligible cost on performance/memory.

For -W default, I guess that your question is the cost on emitting a
warning: overhead when a warning is displayed, and overhead when the
warning is filtered. Right?

IMHO the most expensive check is PYTHONMALLOC=debug which increases a
lot the memory usage.

You can measure the difference using tracemalloc and PYTHONMALLOC:

haypo@selma$ PYTHONMALLOC=debug ./python -X tracemalloc -i -m test test_os
(...)
>>> import tracemalloc; tracemalloc.get_traced_memory()
(10719623, 10981725)

haypo@selma$ PYTHONMALLOC=debug ./python -X tracemalloc -i -m test test_os
(...)
>>> import tracemalloc; tracemalloc.get_traced_memory()
(10724064, 16577338)

For example, on test_os, PYTHONMALLOC=debug increases the peak memory
usage from 10.5 MiB to 15.8 MiB: +50%.

PYTHONMALLOC=debug adds 4 * sizeof(size_t) bytes to each allocated
memory block. For example, an empty tuple uses 64 bytes, but
PYTHONMALLOC=debug allocates 96 bytes (+ 32 bytes) in 64-bit mode.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Antoine Pitrou]

Hi Victor,

Thanks for the answer!

Le 16/11/2017 à 13:48, Victor Stinner a écrit :
> 
> faulthandler has a negligible cost on performance/memory.
> 
> For -W default, I guess that your question is the cost on emitting a
> warning: overhead when a warning is displayed, and overhead when the
> warning is filtered. Right?

-Wdefault means -Wonce or -Walways? If the former, I don't expect many
warnings to be emitted.

> For example, on test_os, PYTHONMALLOC=debug increases the peak memory
> usage from 10.5 MiB to 15.8 MiB: +50%.

I see.  For my use cases, this would be acceptable :-)

But I think this should be documented, for example:

"""Currently, developer mode adds negligible CPU time overhead, but can
increase memory consumption significantly if many small objects are
allocated.  This is subject to change in the future."""

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

2017-11-16 13:43 GMT+01:00 Antoine Pitrou :
>> About asyncio debug mode, if it's too expensive to be used to develop
>> an application, maybe there is an issue with additional checks? Should
>> we remove some of them to be able to use asyncio debug mode in more
>> cases?
>
> Well, I'm sure some people like them, otherwise they wouldn't have been
> added to the codebase in the first place :-)  For example, knowing where
> a Future was created can make debug logs much more informative.

The most expensive part of asyncio debug mode is the code to extracts
the current stack when a coroutine or a handle is created.

Would it make sense to modify asyncio debug mode to skip the traceback
by default, but add a second debug level which extracts the traceback?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

2017-11-16 13:54 GMT+01:00 Antoine Pitrou :
> -Wdefault means -Wonce or -Walways? If the former, I don't expect many
> warnings to be emitted.

It's kind of funny that passing "-W default" changes the "default"
behaviour. "-W default" is documented as:

   "Explicitly request the default behavior (printing each warning
once per source line)."

https://docs.python.org/dev/using/cmdline.html#cmdoption-w

Default warnings filters in release mode:

$ python3 -c 'import pprint, warnings; pprint.pprint(warnings.filters)'
[('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0)]

-Wd adds the a warnings filter with the "default" action matching all
warnings (any kind, any message, any line number) if I understand
correctly:

$ python3 -Wd -c 'import pprint, warnings; pprint.pprint(warnings.filters)'
[('default',
  re.compile('', re.IGNORECASE),
  ,
  re.compile(''),
  0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0)]

"default" and "once" actions are different:

default: "print the first occurrence of matching warnings for **each
location where the warning is issued**"
once: "print only the first occurrence of matching warnings,
**regardless of location**"


>> For example, on test_os, PYTHONMALLOC=debug increases the peak memory
>> usage from 10.5 MiB to 15.8 MiB: +50%.
>
> I see.  For my use cases, this would be acceptable :-)
>
> But I think this should be documented, for example:
>
> """Currently, developer mode adds negligible CPU time overhead, but can
> increase memory consumption significantly if many small objects are
> allocated.  This is subject to change in the future."""

+50% memory is inacceptable to develop on embedded devices, or more
generally with low level. But in that case, you are can enable options
enabled by -X dev manually, without PYTHONMALLOC=debug :-)

Ok, I will document that, I like your proposed paragraph.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Nick Coghlan]

On 16 November 2017 at 22:54, Antoine Pitrou  wrote:

>
> Hi Victor,
>
> Thanks for the answer!
>
> Le 16/11/2017 à 13:48, Victor Stinner a écrit :
> >
> > faulthandler has a negligible cost on performance/memory.
> >
> > For -W default, I guess that your question is the cost on emitting a
> > warning: overhead when a warning is displayed, and overhead when the
> > warning is filtered. Right?
>
> -Wdefault means -Wonce or -Walways? If the former, I don't expect many
> warnings to be emitted.
>

Confusingly, neither of these: default, once, module, and always are all
different settings.

once: once per process (regardless of location)
module: once per module (regardless of line)
default: once per location (line+module combination)
always: every time

Still, even with once-per-location behaviour, the warning overhead should
be minimal.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] IPv6 issues on *.python.org

[Ned Deily]

On Nov 16, 2017, at 15:07, Antoine Pitrou  wrote:
> I'm having IPv6 issues on *.python.org.  Is anyone having the same
> issues or is it just me?  Who should I report this to?

The PSF Infrastructure team is available via email or IRC:

http://psf-salt.readthedocs.io/overview/#the-infrastructure-team

--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Yury Selivanov]

On Thu, Nov 16, 2017 at 7:56 AM, Victor Stinner
 wrote:
> 2017-11-16 13:43 GMT+01:00 Antoine Pitrou :
>>> About asyncio debug mode, if it's too expensive to be used to develop
>>> an application, maybe there is an issue with additional checks? Should
>>> we remove some of them to be able to use asyncio debug mode in more
>>> cases?
>>
>> Well, I'm sure some people like them, otherwise they wouldn't have been
>> added to the codebase in the first place :-)  For example, knowing where
>> a Future was created can make debug logs much more informative.
>
> The most expensive part of asyncio debug mode is the code to extracts
> the current stack when a coroutine or a handle is created.

Probably the most expensive part of asyncio debug mode is all
coroutines wrapped with CoroWrapper.  This makes every "await" and
coroutine instantiation much slower (think 2-3x).

> Would it make sense to modify asyncio debug mode to skip the traceback
> by default, but add a second debug level which extracts the traceback?

Let's keep it simple.  I'm big -1 on adding different "debug levels",
they are always confusing.

Overall I don't see an issue with enabling asyncio debug mode when
python is executed with "-X dev".  If the purpose of the flag is to
make Python super verbose and it will not be recommended to use it in
production -- then why not.


Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Barry Warsaw]

On Nov 16, 2017, at 06:53, Victor Stinner  wrote:

> What do you think? Is it ok to include asyncio in the global "developer mode"?

I’m +1 on that, and the performance hit doesn’t bother me for a developer mode.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Barry Warsaw]

On Nov 15, 2017, at 21:57, Victor Stinner  wrote:
> 
> Since Brett and Nick like the idea and nobody complained against it, I
> implemented the -X dev option:

Cool!  What would you think about printing a summary of the settings under the 
standard banner when you run the REPL under -X dev?  I’d rather not have to 
look it up in some obscure docs page whenever I use it.  If not that, then what 
about having a -msettings module or some such that prints it out?

-Barry




signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Barry Warsaw]

On Nov 16, 2017, at 09:47, Yury Selivanov  wrote:

> Let's keep it simple.  I'm big -1 on adding different "debug levels",
> they are always confusing.

Oh, this one’s easy.

-X dev   == some debugging
-X deve  == a little more
-X devel == give it to me!
-X develo== now you’re talking (literally)
-X develop   == thank you sir, may I have another?
-X develope  == here comes the flood
-X developer == needle inna haystack!

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[brent bejot]

Hello all,

Noticed that "MRO" is not actually defined in the PEP and it seems like it
should be.  Probably in the Performance section where the abbreviation is
first used outside of a function name.

-Brent

On Thu, Nov 16, 2017 at 7:22 AM, Ivan Levkivskyi 
wrote:

> On 16 November 2017 at 07:56, Nick Coghlan  wrote:
>
>> On 16 November 2017 at 04:39, Ivan Levkivskyi 
>> wrote:
>>
>>> Nick is exactly right here. Jim, if you want to propose alternative
>>> wording, then we could consider it.
>>>
>>
>> Jim also raised an important point that needs clarification at the spec
>> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
>> do all such methods get passed the *same* orig_bases tuple? Or do they
>> receive partially resolved ones, such that bases listed before them have
>> already been resolved to their MRO entries by the time they run.
>>
>>
>>
> Yes, they all get the same initial bases tuple as an argument. Passing
> updated ones will cost a bit more and I don't think it will be needed (in
> the worst case a base can resolve another base by calling its
> __mro_entries__ manually).
> I will clarify this in the PEP.
>
> --
> Ivan
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brent.bejot%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

2017-11-16 15:47 GMT+01:00 Yury Selivanov :
> Overall I don't see an issue with enabling asyncio debug mode when
> python is executed with "-X dev".

Cool!

> If the purpose of the flag is to
> make Python super verbose and it will not be recommended to use it in
> production -- then why not.

Running an application in debug mode or "developer mode" was never
recommanded by anyone. Who runs Django in debug mode on production?
:-) (Nobody, I hope.)

I'm working on the -X dev documentation to clarify its purpose.

No, the developer mode must not flood stdout with debug messages. It
should only emit warnings if a potential or real bug is detected. For
example, I don't want to enable the debug mode of ftplib in the
developer mode, since this option logs each FTP command to stdout.
It's a different use case.

About performance, IMHO the purpose of the developer mode is to enable
additional checks to ease debug, checks were are too expensive to be
enabled by default. This definition fits well with the asyncio debug
mode.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Antoine Pitrou]

Le 16/11/2017 à 17:42, Victor Stinner a écrit :
> 
> Running an application in debug mode or "developer mode" was never
> recommanded by anyone.

I don't know.  Almost everyone runs Python with __debug__ set to True :-)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Victor Stinner]

My draft addition to the -X dev doc:
"""
The developer mode is different from the "debug mode" of some modules of the
standard library which enables a lot of debug logs. The developer mode should
only log a warning if a potential bug is detected.

The developer mode has a limited effect. Many modules of the standard library
have their own "debug mode" which is not enabled in the developer mode. See
also the :data:`__debug__` flag disabled by the :option:`-O` option, and
:data:`sys.flags.debug` enabled by the :option:`-d` option.
"""

You forgot the infamous -d option!

Honestly, I never used __debug__ nor sys.flags.debug.

I like adding yet another option to confuse people a little bit more!

Victor

2017-11-16 17:52 GMT+01:00 Antoine Pitrou :
>
> Le 16/11/2017 à 17:42, Victor Stinner a écrit :
>>
>> Running an application in debug mode or "developer mode" was never
>> recommanded by anyone.
>
> I don't know.  Almost everyone runs Python with __debug__ set to True :-)
>
> Regards
>
> Antoine.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Add a developer mode to Python: -X dev command line option

[Glenn Linderman]

On 11/16/2017 7:39 AM, Barry Warsaw wrote:

On Nov 16, 2017, at 09:47, Yury Selivanov  wrote:


Let's keep it simple.  I'm big -1 on adding different "debug levels",
they are always confusing.

Oh, this one’s easy.

-X dev   == some debugging
-X deve  == a little more
-X devel == give it to me!


This is the   "devel level", where you solve those bugs from hell...


-X develo== now you’re talking (literally)
-X develop   == thank you sir, may I have another?
-X develope  == here comes the flood
-X developer == needle inna haystack!

Cheers,
-Barry



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/v%2Bpython%40g.nevcal.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Koos Zevenhoven]

On Thu, Nov 16, 2017 at 6:28 PM, brent bejot  wrote:

> Hello all,
>
> Noticed that "MRO" is not actually defined in the PEP and it seems like it
> should be.  Probably in the Performance section where the abbreviation is
> first used outside of a function name.
>
>
​I don't think it will hurt if I suggest that __bases__, bases, "original
bases"​, mro, __orig_bases__, MRO, __mro__ and "concatenated mro entries"
are all defined as synonyms of each other, except with different meanings
:-)

––Koos



> -Brent
>
> On Thu, Nov 16, 2017 at 7:22 AM, Ivan Levkivskyi 
> wrote:
>
>> On 16 November 2017 at 07:56, Nick Coghlan  wrote:
>>
>>> On 16 November 2017 at 04:39, Ivan Levkivskyi 
>>> wrote:
>>>
 Nick is exactly right here. Jim, if you want to propose alternative
 wording, then we could consider it.

>>>
>>> Jim also raised an important point that needs clarification at the spec
>>> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
>>> do all such methods get passed the *same* orig_bases tuple? Or do they
>>> receive partially resolved ones, such that bases listed before them have
>>> already been resolved to their MRO entries by the time they run.
>>>
>>>
>>>
>> Yes, they all get the same initial bases tuple as an argument. Passing
>> updated ones will cost a bit more and I don't think it will be needed (in
>> the worst case a base can resolve another base by calling its
>> __mro_entries__ manually).
>> I will clarify this in the PEP.
>>
>> --
>> Ivan
>>
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/brent.
>> bejot%40gmail.com
>>
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> k7hoven%40gmail.com
>
>


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Ethan Furman]

On 11/16/2017 04:22 AM, Ivan Levkivskyi wrote:

On 16 November 2017 at 07:56, Nick Coghlan wrote:



Jim also raised an important point that needs clarification at the spec

>> level: given multiple entries in "orig_bases" with __mro_entries__ methods,
>> do all such methods get passed the *same* orig_bases tuple? Or do they

receive partially resolved ones, such that bases listed before them have

>> already been resolved to their MRO entries by the time they run.


Yes, they all get the same initial bases tuple as an argument. Passing

> updated ones will cost a bit more and I don't think it will be needed
> (in the worst case a base can resolve another base by calling its
> __mro_entries__ manually).  I will clarify this in the PEP.

If the extra complexity is to:

> - given orig_bases, a method could avoid injecting bases already listed
>   if it wanted to
> - allowing multiple items to be returned provides a way to programmatically
>   combine mixins without having to define a new subclass for each combination

And each method is passed the same original tuple (without other methods' updates) then don't we end up in a situation 
where we can have duplicates base classes?


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] module customization

[Brett Cannon]

On Wed, 15 Nov 2017 at 16:27 Ethan Furman  wrote:

> So there are currently two ways to customize a module, with PEP 562
> proposing a third.
>
> The first method involves creating a standard class object, instantiating
> it, and replacing the sys.modules entry with it.
>
> The second way is fairly similar, but instead of replacing the entire
> sys.modules entry, its class is updated to be the
> class just created -- something like sys.modules['mymod'].__class__ =
> MyNewClass .
>
> My request:  Can someone write a better example of the second method?  And
> include __getattr__ ?
>

There's actually an example in the stdlib thanks to
importlib.util.LazyLoader
,
although it uses __getattribute__() and not __getattr__():

https://github.com/python/cpython/blob/d505a29a15a6f9315d8c46445b8a0cccfc2048b8/Lib/importlib/util.py#L212


-Brett


> My question:  Does that __getattr__ method have 'self' as the first
> parameter?  If not, why not, and if so, shouldn't
> PEP 562's __getattr__ also take a 'self'?
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 560: bases classes / confusion

[Ivan Levkivskyi]

On 16 November 2017 at 18:58, Ethan Furman  wrote:

> On 11/16/2017 04:22 AM, Ivan Levkivskyi wrote:
>
>> On 16 November 2017 at 07:56, Nick Coghlan wrote:
>>
>
> Jim also raised an important point that needs clarification at the spec
>>>
>> >> level: given multiple entries in "orig_bases" with __mro_entries__
> methods,
> >> do all such methods get passed the *same* orig_bases tuple? Or do they
>
>> receive partially resolved ones, such that bases listed before them have
>>>
>> >> already been resolved to their MRO entries by the time they run.
>
>>
>> Yes, they all get the same initial bases tuple as an argument. Passing
>>
> > updated ones will cost a bit more and I don't think it will be needed
> > (in the worst case a base can resolve another base by calling its
> > __mro_entries__ manually).  I will clarify this in the PEP.
>
> If the extra complexity is to:
>
> > - given orig_bases, a method could avoid injecting bases already listed
> >   if it wanted to
> > - allowing multiple items to be returned provides a way to
> programmatically
> >   combine mixins without having to define a new subclass for each
> combination
>
> And each method is passed the same original tuple (without other methods'
> updates) then don't we end up in a situation where we can have duplicates
> base classes?
>

Not that it is impossible now (in certain sense):

class MultiMeta(type):
def __new__(mcls, name, bases, ns):
return super().__new__(mcls, name, (), ns)

class MultiBase(metaclass=MultiMeta):
pass

class C(MultiBase, list, list, MultiBase, dict, dict, dict):  # OK
pass

What is probably confusing in the current PEP text, is that it doesn't say
clearly that
the substitution happens before any other steps in __build_class__.
Therefore all normal checks (like duplicate bases and MRO consistency)
happen and e.g.

class C(List[int], List[str]):
pass

will fail with:

TypeError: duplicate base class list

(by the way while playing with this I have found a bug in the reference
implementation)

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com