[Python-Dev] Re: Virtual machine bleeds into generator implementation?

2020-04-27 Thread Paul Sokolovsky
Hello,

On Sun, 26 Apr 2020 19:51:18 -0700
Skip Montanaro  wrote:

[]

> I think it's worse that this though, as it seems that in gen_send_ex()
> it actually pushes a value onto the stack. That can't be solved by
> simply adding a state attribute to the generator object struct.

At the higher level, "it doesn't push value on stack", it "sets value
of the yield operator to return". CPython uses stack slots to keep
data, so puts it in a stack slot, you use registers, so would put it in
a ("return value") register. Overall, that seems like minor patching
detail comparing to patching which would be required to implement
register-based calling convention for functions.


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AIMUHRIUBDN63IGLTLIYZH4YXZDBNZTV/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Virtual machine bleeds into generator implementation?

2020-04-27 Thread Skip Montanaro
> > I think it's worse that this though, as it seems that in gen_send_ex()
> > it actually pushes a value onto the stack. That can't be solved by
> > simply adding a state attribute to the generator object struct.
>
> At the higher level, "it doesn't push value on stack", it "sets value
> of the yield operator to return".

Potatoes, potahtoes. :-) The current implementation "sets the value of
the yield operator to return" by pushing a value onto the stack:

/* Push arg onto the frame's value stack */
result = arg ? arg : Py_None;
Py_INCREF(result);
*(f->f_stacktop++) = result;

Thanks for the replies. I will cook up some private API in my cpython
fork. Whether or not my new vm ever sees the light of day, I think it
would be worthwhile to consider a proper API (even a _PyEval macro or
two) for the little dance the two subsystems do.

Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NR66U3TD3BH6K7CQTA4B6HOLOT3KP3VF/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Announcement: pip 20.1b1 beta release

2020-04-27 Thread Miro Hrončok

On 23. 04. 20 21:36, Sumana Harihareswara wrote:

We would be grateful for all the testing that users could do to ensure that, 
when pip 20.1 is released, it's as solid as we can make it.


We are doing some basic testing in Fedora.

So far everyhting looks good.

We've tested:

Basic CI smoke testing with venvs, virtualenvs, tox:
  - CPython 3.5-3.8 (will add 3.9)
  - PyPy 3.6

Bundling in Fedora and running non-network tests during build on CPython 3.8.2 
and 3.9.0a5.


Building couple PEP 517/518 packages (pytest, pluggy, clickit, entyrpoints).

Building and self-testing CPython 3.5-3.9 and PyPy 3.6 using a wheel of this 
pip, passed their ensurepip and venv tests during the build.



I have now experienced some weird permissions:

/usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/INSTALLER 600
/usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/RECORD 600
/usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/direct_url.json 600


Will try to reproduce this outside of our build environment and will report an 
issue.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5EAUIYYIRKXEHTAG5GQ7EJHSXGZIW2F7/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Announcement: pip 20.1b1 beta release

2020-04-27 Thread Paul Moore
On Mon, 27 Apr 2020 at 16:21, Miro Hrončok  wrote:
>
> On 23. 04. 20 21:36, Sumana Harihareswara wrote:
> > We would be grateful for all the testing that users could do to ensure 
> > that, when pip 20.1 is released, it's as solid as we can make it.
>
> We are doing some basic testing in Fedora.
>
> So far everyhting looks good.
>
> We've tested:
>
> Basic CI smoke testing with venvs, virtualenvs, tox:
>- CPython 3.5-3.8 (will add 3.9)
>- PyPy 3.6
>
> Bundling in Fedora and running non-network tests during build on CPython 3.8.2
> and 3.9.0a5.
>
> Building couple PEP 517/518 packages (pytest, pluggy, clickit, entyrpoints).
>
> Building and self-testing CPython 3.5-3.9 and PyPy 3.6 using a wheel of this
> pip, passed their ensurepip and venv tests during the build.
>
>
> I have now experienced some weird permissions:
>
> /usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/INSTALLER 600
> /usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/RECORD 600
> /usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/direct_url.json 600
>
>
> Will try to reproduce this outside of our build environment and will report an
> issue.

I suspect this is https://github.com/pypa/pip/issues/8139 (for which
we have a fix prepared).

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V23MZH5UOG7XE7EC4USJC37C4SCEL56Y/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Announcement: pip 20.1b1 beta release

2020-04-27 Thread Miro Hrončok

On 27. 04. 20 17:31, Paul Moore wrote:

On Mon, 27 Apr 2020 at 16:21, Miro Hrončok  wrote:


On 23. 04. 20 21:36, Sumana Harihareswara wrote:

We would be grateful for all the testing that users could do to ensure that, 
when pip 20.1 is released, it's as solid as we can make it.


We are doing some basic testing in Fedora.

So far everyhting looks good.

We've tested:

Basic CI smoke testing with venvs, virtualenvs, tox:
- CPython 3.5-3.8 (will add 3.9)
- PyPy 3.6

Bundling in Fedora and running non-network tests during build on CPython 3.8.2
and 3.9.0a5.

Building couple PEP 517/518 packages (pytest, pluggy, clickit, entyrpoints).

Building and self-testing CPython 3.5-3.9 and PyPy 3.6 using a wheel of this
pip, passed their ensurepip and venv tests during the build.


I have now experienced some weird permissions:

/usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/INSTALLER 600
/usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/RECORD 600
/usr/lib/python3.8/site-packages/pip-20.1b1.dist-info/direct_url.json 600


Will try to reproduce this outside of our build environment and will report an
issue.


I suspect this is https://github.com/pypa/pip/issues/8139 (for which
we have a fix prepared).


Yes indeed. I have opened https://github.com/pypa/pip/issues/8153 in the 
meantime (the linked issue didn't rally stand out when searching).


Closed as duplicate. This would indeed generate a lot of problems in Fedora if 
not fixed in 20.1.


Thanks!
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3VJ2433NVC7QWUYPHTRFTMYLUICAXTZX/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] test_ssl.py hangs with SSL 1.1 built with no threads

2020-04-27 Thread mig28suarez
Hello!

This is my first time posting to the group. 

I've been running builds of Python 3.7.x on CentOS Linux release 7.7 
(64bit/Intel Core 2 Duo) 
and I ran into hangs with test_ssl.py when using latest SSL 1.1.1d sources. 

I've done a full compilation from source for Python 3.7.7 and SSL 1.1 in my 
workspaces. 

>From what I can tell the problem is when SSL 1.1 is built with no threading
there is no locking enabled by python. 

This one line change will make the hangs in test_ssl.py  go away:

Index: Modules/_ssl.c
===
--- Modules/_ssl.c  (revision 70)
+++ Modules/_ssl.c  (working copy)
@@ -5875,7 +5875,7 @@
 if (!_setup_ssl_threads()) {
 return NULL;
 }
-#elif OPENSSL_VERSION_1_1 && defined(OPENSSL_THREADS)
+#elif OPENSSL_VERSION_1_1
 /* OpenSSL 1.1.0 builtin thread support is enabled */
 _ssl_locks_count++;
 #endif


There appears to be an assumption in _ssl.c and test_ssl.y that SSL 1.1 will be 
threaded
but this may not be true (as in my case). 

Appreciate any feedback. Thanks!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OLCPETITO2FHBKL36H7PPPMHKCXRUVQJ/
Code of Conduct: http://python.org/psf/codeofconduct/


python-dev@python.org

2020-04-27 Thread Victor Stinner
Copy of Brett Cannon's message:
https://discuss.python.org/t/collecting-questions-for-a-steering-council-q-a/4010

---
As part of PyCon US 2020 (*), we are planning to record a Q&A with the
Steering Council. We are using this topic to solicit questions from
the community. We plan to leave this open for about a week.

Thanks for all of those who leave a question (which adhere to the PSF
Code of Conduct )!

P.S. Do make sure to check out the various talks that have been posted
for the conference so far! And thanks in advance to Ewa, Emily, and
the whole PyCon US 2020 staff for making this Q&A possible.

(*) https://www.youtube.com/channel/UCMjMBMGt0WJQLeluw6qNJuA
---

Please ask your question at:
https://discuss.python.org/t/collecting-questions-for-a-steering-council-q-a/4010

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QWLBS7TEPLXHXHGM3MDG5DOHDNDMLNLR/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Announcement: pip 20.1b1 beta release

2020-04-27 Thread Miro Hrončok

On 27. 04. 20 17:17, Miro Hrončok wrote:


Basic CI smoke testing with venvs, virtualenvs, tox:
   - CPython 3.5-3.8 (will add 3.9)


CPython 3.9.0a5 venv, virtualenv, tox OK as well.


I've also observed that we now have the direct_url.json when we build pip, that 
leaks the builddir path:


{"archive_info": {}, "url": 
"file:///builddir/build/BUILD/pip-20.1b1/dist/pip-20.1b1-py2.py3-none-any.whl"}


I will read PEP 610, but may I safely assume that we can just rm it to have the 
previous behavior?


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O6VZGXIZQRLBSUJS5UQ2KSRUSTEOWE3S/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Adding a "call_once" decorator to functools

2020-04-27 Thread tom
Hello,
After a great discussion in python-ideas[1][2] it was suggested that I 
cross-post this proposal to python-dev to gather more comments from those who 
don't follow python-ideas.

The proposal is to add a "call_once" decorator to the functools module that, as 
the name suggests, calls a wrapped function once, caching the result and 
returning it with subsequent invocations. The rationale behind this proposal is 
that:
1. Developers are using "lru_cache" to achieve this right now, which is less 
efficient than it could be
2. Special casing "lru_cache" to account for zero arity methods isn't trivial 
and we shouldn't endorse lru_cache as a way of achieving "call_once" semantics 
3. Implementing a thread-safe (or even non-thread safe) "call_once" method is 
non-trivial
4. It complements the lru_cache and cached_property methods currently present 
in functools.

The specifics of the method would be:
1. The wrapped method is guaranteed to only be called once when called for the 
first time by concurrent threads
2. Only functions with no arguments can be wrapped, otherwise an exception is 
thrown
3. There is a C implementation to keep speed parity with lru_cache

I've included a naive implementation below (that doesn't meet any of the 
specifics listed above) to illustrate the general idea of the proposal:

```
def call_once(func):
sentinel = object()  # in case the wrapped method returns None
obj = sentinel
@functools.wraps(func)
def inner():
nonlocal obj, sentinel
if obj is sentinel:
obj = func()
return obj
return inner
```

I'd welcome any feedback on this proposal, and if the response is favourable 
I'd love to attempt to implement it.

1. 
https://mail.python.org/archives/list/python-id...@python.org/thread/5OR3LJO7LOL6SC4OOGKFIVNNH4KADBPG/#5OR3LJO7LOL6SC4OOGKFIVNNH4KADBPG
2. 
https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5CFUCM4W3Z36U3GZ6Q3XBLDEVZLNFS63/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Adding a "call_once" decorator to functools

2020-04-27 Thread Steve Dower

On 27Apr2020 2237, t...@tomforb.es wrote:

2. Special casing "lru_cache" to account for zero arity methods isn't trivial and we 
shouldn't endorse lru_cache as a way of achieving "call_once" semantics


Why not? It's a decorator, isn't it? Just make it check for number of 
arguments at decoration time and return a different object.


That way, people can decorate their functions now and get correct 
behaviour (I assume?) on 3.8 and earlier, and also a performance 
improvement on 3.9, without having to do any version checking.


This part could even be written in Python.


3. Implementing a thread-safe (or even non-thread safe) "call_once" method is 
non-trivial


Agree that this is certainly true. But presumably we should be making 
lru_cache thread safe if it isn't.



4. It complements the lru_cache and cached_property methods currently present 
in functools.


It's unfortunate that cached_property doesn't work at module level (as 
was pointed out on the other threads - thanks for linking those, BTW).


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JNIOOBGOMTNGQTSRCBDBS7WAT4H65A4P/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Adding a "call_once" decorator to functools

2020-04-27 Thread Tom Forbes
> Why not? It's a decorator, isn't it? Just make it check for number of 
> arguments at decoration time and return a different object.

It’s not that it’s impossible, but I didn’t think the current implementation 
doesn’t make it easy 
(https://github.com/python/cpython/blob/cecf049673da6a24435acd1a6a3b34472b323c97/Lib/functools.py#L771
 
).
 You’d ideally want to skip creating all these objects and special case 
`user_function` having no parameters, but then you have an issue with 
`cache_info()` being passed `cache_len()`. So maybe it’s simplest to use the 
`cache` dictionary with a single static key, but then you’re not really helping 
much, or avoiding this method altogether, which seemed pretty messy.

The C implementation seemed easier to implement - you could re-use the `cache` 
member 
(https://github.com/python/cpython/blob/cecf049673da6a24435acd1a6a3b34472b323c97/Modules/_functoolsmodule.c#L1192
 
)
 and store the result of the function call, but that also seemed sub-optimal as 
the `root` member doesn’t make much sense to be there.

At least, that was my line of thought. It basically seemed that it would be 
more trouble than it was potentially worth, and it might be better to spend my 
time on `call_once` than special-casing `lru_cache`.

> But presumably we should be making lru_cache thread safe if it isn’t.

lru_cache is indeed thread-safe but it doesn’t guarantee that the wrapped 
method is only called _once_ per unique set of arguments. It apparently just 
ensures that the internal state of the cache is not corrupted by concurrent 
accesses.

> It's unfortunate that cached_property doesn't work at module level

It is indeed, but a solution that works generally in any function defined at 
the module level or not would be good to have.

> On 27 Apr 2020, at 22:55, Steve Dower  wrote:
> 
> On 27Apr2020 2237, t...@tomforb.es wrote:
>> 2. Special casing "lru_cache" to account for zero arity methods isn't 
>> trivial and we shouldn't endorse lru_cache as a way of achieving "call_once" 
>> semantics
> 
> Why not? It's a decorator, isn't it? Just make it check for number of 
> arguments at decoration time and return a different object.
> 
> That way, people can decorate their functions now and get correct behaviour 
> (I assume?) on 3.8 and earlier, and also a performance improvement on 3.9, 
> without having to do any version checking.
> 
> This part could even be written in Python.
> 
>> 3. Implementing a thread-safe (or even non-thread safe) "call_once" method 
>> is non-trivial
> 
> Agree that this is certainly true. But presumably we should be making 
> lru_cache thread safe if it isn't.
> 
>> 4. It complements the lru_cache and cached_property methods currently 
>> present in functools.
> 
> It's unfortunate that cached_property doesn't work at module level (as was 
> pointed out on the other threads - thanks for linking those, BTW).
> 
> Cheers,
> Steve



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YE672NOPVB3AY2VKTL7JWJFLWAYMRNDK/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Adding a "call_once" decorator to functools

2020-04-27 Thread Steve Dower

On 27Apr2020 2311, Tom Forbes wrote:
Why not? It's a decorator, isn't it? Just make it check for number of 
arguments at decoration time and return a different object.


It’s not that it’s impossible, but I didn’t think the current 
implementation doesn’t make it easy 


This is the line I'd change: 
https://github.com/python/cpython/blob/cecf049673da6a24435acd1a6a3b34472b323c97/Lib/functools.py#L763


At this point, you could inspect the user_function object and choose a 
different wrapper than _lru_cache_wrapper if it takes zero arguments. 
Though you'd likely still end up with a lot of the code being replicated.


You're probably right to go for the C implementation. If the Python 
implementation is correct, then best to leave the inefficiencies there 
and improve the already-fast version.


Looking at 
https://github.com/python/cpython/blob/master/Modules/_functoolsmodule.c 
it seems the fast path for no arguments could be slightly improved, but 
it doesn't look like it'd be much. (I'm deliberately not saying how I'd 
improve it in case you want to do it anyway as a learning exercise, and 
because I could be wrong :) )


Equally hard to say how much more efficient a new API would be, so 
unless it's written already and you have benchmarks, that's probably not 
the line of reasoning to use. An argument that people regularly get this 
wrong and can't easily get it right with what's already there is most 
compelling - see the recent removeprefix/removesuffix discussions if you 
haven't.


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HTOONP2GW3WCMWHEKHOBWNGJUYGUCACS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Announcement: pip 20.1b1 beta release

2020-04-27 Thread Miro Hrončok

On 27. 04. 20 20:56, Miro Hrončok wrote:


I've also observed that we now have the direct_url.json when we build pip, that 
leaks the builddir path:


{"archive_info": {}, "url": 
"file:///builddir/build/BUILD/pip-20.1b1/dist/pip-20.1b1-py2.py3-none-any.whl"}


I will read PEP 610, but may I safely assume that we can just rm it to have the 
previous behavior?


Followup in https://discuss.python.org/t/4012

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CK43ARE6O7QARF75OHYFGN3NTPZRD2M7/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Improvement to SimpleNamespace

2020-04-27 Thread Rob Cliffe via Python-Dev
Here's another revolutionary thought:  add a new operator and associated 
dunder method (to object?)
whose meaning is *undefined*.  Its default implementation would do 
*nothing* useful (raise an error? return None?).

E.g. suppose the operator were `..`
Then in a specific class you could implement x..y to mean x['y']
and then you could write
    obj..abc..def..ghi
Still fairly concise, but warns that what is happening is not normal 
attribute lookup.


On 15/04/2020 22:34, Glenn Linderman wrote:

On 4/15/2020 12:47 PM, Ivan Pozdeev via Python-Dev wrote:
When writing a proof-of-concept implementation, however, I bumped 
into the need to distinguish which of the child objects are 
containers (thus need to be wrapped as well) and which are the leaves 
(thus need to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any 
implementation, not only a wrapper.


Do the child objects truly need to be wrapped, or just accessed?

Thanks for your comments though, they inspired a thought.

The problem with the glom syntax versus the dotted syntax is that the 
minimal syntax is bulky.


obj.abc.def.ghi versus   glom( obj, 'abc.def.ghi')

The problem with attribute syntax is the conflict with regular 
attributes, and the limitation of valid identifier characters. Glom 
probably doesn't have these problems.


So my new thought is that we need a new syntax.  The minimal existing 
syntax might be


obj._('abc.def.ghi')     or maybe   obj['abc.def.ghi']

although the latter would conflict with regular dict lookups, but obj 
isn't a regular dict, so that might not matter.



A new syntax might be:

obj.'abc.def.ghi'


In any of these syntaxes, one could enhance it to use variables in 
some spots similarly to f-strings (but from the beginning, so no 'f' 
would be required)


foo = 'def'
obj.'abc.{foo}.ghi'

would access the same item as previous examples.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3A5V62LY3DAPAREZMB7MVRYX4432NR7F/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KAAQISHADS5S5WUCYESHSCEBND7FHF2F/
Code of Conduct: http://python.org/psf/codeofconduct/