Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Paul Moore
On 16 October 2017 at 02:33, Yury Selivanov wrote: > Stage 1. A new execution context PEP to solve the problem *just for > async code*. The PEP will target Python 3.7 and completely ignore > synchronous generators and asynchronous generators. It will be based > on PEP 550 v1 (no chained lookups,

[Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
Hi, While discussions on this PEP are not over on python-ideas, I proposed this PEP directly on python-dev since I consider that my PEP already summarizes current and past proposed alternatives. python-ideas threads: * Add time.time_ns(): system clock with nanosecond resolution * Why not picosec

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Nick Coghlan
On 16 October 2017 at 18:26, Paul Moore wrote: > On 16 October 2017 at 02:33, Yury Selivanov > wrote: > > Stage 1. A new execution context PEP to solve the problem *just for > > async code*. The PEP will target Python 3.7 and completely ignore > > synchronous generators and asynchronous generat

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Paul Moore
On 16 October 2017 at 12:44, Nick Coghlan wrote: > The downside is that you'll still need to explicitly revert the decimal > context before yielding from a generator if you didn't want the context > change to "leak", but that's not a new constraint - it's one that already > exists for the thread-l

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
I read again the discussions on python-ideas and noticed that I forgot to mention the "time_ns module" idea. I also added a section to give concrete examples of the precision loss. https://github.com/python/peps/commit/a4828def403913dbae7452b4f9b9d62a0c83a278 Issues caused by precision loss -

[Python-Dev] Preprocessing the CPython Source Tree

2017-10-16 Thread Paul Ross
I have implemented a C preprocessor written in Python which gives some useful visualisations of source code, particularly macro usage: https://github.com/paulross/cpip I have been running this on the CPython source code and it occurs to me that this might be useful to the python-dev community. Fo

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Antoine Pitrou
Hi, On Mon, 16 Oct 2017 12:42:30 +0200 Victor Stinner wrote: > > ``time.time()`` returns seconds elapsed since the UNIX epoch: January > 1st, 1970. This function loses precision since May 1970 (47 years ago):: This is a funny sentence. I doubt computers (Unix or not) had nanosecond clocks in

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
2017-10-16 17:06 GMT+02:00 Antoine Pitrou : >> This PEP adds five new functions to the ``time`` module: >> >> * ``time.clock_gettime_ns(clock_id)`` >> * ``time.clock_settime_ns(clock_id, time: int)`` >> * ``time.perf_counter_ns()`` >> * ``time.monotonic_ns()`` >> * ``time.time_ns()`` > > Why not ``

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Ben Hoyt
I've read the examples you wrote here, but I'm struggling to see what the real-life use cases are for this. When would you care about *both* very long-running servers (104 days+) and nanosecond precision? I'm not saying it could never happen, but would want to see real "experience reports" of when

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Guido van Rossum
On Sun, Oct 15, 2017 at 10:26 PM, Nathaniel Smith wrote: > On Sun, Oct 15, 2017 at 10:10 PM, Guido van Rossum > wrote: > > Yes, that's what I meant by "ignoring generators". And I'd like there to > be > > a "current context" that's a per-thread MutableMapping with ContextVar > keys. > > Maybe th

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Antoine Pitrou
On Mon, 16 Oct 2017 17:23:15 +0200 Victor Stinner wrote: > 2017-10-16 17:06 GMT+02:00 Antoine Pitrou : > >> This PEP adds five new functions to the ``time`` module: > >> > >> * ``time.clock_gettime_ns(clock_id)`` > >> * ``time.clock_settime_ns(clock_id, time: int)`` > >> * ``time.perf_counter_ns()

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Guido van Rossum
On Mon, Oct 16, 2017 at 8:37 AM, Ben Hoyt wrote: > I've read the examples you wrote here, but I'm struggling to see what the > real-life use cases are for this. When would you care about *both* very > long-running servers (104 days+) and nanosecond precision? I'm not saying > it could never happe

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
2017-10-16 17:37 GMT+02:00 Ben Hoyt : > I've read the examples you wrote here, but I'm struggling to see what the > real-life use cases are for this. When would you care about *both* very > long-running servers (104 days+) and nanosecond precision? I'm not saying it > could never happen, but would

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Ben Hoyt
Got it -- fair enough. We deploy so often where I work (a couple of times a week at least) that 104 days seems like an eternity. But I can see where for a very stable file server or something you might well run it that long without deploying. Then again, why are you doing performance tuning on a "

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
2017-10-16 17:42 GMT+02:00 Antoine Pitrou : > Restricting this PEP to the time module would be fine with me. Maybe I should add a short sentence to keep the question open, but exclude it from the direct scope of the PEP? For example: "New nanosecond flavor of these functions may be added later, i

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Yury Selivanov
On Mon, Oct 16, 2017 at 11:49 AM, Guido van Rossum wrote: > On Sun, Oct 15, 2017 at 10:26 PM, Nathaniel Smith wrote: >> >> On Sun, Oct 15, 2017 at 10:10 PM, Guido van Rossum >> wrote: >> > Yes, that's what I meant by "ignoring generators". And I'd like there to >> > be >> > a "current context" t

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Antoine Pitrou
On Mon, 16 Oct 2017 18:06:06 +0200 Victor Stinner wrote: > 2017-10-16 17:42 GMT+02:00 Antoine Pitrou : > > Restricting this PEP to the time module would be fine with me. > > Maybe I should add a short sentence to keep the question open, but > exclude it from the direct scope of the PEP? For exa

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
2017-10-16 18:14 GMT+02:00 Ben Hoyt : > Got it -- fair enough. > > We deploy so often where I work (a couple of times a week at least) that 104 > days seems like an eternity. But I can see where for a very stable file > server or something you might well run it that long without deploying. Then > a

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Yury Selivanov
On Mon, Oct 16, 2017 at 7:44 AM, Nick Coghlan wrote: [..] > So going down this path would lock in the *default* semantics for the > interaction between context variables and generators as being the same as > the interaction between thread locals and generators, but would still leave > the door ope

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
2017-10-16 18:28 GMT+02:00 Antoine Pitrou : >> What do you think? > > It sounds fine to me! Ok fine, I updated the PEP. Let's start simple with the few functions (5 "clock" functions) which are "obviously" impacted by the precission loss. Victor ___ Pyt

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Ben Hoyt
Makes sense, thanks. -Ben On Mon, Oct 16, 2017 at 12:28 PM, Victor Stinner wrote: > 2017-10-16 18:14 GMT+02:00 Ben Hoyt : > > Got it -- fair enough. > > > > We deploy so often where I work (a couple of times a week at least) that > 104 > > days seems like an eternity. But I can see where for a v

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Antoine Pitrou
On Mon, 16 Oct 2017 18:53:18 +0200 Victor Stinner wrote: > 2017-10-16 18:28 GMT+02:00 Antoine Pitrou : > >> What do you think? > > > > It sounds fine to me! > > Ok fine, I updated the PEP. Let's start simple with the few functions > (5 "clock" functions) which are "obviously" impacted by the

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Guido van Rossum
On Mon, Oct 16, 2017 at 9:11 AM, Yury Selivanov wrote: > On Mon, Oct 16, 2017 at 11:49 AM, Guido van Rossum > wrote: > > On Sun, Oct 15, 2017 at 10:26 PM, Nathaniel Smith wrote: > >> We don't need it to be abstract (it's fine to have a single concrete > >> mapping type that we always use intern

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Victor Stinner
Oh, now I'm confused. I misunderstood your previous message. I understood that you changed you mind and didn't want to add process_time_ns(). Can you elaborate why you consider that time.process_time_ns() is needed, but not the nanosecond flavor of os.times() nor resource.getrusage()? These functi

Re: [Python-Dev] PEP 564: Add new time functions with nanosecond resolution

2017-10-16 Thread Antoine Pitrou
On Mon, 16 Oct 2017 19:20:44 +0200 Victor Stinner wrote: > Oh, now I'm confused. I misunderstood your previous message. I understood > that you changed you mind and didn't want to add process_time_ns(). > > Can you elaborate why you consider that time.process_time_ns() is needed, > but not the na

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Guido van Rossum
On Mon, Oct 16, 2017 at 9:53 AM, Yury Selivanov wrote: > I think we can still implement context isolation in generators in > later versions for ContextVars. In 3.7, ContextVars will only support > async tasks and threads. Using them in generators will be > *documented* as unsafe, as the context

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Ethan Furman
On 10/16/2017 09:11 AM, Yury Selivanov wrote: Question: why do we want EC objects to be mappings? I'd rather make them opaque, which will result in less code and make it more future-proof. The key arguments for keeping ContextVar abstraction: * Naturally avoids name clashes. * Allows to impl

[Python-Dev] Compiling Python-3.6.3 fails two tests test_math and test_cmath

2017-10-16 Thread Richard Hinerfeld
Compiling Python-3.6.3 on Linux fails two tests: test_math and test_cmatg running build running build_ext The following modules found by detect_modules() in setup.py, have been built by the Makefile instead, as configured by the Setup files: atexitpwd time

Re: [Python-Dev] Compiling Python-3.6.3 fails two tests test_math and test_cmath

2017-10-16 Thread Tim Peters
[Richard Hinerfeld ] > Compiling Python-3.6.3 on Linux fails two tests: test_math and test_cmatg Precisely which version of Linux? The same failure has already been reported on OpenBSD here: https://bugs.python.org/issue31630 ___ Python-Dev mailing

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Nathaniel Smith
On Mon, Oct 16, 2017 at 11:12 AM, Ethan Furman wrote: > What would be really nice is to have attribute access like thread locals. > Instead of working with individual ContextVars you grab the LocalContext and > access the vars as attributes. I don't recall reading in the PEP why this > is a bad i

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Nathaniel Smith
On Mon, Oct 16, 2017 at 8:49 AM, Guido van Rossum wrote: > On Sun, Oct 15, 2017 at 10:26 PM, Nathaniel Smith wrote: >> >> On Sun, Oct 15, 2017 at 10:10 PM, Guido van Rossum >> wrote: >> > Yes, that's what I meant by "ignoring generators". And I'd like there to >> > be >> > a "current context" th

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Ethan Furman
On 10/16/2017 05:29 PM, Nathaniel Smith wrote: On Mon, Oct 16, 2017 at 11:12 AM, Ethan Furman wrote: What would be really nice is to have attribute access like thread locals. Instead of working with individual ContextVars you grab the LocalContext and access the vars as attributes. I don't re

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Guido van Rossum
Hm. I really like the idea that you can implement and demonstrate all of ContextVar by manipulating the underlying mapping. And surely compared to the effort of implementing the HAMT itself (including all its edge cases) surely implementing the mutable mapping API should be considered recreational

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Nick Coghlan
On 17 October 2017 at 03:00, Guido van Rossum wrote: > On Mon, Oct 16, 2017 at 9:11 AM, Yury Selivanov > wrote: > >> > I agree, but I don't see how making the type a subtype (or duck type) of >> > MutableMapping prevents any of those strategies. (Maybe you were >> equating >> > MutableMapping wi

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Guido van Rossum
No, that version just defers to magic in ContextVar.get/set, whereas what I'd like to see is that the latter are just implemented in terms of manipulating the mapping directly. The only operations for which speed matters would be __getitem__ and __setitem__; most other methods just defer to those.

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Nick Coghlan
On 17 October 2017 at 14:31, Guido van Rossum wrote: > No, that version just defers to magic in ContextVar.get/set, whereas what > I'd like to see is that the latter are just implemented in terms of > manipulating the mapping directly. The only operations for which speed > matters would be __geti

Re: [Python-Dev] Timeout for PEP 550 / Execution Context discussion

2017-10-16 Thread Nick Coghlan
On 17 October 2017 at 15:02, Nick Coghlan wrote: > On 17 October 2017 at 14:31, Guido van Rossum wrote: > >> No, that version just defers to magic in ContextVar.get/set, whereas what >> I'd like to see is that the latter are just implemented in terms of >> manipulating the mapping directly. The