[Python-Dev] Re: Keeping Python a Duck Typed Language.

2021-04-20 Thread Nathaniel Smith
On Tue, Apr 20, 2021 at 10:07 AM Mark Shannon wrote: > > Hi everyone, > > Once upon a time Python was a purely duck typed language. > > Then came along abstract based classes, and some nominal typing starting > to creep into the language. > > If you guarded your code with `isinstance(foo, Sequence

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-21 Thread Nathaniel Smith
On Tue, Apr 20, 2021 at 2:15 PM srku...@mail.de wrote: > > So, forgive me my relatively simple mental model about ExceptionGroup. I > still try to create one for daily use. > > As noted in the discussion, an EG provides a way to collect exceptions from > different sources and raise them as a bun

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-21 Thread Nathaniel Smith
On Tue, Apr 20, 2021 at 3:15 AM Irit Katriel wrote: > On Tue, Apr 20, 2021 at 2:48 AM Nathaniel Smith wrote: >> >> >> The problem is that most of the time, even if you're using concurrency >> internally so multiple things *could* go wrong at once, only one thing

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-23 Thread Nathaniel Smith
On Wed, Apr 21, 2021 at 4:50 PM Guido van Rossum wrote: > On Wed, Apr 21, 2021 at 3:26 PM Nathaniel Smith wrote: >> Sure. This was in my list of reasons why the backwards compatibility >> tradeoffs are forcing us into awkward compromises. I only elaborated >> on it b/c i

[Python-Dev] Anyone else gotten bizarre personal replies to mailing list posts?

2021-04-23 Thread Nathaniel Smith
nto? Or... something? -- Forwarded message - From: Hoi lam Poon Date: Fri, Apr 23, 2021, 02:01 Subject: Re: [Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST] To: Nathaniel Smith Stop pretending, I can definitely get the key control file, your working group, all past acti

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-25 Thread Nathaniel Smith
On Fri, Apr 23, 2021 at 2:45 AM Chris Angelico wrote: > > On Fri, Apr 23, 2021 at 6:25 PM Nathaniel Smith wrote: > > The main possibility that I don't think we've examined fully is to > > make 'except' blocks fire multiple times when there are multiple >

[Python-Dev] Re: Existing asyncio features scheduled for removal in Python 3.9 and 3.10

2021-04-26 Thread Nathaniel Smith
@asyncio.coroutine and @types.coroutine are different beasts. @asyncio.coroutine is the deprecated one; @types.coroutine is lower-level and not deprecated. On Mon, Apr 26, 2021 at 2:48 PM Luciano Ramalho wrote: > > I don't understand how it's possible to "Deprecate @coroutine for sake > of async

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-28 Thread Nathaniel Smith
On Fri, Apr 23, 2021 at 4:08 AM Irit Katriel wrote: > > On Fri, Apr 23, 2021 at 9:22 AM Nathaniel Smith wrote: >> I'm not trying to filibuster here -- I really want some form of EGs to >> land. > > I'm very glad to hear that. It's been hard to know wher

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-05-06 Thread Nathaniel Smith
On Thu, Apr 29, 2021 at 9:14 AM Yury Selivanov wrote: > Nathaniel, at this point it's clear that this thread somehow does not > help us understand what you want. Could you please just write your own > PEP clearly outlining your proposal, its upsides and downsides? > Without a PEP from you this thr

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-05-06 Thread Nathaniel Smith
On Thu, May 6, 2021 at 2:17 AM Nathaniel Smith wrote: > > On Thu, Apr 29, 2021 at 9:14 AM Yury Selivanov > wrote: > > Nathaniel, at this point it's clear that this thread somehow does not > > help us understand what you want. Could you please just write your own >

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Nathaniel Smith
On Fri, May 7, 2021 at 8:14 PM Neil Schemenauer wrote: > > On 2021-05-07, Pablo Galindo Salgado wrote: > > Technically the main concern may be the size of the unmarshalled > > pyc files in memory, more than the storage size of disk. > > It would be cool if we could mmap the pyc files and have the

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Nathaniel Smith
On Mon, May 17, 2021 at 6:18 AM Mark Shannon wrote: > 2. Repeated binary operations on the same line. > > A single location can also be clearer when all the code is on one line. > > i1 + i2 + s1 > > PEP 657: > > i1 + i2 + s1 > > > Using a single location: > > i1 + i2 + s1 > ^

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-19 Thread Nathaniel Smith
On Tue, May 18, 2021 at 2:49 PM Pablo Galindo Salgado wrote: > * It actually doesn't have more advantages. The current solution in the PEP > can do exactly the same as this solution if you allow reparsing when > displaying tracebacks. This is because with the start line, end line, start > offset

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-20 Thread Nathaniel Smith
On Wed, May 19, 2021 at 7:28 PM Pablo Galindo Salgado wrote: >> >> Excellent point! Do you know how reliable this is in practice, i.e. >> what proportion of bytecode source spans are something you can >> successfully pass to ast.parse? If it works it's obviously nicer, but >> I can't tell how ofte

[Python-Dev] Re: Roundup to GitHub Issues migration

2021-06-22 Thread Nathaniel Smith
On Tue, Jun 22, 2021 at 4:42 AM Sebastian Rittau wrote: > > Am 22.06.21 um 10:00 schrieb Tiziano Zito: > > I think it is important to notice that GitHub actively blocks user > > registration and activity from countries that are sanctioned by the US > > government. At least in 2019 GitHub was block

[Python-Dev] Re: PEP 558, the simplest thing I could come up with

2021-07-29 Thread Nathaniel Smith
On Thu, Jul 29, 2021 at 4:52 PM Nick Coghlan wrote: > > On Fri, 30 Jul 2021, 6:05 am Mark Shannon, wrote: >> >> Hi Nick, >> >> Our discussion on PEP 558 got me thinking >> "What is the simplest thing that would work?". >> >> This is what I came up (in the form of a draft PEP): >> https://github.c

[Python-Dev] Re: python-dev thread w/ Marco Sulla

2021-08-16 Thread Nathaniel Smith
Was this post intended to go to python-dev or...? On Mon, Aug 16, 2021 at 9:53 AM Brett Cannon wrote: > > https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/ > > I can't be objective with Marco as I believe we have recorded issues with him > previ

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-25 Thread Nathaniel Smith
On Sat, Sep 25, 2021 at 5:40 PM wrote: > PyPI packages and wheels are targeted to specific Python versions, which > means that any project that depends on some of the larger extension packages > (of which there are many, and many of which are must-have for many projects) > now start lagging Pyt

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-27 Thread Nathaniel Smith
On Sun, Sep 26, 2021 at 3:38 AM wrote: > Open3D is an example. Will finally move to Python 3.9 some time the coming > month. Its dependency graph contains about 70 other packages. > > In this specific case, the underlying problem was that TensorFlow was stuck > at 3.8. The TensorFlow codebase go

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-28 Thread Nathaniel Smith
On Tue, Sep 28, 2021 at 12:40 AM Guido van Rossum wrote: > > What I have heard repeatedly, from people who are paid to know, is that most > users don’t care about the latest features, and would rather stick to a > release until it becomes unsupported. (Extreme example: Python 2.) > > Numpy isn’t

[Python-Dev] Re: Python multithreading without the GIL

2021-10-08 Thread Nathaniel Smith
On Thu, Oct 7, 2021 at 7:54 PM Sam Gross wrote: > Design overview: > https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit Whoa, this is impressive work. I notice the fb.com address -- is this a personal project or something facebook is working on? what's the relat

Re: [Python-Dev] C99

2016-06-07 Thread Nathaniel Smith
On Tue, Jun 7, 2016 at 12:37 PM, Sturla Molden wrote: > Victor Stinner wrote: > >> Is it worth to support a compiler that in 2016 doesn't support the C >> standard released in 1999, 17 years ago? > > MSVC only supports C99 when its needed for C++11 or some MS extension to C. > > Is it worth suppo

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Nathaniel Smith
On Thu, Jun 9, 2016 at 3:22 PM, Larry Hastings wrote: > > On 06/09/2016 08:52 AM, Guido van Rossum wrote: > >> That leaves direct calls to os.urandom(). I don't think this should block >> either. > > > Then it's you and me against the rest of the world ;-) > > > Okay, it's decided: os.urandom() mu

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Nathaniel Smith
On Thu, Jun 9, 2016 at 6:53 PM, Barry Warsaw wrote: > On Jun 09, 2016, at 03:22 PM, Larry Hastings wrote: > >>On 06/09/2016 08:52 AM, Guido van Rossum wrote: >>> That leaves direct calls to os.urandom(). I don't think this should > block >>> either. >> >>Then it's you and me against the rest of t

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Nathaniel Smith
On Thu, Jun 9, 2016 at 8:11 PM, Larry Hastings wrote: > > On 06/09/2016 07:58 PM, Nathaniel Smith wrote: > > I suspect the crypto folks would be okay with pushing this back to > 3.6, so long as the final resolution is that os.urandom remains the > standard interface for, as

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-12 Thread Nathaniel Smith
On Jun 11, 2016 11:13 PM, "Theodore Ts'o" wrote: > > On Sat, Jun 11, 2016 at 05:46:29PM -0400, Donald Stufft wrote: > > > > It was a RaspberryPI that ran a shell script on boot that called > > ssh-keygen. That shell script could have just as easily been a > > Python script that called os.urandom

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-12 Thread Nathaniel Smith
On Sun, Jun 12, 2016 at 4:28 PM, Theodore Ts'o wrote: > P.S. BTW, I probably won't change the behaviour of /dev/urandom to > make it be blocking. Before I found out about Pyhton Bug #26839, I > actually had patches that did make /dev/urandom blocking, and they > were planned to for the next kern

Re: [Python-Dev] PEP 468

2016-06-13 Thread Nathaniel Smith
On Jun 13, 2016 6:16 PM, "MRAB" wrote: > > On 2016-06-14 01:47, Larry Hastings wrote: >> >> On 06/13/2016 05:05 PM, MRAB wrote: >>> >>> This could be avoided by expanding the items to include the index of >>> the 'previous' and 'next' item, so that they could be handled like a >>> doubly-linked li

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-15 Thread Nathaniel Smith
On Wed, Jun 15, 2016 at 1:01 PM, Nick Coghlan wrote: [...] > For 3.6+, we can instead make it so that the only things that actually > rely on cryptographic quality randomness being available are: > > - calling a secrets module API > - calling a random.SystemRandom method > - calling os.urandom dir

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-16 Thread Nathaniel Smith
On Wed, Jun 15, 2016 at 10:25 PM, Theodore Ts'o wrote: > On Wed, Jun 15, 2016 at 04:12:57PM -0700, Nathaniel Smith wrote: >> - It's not exactly true that the Python interpreter doesn't need >> cryptographic randomness to initialize SipHash -- it's more tha

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-16 Thread Nathaniel Smith
On Wed, Jun 15, 2016 at 11:45 PM, Barry Warsaw wrote: > On Jun 15, 2016, at 01:01 PM, Nick Coghlan wrote: > >>No, this is a bad idea. Asking novice developers to make security >>decisions they're not yet qualified to make when it's genuinely >>possible for us to do the right thing by default is th

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-16 Thread Nathaniel Smith
On Jun 16, 2016 1:23 AM, "Stefan Krah" wrote: > > Nathaniel Smith pobox.com> writes: > > In practice, your proposal means that ~all existing code that uses > > os.urandom becomes incorrect and should be switched to either secrets > > or random. This is *far*

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-16 Thread Nathaniel Smith
On Jun 16, 2016 10:01 AM, "David Mertz" wrote: > Python 3.6 is introducing a NEW MODULE, with new APIs. The 'secrets' module is the very first time that Python has ever really explicitly addressed cryptography in the standard library. This is completely, objectively untrue. If you look up os.ura

Re: [Python-Dev] Compact ordered dict is not ordered for split table. (was: PEP XXX: Compact ordered dict

2016-06-23 Thread Nathaniel Smith
On Tue, Jun 21, 2016 at 8:40 PM, INADA Naoki wrote: > There are three options I can think. > > > 1) Revert key-shared dict (PEP412). > > pros: Removing key-shared dict makes dict implementation simple. > > cons: In some applications, PEP 412 is far more compact than compact > ordered dict. (Note:

Re: [Python-Dev] release cadence (was: Request for CPython 3.5.3 release)

2016-07-03 Thread Nathaniel Smith
On Jul 3, 2016 1:45 PM, "Paul Moore" wrote: > [...] > Furthermore, pip/setuptools are just getting to the point of allowing > for dependencies conditional on Python version. If independent stdlib > releases were introduced, we'd need to implement dependencies based on > stdlib version as well - co

Re: [Python-Dev] PEP 525, third round, better finalization

2016-09-02 Thread Nathaniel Smith
On Thu, Sep 1, 2016 at 3:34 PM, Yury Selivanov wrote: > Hi, > > I've spent quite a while thinking and experimenting with PEP 525 trying to > figure out how to make asynchronous generators (AG) finalization reliable. > I've tried to replace the callback for GCed with a callback to intercept > first

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-10 Thread Nathaniel Smith
On Fri, Sep 9, 2016 at 11:39 AM, Barry Warsaw wrote: > On Sep 09, 2016, at 01:08 PM, Elvis Pranskevichus wrote: > >>Are there any downsides to explicitly specifying that all dicts are ordered? >>People will inevitably start relying on this behaviour, and this will >>essentially become the *de-fact

Re: [Python-Dev] C99

2016-10-06 Thread Nathaniel Smith
On Fri, Aug 5, 2016 at 8:02 PM, Nick Coghlan wrote: > On 6 August 2016 at 12:15, Steve Dower wrote: >> FYI, it's MSVC 14.0 (which is included in VS 2015). >> >> Personally I'd like to see it restricted to the common subset of C99 and >> some version of C++ (which is obviously mostly C and include

Re: [Python-Dev] C99

2016-10-06 Thread Nathaniel Smith
On Thu, Oct 6, 2016 at 10:10 PM, Nathaniel Smith wrote: [...] > And in practice this almost certainly doesn't matter -- the only > reason people jump through hoops to get gcc 4.8 is for its improved > C++ support. I just tried my c99 test file on CentOS 5's default gcc > 4.

Re: [Python-Dev] C99

2016-10-08 Thread Nathaniel Smith
On Sat, Oct 8, 2016 at 9:23 AM, Steven D'Aprano wrote: > On Thu, Oct 06, 2016 at 10:10:54PM -0700, Nathaniel Smith wrote: > >> The reason this matters is that the official vendor compiler on RHEL 5 >> is gcc 4.1, but there's also a separately-distributed version of gcc

Re: [Python-Dev] PyWeakref_GetObject() borrows its reference from... whom?

2016-10-10 Thread Nathaniel Smith
On Mon, Oct 10, 2016 at 10:03 AM, Paul Moore wrote: > On 10 October 2016 at 17:49, MRAB wrote: >> If you lookup something in a dict, it'll be a borrowed reference. >> >> If the dict is globals() and there's no GIL, another thread could delete the >> item while your code had the borrowed reference

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-10 Thread Nathaniel Smith
On Mon, Oct 10, 2016 at 1:42 PM, Elliot Gorokhovsky wrote: > *** 10 strings *** > F.fastsort(): 1.6689300537109375e-06 > F.sort(): 1.6689300537109375e-06 I think something has gone wrong with your timing harness... For accurately timing microbenchmarks, you should use timeit, or better yet Victo

Re: [Python-Dev] Optimizing list.sort() by checking type in advance

2016-10-10 Thread Nathaniel Smith
On Mon, Oct 10, 2016 at 2:16 PM, Elliot Gorokhovsky wrote: > Hm... that is strange, but I don't think there's anything wrong with the way > I'm timing, though I agree perf/timeit would be better. I ran the benchmark > a couple of times and the numbers seem to exactly line up something like one > i

Re: [Python-Dev] PyWeakref_GetObject() borrows its reference from... whom?

2016-10-10 Thread Nathaniel Smith
On Mon, Oct 10, 2016 at 3:27 PM, Greg Ewing wrote: > Nathaniel Smith wrote: >> >> IIRC to handle >> this gilectomy adds per-object mutexes that you have to hold whenever >> you're mucking around with that object's internals. > > > What counts

Re: [Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

2016-10-11 Thread Nathaniel Smith
On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki wrote: > From Python 3.4, bytearray is good solution for I/O buffer, thanks to > #19087 [1]. > Actually, asyncio uses bytearray as I/O buffer often. Whoa what?! This is awesome, I had no idea that bytearray had O(1) deletes at the front. I literally re

[Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-11 Thread Nathaniel Smith
On Tue, Oct 11, 2016 at 10:53 PM, Serhiy Storchaka wrote: > On 12.10.16 08:03, Nathaniel Smith wrote: >> >> On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki >> wrote: >>> >>> From Python 3.4, bytearray is good solution for I/O buffer, thanks to >>> #19

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Nathaniel Smith
On Wed, Oct 12, 2016 at 12:17 AM, Serhiy Storchaka wrote: > On 12.10.16 09:31, Nathaniel Smith wrote: >> >> But amortized O(1) deletes from the front of bytearray are totally >> different, and more like amortized O(1) appends to list: there are >> important use cas

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Nathaniel Smith
On Wed, Oct 12, 2016 at 4:55 AM, Victor Stinner wrote: > 2016-10-12 10:01 GMT+02:00 Nathaniel Smith : >> It's more complicated than that -- the right algorithm is the one that >> Antoine implemented in 3.4. >> (...) >> My point is that >> forcing everyone w

Re: [Python-Dev] O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

2016-10-12 Thread Nathaniel Smith
On Wed, Oct 12, 2016 at 3:28 AM, INADA Naoki wrote: > When Tornado drop Python 2.7 support, they can use bytearray, and > iostream can be more simple and fast. FYI 2.7 does have bytearray. (You still have to implement the O(1) deletion part as a layer on top, like Victor points out, but I suspect

[Python-Dev] Is there any remaining reason why weakref callbacks shouldn't be able to access the referenced object?

2016-10-21 Thread Nathaniel Smith
Hi all, It's an old feature of the weakref API that you can define an arbitrary callback to be invoked when the referenced object dies, and that when this callback is invoked, it gets handed the weakref wrapper object -- BUT, only after it's been cleared, so that the callback can't access the orig

Re: [Python-Dev] Is there any remaining reason why weakref callbacks shouldn't be able to access the referenced object?

2016-10-21 Thread Nathaniel Smith
On Fri, Oct 21, 2016 at 8:32 PM, Nick Coghlan wrote: > On 21 October 2016 at 17:09, Nathaniel Smith wrote: >> But that was 2.4. In the mean time, of course, PEP 442 fixed it so >> that finalizers and weakrefs mix just fine. In fact, weakref callbacks >> are now run *befor

Re: [Python-Dev] Is there any remaining reason why weakref callbacks shouldn't be able to access the referenced object?

2016-10-22 Thread Nathaniel Smith
On Sat, Oct 22, 2016 at 3:01 AM, Nick Coghlan wrote: > On 22 October 2016 at 16:05, Nathaniel Smith wrote: >> On Fri, Oct 21, 2016 at 8:32 PM, Nick Coghlan wrote: >> But PEP 442 already broke all that :-). Now weakref callbacks can >> happen before __del__, and they can h

Re: [Python-Dev] Recent changes to PyCodeObject

2016-11-16 Thread Nathaniel Smith
Didn't 3.5 have to roll an extra last minute RC for an emergency abi-breaking bug fix, though? (Thinking of the windows runtime stuff.) On Nov 16, 2016 7:51 PM, "Nick Coghlan" wrote: > On 17 November 2016 at 10:44, Steve Dower wrote: > > On 16Nov2016 1618, Ned Batchelder wrote: > >> Am I doing

Re: [Python-Dev] Issue #23903 - stable API is incomplete

2016-12-21 Thread Nathaniel Smith
On Dec 21, 2016 7:43 AM, "Steve Dower" wrote: "Ok, now why should _Py_PrintReferences() function be exported?" It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is excluded from the headers (my list was automatically generated). It sounds like the opt-out approach isn't

[Python-Dev] --with-fpectl changes the CPython ABI

2016-12-24 Thread Nathaniel Smith
Hi all, Well, we finally got that ucs2/ucs4 stuff all sorted out (yay), but I just learned that there's another CPython build flag that also changes the ABI: --with-fpectl Specifically, it seems that if you build CPython with --with-fpectl, and then use that CPython to build an extension module,

Re: [Python-Dev] --with-fpectl changes the CPython ABI

2017-01-02 Thread Nathaniel Smith
On Sun, Dec 25, 2016 at 5:55 PM, Nick Coghlan wrote: > On 25 December 2016 at 09:48, Nathaniel Smith wrote: >> >> Or maybe make it so that even no-fpectl builds still export the >> necessary symbols so that yes-fpectl extensions don't crash on import? >> (This

Re: [Python-Dev] --with-fpectl changes the CPython ABI

2017-01-02 Thread Nathaniel Smith
e... > > On Mon, Jan 2, 2017 at 4:22 AM, Nick Coghlan wrote: >> >> On 2 January 2017 at 18:27, Nathaniel Smith wrote: >>> >>> So I guess that yeah, my suggestion would be to drop this feature >>> entirely in 3.7, given that it's never been enabl

Re: [Python-Dev] Can we use "designated initializer" widely in coremodules?

2017-01-17 Thread Nathaniel Smith
On Tue, Jan 17, 2017 at 4:48 PM, INADA Naoki wrote: > On Wed, Jan 18, 2017 at 8:48 AM, Larry Hastings wrote: >> >> On 01/17/2017 12:02 PM, Steve Dower wrote: >> >> Avoiding header files would be my only request. As Brett says, the C99 >> requirement should not be enforced on all embedders or exte

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

2017-01-24 Thread Nathaniel Smith
On Jan 24, 2017 3:35 AM, "Thomas Wouters" wrote: On Fri, Jan 20, 2017 at 1:40 PM, Christian Heimes wrote: > On 2017-01-20 13:15, INADA Naoki wrote: > >> > >> "this script counts static memory usage. It doesn’t care about dynamic > >> memory usage of processing real request" > >> > >> You may

Re: [Python-Dev] Generator objects and list comprehensions?

2017-01-25 Thread Nathaniel Smith
On Jan 25, 2017 8:16 AM, "Joe Jevnik" wrote: List, set, and dict comprehensions compile like: # input result = [expression for lhs in iterator_expression] # output def comprehension(iterator): out = [] for lhs in iterator: out.append(expression) return out result = comprehe

Re: [Python-Dev] API design: where to add async variants of existing stdlib APIs?

2017-02-28 Thread Nathaniel Smith
On Tue, Feb 28, 2017 at 9:42 PM, Nick Coghlan wrote: > Short version: > > - there are some reasonable requests for async variants of contextlib APIs > for 3.7 > - prompted by Raymond, I'm thinking it actually makes more sense to add > these in a new `asyncio.contextlib` module than it does to add

Re: [Python-Dev] API design: where to add async variants of existing stdlib APIs?

2017-03-07 Thread Nathaniel Smith
On Tue, Mar 7, 2017 at 9:41 AM, Brett Cannon wrote: > I don't think a common practice has bubbled up yet for when there's both > synchronous and asynchronous versions of an API (closest I have seen is > appending an "a" to the async version but that just looks like a spelling > mistake to me most

Re: [Python-Dev] __del__ is not called after creating a new reference

2017-03-20 Thread Nathaniel Smith
On Mar 20, 2017 1:26 PM, "Antoine Pitrou" wrote: Hello Oleg, On Mon, 20 Mar 2017 18:28:29 +0100 Oleg Nesterov wrote: > I started to learn python a few days ago and I am trying to understand what > __del__() actually does. https://docs.python.org/3/ reference/datamodel.html > says: > > ob

Re: [Python-Dev] What version is an extension module binary compatible with

2017-03-28 Thread Nathaniel Smith
On Mar 28, 2017 8:29 AM, "Serhiy Storchaka" wrote: On 28.03.17 14:24, Miro Hrončok wrote: > However, recently we found an issue with this approach [1]: an extension > module built against Python 3.6.1 cannot be run on Python 3.6.0, because > it uses a macro that, in 3.6.1, uses the new PySlice_A

Re: [Python-Dev] What version is an extension module binary compatible with

2017-03-28 Thread Nathaniel Smith
On Mar 28, 2017 10:54 AM, "Steve Dower" wrote: On 28Mar2017 1035, Paul Moore wrote: > On 28 March 2017 at 18:05, Glenn Linderman wrote: > >> Somewhere I got the idea that extension authors were supposed to build >> against the n.m.0 releases, expressly so that the extensions would then be >> co

Re: [Python-Dev] What version is an extension module binary compatible with

2017-03-29 Thread Nathaniel Smith
On Wed, Mar 29, 2017 at 8:22 AM, Paul Moore wrote: > On 28 March 2017 at 17:31, Nathaniel Smith wrote: >> IMO this is a bug, and depending on how many packages are affected it might >> even call for an emergency 3.6.2. The worst case is that we start getting >> large numbers

Re: [Python-Dev] why _PyGen_Finalize(gen) propagates close() to _PyGen_yf() ?

2017-03-30 Thread Nathaniel Smith
On Thu, Mar 30, 2017 at 11:05 AM, Oleg Nesterov wrote: > On 03/28, Eric Snow wrote: >> >> On Mon, Mar 20, 2017 at 11:30 AM, Oleg Nesterov wrote: >> > Hello, >> > >> > Let me first clarify, I do not claim this is a bug, I am trying to learn >> > python and now I trying to understand yield-from. >>

Re: [Python-Dev] windows installer and python list mention

2017-04-10 Thread Nathaniel Smith
On Mon, Apr 10, 2017 at 10:32 AM, Ethan Furman wrote: > Some people find it easier to follow this and other lists via gmane > (http://news.gmane.org/gmane.comp.python.general), a service which > offers a newsgroup interface to many online mailing lists. Also, gmane has been dead for a few months

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

2017-05-23 Thread Nathaniel Smith
Well, it sounds like you have an answer to the question of whether the cpython developers are interested in making official snap releases, but of course this doesn't stop you making unofficial snap releases, and simple standalone python distributions can be pretty handy. I just wanted to point out

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Nathaniel Smith
On Tue, May 23, 2017 at 8:34 PM, David Wilson wrote: > On Tue, May 23, 2017 at 06:13:17PM -0700, Cory Benfield wrote: > >> There are discussions around Requests unvendoring its dependencies >> thanks to the improved nature of pip. This might be a year of pretty >> big changes for Requests. > > In

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Nathaniel Smith
On Jun 1, 2017 9:20 AM, "Chris Angelico" wrote: On Fri, Jun 2, 2017 at 1:01 AM, Cory Benfield wrote: > The answer to that is honestly not clear to me. I chatted with the pip developers, and they have 90%+ of their users currently on Python 2, but more than half of those are on 2.7.9 or later. Th

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nathaniel Smith
On Jun 2, 2017 7:24 AM, "Ben Darnell" wrote: The PEP's rationale is now "This PEP will help facilitate the future adoption of :pep:`543` across all supported Python versions, which will improve security for both Python 2 and Python 3 users." What exactly are these security improvements? My under

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nathaniel Smith
On Fri, Jun 2, 2017 at 1:29 PM, Terry Reedy wrote: > On 6/2/2017 12:21 PM, Barry Warsaw wrote: >> >> On Jun 03, 2017, at 02:10 AM, Nick Coghlan wrote: > > >>> The benefit of making any backport a private API is that it would mean >>> we weren't committing to support that API for general use: it wo

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-05 Thread Nathaniel Smith
On Jun 5, 2017 7:01 AM, "Nick Coghlan" wrote: On 5 June 2017 at 21:44, Donald Stufft wrote: > Is pip allowed to use the hypothetical _ensurepip_ssl outside of ensurepip? Yes, so something like _tls_bootstrap would probably be a better name for the helper module (assuming we go down the "private

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-06 Thread Nathaniel Smith
On Mon, Jun 5, 2017 at 8:49 PM, Nick Coghlan wrote: > The reason this kind of approach is really attractive to > redistributors from a customer risk management perspective is that > like gevent's monkeypatching of synchronous networking APIs, it's > *opt-in at runtime*, so the risk of our accident

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-07 Thread Nathaniel Smith
On Tue, Jun 6, 2017 at 10:49 AM, Jim Baker wrote: > With Nick's suggestion of _tls_bootstrap, this has certainly become more > complicated. I'm still thinking of the ramifications for future Jython 2.7 > support, if Python dev goes down this path. It still seems easier to me to > avoid exposing th

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-07 Thread Nathaniel Smith
On Jun 7, 2017 6:29 AM, "Victor Stinner" wrote: 2017-06-07 10:56 GMT+02:00 Nathaniel Smith : > Another testing challenge is that the stdlib ssl module has no way to > trigger a renegotiation, and therefore there's no way to write tests > to check that it properly handl

Re: [Python-Dev] PEP 538 warning at startup: please remove it

2017-06-12 Thread Nathaniel Smith
On Jun 12, 2017 10:50, "Gregory P. Smith" wrote: The problem, as with all warnings, is that it isn't the user who has control over the problem who sees the warning. It is the end use of an application on a system that sees it. I don't think I understand the distinction you're making here. This

Re: [Python-Dev] Handle errors in cleanup code

2017-06-12 Thread Nathaniel Smith
On Mon, Jun 12, 2017 at 6:29 AM, Stefan Ring wrote: > > > Yury in the comment for PR 2108 [1] suggested more complicated code: > > > > do_something() > > try: > > do_something_other() > > except BaseException as ex: > > try: > > undo_something() > >

Re: [Python-Dev] Handle errors in cleanup code

2017-06-12 Thread Nathaniel Smith
On Mon, Jun 12, 2017 at 1:07 AM, Nick Coghlan wrote: > Aye, agreed. The key challenge we have is that we're trying to > represent the exception state as a linked list, when what we really > have once we start taking cleanup errors into account is an exception > *tree*. [...] > P.S. trio's MultiErr

Re: [Python-Dev] Handle errors in cleanup code

2017-06-13 Thread Nathaniel Smith
On Tue, Jun 13, 2017 at 12:10 AM, Nick Coghlan wrote: > > reporting failures from concurrent.futures.wait: > https://pythonhosted.org/futures/#concurrent.futures.wait Yeah, and asyncio.gather is another example in the stdlib. Or there's twisted's DeferredList. Trio is unusual in effectively for

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-25 Thread Nathaniel Smith
On Jun 25, 2017 08:12, "Jakub Wilk" wrote: * Paul Sokolovsky , 2017-06-25, 11:47: A GitHub PR is just a git branch (in somebody else's repository, but also > in the repository it's submitted to). So, like any git branch, you can > fetch it, re-branch to your local branch, apply any changes to it

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-25 Thread Nathaniel Smith
On Jun 25, 2017 10:27, "Larry Hastings" wrote: On 06/25/2017 10:02 AM, Nathaniel Smith wrote: My dudes, in a previous life I helped invent distributed VCS, but I still get confused by fiddly git BS just like everyone else. Really? I thought Bitkeeper was out before the monoto

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Nathaniel Smith
On Jul 17, 2017 5:28 PM, "Steven D'Aprano" wrote: On Mon, Jul 17, 2017 at 09:31:20PM +, Brett Cannon wrote: > As for removing exec() as a goal, I'll back up Christian's point and the > one Steve made at the language summit that removing the use of exec() from > the critical path in Python is

Re: [Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Nathaniel Smith
I'd probably start with a regular C-level profiler, like perf or callgrind. They're not very useful for comparing two versions of code written in Python, but here the Python code is the same (modulo changes in the stdlib), and it's changes in the interpreter's C code that probably make the differen

Re: [Python-Dev] startup time repeated? why not daemon

2017-07-20 Thread Nathaniel Smith
On Jul 20, 2017 14:18, "Eric Snow" wrote: On Thu, Jul 20, 2017 at 11:53 AM, Jim J. Jewett wrote: > I agree that startup time is a problem, but I wonder if some of the pain > could be mitigated by using a persistent process. > > [snip] > > Is it too hard to create a daemon server? > Is the commun

Re: [Python-Dev] Interrupt thread.join() with Ctrl-C / KeyboardInterrupt on Windows

2017-08-08 Thread Nathaniel Smith
On Tue, Aug 8, 2017 at 2:54 AM, Jonathan Slenders wrote: > Hi all, > > Is it possible that thread.join() cannot be interrupted on Windows, while it > can be on Linux? > Would this be a bug, or is it by design? > > > import threading, time > def wait(): > time.sleep(1000) > t = threading.Thread

Re: [Python-Dev] Interrupt thread.join() with Ctrl-C / KeyboardInterrupt on Windows

2017-08-08 Thread Nathaniel Smith
On Tue, Aug 8, 2017 at 2:29 PM, Steve Dower wrote: > On 08Aug2017 1151, Nathaniel Smith wrote: >> >> It looks like Thread.join ultimately ends up blocking in >> Python/thread_nt.h:EnterNonRecursiveMutex, which has a maze of #ifdefs >> behind it -- I think there are 3 d

Re: [Python-Dev] PEP 550 v3 naming

2017-08-22 Thread Nathaniel Smith
On Tue, Aug 22, 2017 at 8:22 AM, Guido van Rossum wrote: > As I understand the key APIs and constraints of the proposal better, I'm > leaning towards FooContext (LC) and FooContextStack (EC), for some value of > Foo that I haven't determined yet. Perhaps the latter can be shortened to > just Conte

Re: [Python-Dev] PEP 550 v3 naming

2017-08-22 Thread Nathaniel Smith
On Tue, Aug 22, 2017 at 8:51 PM, Guido van Rossum wrote: > On Tue, Aug 22, 2017 at 7:12 PM, Nathaniel Smith wrote: >> >> We could do worse than just plain Context and ContextStack, for that >> matter. > > > I worry that that's going to lead more people astray

Re: [Python-Dev] PEP 550 leak-in vs leak-out, why not just a ChainMap

2017-08-24 Thread Nathaniel Smith
On Wed, Aug 23, 2017 at 9:32 PM, Jim J. Jewett wrote: >> While the context is defined conceptually as a nested chain of >> key:value mappings, we avoid using the mapping syntax because of the >> way the values can shift dynamically out from under you based on who >> called you > ... >> instead of

Re: [Python-Dev] PEP 550 v3 naming

2017-08-24 Thread Nathaniel Smith
On Thu, Aug 24, 2017 at 1:22 AM, Nick Coghlan wrote: > On 24 August 2017 at 02:19, Yury Selivanov wrote: >> I think that "implicit context" is not an accurate description of what >> LogicalContext is. >> >> "implicit context" only makes sense when we talk about decimal >> context. For instance,

Re: [Python-Dev] PEP 550 v4

2017-08-25 Thread Nathaniel Smith
On Fri, Aug 25, 2017 at 3:32 PM, Yury Selivanov wrote: > Coroutines and Asynchronous Tasks > - > > In coroutines, like in generators, context variable changes are local > and are not visible to the caller:: > > import asyncio > > var = new_context_var() > >

Re: [Python-Dev] PEP 550 v4

2017-08-26 Thread Nathaniel Smith
On Sat, Aug 26, 2017 at 10:25 AM, Eric Snow wrote: > With threads we have a directed graph of execution, rooted at the root > thread, branching with each new thread and merging with each .join(). > Each thread gets its own copy of each threading.local, regardless of > the relationship between bran

Re: [Python-Dev] PEP 550 v4

2017-08-26 Thread Nathaniel Smith
On Sat, Aug 26, 2017 at 7:58 AM, Elvis Pranskevichus wrote: > On Saturday, August 26, 2017 2:34:29 AM EDT Nathaniel Smith wrote: >> On Fri, Aug 25, 2017 at 3:32 PM, Yury Selivanov > wrote: >> > Coroutines and Asynchronous Tasks >> > -

Re: [Python-Dev] Pep 550 and None/masking

2017-08-27 Thread Nathaniel Smith
I believe that the current status is: - assigning None isn't treated specially – it does mask any underlying values (which I think is what we want) - there is currently no way to "unmask" - but it's generally agreed that there should be a way to do that, at least in some cases, to handle the sav

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Nathaniel Smith
On Mon, Aug 28, 2017 at 3:14 PM, Eric Snow wrote: > On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith wrote: >> You might be interested in these notes I wrote to motivate why we need >> a chain of namespaces, and why simple "async task locals" aren't >> suff

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Nathaniel Smith
On Tue, Aug 29, 2017 at 12:32 PM, Antoine Pitrou wrote: > > > Le 29/08/2017 à 21:18, Yury Selivanov a écrit : >> On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou wrote: >>> On Mon, 28 Aug 2017 17:24:29 -0400 >>> Yury Selivanov wrote: Long story short, I think we need to rollback our last dec

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Nathaniel Smith
On Tue, Aug 29, 2017 at 12:59 PM, Yury Selivanov wrote: > b2 = wait_for(bar()) > # bar() was wrapped into a Task and is being running right now > await b2 Ah not quite. wait_for is itself implemented as a coroutine, so it doesn't spawn off bar() into its own task until you await b

Re: [Python-Dev] To reduce Python "application" startup time

2017-09-05 Thread Nathaniel Smith
On Tue, Sep 5, 2017 at 11:13 AM, Jelle Zijlstra wrote: > > > 2017-09-05 6:02 GMT-07:00 INADA Naoki : >> With this profile, I tried optimize `python -c 'import asyncio'`, logging >> and http.client. >> >> >> https://gist.github.com/methane/1ab97181e74a33592314c7619bf34233#file-0-optimize-import-pat

<    1   2   3   4   5   >