Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-05 Thread Nathaniel Smith
On Thu, Jun 5, 2014 at 11:12 PM, Paul Moore wrote: > On 5 June 2014 22:47, Nathaniel Smith wrote: >> To make sure I understand correctly, you're suggesting something like >> adding a new set of special method slots, __te_add__, __te_mul__, >> etc. > > I wasn

Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-05 Thread Nathaniel Smith
On 6 Jun 2014 02:16, "Nikolaus Rath" wrote: > > Nathaniel Smith writes: > > Such optimizations are important enough that numpy operations always > > give the option of explicitly specifying the output array (like > > in-place operators but more general and wi

Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-05 Thread Nathaniel Smith
On 5 Jun 2014 23:58, "Terry Reedy" wrote: > > On 6/5/2014 4:51 PM, Nathaniel Smith wrote: > >> In fact, AFAICT it's 100% correct for libraries being called by >> regular python code (which is why I'm able to quote benchmarks at you >> :-)). The byt

Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Nathaniel Smith
On 6 Jun 2014 17:07, "Sturla Molden" wrote: > We would in total need two mutexes, one condition variable, a pthread, and > a heap. The proposal in my initial email requires zero pthreads, and is substantially more effective. (Your proposal reduces only the alloc overhead for large arrays; mine re

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-06 Thread Nathaniel Smith
On Fri, Jun 6, 2014 at 11:43 PM, Sturla Molden wrote: > Brett Cannon wrote: > >> Nope. A new minor release of Python is a massive undertaking which is why >> we have saved ourselves the hassle of doing a Python 2.8 or not giving a >> clear signal as to when Python 2.x will end as a language. > >

Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-06 Thread Nathaniel Smith
On Fri, Jun 6, 2014 at 11:33 PM, Sturla Molden wrote: > Nathaniel Smith wrote: > >> The proposal in my initial email requires zero pthreads, and is >> substantially more effective. (Your proposal reduces only the alloc >> overhead for large arrays; mine reduces both

Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler

2014-06-07 Thread Nathaniel Smith
Once 7 Jun 2014 06:19, "Nick Coghlan" wrote: > > On 7 June 2014 15:05, Donald Stufft wrote: > > I don’t particularly care too much though, I just think that bumping > > the compiler in a 2.7.Z release is a really bad idea and that either > > of the other two options are massively better. > > It i

Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

2014-06-07 Thread Nathaniel Smith
On Sat, Jun 7, 2014 at 1:37 AM, Greg Ewing wrote: > Julian Taylor wrote: >> >> tp_can_elide receives two objects and returns one of three values: >> * can work inplace, operation is associative >> * can work inplace but not associative >> * cannot work inplace > > > Does it really need to be that

Re: [Python-Dev] Examples for PEP 572

2018-07-04 Thread Nathaniel Smith
On Tue, Jul 3, 2018 at 11:07 PM, Serhiy Storchaka wrote: > 04.07.18 00:51, Chris Angelico пише: >> >> On Wed, Jul 4, 2018 at 7:37 AM, Serhiy Storchaka >> wrote: >>> >>> I believe most Python users are not >>> professional programmers -- they are sysadmins, scientists, hobbyists and >>> kids -- >>

Re: [Python-Dev] Examples for PEP 572

2018-07-04 Thread Nathaniel Smith
On Tue, Jul 3, 2018 at 11:14 PM, Serhiy Storchaka wrote: > 04.07.18 04:26, Tim Peters пише: >> >> I really don't know what Guido likes best about this, but for me it's the >> large number of objectively small wins in `if` and `while` contexts. They >> add up. That conclusion surprised me. That

Re: [Python-Dev] Examples for PEP 572

2018-07-04 Thread Nathaniel Smith
On Wed, Jul 4, 2018, 09:09 Steven D'Aprano wrote: > On Wed, Jul 04, 2018 at 12:10:11AM -0700, Nathaniel Smith wrote: > > > Right, Python has a *very strong* convention that each line should > > have at most one side-effect, > > import math, fractions, decimal &g

Re: [Python-Dev] Assignment expression and coding style: the while True case

2018-07-04 Thread Nathaniel Smith
On Wed, Jul 4, 2018 at 3:51 PM, Victor Stinner wrote: > My question is now: for which "while True" patterns are the assignment > expression appropriate? There identified different patterns. > > > == Pattern 1, straighforward == > > while True: > line = input.readline() > if not line: >

Re: [Python-Dev] Don't assign to a variable used later in the expression

2018-07-04 Thread Nathaniel Smith
On Wed, Jul 4, 2018 at 4:10 PM, Ivan Pozdeev via Python-Dev wrote: > On 04.07.2018 10:10, Nathaniel Smith wrote: >> Right, Python has a *very strong* convention that each line should >> have at most one side-effect, and that if it does have a side-effect >> it should be a

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-07 Thread Nathaniel Smith
On Sat, Jul 7, 2018 at 12:19 AM, Stefan Behnel wrote: > Typically, it's calls with 1 to ~3 positional arguments that occur in > performance critical situations. Often just one, rarely more, and zero > arguments is a fast case anyway. Keyword arguments will always suffer some > kind of penalty comp

Re: [Python-Dev] PEP 575, 576, 579 and 580

2018-07-07 Thread Nathaniel Smith
On Sat, Jul 7, 2018 at 6:38 AM, Mark Shannon wrote: > 1. The new API should be fully backwards compatible and shouldn't break the > ABI Which ABI? The stable ABI (PEP 384)? I don't think object layout is exposed there, though I'm not sure of the details. The regular ABI that almost everyone actu

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Nathaniel Smith
On Mon, Jul 9, 2018 at 10:20 AM, Steve Dower wrote: > On 09Jul2018 0922, Antoine Pitrou wrote: >> >> On Mon, 9 Jul 2018 09:01:00 -0700 >> Steve Dower wrote: >>> >>> I've thought a bit about making a single installer that can offer the >>> option of 32-bit/64-bit at install time, but I don't actua

Re: [Python-Dev] Fuzzing the Python standard library

2018-07-17 Thread Nathaniel Smith
On Tue, Jul 17, 2018 at 9:44 AM, Jussi Judin wrote: > * Exceptions that are something else than the documented ones. These usually > indicate an internal implementation issue. For example one would not expect > an UnicodeDecodeError from netrc.netrc() function when the documentation[3] > promis

Re: [Python-Dev] [Windows] how to prevent the wrong version of zlib1.dll to be used by lib-dynload modules

2018-07-23 Thread Nathaniel Smith
On Mon, Jul 23, 2018, 08:22 Eric Le Lay wrote: > Hello list, > > I encountered a problem with the Windows packaging of gPodder[1] > using msys2: > > basic libraries (zlib, openssl) depended upon by python > platform-specific modules are loaded preferably : > 1. from lib-dynload (where they are n

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register

2018-09-17 Thread Nathaniel Smith
Hi Wes, It's great you're passionate about python security, but this is the wrong way to go about it. Spectre is inherently super subtle and confusing, so if there's something that people need to do, then we need a clear, comprehensive write-up of what the threat is and how to address it. Perhaps

Re: [Python-Dev] What is the purpose of the _PyThreadState_Current symbol in Python 3?

2018-09-28 Thread Nathaniel Smith
What information do you wish the interpreter provided, that would make your program simpler and more reliable? On Fri, Sep 28, 2018, 07:21 Gabriele wrote: > Hi Victor, > > > I understand that you are writing a debugger and you can only *read* > > modify, not execute code, right? > > I'm working

Re: [Python-Dev] What is the purpose of the _PyThreadState_Current symbol in Python 3?

2018-09-29 Thread Nathaniel Smith
On Fri, Sep 28, 2018 at 3:29 PM, Gabriele wrote: > On Fri, 28 Sep 2018 at 23:12, Nathaniel Smith wrote: >> What information do you wish the interpreter provided, that would make your >> program simpler and more reliable? > > An exported global variable that poin

Re: [Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

2018-10-04 Thread Nathaniel Smith
On Wed, Oct 3, 2018 at 6:30 PM, Sean Harrington wrote: > with Pool(func_kwargs={"big_cache": big_cache}) as pool: > pool.map(func, ls) I feel like it would be nicer to spell this: with Pool() as pool: pool.map(functools.partial(func, big_cache=big_cache), ls) And this might also solve y

Re: [Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

2018-10-12 Thread Nathaniel Smith
On Fri, Oct 12, 2018, 06:09 Antoine Pitrou wrote: > On Fri, 12 Oct 2018 08:33:32 -0400 > Sean Harrington wrote: > > Hi Nathaniel - this if this solution can be made performant, than I would > > be more than satisfied. > > > > I think this would require removing "func" from the "task tuple", and

Re: [Python-Dev] short-circuiting runtime errors/exceptions in python debugger.

2018-10-29 Thread Nathaniel Smith
On Mon, Oct 29, 2018 at 11:59 AM, Chris Jerdonek wrote: > A simpler feature that could possibly help him (assuming there isn't any > external state to deal with) would be the ability to save everything at a > certain point in time, and then resume it later. He could rig things up to > save the sta

Re: [Python-Dev] Implementing an awaitable

2018-11-07 Thread Nathaniel Smith
"Awaitable" is a language-level concept. To actually use awaitables, you also need a coroutine runner library, and each library defines additional restrictions on the awaitables it works with. So e.g. when using asyncio as your coroutine runner, asyncio expects your awaitables to follow particular

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-09 Thread Nathaniel Smith
On Fri, Nov 9, 2018 at 4:30 PM, Victor Stinner wrote: > Ah, important points. I don't want to touch the current C API nor make > it less efficient. And compatibility in both directions (current C API > <=> new C API) is very important for me. There is no such plan as > "Python 4" which would break

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-09 Thread Nathaniel Smith
On Fri, Nov 9, 2018 at 6:03 PM, Victor Stinner wrote: > Le sam. 10 nov. 2018 à 02:50, Nathaniel Smith a écrit : >> Doesn't this mean that you're just making the C API larger and more >> complicated, rather than simplifying it? You cite some benefits >> (tagged p

Re: [Python-Dev] Signalling NANs

2018-11-10 Thread Nathaniel Smith
On Sat, Nov 10, 2018 at 3:26 PM, Steven D'Aprano wrote: > On Fri, Nov 09, 2018 at 01:17:09PM -0800, Chris Barker via Python-Dev wrote: >> works for me, too: >> >> In [9]: x = cast_int2float(0x7ff80001) >> In [10]: hex(cast_float2int(x)) >> Out[10]: '0x7ff80001' >> >> In [11]: x = c

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-13 Thread Nathaniel Smith
On Mon, Nov 12, 2018 at 10:46 PM, Gregory P. Smith wrote: > > On Fri, Nov 9, 2018 at 5:50 PM Nathaniel Smith wrote: >> >> On Fri, Nov 9, 2018 at 4:30 PM, Victor Stinner >> wrote: >> > Ah, important points. I don't want to touch the current C AP

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-13 Thread Nathaniel Smith
On Sun, Nov 11, 2018 at 3:19 PM, Victor Stinner wrote: > I'm not sure yet how far we should go towards a perfect API which > doesn't leak everything. We have to move slowly, and make sure that we > don't break major C extensions. We need to write tools to fully > automate the conversion. If it's n

Re: [Python-Dev] General concerns about C API changes

2018-11-13 Thread Nathaniel Smith
To me, the "new C API" discussion and the "converting macros into inline functions" discussions are very different, almost unrelated. There are always lots of small C API changes happening, and AFAIK the macros->inline changes fall into that category. It sounds like you want to discuss whether inli

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-16 Thread Nathaniel Smith
On Fri, Nov 16, 2018 at 3:12 PM Neil Schemenauer wrote: > Also, the extension module should not take a big performance hit. > So, you can't change all APIs that were macros into non-inlined > functions. People are not going to accept that and rightly so. > However, it could be that we introduce a

Re: [Python-Dev] General concerns about C API changes

2018-11-18 Thread Nathaniel Smith
On Sun, Nov 18, 2018 at 8:52 AM Stefan Behnel wrote: > > Gregory P. Smith schrieb am 15.11.18 um 01:03: > > From my point of view: A static inline function is a much nicer modern code > > style than a C preprocessor macro. > > It's also slower to compile, given that function inlining happens at a

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-20 Thread Nathaniel Smith
On Tue, Nov 20, 2018 at 1:34 AM Petr Viktorin wrote: > > On 11/19/18 12:14 PM, Victor Stinner wrote: > > To design a new C API, I see 3 options: > > > > (1) add more functions to the existing Py_LIMITED_API > > (2) "fork" the current public C API: remove functions and hide as much > > implementati

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-20 Thread Nathaniel Smith
On Tue, Nov 20, 2018 at 6:05 PM Glenn Linderman wrote: > > On 11/20/2018 2:17 PM, Victor Stinner wrote: >> IMHO performance and hiding implementation details are exclusive. You >> should either use the C API with impl. details for best performances, >> or use a "limited" C API for best compatibili

Re: [Python-Dev] C API changes

2018-11-26 Thread Nathaniel Smith
On Mon, Nov 26, 2018 at 6:12 PM Eric V. Smith wrote: > I thought the important part of the proposal was to have multiple > PyHandles that point to the same PyObject (you couldn't "directly > compare handles with each other to learn about object identity"). But > I'll admit I'm not sure why this wo

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-11-28 Thread Nathaniel Smith
On Wed, Nov 28, 2018, 12:08 Antoine Pitrou On Wed, 28 Nov 2018 19:35:31 + > Jonathan Underwood wrote: > > On Wed, 28 Nov 2018 at 18:57, Antoine Pitrou > wrote: > > > > > > On Wed, 28 Nov 2018 10:43:04 -0800 > > > "Gregory P. Smith" wrote: > > [snip] > > > > I don't think adding lz4 to the s

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-11-29 Thread Nathaniel Smith
On Thu, Nov 29, 2018, 08:34 Antoine Pitrou > Le 29/11/2018 à 17:25, Steve Dower a écrit : > > > > My experience is that the first group would benefit from a larger > > _standard distribution_, which is not necessarily the same thing as a > > larger stdlib. > > > > I'm firmly on the "smaller core,

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-11-29 Thread Nathaniel Smith
On Thu, Nov 29, 2018, 10:32 Antoine Pitrou On Thu, 29 Nov 2018 09:49:32 -0800 > Nathaniel Smith wrote: > > > > There are a lot of challenges to switching to a "standard distribution" > > model. I'm not certain it's the best option. But what I like about i

Re: [Python-Dev] Standard library vs Standard distribution?

2018-11-29 Thread Nathaniel Smith
On Thu, Nov 29, 2018 at 10:11 AM Christian Heimes wrote: > You are assuming that you can convince or force upstream developers to > change their project and development style. Speaking from personal > experience, that is even unrealistic for projects that are already > developed and promoted by of

Re: [Python-Dev] Standard library vs Standard distribution?

2018-11-29 Thread Nathaniel Smith
On Thu, Nov 29, 2018 at 10:22 AM Antoine Pitrou wrote: > > Le 29/11/2018 à 19:07, Steve Dower a écrit : > > On 29Nov2018 0923, Antoine Pitrou wrote: > >> I think the whole argument amounts to hand waving anyway. You are > >> inventing an extended distribution which doesn't exist (except as > >> A

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-12-01 Thread Nathaniel Smith
On Sat, Dec 1, 2018, 06:56 Steve Holden We* should probably do more collectively to point people at > production-quality third-party modules, as I believe we currently do with > pipenv which, while not a part of the standard library, is still > recommended in the documentation as the preferred met

Re: [Python-Dev] test_urllib2net fixed to repair Travis CI

2018-12-05 Thread Nathaniel Smith
Travis is in the middle of moving everything from AWS to GCE, which is probably related: https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration As noted there, GCE has different IP addresses. But I suspect it's not the new IP address that's the problem, but rather the fact

Re: [Python-Dev] Usage of the multiprocessing API and object lifetime

2018-12-11 Thread Nathaniel Smith
On Tue, Dec 11, 2018, 07:13 Antoine Pitrou > What you are proposing here starts to smell like an anti-pattern to > me. Python _is_ a garbage-collected language, so by definition, there > _are_ going to be resources that are automatically collected when an > object disappears. If I'm allocating a

Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

2019-01-17 Thread Nathaniel Smith
n Thu, Jan 17, 2019 at 4:51 PM Gregory P. Smith wrote: > > I've heard that libraries using ctypes, cffi, or cython code of various sorts > in the real world wild today does abuse the unfortunate side effect of > CPython's implementation of id(). I don't have specific instances of this in > mind

Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

2019-01-18 Thread Nathaniel Smith
On Thu, Jan 17, 2019, 22:11 Steve Dower For everyone who managed to reply *hours* after Eryk Sun posted the > correct answer and still get it wrong, here it is again in full. > > As a bonus, here's a link to the place where this answer appears in the > documentation: > https://docs.python.org/3/li

Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

2019-01-18 Thread Nathaniel Smith
On Fri, Jan 18, 2019 at 1:51 AM Steven D'Aprano wrote: > Across the entire Python ecosystem, no it isn't, as Jython and > IronPython return consecutive integers. But should we consider it an > intentional part of the CPython API? It's always worked, there's substantial code in the wild that depen

Re: [Python-Dev] Sub-interpreters: importing numpy causes hang

2019-01-22 Thread Nathaniel Smith
There are currently numerous incompatibilities between numpy and subinterpreters, and no concrete plan for fixing them. The numpy team does not consider subinterpreters to be a supported configuration, and can't help you with any issues you run into. I know the concept of subinterpreters is really

Re: [Python-Dev] Sub-interpreters: importing numpy causes hang

2019-01-22 Thread Nathaniel Smith
On Tue, Jan 22, 2019 at 6:33 PM Stephan Reiter wrote: > > Thanks for the answers so far. I appreciate them! > > Nathaniel, I'd like to allow Python plugins in my application. A > plugin should be allowed to bring its own modules along (i.e. > plugin-specific subdir is in sys.path when the plugin i

Re: [Python-Dev] Sub-interpreters: importing numpy causes hang

2019-01-27 Thread Nathaniel Smith
On Sun, Jan 27, 2019, 06:34 Stephan Reiter Cool. Thanks, Nick! > > I did experiments based on this idea ( > https://github.com/stephanreiter/cpython/commit/3bca91c26ac81e517b4aa22302be1741b3315622) > and haven't rejected it yet. :-) > > Together with the other fix ( > https://github.com/stephanrei

Re: [Python-Dev] How to update namedtuple asdict() to use dict instead of OrderedDict

2019-01-30 Thread Nathaniel Smith
On Wed, Jan 30, 2019 at 2:34 PM Raymond Hettinger wrote: > Now that regular dicts are ordered and compact, it makes more sense for the > _asdict() method to create a regular dict (as it did in its early days) > rather than an OrderedDict. The regular dict is much smaller, much faster, > and ha

Re: [Python-Dev] How to update namedtuple asdict() to use dict instead of OrderedDict

2019-01-31 Thread Nathaniel Smith
On Thu, Jan 31, 2019, 05:26 Nick Coghlan On Thu, 31 Jan 2019 at 16:40, Glenn Linderman > wrote: > >> On 1/30/2019 8:45 PM, Raymond Hettinger wrote: > >>> On Jan 30, 2019, at 3:41 PM, Glenn Linderman > wrote: > >>> Would it be practical to add deprecated methods to regular dict for > the OrderedD

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-13 Thread Nathaniel Smith
On Wed, Feb 13, 2019 at 3:02 PM Steven D'Aprano wrote: > > On Wed, Feb 13, 2019 at 05:16:54PM -0500, Terry Reedy wrote: > > > It appears python is already python3 for a large majority of human users > > (as opposed to machines). > > > > https://www.jetbrains.com/research/python-developers-survey-2

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-14 Thread Nathaniel Smith
On Thu, Feb 14, 2019 at 12:43 AM Antoine Pitrou wrote: > > On Wed, 13 Feb 2019 17:32:44 -0800 > Nathaniel Smith wrote: > > On Wed, Feb 13, 2019 at 3:02 PM Steven D'Aprano wrote: > > > > > > On Wed, Feb 13, 2019 at 05:16:54PM -0500, Terry Reedy wrote: >

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-18 Thread Nathaniel Smith
I noticed that your list doesn't include "add a DOM equality operator". That seems potentially simpler to implement than canonical XML serialization, and like a useful thing to have in any case. Would it make sense as an option? On Mon, Mar 18, 2019, 15:46 Raymond Hettinger wrote: > We're having

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Nathaniel Smith
On Tue, Mar 19, 2019 at 3:43 PM Giampaolo Rodola' wrote: > > On Tue, Mar 19, 2019 at 6:21 PM Guido van Rossum wrote: > > >> On Tue, Mar 19, 2019 at 10:14 AM Giampaolo Rodola' > >> wrote: > >> Technically you cannot make it 100% safe, only less likely to occur. > > That seems unnecessarily pedan

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-03-29 Thread Nathaniel Smith
That does sound strange. How easily can you reproduce it? That majorly effects how I would try to debug something like this... If you're able to get an strace of a failed run then that would probably tell us a *lot*. On Fri, Mar 29, 2019, 16:28 Steve Dower wrote: > Hi > > I'm trying to track do

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Nathaniel Smith
On Wed, Apr 10, 2019, 04:04 Victor Stinner wrote: > Le mar. 9 avr. 2019 à 22:16, Steve Dower a écrit > : > > What are the other changes that would be required? > > I don't know. > > > And is there another > > way to get the same functionality without ABI modifications? > > Py_TRACE_REFS is a dou

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Nathaniel Smith
On Wed, Apr 10, 2019 at 1:50 PM Steve Dower wrote: > > On 10Apr2019 1227, Nathaniel Smith wrote: > > On Wed, Apr 10, 2019, 04:04 Victor Stinner > <mailto:vstin...@redhat.com>> wrote: > > I don't think that I ever used sys.getobjects(), whereas many projects

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-11 Thread Nathaniel Smith
On Thu, Apr 11, 2019 at 8:32 AM Serhiy Storchaka wrote: > On other hand, since using the debug allocator doesn't cause problems > with compatibility, it may be possible to use similar technique for the > objects double list. Although this is not easy because of objects placed > at static memory.

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Nathaniel Smith
On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: > > On 10Apr2019 1917, Nathaniel Smith wrote: > > It sounds like --with-pydebug has accumulated a big grab bag of > > unrelated features, mostly stuff that was useful at some point for > > some CPython dev trying to de

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Nathaniel Smith
On Fri, Apr 12, 2019 at 5:05 PM Steve Dower wrote: > > On 12Apr.2019 1643, Nathaniel Smith wrote: > > On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: > >> > >> On 10Apr2019 1917, Nathaniel Smith wrote: > > I don't know how many people use Py_TRACE

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-15 Thread Nathaniel Smith
On Mon, Apr 15, 2019, 15:27 Michael Sullivan wrote: > > The main question is if anyone ever used Py_TRACE_REFS? Does someone > > use sys.getobjects() or PYTHONDUMPREFS environment variable? > > I used sys.getobjects() today to track down a memory leak in the > mypyc-compiled version of mypy. > >

Re: [Python-Dev] PEP 591 discussion (final qualifier) happening at typing-sig@

2019-04-15 Thread Nathaniel Smith
On Mon, Apr 15, 2019 at 5:00 PM Michael Sullivan wrote: > > I've submitted PEP 591 (Adding a final qualifier to typing) for discussion to > typing-sig [1]. I'm not on typing-sig [1] so I'm replying here. > Here's the abstract: > This PEP proposes a "final" qualifier to be added to the ``typing`

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-16 Thread Nathaniel Smith
On Mon, Apr 15, 2019 at 8:58 PM Michael Sullivan wrote: > > On Mon, Apr 15, 2019 at 4:06 PM Nathaniel Smith wrote: >> >> On Mon, Apr 15, 2019, 15:27 Michael Sullivan wrote: >>> >>> > The main question is if anyone ever used Py_TRACE_REFS? Doe

Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm

2019-04-20 Thread Nathaniel Smith
On Sat, Apr 20, 2019 at 2:11 PM Inada Naoki wrote: > > "import typing" is slow too. Many static analysis tools will also accept: TYPE_CHECKING = False if TYPE_CHECKING: ... At least mypy and pylint both treat all variables named TYPE_CHECKING as true, regardless of where they came from. I'm

Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm

2019-04-23 Thread Nathaniel Smith
On Tue, Apr 23, 2019, 05:09 Andrew Svetlov wrote: > I agree that `from typing import TYPE_CHECKING` is not desirable from > the import time reduction perspective. > > From my understanding code completion *can* be based on type hinting > to avoid actual code execution. > That's why I've mentioned

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-24 Thread Nathaniel Smith
You don't necessarily need rpath actually. The Linux loader has a bug/feature where once it has successfully loaded a library with a given soname, then any future requests for that soname within the same process will automatically return that same library, regardless of rpath settings etc. So as lo

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-27 Thread Nathaniel Smith
On Sat, Apr 27, 2019, 04:27 Armin Rigo wrote: > Hi Neil, > > On Wed, 24 Apr 2019 at 21:17, Neil Schemenauer > wrote: > > Regarding the Py_TRACE_REFS fields, I think we can't do them without > > breaking the ABI because of the following. For GC objects, they are > > always allocated by _PyObject

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-29 Thread Nathaniel Smith
On Mon, Apr 29, 2019 at 5:01 PM Neil Schemenauer wrote: > As far as I understand, we have a similar problem already for > gc.get_objects() because those static type objects don't have a > PyGC_Head. My 2-cent proposal for fixing things in the long term > would be to introduce a function like PyTy

Re: [Python-Dev] deprecation of abstractstaticmethod and abstractclassmethod

2019-05-15 Thread Nathaniel Smith
I don't care about the deprecation either way. But can we fix the individual decorators so both orders work? Even if it requires a special case in the code, it seems worthwhile to remove a subtle user-visible footgun. On Wed, May 15, 2019, 12:39 Ethan Furman wrote: > In issue 11610* abstractclas

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-15 Thread Nathaniel Smith
On Wed, May 15, 2019 at 6:25 PM Victor Stinner wrote: > I proposed a different approach: add a new sys.unraisablehook hook > which is called to handle an "unraisable exception". To handle them > differently, replace the hook. For example, I wrote a custom hook to > log these exceptions into a file

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019, 09:07 Steve Dower wrote: > > Actually, if the default implementation prints the exception message, > how is this different from sys.excepthook? Specifically, from the point > of customizing the hooks. > sys.excepthook means the program has fully unwound and is about to exit

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019 at 2:17 PM Steve Dower wrote: > You go on to say "pass an error message" and "keep repr(obj) if you > want", but how is this different from creating an exception that > contains the custom message, the repr of the object, and chains the > exception that triggered it? A clever

Re: [Python-Dev] Parser module in the stdlib

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019 at 2:13 PM Pablo Galindo Salgado wrote: > I propose to remove finally the parser module as it has been "deprecated" for > a long time, is almost clear that nobody uses it and has very limited > usability and replace it (maybe with a different name) > with pgen2 (maybe with a

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019 at 1:23 PM Victor Stinner wrote: > > Le jeu. 16 mai 2019 à 20:58, Petr Viktorin a écrit : > > I always thought the classic (exc_type, exc_value, exc_tb) triple is a > > holdover from older Python versions, and all the information is now in > > the exception instance. > > Is t

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-21 Thread Nathaniel Smith
On Tue, May 21, 2019 at 2:40 AM Walter Dörwald wrote: > > On 20 May 2019, at 22:15, Christian Heimes wrote: > > > Hi, > > > > here is the first version of my PEP 594 to deprecate and eventually > > remove modules from the standard library. The PEP started last year > > with talk during the Python

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-21 Thread Nathaniel Smith
On Tue, May 21, 2019 at 10:43 AM Glenn Linderman wrote: > After maintaining my own version of http.server to fix or workaround some of > its deficiencies for some years, I discovered bottle.py. It has far more > capability, is far better documented, and is just as quick to deploy. While I > hav

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-21 Thread Nathaniel Smith
On Tue, May 21, 2019 at 4:25 AM Victor Stinner wrote: > > Le mar. 21 mai 2019 à 13:18, André Malo a écrit : > > There's software in production using both. (It doesn't mean it's on pypi or > > even free software). > > > > What would be the maintenance burden of those modules anyway? (at least for

Re: [Python-Dev] PEP 594: update 1

2019-05-22 Thread Nathaniel Smith
On Wed, May 22, 2019, 12:14 Sean Wallitsch wrote: > Dear python-dev, > > I'm writing to provide some feedback on PEP-594, primarily the proposed > deprecation and reason for the removal of the aifc and audioop libraries. > > The post production film industry continues to make heavy use of AIFFs,

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-22 Thread Nathaniel Smith
On Wed, May 22, 2019, 04:32 Christian Heimes wrote: > On 22/05/2019 12.19, Steven D'Aprano wrote: > > I don't think this PEP should become a document about "Why you should > > use PAM". I appreciate that from your perspective as a Red Hat security > > guy, you want everyone to use best practices

Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done

2019-05-24 Thread Nathaniel Smith
On Fri, May 24, 2019, 08:08 Ben Cail wrote: > > Why not have the PSF hire someone (or multiple people) to be paid to > work on the maintenance burden? This could be similar to the Django > fellows: > https://www.djangoproject.com/fundraising/#who-is-the-django-fellow. It > seems like a good thing

Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done (was: PEP 594: Removing dead batteries from the standard library)

2019-05-24 Thread Nathaniel Smith
On Thu, May 23, 2019 at 2:18 PM Brett Cannon wrote: > I'm personally viewing it as a first step in addressing the maintenance > burden we have with such a large stdlib. Christian started this work over a > year ago and I think it's worth seeing through. After that we should probably > have a di

Re: [Python-Dev] PEP 558: Defined semantics for locals()

2019-05-25 Thread Nathaniel Smith
On Sat, May 25, 2019, 07:38 Guido van Rossum wrote: > This looks great. > > I only have two nits with the text. > > First, why is the snapshot called a "dynamic snapshot"? What exactly is > dynamic about it? > It's dynamic in that it can spontaneously change when certain other events happen. For

[Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-27 Thread Nathaniel Smith
First, I want to say: I'm very happy with PEP 558's changes to f_locals. It solves the weird threading bugs, and exposes the fundamental operations you need for debugging in a simple and clean way, while leaving a lot of implementation flexibility for future Python VMs. It's a huge improvement over

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-27 Thread Nathaniel Smith
On Mon, May 27, 2019 at 9:16 AM Guido van Rossum wrote: > > I re-ran your examples and found that some of them fail. > > On Mon, May 27, 2019 at 8:17 AM Nathaniel Smith wrote: [...] >> The interaction between f_locals and and locals() is also subtle: >> >> def f(

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-27 Thread Nathaniel Smith
On Mon, May 27, 2019 at 9:18 PM Guido van Rossum wrote: > > Note that the weird, Action At A Distance behavior is also visible for > locals() called at module scope (since there, locals() is globals(), which > returns the actual dict that's the module's __dict__, i.e. the Source Of > Truth. So

Re: [Python-Dev] [SPAM?] Re: PEP 558: Defined semantics for locals()

2019-05-28 Thread Nathaniel Smith
On Tue, May 28, 2019 at 6:02 PM Guido van Rossum wrote: > > On Tue, May 28, 2019 at 5:25 PM Greg Ewing > wrote: >> >> Terry Reedy wrote: >> > I believe that the situation is or can be thought of as this: there is >> > exactly 1 function locals dict. Initially, it is empty and inaccessible >> >

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-28 Thread Nathaniel Smith
On Tue, May 28, 2019 at 6:48 PM Greg Ewing wrote: > > Nathaniel Smith wrote: > > - [proxy]: Simply return the .f_locals object, so in all contexts > > locals() returns a live mutable view of the actual environment: > > > > def locals(): > > return get_

Re: [Python-Dev] [SPAM?] Re: PEP 558: Defined semantics for locals()

2019-05-28 Thread Nathaniel Smith
On Tue, May 28, 2019 at 5:24 PM Greg Ewing wrote: > > Terry Reedy wrote: > > I believe that the situation is or can be thought of as this: there is > > exactly 1 function locals dict. Initially, it is empty and inaccessible > > (unusable) from code. Each locals() call updates the dict to a curre

Re: [Python-Dev] Should I postpone PEP 558 (locals() semantics) to Python 3.9?

2019-05-31 Thread Nathaniel Smith
I wouldn't mind having a little more breathing room. It's frustrating to miss the train, but these bugs are several decades old so I guess nothing terrible will happen if their fixes get delayed to 3.9. On Thu, May 30, 2019 at 4:23 PM Nick Coghlan wrote: > > Hi folks, > > The reference implementa

Re: [Python-Dev] PEP 595: Improving bugs.python.org

2019-05-31 Thread Nathaniel Smith
On Fri, May 31, 2019 at 11:39 AM Barry Warsaw wrote: > > On May 31, 2019, at 01:22, Antoine Pitrou wrote: > > > I second this. > > > > There are currently ~7000 bugs open on bugs.python.org. The Web UI > > makes a good job of actually being able to navigate through these bugs, > > search through

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-06 Thread Nathaniel Smith
On Tue, Aug 6, 2019 at 3:44 PM Brett Cannon wrote: > I think this is a good example of how the community is not running tests with > warnings on and making sure that their code is warnings-free. This warning > has existed for at least one full release and fixing it doesn't require some > crazy

[Python-Dev] Re: Speeding up CPython

2020-10-21 Thread Nathaniel Smith
Hi Mark, This sounds really cool. Can you give us more details? Some questions that occurred to me while reading: - You're suggesting that the contractor would only be paid if the desired 50% speedup is achieved, so I guess we'd need some objective Python benchmark that boils down to a single spe

[Python-Dev] Re: Why aren't we allowing the use of C11?

2021-01-28 Thread Nathaniel Smith
On Thu, Jan 28, 2021 at 9:03 PM Gregory P. Smith wrote: > > On Thu, Jan 28, 2021 at 10:52 AM Charalampos Stratakis > wrote: >> >> >> >> - Original Message - >> > From: "Mark Shannon" >> > To: "Python Dev" >> > Sent: Thursday, January 28, 2021 5:26:37 PM >> > Subject: [Python-Dev] Why a

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Nathaniel Smith
On Thu, Feb 25, 2021 at 2:13 PM Guido van Rossum wrote: > > So is "fail-fast if you forget to handle an ExceptionGroup" really a feature? > (Do we call this out in the PEP?) > > We may believe that "except Exception" is an abuse, but it is too common to > dismiss out of hand. I think if some app

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Nathaniel Smith
On Thu, Feb 25, 2021 at 10:23 PM Glenn Linderman wrote: > So then why do you need except* at all? Only to catch unwrapped > ExceptionGroup before it gets wrapped? > > So why not write except ExceptionGroup, and let it catch unwrapped > ExceptionGroup? > > That "CUTE BIT" could be done only when

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Nathaniel Smith
On Fri, Feb 26, 2021 at 5:05 AM Irit Katriel wrote: > I'm not sure it's safe to assume that it is necessarily a programming error, > and that the interpreter can essentially break the program in this case. > Is this not allowed? > > try: > try: > obj.func()# function that raises E

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-05 Thread Nathaniel Smith
OK, better late than never... here's a much-delayed review of the PEP. Thank you Irit and Guido for carrying this forward while I've been AWOL! It's fantastic to see my old design sketches turned into something like, actually real. == Overall feelings == Honestly, I have somewhat mixed feelings E

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-19 Thread Nathaniel Smith
On Mon, Apr 5, 2021 at 9:48 AM Irit Katriel wrote: > On Mon, Apr 5, 2021 at 11:01 AM Nathaniel Smith wrote: >> - I'm uncomfortable with how in some contexts we treat EG's as placeholders >> for the contained exceptions, and other places we treat them like a single

<    1   2   3   4   5   >