Re: [Python-Dev] __del__ is not called after creating a new reference

2017-03-20 Thread Antoine Pitrou
Hello Oleg, On Mon, 20 Mar 2017 18:28:29 +0100 Oleg Nesterov wrote: > I started to learn python a few days ago and I am trying to understand what > __del__() actually does. https://docs.python.org/3/reference/datamodel.html > says: > > object.__del__(self) > ... > Note that it

Re: [Python-Dev] Exact date of Python 2 EOL?

2017-03-23 Thread Antoine Pitrou
On Thu, 23 Mar 2017 09:41:11 +0100 Victor Stinner wrote: > Hi, > > I looked at the "Status of Python branches" to check if it was up to > date. It's the case, thanks :-) But it recalled me that no exact date > was decided for the official end of line of the the Python 2 branch > (2.7 EOL). > > h

Re: [Python-Dev] Distutils frozen?

2017-03-27 Thread Antoine Pitrou
On Mon, 27 Mar 2017 12:32:09 -0500 Ryan Gonzalez wrote: > So, I had opened up a PR (#563) to add README.rst to the distutils readme > list. Turns out, I didn't read the devguide correctly, and there needed to > be an open issue first. Oops. > > Then I found bpo-11913 (https://bugs.python.org/iss

Re: [Python-Dev] TLS issue (unknown CA) on test_imaplib

2017-04-27 Thread Antoine Pitrou
There's also a failure in test_nntplib: https://travis-ci.org/python/cpython/jobs/226384811#L3157-L3171 Regards Antoine. On Thu, 27 Apr 2017 14:50:43 +0200 Victor Stinner wrote: > Hi, > > test_imaplib is starting to fail randomly causing CI tests to fail and > buildbots to fail randomly: >

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

2017-05-04 Thread Antoine Pitrou
On Thu, 4 May 2017 11:24:27 +0900 INADA Naoki wrote: > Hi, Nick and all core devs who are interested in this PEP. > > I'm reviewing PEP 538 and I want to accept it in this month. > It will reduces much UnicodeError pains which server-side OPs facing. > Thank you Nick for working on this PEP. > >

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

2017-05-23 Thread Antoine Pitrou
On Mon, 22 May 2017 23:56:43 + Matt Billenstein wrote: > On Tue, May 16, 2017 at 11:31:42AM +0100, Martin Wimpress wrote: > > Is there someone here who'd be interested in doing the same for Python? > > Do snaps support Windows and/or MacOS? If you want a package manager that supports insta

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

2017-05-23 Thread Antoine Pitrou
On Mon, 22 May 2017 11:17:18 -0700 Steve Dower wrote: > > I'd like to propose a highly-accelerated deprecation period for this > specific feature, starting in CPython 3.6.2 and being "completed" in > 3.7.0, when we will hopefully move onto a newer libffi. > > In general, the "feature" is a mis

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-25 Thread Antoine Pitrou
On Tue, 23 May 2017 23:09:31 -0500 Victor Stinner wrote: > Le 23 mai 2017 20:43, "David Wilson" a écrit : > In which case, what is to prevent Requests from just depending on > > pyOpenSSL as usual? > > > From what I heard, pyOpenSSL development is slowing down, so I'm not sure > that it's real

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-25 Thread Antoine Pitrou
On Wed, 24 May 2017 23:31:47 +0200 Ivan Levkivskyi wrote: > Hi all, > > After collecting suggestions in the previous discussion on python-dev > https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629 > and playing with implementation, here is an updated version of PEP 544. >

Re: [Python-Dev] Aligning the packaging.python.org theme with the rest of the docs

2017-05-28 Thread Antoine Pitrou
On Sat, 27 May 2017 14:26:54 +1000 Nick Coghlan wrote: > > Thoughts? Should we stick with pypa-theme as the name? Switch to > psf-docs-theme? Publish both, with pypa-theme adding PyPA specific > elements to a more general psf-docs-theme? > [...] > > Future requests to use the theme (beyond CPyt

[Python-Dev] "Micro-optimisations can speed up CPython"

2017-05-29 Thread Antoine Pitrou
Hi, I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/mi

Re: [Python-Dev] Aligning the packaging.python.org theme with the rest of the docs

2017-05-30 Thread Antoine Pitrou
On Tue, 30 May 2017 21:49:19 +1000 Nick Coghlan wrote: > > Here's an alternate wording for the README that would focus on those > considerations without explicitly asking folks not to use the theme: > > "Note that when adopting this theme, you're also borrowing an element > of the trust and cred

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 00:38:09 -0700 Larry Hastings wrote: > * CPython programs would use more memory. How much? Hard to say. It > depends on their allocation strategy. I suspect most of the time it > would be < 3mb additional memory. But for pathological allocation > strategies th

[Python-Dev] "Global freepool"

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 09:57:04 +0200 Victor Stinner wrote: > > By the way, Naoki INADA also proposed a different idea: > > "Global freepool: Many types has it’s own freepool. Sharing freepool > can increase memory and cache efficiency. Add PyMem_FastFree(void* > ptr, size_t size) to store memory bl

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 01:41:15 -0700 Larry Hastings wrote: > On 06/01/2017 01:19 AM, Antoine Pitrou wrote: > > If you'd like to go that way anyway, I would suggest 1MB as a starting > > point in 3.7. > > I understand the desire for caution. But I was hoping maybe we co

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Wed, 31 May 2017 14:09:20 -0600 Jim Baker wrote: > > But I object to a completely new feature being added to 2.7 to support the > implementation of event loop SSL usage. This feature cannot be construed as > a security fix, and therefore does not qualify as a feature that can be > added to CPy

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
Le 01/06/2017 à 11:13, Victor Stinner a écrit : > That's why pip uses Requests which uses certifi (Mozilla > bundled root certificate authorities.) pip could use certifi without using Requests. My guess is that Requests is used mostly because it eases coding. > pip and so Requests are part of t

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 18:37:17 +0900 INADA Naoki wrote: > x86's hugepage is 2MB. > And some Linux enables "Transparent Huge Page" feature. > > Maybe, 2MB arena size is better for TLB efficiency. > Especially, for servers having massive memory. But, since Linux is able to merge pages transparently,

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 19:50:22 +1000 Chris Angelico wrote: > On Thu, Jun 1, 2017 at 7:23 PM, Antoine Pitrou wrote: > >> Do you also disagree on the need of the need of the PEP 546 > >> (backport) to make the PEP 543 (new TLS API) feasible in practice? > > > > Yes,

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 20:05:48 +1000 Chris Angelico wrote: > > As stated in this thread, OS-provided certificates are not handled by > that. For instance, if a local administrator distributes a self-signed > cert for the intranet server, web browsers will use it, but pip will > not. That's true. B

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
Le 01/06/2017 à 12:23, Cory Benfield a écrit : > > No it can’t. > > OpenSSL builds chains differently, and disregards some metadata that Windows > and macOS store, which means that cert validation will work differently than > in the system store. This can lead to pip accepting a cert marked as

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 11:22:21 +0100 Cory Benfield wrote: > > Who is the “we” that should move on? Python core dev? Or the Python ecosystem? Sorry. Python core dev certainly. As for the rest of the ecosystem, it is moving on as well. > Requests is stuck in a place from which it cannot move. > We

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 11:45:14 +0100 Cory Benfield wrote: > > I am claiming that using OpenSSL certificate validation with root stores that > are not intended for OpenSSL can be. This is because trust of a certificate > is non-binary. For example, consider WoSign. The Windows TLS implementation >

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 12:01:41 +0100 Cory Benfield wrote: > In principle, sure. In practice, that means most of our users don’t use those > features and so we don’t get any feedback on whether they’re good solutions > to the problem. On bugs.python.org we get plenty of feedback from people using P

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
Le 01/06/2017 à 15:12, Cory Benfield a écrit : > > I don’t know what to do with that answer, really. I gave you some data (80%+ > of requests downloads over the last month were Python 2), and you responded > with “it doesn’t cause us problems”. And indeed it doesn't. Unless the target user ba

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 14:37:55 +0100 Cory Benfield wrote: > > > > And indeed it doesn't. Unless the target user base for pip is widely > > different than Python's, it shouldn't cause you any problems either. > > Maybe not now, but I think it’s fair to say that it did, right? Until Python 3.2 an

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Antoine Pitrou
On Thu, 1 Jun 2017 15:09:41 +0100 Cory Benfield wrote: > > As to the plans to move Requests to a more event loop-y model, I think that > it does stand in the way of this, but only insomuch as, again, we want our > event loopy model to be as bug-free as possible. But I can concede that > reword

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Antoine Pitrou
On Thu, 1 Jun 2017 20:21:12 -0700 Larry Hastings wrote: > On 06/01/2017 02:50 AM, Antoine Pitrou wrote: > > Another possible strategy is: allocate several arenas at once (using a > > larger mmap() call), and use MADV_DONTNEED to relinquish individual > > arenas. >

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Antoine Pitrou
Thanks you for all the explanations. So to summarize my opinion, I'm still -0.5 on this PEP. I would also like to see the issues Jython, Ubuntu et al. have mentioned solved before this is accepted. Regards Antoine. On Fri, 2 Jun 2017 11:42:58 +0200 Victor Stinner wrote: > Thanks Cory for t

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Antoine Pitrou
On Fri, 2 Jun 2017 12:22:06 -0400 Donald Stufft wrote: > > It’s not just bootstrapping that pip has a problem with for C extensions, it > also prevents upgrading PyOpenSSL on Windows because having pip import > PyOpenSSL locks the .dll, and we can’t delete it or overwrite it until the > pip pr

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Antoine Pitrou
On Fri, 2 Jun 2017 13:23:05 -0500 Tim Peters wrote: > > While I would like to increase the pool size, it's fraught with > danger. What would be the point of increasing the pool size? Apart from being able to allocate 4KB objects out of it, I mean. Since 4KB+ objects are relatively uncommon (I

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-03 Thread Antoine Pitrou
On Fri, 2 Jun 2017 12:31:19 -0700 Larry Hastings wrote: > > Anyway, I'm not super excited by the prospect of using obmalloc for > larger objects. There's an inverse relation between the size of > allocation and the frequency of allocation. In Python there are lots of > tiny allocations, but

[Python-Dev] Extremely slow test modules

2017-06-03 Thread Antoine Pitrou
Hi, Is there a reason some of our tests are excruciatingly slow in `-uall` mode? `test_multiprocessing_spawn` is understandable (after all, it will spawn a new executable for each subprocess), but other tests leave me baffled: - test_tools: 7 min 41 sec - test_tokenize: 6 min 23 sec - test_date

Re: [Python-Dev] Extremely slow test modules

2017-06-03 Thread Antoine Pitrou
On Sat, 3 Jun 2017 15:28:18 +0300 Serhiy Storchaka wrote: > > test_tools (in particular the test for the unparse.py script), > test_tokenize, and test_lib2to3 read and proceed every Python file in > the stdlib. This is necessary in full test run because some syntax > constructs are very rarely

Re: [Python-Dev] Extremely slow test modules

2017-06-03 Thread Antoine Pitrou
On Sat, 3 Jun 2017 15:01:07 +0200 Antoine Pitrou wrote: > > > This is controlled by the cpy resource. > > I suggested to disable it on the slowest buildbots (-uall,-cpu). In that > > case tests are ran only for few random files. > > I don't really care about

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-04 Thread Antoine Pitrou
On Sat, 3 Jun 2017 21:46:09 -0500 Tim Peters wrote: > > A virtual address space span of a terabyte could hold 1M pools, so > would "only" need a 1M/8 = 128KB bit vector. That's minor compared to > a terabyte (one bit per megabyte). The virtual address space currently supported by x86-64 is 48 b

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-04 Thread Antoine Pitrou
On Sun, 4 Jun 2017 09:46:10 -0500 Tim Peters wrote: > [Tim] > >> A virtual address space span of a terabyte could hold 1M pools, so > >> would "only" need a 1M/8 = 128KB bit vector. That's minor compared to > >> a terabyte (one bit per megabyte). > > [Antoine] > > The virtual address space cur

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-08 Thread Antoine Pitrou
On Thu, 8 Jun 2017 09:30:37 +0100 Cory Benfield wrote: > > Ah, yes, we do. In our defense, this is the semantic of the listen > syscall,[...] According to POSIX, the backlog is only a hint, i.e. Jython is probably ok in not observing its value: """The backlog argument provides a hint to the im

Re: [Python-Dev] PEP 538 warning at startup: please remove it

2017-06-12 Thread Antoine Pitrou
On Mon, 12 Jun 2017 10:56:31 +0200 Victor Stinner wrote: > Hi, > > Nick Coghlan pushed his implementation of his PEP 538: nice! Nice step > forward to UTF-8 everywhere ;-) > > I would prefer to not be annoyed by warning messages about encodings > at startup if possible: > > "Python detected LC_

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-24 Thread Antoine Pitrou
On Fri, 23 Jun 2017 20:24:05 -0700 Larry Hastings wrote: > > We've been talking about addressing this for years. Fixing this was one > of the goals of the new workflow. And finally, as of right now, the > future is here. Ladies and gentlemen, I present: blurb. > > https://github.com/pyt

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-24 Thread Antoine Pitrou
On Sat, 24 Jun 2017 08:57:24 -0700 Larry Hastings wrote: > On 06/24/2017 01:53 AM, Antoine Pitrou wrote: > > Would you like to make it 3.5-compatible? 3.6 is quite young and not > > all systems have it (e.g. Ubuntu 16.04, which many people use, has 3.5). > > Well, tbh

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-24 Thread Antoine Pitrou
Larry, I have just used blurb on https://github.com/python/cpython/pull/2010 and it was absolutely painless. Thank you! One minor thing: `make patchcheck` doesn't know to examine the NEWS.d directory when checking for NEWS entries. Regards Antoine. On Fri, 23 Jun 2017 20:24:05 -0700 Larry

[Python-Dev] For all branches? [was Re: New workflow change: Welcome to blurb]

2017-06-24 Thread Antoine Pitrou
One thing you didn't mention: is the switch on all branches or only master? Regards Antoine. On Fri, 23 Jun 2017 20:24:05 -0700 Larry Hastings wrote: > One minor but ongoing problem we've had in CPython core development has > been the mess of updating Misc/NEWS. Day-to-day developers may h

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-24 Thread Antoine Pitrou
On Sat, 24 Jun 2017 23:48:03 + Brett Cannon wrote: > On Sat, 24 Jun 2017 at 10:07 Antoine Pitrou wrote: > > > > > Larry, > > > > I have just used blurb on https://github.com/python/cpython/pull/2010 > > and it was absolutely painless. Thank you! >

[Python-Dev] Helping contributors with chores (do we have to?)

2017-06-25 Thread Antoine Pitrou
On Sat, 24 Jun 2017 21:37:46 -0700 Larry Hastings wrote: > On 06/24/2017 09:14 PM, Serhiy Storchaka wrote: > > Not only core developers make PRs for CPython. Since all non-trivial > > changes need to be mentioned in Misc/NEWS, blurb becomes a required > > tool for all committers. > > Well, no

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-25 Thread Antoine Pitrou
On Sun, 25 Jun 2017 11:47:10 +0300 Paul Sokolovsky wrote: > > > > This is touching a more general problem, though. Before GitHub, we > > (core developers) would take the patch submitted by a contributor, > > make whatever minor changes were needed (e.g. Misc/NEWS) and push the > > aggregate ours

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-25 Thread Antoine Pitrou
On Mon, 26 Jun 2017 01:27:20 +1000 Chris Angelico wrote: > On Sun, Jun 25, 2017 at 7:25 PM, Antoine Pitrou wrote: > >> There're also various tools for dealing specifically with git branch > >> layout as used by Github, and every real man writes their own (because >

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-25 Thread Antoine Pitrou
On Sun, 25 Jun 2017 20:38:51 +0200 Stefan Krah wrote: > On Sun, Jun 25, 2017 at 10:02:44AM -0700, Nathaniel Smith wrote: > > https://help.github.com/articles/committing-changes-to-a-pull-request-branch-created-from-a-fork/ > > > > However, like Antoine, when I've tried to do this then all I've ma

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

2017-06-28 Thread Antoine Pitrou
On Wed, 28 Jun 2017 09:50:10 -0400 Ben Hoyt wrote: > I've now got a patch to add new COMPARE_IS_NONE and > COMPARE_IS_NOT_NONE bytecodes to CPython to attempt to speed up the > very common "is None" and "is not None" expressions. It's a very > simple change overall. Here is my patch: > https://gis

[Python-Dev] Bug in buildbot view (changes column)

2017-06-30 Thread Antoine Pitrou
Hello, If you look at this page (waterfall view of 3.x unstable buildbots), you'll notice the "changes" column on the left lists the recent changesets on master: http://buildbot.python.org/all/waterfall?tag=3.x.unstable However, on the equivalent view for 3.x stable buildbots, the "changes" colu

Re: [Python-Dev] Remove own implementation for thread-local storage

2017-07-03 Thread Antoine Pitrou
Also note that C11, MSVC and some Unix C compilers have built-in support for thread-local variables. Example: https://github.com/numba/numba/blob/master/numba/_random.c#L114-L119 Regards Antoine. On Mon, 3 Jul 2017 10:07:06 +0200 Victor Stinner wrote: > I'm in favor of removing it. I know t

Re: [Python-Dev] Pure pickle bechmark.

2017-07-09 Thread Antoine Pitrou
Hi, On Sun, 9 Jul 2017 19:38:09 +0530 Bhavishya wrote: > Hello, > > 1).I was going through the code of *python pickle* to search any > optimization possibility.But the only thing that I found very alarming was > again the import time(I tried with lazy-import but it didn't helped much.) > > I f

[Python-Dev] Extendability of C vs Python pickle

2017-07-10 Thread Antoine Pitrou
On Mon, 10 Jul 2017 02:35:37 +0200 Victor Stinner wrote: > > I already proposed to remove this benchmark: > https://mail.python.org/pipermail/speed/2017-April/000554.html > > *but* Antoine Pitrou mentionned that the cloudpickle project uses it. > > Maybe we should try

[Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Antoine Pitrou
Hello, Cost of creating a namedtuple has been identified as a contributor to Python startup time. Not only Python core and the stdlib, but any third-party library creating namedtuple classes (there are many of them). An issue was created for this: https://bugs.python.org/issue28638 Raymond dec

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Antoine Pitrou
On Mon, 17 Jul 2017 14:43:19 +0200 Antoine Pitrou wrote: > Hello, > > Cost of creating a namedtuple has been identified as a contributor to > Python startup time. Imprecise wording: that's the cost of creating a namedtuple *class*, i.e. anytime someone writes `MyClass = na

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Antoine Pitrou
On Mon, 17 Jul 2017 15:03:26 +0200 Ivan Levkivskyi wrote: > Interesting coincidence, just two days ago I have heard that a team at one > large company completely abandoned namedtuple because of the creation time > problem. > > Concerning _source, why it is not possible to make it a property so th

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Antoine Pitrou
Le 17/07/2017 à 15:26, Isaac Morland a écrit : > > I think I understand well enough to say something intelligent… > > While actual references to _source are likely rare (certainly I’ve never > used it), my understanding is that the way namedtuple works is to > construct _source, and then exec it

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-18 Thread Antoine Pitrou
On Tue, 18 Jul 2017 22:13:01 +1000 Nick Coghlan wrote: > > As another example of this: while trading the global import lock for > per-module locks eliminated most of the old import deadlocks, it turns > out that it *also* left us with some fairly messy race conditions and > more fragile code (I s

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Antoine Pitrou
On Tue, 18 Jul 2017 09:08:08 -0700 Ethan Furman wrote: > > Nick Coughlan: > - It is "Nick Coghlan" not "Coughlan". > > As another example of this: while trading the global import lock for > > per-module locks eliminated most of the old import deadlocks, it turns > > out that it *als

Re: [Python-Dev] Program runs in 12s on Python 2.7, but 5s on Python 3.5 -- why so much difference?

2017-07-18 Thread Antoine Pitrou
On Tue, 18 Jul 2017 12:03:36 -0400 Ben Hoyt wrote: > Hi folks, > > (Not entirely sure this is the right place for this question, but hopefully > it's of interest to several folks.) > > A few days ago I posted a note in response to Victor Stinner's articles on > his CPython contributions, noting

Re: [Python-Dev] anecdotal data

2017-07-19 Thread Antoine Pitrou
On Wed, 19 Jul 2017 14:59:52 +0200 Victor Stinner wrote: > Hi, > > On Twitter, Raymond Hettinger wrote: > >"The decision making process on Python-dev is an anti-pattern, > governed by anecdotal data and ambiguity over what problem is solved." > > https://twitter.com/raymondh/status/8870694

Re: [Python-Dev] [OT] Twitter echo chamber (Python startup time)

2017-07-19 Thread Antoine Pitrou
On Wed, 19 Jul 2017 14:59:52 +0200 Victor Stinner wrote: > Hi, > > On Twitter, Raymond Hettinger wrote: > >"The decision making process on Python-dev is an anti-pattern, > governed by anecdotal data and ambiguity over what problem is solved." > > https://twitter.com/raymondh/status/8870694

Re: [Python-Dev] Python startup time

2017-07-19 Thread Antoine Pitrou
On Wed, 19 Jul 2017 15:26:47 -0400 Ben Hoyt wrote: > Yes, agreed that startup time matters for scripting. I was talking to > someone on the Google Cloud SDK (CLI) team recently, and they said startup > time is a big deal for them ... it's especially problematic for shell tab > completion helpers,

Re: [Python-Dev] Python startup time

2017-07-20 Thread Antoine Pitrou
On Thu, 20 Jul 2017 21:29:18 +0900 INADA Naoki wrote: > > WeakSet should be cared specially. > Maybe, ABCMeta can be optimized first. > > Currently, ABCMeta use three WeakSets. But it can be delayed until > `register` or > `issubclass` is called. > So even if WeakSet is implemented in Python, I

Re: [Python-Dev] Python startup time

2017-07-21 Thread Antoine Pitrou
On Fri, 21 Jul 2017 00:12:20 -0700 David Mertz wrote: > How implausible is it to write out the actual memory image of a loaded > Python process? I.e. on a specific machine, OS, Python version, etc? This > can only be overhead initially, of course, but on subsequent runs it's just > one memory map,

Re: [Python-Dev] Python startup time

2017-07-23 Thread Antoine Pitrou
On Sat, 22 Jul 2017 16:35:31 -0700 Steve Dower wrote: > > Yes, I’m aware of that, which is why I don’t have any specific suggestions > off-hand. But given the differences in file systems between Windows and other > OSs, it wouldn’t surprise me if there were a more optimal approach for NTFS > t

Re: [Python-Dev] first post introduction and question regarding lto

2017-08-09 Thread Antoine Pitrou
On Wed, 9 Aug 2017 13:36:28 +1000 Nick Coghlan wrote: > On 8 August 2017 at 10:12, Gregory P. Smith wrote: > > I don't know whether it is beneficial or not - but having the capability to > > build LTO without PGO seems reasonable. I can review any pull requests > > altering configure.ac and Makef

Re: [Python-Dev] Interrupt thread.join() with Ctrl-C / KeyboardInterrupt on Windows

2017-08-09 Thread Antoine Pitrou
On Tue, 8 Aug 2017 14:29:41 -0700 Steve Dower wrote: > On 08Aug2017 1151, Nathaniel Smith wrote: > > It looks like Thread.join ultimately ends up blocking in > > Python/thread_nt.h:EnterNonRecursiveMutex, which has a maze of #ifdefs > > behind it -- I think there are 3 different implementation you

Re: [Python-Dev] first post introduction and question regarding lto

2017-08-09 Thread Antoine Pitrou
On Wed, 9 Aug 2017 11:16:36 +0200 Victor Stinner wrote: > There is already a ./configure --with-lto flag, why not using it? > > I'm using --with-lto without PGO for months, I never noticed that the > option is fully ignored! What are the reasons it is ignored? IIRC some compilers have buggy LTO

Re: [Python-Dev] PEP 550 v3

2017-08-19 Thread Antoine Pitrou
On Fri, 18 Aug 2017 16:33:27 -0400 Yury Selivanov wrote: > > There are a few open questions left, namely the terminology > and design of ContextKey API. On the former topic, I'm quite > happy with the latest version: Execution Context, Logical > Context, and Context Key. I don't really like it.

Re: [Python-Dev] PEP 550 v3

2017-08-20 Thread Antoine Pitrou
On Sat, 19 Aug 2017 17:21:03 -0700 Guido van Rossum wrote: > The way we came to "logical context" was via "logical thread (of control)", > which is distinct from OS thread. But I think we might need to search for > another term... Perhaps "task context"? A "task" might be a logical thread, OS th

Re: [Python-Dev] PEP 550 v3 naming

2017-08-21 Thread Antoine Pitrou
On Mon, 21 Aug 2017 01:45:05 -0400 "Jim J. Jewett" wrote: > Building on Brett's suggestion: > > FrameContext: used in/writable by one frame It's not frame-specific, it's actually shared by an arbitrary number of frames (by default, all frames in a given thread). Regards Antoine.

Re: [Python-Dev] PEP 550 v3 naming

2017-08-23 Thread Antoine Pitrou
For the record, I'm starting to think that "implicit context" is a reasonable name. (in case anyone is interested in those 2 cents of mine :-)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listin

Re: [Python-Dev] PEP 550 v3 naming

2017-08-23 Thread Antoine Pitrou
On Wed, 23 Aug 2017 12:19:40 -0400 Yury Selivanov wrote: > PEP 550 calls generators and asynchronous tasks as "logical threads", > and "logical context" stems directly from that notion. I wouldn't refer to a generator as a "thread" personally. A thread essentially executes without intervention f

Re: [Python-Dev] PEP 550 v3 naming

2017-08-24 Thread Antoine Pitrou
On Wed, 23 Aug 2017 14:27:55 -0400 Yury Selivanov wrote: > > Yeah.. I like TaskContext when it's applied to asynchronous code. It > doesn't really work for generators because we never refer to > generators as tasks. > > Out of what was proposed so far to replace Logical Context: > > 1. DynamicC

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-24 Thread Antoine Pitrou
On Thu, 24 Aug 2017 09:52:47 -0400 Barry Warsaw wrote: > > To me, the functionality proposed in PEP 550 feels more like a "scope" > than a "context". I would call it "environment" myself, but that risks confusion with environment variables. Perhaps "dynamic environment" would remove the confusi

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-24 Thread Antoine Pitrou
On Thu, 24 Aug 2017 17:06:09 -0400 Barry Warsaw wrote: > On Aug 24, 2017, at 16:01, francismb wrote: > > On 08/24/2017 03:52 PM, Barry Warsaw wrote: > >> Guido van Rossum wrote: > >>> On Tue, Aug 22, 2017 at 7:12 PM, Nathaniel Smith wrote: > >>> > >>> I worry that that's going to lead more

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-25 Thread Antoine Pitrou
On Fri, 25 Aug 2017 15:36:55 +1000 Nick Coghlan wrote: > On 24 August 2017 at 23:52, Barry Warsaw wrote: > > Guido van Rossum wrote: > >> On Tue, Aug 22, 2017 at 7:12 PM, Nathaniel Smith wrote: > >> > >> I worry that that's going to lead more people astray thinking this has > >> something to d

Re: [Python-Dev] PEP 550 v4

2017-08-26 Thread Antoine Pitrou
On Sat, 26 Aug 2017 13:23:00 -0400 Yury Selivanov wrote: > > > > A good point; however, ChainMap, which behaves similarly as far as lookup > > goes, uses "get" and does not have a "lookup" method. I think we lose more > > than we gain by changing that method name. > > ChainMap is constrained t

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
On Mon, 28 Aug 2017 17:24:29 -0400 Yury Selivanov wrote: > Long story short, I think we need to rollback our last decision to > prohibit context propagation up the call stack in coroutines. In PEP > 550 v3 and earlier, the following snippet would work just fine: > >var = new_context_var() >

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
Le 29/08/2017 à 21:18, Yury Selivanov a écrit : > On Tue, Aug 29, 2017 at 2:40 PM, Antoine Pitrou wrote: >> On Mon, 28 Aug 2017 17:24:29 -0400 >> Yury Selivanov wrote: >>> Long story short, I think we need to rollback our last decision to >>> prohibit context

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
Le 29/08/2017 à 21:59, Yury Selivanov a écrit : > > This absolutely needs to be fixed, and the only way (that I know) it > can be fixed is to revert the "every coroutine has its own LC" > statement (going back to the semantics coroutines had in PEP 550 v2 > and v3). I completely agree with this.

Re: [Python-Dev] PEP 550 v4: coroutine policy

2017-08-29 Thread Antoine Pitrou
Le 29/08/2017 à 22:20, Yury Selivanov a écrit : > > 2) > >gvar = new_context_var() >var = new_context_var() > >async def bar(): ># EC = [current_thread_LC_copy, Task_foo_LC_copy, Task_wait_for_LC] Ah, thanks!... That explains things, though I don't expect most users to spo

Re: [Python-Dev] bpo-5001: More-informative multiprocessing error messages (#3079)

2017-08-30 Thread Antoine Pitrou
On Wed, 30 Aug 2017 08:48:56 +0300 Serhiy Storchaka wrote: > 30.08.17 01:52, Antoine Pitrou пише: > > https://github.com/python/cpython/commit/bd73e72b4a9f019be514954b1d40e64dc3a5e81c > > commit: bd73e72b4a9f019be514954b1d40e64dc3a5e81c > > branch: master > > a

Re: [Python-Dev] [Python-checkins] bpo-5001: More-informative multiprocessing error messages (#3079)

2017-08-30 Thread Antoine Pitrou
On Wed, 30 Aug 2017 03:16:42 -0700 Chris Jerdonek wrote: > https://github.com/python/cpython/commit/bd73e72b4a9f019be514954b1d40e64dc3a5e81c > > commit: bd73e72b4a9f019be514954b1d40e64dc3a5e81c > > branch: master > > author: Allen W. Smith, Ph.D > > committer: Antoin

Re: [Python-Dev] [Python-checkins] bpo-5001: More-informative multiprocessing error messages (#3079)

2017-08-30 Thread Antoine Pitrou
On Wed, 30 Aug 2017 03:21:42 -0700 Chris Jerdonek wrote: > > https://github.com/python/cpython/commit/bd73e72b4a9f019be514954b1d40e64dc3a5e81c > > commit: bd73e72b4a9f019be514954b1d40e64dc3a5e81c > > branch: master > > author: Allen W. Smith, Ph.D > > committer: Ant

Re: [Python-Dev] Inplace operations for PyLong objects

2017-08-31 Thread Antoine Pitrou
On Thu, 31 Aug 2017 23:35:34 + "Manciu, Catalin Gabriel" wrote: > > The focus of this experiment was inplace adds in general. While, as you've > shown, there are ways to write the loop optimally, the benchmark was written > as a huge loop just to showcase that there is an improvement using th

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Antoine Pitrou
On Fri, 1 Sep 2017 22:15:29 +0900 INADA Naoki wrote: > FYI, there is issue report for it. > http://psf.upfronthosting.co.za/roundup/meta/issue463 > INADA Naoki That issue is about making the tracker HTTPS-only, but fixing internal links to point to the HTTPS site would already go a long way, ev

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Antoine Pitrou
And by the way the problem goes away if you use the "HTTPS Everywhere" plugin for Firefox. Regards Antoine. On Fri, 1 Sep 2017 15:29:58 +0200 Antoine Pitrou wrote: > On Fri, 1 Sep 2017 22:15:29 +0900 > INADA Naoki wrote: > > FYI, there is issue r

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Antoine Pitrou
Le 01/09/2017 à 16:32, Victor Stinner a écrit : > > In short, it doesn't work :-) I'm using Firefox 55 on Ubuntu 16.04 and it works here. You may be misunderstading what happens :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Antoine Pitrou
On Fri, 1 Sep 2017 17:03:59 +0200 Victor Stinner wrote: > > > And by the way the problem goes away if you use the "HTTPS Everywhere" > > plugin for Firefox. > > Try for example this page: > > https://bugs.python.org/issue31234?@ok_message=msg%20301118%20created > > For me, the "clear this me

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Antoine Pitrou
On Fri, 1 Sep 2017 17:31:00 +0200 Oleg Broytman wrote: > On Fri, Sep 01, 2017 at 05:27:59PM +0200, Antoine Pitrou > wrote: > > On Fri, 1 Sep 2017 17:03:59 +0200 > > Victor Stinner wrote: > > > > > > > And by the way the problem goes away if you us

Re: [Python-Dev] Inplace operations for PyLong objects

2017-09-02 Thread Antoine Pitrou
On Sat, 2 Sep 2017 10:38:05 -0700 Chris Barker wrote: > On Fri, Sep 1, 2017 at 6:16 PM, Joe Jevnik via Python-Dev < > python-dev@python.org> wrote: > > > The string concat optimization happens in the interpreter dispatch for > > INPLACE_ADD > > > > In that case, isn't there a new string being

Re: [Python-Dev] [RFC] Removing pure Python implementation of OrderedDict

2017-09-05 Thread Antoine Pitrou
On Tue, 5 Sep 2017 17:38:06 +0900 INADA Naoki wrote: > > Like that, how about removing OrderedDict Pure Python implementation > from stdlib and require it to implementation? I don't like this. The C version of OrderedDict is probably very hard to read, while the Python version helps understand

[Python-Dev] Compiling without multithreading support -- still useful?

2017-09-05 Thread Antoine Pitrou
Hello, It's 2017 and we are still allowing people to compile CPython without threads support. It adds some complication in several places (including delicate parts of our internal C code) without a clear benefit. Do people still need this? Regards Antoine. __

Re: [Python-Dev] [RFC] Removing pure Python implementation of OrderedDict

2017-09-05 Thread Antoine Pitrou
On Wed, 6 Sep 2017 03:20:43 +0900 INADA Naoki wrote: > What is *idiomatic*? For example, in this test case: > > https://github.com/python/cpython/blob/564a2c68add64ebf2e558a54f5697513b19293cb/Lib/test/test_ordered_dict.py#L575-L582 > > def test_dict_delitem(self): > OrderedDict = se

[Python-Dev] Consolidate stateful runtime globals

2017-09-06 Thread Antoine Pitrou
Hello, I'm a bit concerned about https://github.com/python/cpython/commit/76d5abc8684bac4f2fc7cccfe2cd940923357351 My main gripe is that makes writing C code more tedious. Simple C global variables such as "_once_registry" are now spelled "_PyRuntime.warnings.once_registry". The most egregious

Re: [Python-Dev] [RFC] Removing pure Python implementation of OrderedDict

2017-09-06 Thread Antoine Pitrou
On Wed, 6 Sep 2017 11:26:52 +0900 INADA Naoki wrote: > > Like that, should we say "atomic & threadsafe __setitem__ for simple > key is implementation detail of CPython and PyPy. We recommend > using mutex when using OrderedDict from multiple thread."? I think you may be overstating the importan

Re: [Python-Dev] Consolidate stateful runtime globals

2017-09-06 Thread Antoine Pitrou
On Wed, 06 Sep 2017 09:42:29 -0700 Benjamin Peterson wrote: > On Wed, Sep 6, 2017, at 03:14, Antoine Pitrou wrote: > > > > Hello, > > > > I'm a bit concerned about > > https://github.com/python/cpython/commit/76d5abc8684bac4f2fc7cccfe2cd940923357351 > &

Re: [Python-Dev] Compiling without multithreading support -- still useful?

2017-09-06 Thread Antoine Pitrou
I made an experimental PR to remove support for threads-less builds: https://github.com/python/cpython/pull/3385 The next effect is to remove almost 2000 lines of code (including many #ifdef sections in C code). Regards Antoine. On Tue, 5 Sep 2017 18:36:51 +0200 Antoine Pitrou wrote

<    14   15   16   17   18   19   20   21   22   23   >