Re: [Python-Dev] Need testing audio files

2013-09-11 Thread Victor Stinner
Use your microphone, say "python" and save the file in your favorite file format. Try for example Audacity. I suppose that you don't need specific audio content and you don't need a huge file. Victor 2013/9/11 Serhiy Storchaka : > I work on enhancement of audio modules testing [1], and I need fre

Re: [Python-Dev] Add a "transformdict" to collections

2013-09-11 Thread Victor Stinner
2013/9/11 Steven D'Aprano : > But the proposal is not for a case-insensitive dict. It is more general > than that, with case-insensitivity just one specific use-case for such > a transformative dict. Arguably the most natural, or at least obvious, > such transformation, but there are others. > > I

Re: [Python-Dev] Add a "transformdict" to collections

2013-09-11 Thread Victor Stinner
2013/9/11 Ethan Furman : > He isn't keeping the key unchanged (notice no white space in MAPPING), he's > merely providing a function that will automatically strip the whitespace > from key lookups. transformdict keeps the key unchanged, see the first message: >>> d = transformdict(str.lower)

Re: [Python-Dev] cpython: Issue #18571: Implementation of the PEP 446: file descriptors and file handles

2013-09-15 Thread Victor Stinner
Yes, but I'm not interested to write such doc. Victor Le 15 sept. 2013 10:34, "Georg Brandl" a écrit : > On 08/28/2013 01:20 AM, victor.stinner wrote: > > http://hg.python.org/cpython/rev/ef889c3d5dc6 > > changeset: 85420:ef889c3d5dc6 > > user:Vict

Re: [Python-Dev] PEP 428: Pathlib -> stat caching

2013-09-16 Thread Victor Stinner
2013/9/16 Brett Cannon : > Any reason why stat() can't get a keyword-only cached=True argument instead? > Or have stat() never cache() but stat_cache() always so that people can > choose if they want fresh or cached based on API and not whether some > library happened to make a decision for them?

[Python-Dev] PEP 454: add a new tracemalloc module (second round)

2013-09-16 Thread Victor Stinner
e to trace Python memory allocations Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 3-September-2013 Python-Version: 3.4 Abstract Add a new ``tracemalloc`` module to trace memory blocks allocated

Re: [Python-Dev] PEP 454: add a new tracemalloc module (second round)

2013-09-17 Thread Victor Stinner
2013/9/17 Victor Stinner : > Issue tracking the implementation: > http://bugs.python.org/issue18874 If you want to test the implementation, you can try the following repository: http://hg.python.org/features/tracemalloc Or try the patch attached on the issue #18874 on the Python default v

[Python-Dev] benchmark

2013-09-17 Thread Victor Stinner
benchmark.patch Description: Binary data ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] benchmark

2013-09-18 Thread Victor Stinner
Oops, I wanted to send the patch to myself, not to the python-dev mailing list, sorry :-) (It's a set of patch to try to optimize the tracemalloc module.) Victor 2013/9/18 Victor Stinner : > ___ Python-Dev mailing list Python-Dev@python.o

Re: [Python-Dev] sys.intern should work on bytes

2013-09-20 Thread Victor Stinner
2013/9/20 Jesus Cea : > """ > sys.intern(b'12121212') > Traceback (most recent call last): > File "", line 1, in > TypeError: must be str, not bytes > """ > > I wonder why. Intern strings optimize dictionary lookup. In Python 3, most dictionaries use str keys (ex: __dict__ of classes). What wo

Re: [Python-Dev] Best practice for documentation for std lib

2013-09-22 Thread Victor Stinner
I don't like having to browse the source code to read the documentation. I prefer short docstring and longer reST documentation. In ReST, you get a table of content and nice links to functions, modules, etc. When I read doc, I like having two levels: tutorial listing most important functions and p

Re: [Python-Dev] cpython: Use cached builtins.

2013-10-02 Thread Victor Stinner
I don't remember where, but I remember that I also saw things like "str=str, len=len, ...". So you keep the same name, but you use fast local lookups instead of slow builtin lookups. Victor 2013/10/2 Antoine Pitrou : > On Wed, 2 Oct 2013 18:16:48 +0200 (CEST) > serhiy.storchaka wrote: >> http:/

[Python-Dev] PEP 454: Add a new tracemalloc module (final version)

2013-10-03 Thread Victor Stinner
nd releases the memory earlier thanks to a new memory pool (only used by the module). PEP: 454 Title: Add a new tracemalloc module to trace Python memory allocations Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Cr

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-03 Thread Victor Stinner
2013/10/3 Christian Heimes : > A hash algorithm can be added and one avaible hash > algorithm can be set before Py_Initialize() is called for the first > time. "Py_Initialize" is not the good guard. Try for example "python3 -X faulthandler": PyObject_Hash() is called before Py_Initialize() to add

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-04 Thread Victor Stinner
2013/10/4 Armin Rigo : > The current hash randomization is > simply not preventing anything; someone posted long ago a way to > recover bit-by-bit the hash randomized used by a remote web program in > Python running on a server. Oh interesting, is it public? If yes, could we please search the URL

Re: [Python-Dev] PEP 455: TransformDict

2013-10-04 Thread Victor Stinner
2013/10/4 Raymond Hettinger : > * Another issue is that we're accumulating too many dictionary > variants and that is making it difficult to differentiate and choose > between them. I haven't found anyone (even in advanced classes > with very experienced pythonistas) would knew about > all the var

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-04 Thread Victor Stinner
2013/10/5 Larry Hastings : > On 10/04/2013 11:15 AM, Victor Stinner wrote: > > 2013/10/4 Armin Rigo : > > The current hash randomization is > simply not preventing anything; someone posted long ago a way to > recover bit-by-bit the hash randomized used by a remote web progra

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-04 Thread Victor Stinner
2013/10/4 : > > Quoting Victor Stinner : > >> I still fail to understand the real impact of a hash DoS compared to >> other kinds of DoS. > > > I think the key question is: how many attacking nodes do you need to > control to effectively make some system deny ser

[Python-Dev] PEP 455: TransformDict

2013-10-05 Thread Victor Stinner
2013/10/4 Raymond Hettinger >: > Please do conduct your own API tests and report back. I don't understand why there is a need to evaluate a "new" API: TransformDict has the same API than dict. The only differences are that the constructor takes an extra mandatory parameter and there is a new getit

Re: [Python-Dev] Optimization

2013-10-05 Thread Victor Stinner
The str type is immutable, bytearray is not. It's easier to justify optimisations on mutable types, like overallocate lists for example. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscr

[Python-Dev] Reduce memory footprint of Python

2013-10-06 Thread Victor Stinner
Hi, Slowly, I'm trying to see if it would be possible to reduce the memory footprint of Python using the tracemalloc module. First, I noticed that linecache can allocate more than 2 MB. What do you think of adding a registry of "clear cache" functions? For exemple, re.purge() and linecache.clearc

Re: [Python-Dev] Reduce memory footprint of Python

2013-10-06 Thread Victor Stinner
2013/10/6 Benjamin Peterson : > 2013/10/6 Victor Stinner : >> Hi, >> >> Slowly, I'm trying to see if it would be possible to reduce the memory >> footprint of Python using the tracemalloc module. >> >> First, I noticed that linecache can allocate more

Re: [Python-Dev] Reduce memory footprint of Python

2013-10-06 Thread Victor Stinner
Benjamin wrote: > Is it important to optimize unittests for memory usage? 2013/10/6 Georg Brandl : > That does not seem very important, except if people execute test_import > on every interpreter startup :) Oh, I just realized that I forgot to explain why I'm starting with unit tests. I ran the P

Re: [Python-Dev] Reduce memory footprint of Python

2013-10-06 Thread Victor Stinner
2013/10/6 : > Quoting Antoine Pitrou : > >> The linecache module is used implicitly by the traceback and warnings >> module, so perhaps quite a bit more than one would imagine :-) >> >> I think limiting the linecache cache size would be good enough. > > So what specific limit would you suggest? B

Re: [Python-Dev] PEP 454: Add a new tracemalloc module (final version)

2013-10-06 Thread Victor Stinner
2013/10/3 Victor Stinner : > I worked on the implementation of the tracemalloc module and its PEP > 454. I consider that this third version of the PEP is ready for a > final review. > > HTML version of the PEP 454: > http://www.python.org/dev/peps/pep-0454/ About the implementa

Re: [Python-Dev] PEP 454: Add a new tracemalloc module (final version)

2013-10-09 Thread Victor Stinner
2013/10/3 Victor Stinner : > I worked on the implementation of the tracemalloc module and its PEP > 454. I consider that this third version of the PEP is ready for a > final review. > > What should be done to finish the PEP? I splitted the module into two parts. Do you prefer the n

Re: [Python-Dev] Python startup time

2013-10-10 Thread Victor Stinner
Hi, In an old issue, I proposed a change to not load the sysconfig module when it's not needed. Nobody reviewed the patch, the issue was closed. http://bugs.python.org/issue14057 When -s or -I option is used, we may skip completly the sysconfig module. (It's already the case when -S is used.) B

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-11 Thread Victor Stinner
[We are discussing issue #15806.] 2013/10/11 Antoine Pitrou : > I don't think that this contextlib.ignore() thing has been discussed a > lot. If we decide to keep the feature, I would prefer a less generic name: contextlib.ignore_excep(), contextlib.ignore_exception() or contextlib.ignore_excepti

Re: [Python-Dev] C extension import time

2013-10-11 Thread Victor Stinner
2013/10/11 Antoine Pitrou : >> So the first step I tried is something horrible (typing from memory): >> >> try: >> import _decimal >> except ImportError: >> >> else: >> from _decimal import * >> >> That way the 2.21 msec are reduced to 912 usec, but the indentation is >> an abomination

[Python-Dev] Support keyword in PEP URL?

2013-10-11 Thread Victor Stinner
Hi, What do you think of adding an optional identifier to a PEP to get a readable URL? Example: http://www.python.org/dev/peps/qualname/ instead of http://www.python.org/dev/peps/pep-0395/ Other examples: 305: csv 450: statistics 3156: asyncio An identifier must only contain lower case l

Re: [Python-Dev] Support keyword in PEP URL?

2013-10-11 Thread Victor Stinner
2013/10/12 Guido van Rossum : > What's the use case? I just use Google search if I don't recall the PEP > number. The final goal would be to identify PEPs using a textual identifier instead of a number identifier. We now have 206 PEPs (341 if you count also deferred and rejected PEPs). It's not e

Re: [Python-Dev] Support keyword in PEP URL?

2013-10-11 Thread Victor Stinner
2013/10/12 Ben Finney : > Victor Stinner writes: > >> For draft PEP, the identifier may change. > > For an idea implemented in several PEPs, the obvious identifier may be > taken first, but the preferred PEP for that identifier may later change. > > For example, PEP

Re: [Python-Dev] Support keyword in PEP URL?

2013-10-11 Thread Victor Stinner
2013/10/12 Ethan Furman : >> What do you propose in cases like this? Should the keyword always refer >> to the same PEP it did in the past, even when that PEP is no longer as >> relevant given later PEPs? Or should the keyword reach a different, >> newer PEP if that newer PEP becomes a “more releva

Re: [Python-Dev] [Python-checkins] cpython: Issue #19209: fix structseq test

2013-10-11 Thread Victor Stinner
2013/10/12 Nick Coghlan : >> summary: >> Issue #19209: fix structseq test >> >> diff --git a/Lib/test/test_structseq.py b/Lib/test/test_structseq.py >> --- a/Lib/test/test_structseq.py >> +++ b/Lib/test/test_structseq.py >> @@ -38,7 +38,7 @@ >> # os.stat() gives a complicated struct sequ

[Python-Dev] bugs.python.org not reachable in IPv6?

2013-10-12 Thread Victor Stinner
Hi, The DNS server of python.org announce the IP address 2a01:4f8:131:2480::3: $ host -t bugs.python.org bugs.python.org has IPv6 address 2a01:4f8:131:2480::3 The problem is that I cannot connect to this IP address: $ ping6 -c 4 2a01:4f8:131:2480::3 PING 2a01:4f8:131:2480::3(2a01:4f8:131:

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-13 Thread Victor Stinner
2013/10/13 Antoine Pitrou : > On Sun, 13 Oct 2013 22:50:07 +1000 > Nick Coghlan wrote: >> That's the way this works (unless you think this is such a disastrous >> addition that you want to appeal to Guido to strip me of my >> responsibilities as contextlib maintainer and go hunting for a new >> on

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-13 Thread Victor Stinner
2013/10/13 Ethan Furman : > True, but Raymond's example of > > with ignore(OSError): > os.remove('somefile') And what about: with ignore(OSError): os.remove('file1') os.remove('file2') ? Victor ___ Python-Dev mailing list Python-Dev@py

Re: [Python-Dev] Change PEP 399 import recommendation

2013-10-13 Thread Victor Stinner
Le 13 oct. 2013 10:19, "Stefan Behnel" a écrit : > I agree. I find it much easier to read a plain and obvious > > try: >from _cmodule import * > except ImportError: >from _pymodule import * > > in a facade module ... I miss maybe something. I don't understand why you would

Re: [Python-Dev] non-US zip archives support in zipfile.py

2013-10-14 Thread Victor Stinner
I opened an issue proposing exactly the same change, but I didn't provide a patch. Thanks for working on the issue. (Sorry I don't know the issue number. Search for "zipfile encoding".) Victor ___ Python-Dev mailing list Python-Dev@python.org https://ma

Re: [Python-Dev] non-US zip archives support in zipfile.py

2013-10-15 Thread Victor Stinner
Belopolsky : > On Mon, Oct 14, 2013 at 6:13 PM, Victor Stinner > wrote: >> I opened an issue proposing exactly the same change, but I didn't provide a >> patch. > > I found this: > > http://bugs.python.org/issue10614 > > but it has (seemingly inc

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-16 Thread Victor Stinner
CONGRATULATION! This thread reached 100 mails, it's now time to summarize it into a PEP. Is there a candidate to write it? If no PEP is written, the thread will never die and people will continue to feed it. Victor 2013/10/11 Antoine Pitrou : > > Hello, > > On Fri, 11 Oct 2013 07:39:48 +0200 (CE

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-16 Thread Victor Stinner
2013/10/16 Raymond Hettinger : > FWIW, here's a little history: Thank you! It helped me to understand the story. > * In February, I presented ignore() in the keynote for the U.S. Pycon. > Again, the feedback was positive. I missed this edition of Pycon US. How did you feedback on the keynote?

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-16 Thread Victor Stinner
I might agree with idea of contextlib.ignore() (I'm still opposed to the idea), but I don't understand the purpose of adding a new syntax doing exactly the same than try/except: > with trap(OSError) as cm: > os.unlink('missing.txt') > if cm.exc: > do_something() Nobody not

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-16 Thread Victor Stinner
2013/10/16 Antoine Pitrou : >> By the way, what are the performances of contextlib.ignore()? >> Exceptions can be slow in some cases. Adding something even slower >> would not be a good idea. > > A "try" block which succeeds is fast. Ah yes, I never reminder this fact. I try to not care too much o

Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().

2013-10-16 Thread Victor Stinner
I would rewrite your examples using try/finally: try: try: os.unlink('missing.txt') finally: some other code except OSError as exc: do_something() It's differently than yours, because it catchs OSError on "some; other

Re: [Python-Dev] Help needed: problems getting the expected default encoding for embedded IO config test

2013-10-17 Thread Victor Stinner
sys.stdout.encoding must never be None, it must be a str. If it is None, it is a regression. I modified Python (maybe in 3.2) to ensure that .encoding is always set. For your failure: what is the locale encoding? What are the values of LC_ALL, LANG, LC_CTYPES and PYTHONIOENCODING env vars? I will

[Python-Dev] PEP 454 (tracemalloc): new minimalist version

2013-10-18 Thread Victor Stinner
memory allocations Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 3-September-2013 Python-Version: 3.4 Abstract This PEP proposes to add a new ``tracemalloc`` module to trace memory blocks allocated by

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

2016-06-08 Thread Victor Stinner
> Abstract > > > This PEP changes the default class definition namespace to ``OrderedDict``. > Furthermore, the order in which the attributes are defined in each class > body will now be preserved in ``type.__definition_order__``. This allows > introspection of the original definition ord

Re: [Python-Dev] PEP 467: Minor API improvements to bytes, bytearray, and memoryview

2016-06-08 Thread Victor Stinner
Hi, > Currently, the ``bytes`` and ``bytearray`` constructors accept an integer > argument and interpret it as meaning to create a zero-initialised sequence > of the given size:: > (...) > This PEP proposes to deprecate that behaviour in Python 3.6, and remove it > entirely in Python 3.7. I'm opp

Re: [Python-Dev] C99

2016-06-08 Thread Victor Stinner
keep supporting MSVC. > > --Guido (mobile) > > On Jun 7, 2016 12:39 PM, "Sturla Molden" wrote: >> >> Victor Stinner wrote: >> >> > Is it worth to support a compiler that in 2016 doesn't support the C >> > standard released in 1999, 1

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

2016-06-08 Thread Victor Stinner
2016-06-08 23:01 GMT+02:00 Neil Schemenauer : > - code coming out of 2to3 runs correctly on this modified Python Stop using 2to3. This tool adds many useless changes when you only care of Python 2.7 and Python 3.4+. I suggest to use better tools like 2to6, modernize or my own tool: https://pypi.py

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Victor Stinner
I understood that Christian Heimes and/or Donald Stufft are interested to work on a PEP. 2016-06-09 13:25 GMT+02:00 Larry Hastings : > A problem has surfaced just this week in 3.5.1. Obviously this is a good > time to fix it for 3.5.2. But there's a big argument over what is "broken" > and what

[Python-Dev] Stop using timeit, use perf.timeit!

2016-06-10 Thread Victor Stinner
Hi, Last weeks, I made researchs on how to get stable and reliable benchmarks, especially for the corner case of microbenchmarks. The first result is a serie of article, here are the first three: https://haypo.github.io/journey-to-stable-benchmark-system.html https://haypo.github.io/journey-to-st

Re: [Python-Dev] Stop using timeit, use perf.timeit!

2016-06-10 Thread Victor Stinner
I started to work on visualisation. IMHO it helps to understand the problem. Let's create a large dataset: 500 samples (100 processes x 5 samples): --- $ python3 telco.py --json-file=telco.json -p 100 -n 5 --- Attached plot.py script creates an histogram: --- avg: 26.7 ms +- 0.2 ms; min = 26.2 ms

Re: [Python-Dev] Stop using timeit, use perf.timeit!

2016-06-10 Thread Victor Stinner
2016-06-10 17:09 GMT+02:00 Paul Moore : > Also, the way people commonly use > micro-benchmarks ("hey, look, this way of writing the expression goes > faster than that way") doesn't really address questions like "is the > difference statistically significant". If you use the "python3 -m perf compar

Re: [Python-Dev] Stop using timeit, use perf.timeit!

2016-06-10 Thread Victor Stinner
2016-06-10 20:47 GMT+02:00 Meador Inge : > Apologies in advance if this is answered in one of the links you posted, but > out of curiosity was geometric mean considered? > > In the compiler world this is a very common way of aggregating performance > results. FYI I chose to store all timings in th

Re: [Python-Dev] Stop using timeit, use perf.timeit!

2016-06-10 Thread Victor Stinner
Hi, 2016-06-10 20:37 GMT+02:00 Kevin Modzelewski via Python-Dev : > Hi all, I wrote a blog post about this. > http://blog.kevmod.com/2016/06/benchmarking-minimum-vs-average/ Oh nice, it's even better to have different articles to explain the problem of using the minimum ;-) It added it to my doc.

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-11 Thread Victor Stinner
> I repeat for like the fifth time: So, is there a candidate to write a PEP? I didn't read the thread. As expected, the discussion restarted for the 3rd time, there are almost 100 emails in this thread. Victor ___ Python-Dev mailing list Python-Dev@py

Re: [Python-Dev] [Python-checkins] cpython (3.5): Fix os.urandom() using getrandom() on Linux

2016-06-14 Thread Victor Stinner
uin 2016 5:11 PM, "Steven D'Aprano" a écrit : > Is this right? I thought we had decided that os.urandom should *not* > fall back on getrandom on Linux? > > > > On Tue, Jun 14, 2016 at 02:36:27PM +, victor. stinner wrote: > > https://hg.python.org

Re: [Python-Dev] Why does base64 return bytes?

2016-06-14 Thread Victor Stinner
To port OpenStack to Python 3, I wrote 4 (2x2) helper functions which accept bytes *and* Unicode as input. xxx_as_bytes() functions return bytes, xxx_as_text() return Unicode: http://docs.openstack.org/developer/oslo.serialization/api.html Victor Le 14 juin 2016 5:21 PM, "Steven D'Aprano" a écrit

Re: [Python-Dev] [Python-checkins] cpython (3.5): Fix os.urandom() using getrandom() on Linux

2016-06-14 Thread Victor Stinner
Le 14 juin 2016 5:28 PM, "Jelle Zijlstra" a écrit : >The problem isn't that os.urandom() uses getrandom(), it's that it calls it in a mode that may block. Except if it changed very recently, os.urandom() doesn't block anymore thanks to my previous change ;-) Victor __

Re: [Python-Dev] JUMP_ABSOLUTE in nested if statements

2016-06-18 Thread Victor Stinner
Python has a peephole optimizer which does not remove dead code that it just created. Victor Le 18 juin 2016 23:14, "Obiesie ike-nwosu via Python-Dev" < python-dev@python.org> a écrit : > Hi, > > Could some one give a hand with explaining to me why we have a > JUMP_ABSOLUTE followed by a JUMP_FOR

Re: [Python-Dev] When to use EOFError?

2016-06-21 Thread Victor Stinner
When loading truncated data with pickle, I expect a pickle error, not a generic ValueError nor EOFError. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/ma

Re: [Python-Dev] Breaking up the stdlib (Was: release cadence)

2016-07-05 Thread Victor Stinner
Hi, asyncio is a good example because it wants to evolve faster than the whole "CPython package". Each minor version of CPython adds news features in asyncio. It is not always easy to document these changes. Moreover, the behaviour of some functions also changed in minor versions. asyncio doesn't

[Python-Dev] Status of Python 3.6 PEPs?

2016-07-12 Thread Victor Stinner
Hi, I see many PEPs accepted for Python 3.6, or stil in draft status, but only a few final PEPs. What is happening? Reminder: the deadline for new features in Python 3.6 is 2016-09-12, only in 2 months and these 2 months are summer in the northern hemisphere which means holiday for many of them..

Re: [Python-Dev] Status of Python 3.6 PEPs?

2016-07-12 Thread Victor Stinner
rg/dev/peps/pep-0451/ (this one was already implemented in Python 3.4) Victor 2016-07-12 11:30 GMT+02:00 Chris Angelico : > On Tue, Jul 12, 2016 at 7:26 PM, Victor Stinner > wrote: >> "PEP 499 -- python -m foo should bind sys.modules['foo'] in addition >> to sys.mo

Re: [Python-Dev] Should we fix these errors?

2016-07-22 Thread Victor Stinner
Oh, the first one is a regression that I introduced in the implementation of the PEP 475 (retry syscall on EINTR). I don't think that it can be triggered in practice, because socket handles on Windows are small numbers, so unlikely to be seen as negative. I just fixed it: https://hg.python.org/cpy

Re: [Python-Dev] Should we fix these errors?

2016-07-22 Thread Victor Stinner
2016-07-22 18:21 GMT+02:00 Random832 : >> I just fixed it: >> https://hg.python.org/cpython/rev/6c11f52ab9db > > Does INVALID_SOCKET exist on non-windows systems? Yes, it was already used in almost all places. When I read again the code, in fact I found other places with "fd < 0" or "fd = -1". I

[Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-08 Thread Victor Stinner
Hi, tl;dr I found a way to make CPython 3.6 faster and I validated that there is no performance regression. I'm requesting approval of core developers to start pushing changes. In 2014 during a lunch at Pycon, Larry Hasting told me that he would like to get rid of temporary tuples to call functio

Re: [Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-08 Thread Victor Stinner
2016-08-09 0:40 GMT+02:00 Guido van Rossum : >> tl;dr I found a way to make CPython 3.6 faster and I validated that >> there is no performance regression. > > But is there a performance improvement? Sure. On micro-benchmarks, you can see nice improvements: * getattr(1, "real") becomes 44% faste

Re: [Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-08 Thread Victor Stinner
2016-08-09 0:40 GMT+02:00 Guido van Rossum : > Hm, I agree that those tuples are probably expensive. I recall that > IronPython boasted faster Python calls by doing something closer to the > platform (in their case I'm guessing C# or the CLR :-). To be honest, I didn't expect *any* speedup just by

Re: [Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-08 Thread Victor Stinner
2016-08-09 1:36 GMT+02:00 Brett Cannon : > I just wanted to say I'm excited about this and I'm glad someone is taking > advantage of what Argument Clinic allows for and what I know Larry had > initially hoped AC would make happen! To make "Python" faster, not only a few specific functions, "all" C

[Python-Dev] Subprocess destructor now logs a ResourceWarning

2016-08-17 Thread Victor Stinner
Hi, FYI I made a tiny enhancement in the subprocess module in Python 3.6: Popen destructor now logs a ResourceWarning if the child process is still running when the destructor is called. Keeping a child process in background is bad because it is likely to create zombi process, because Python will

Re: [Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-22 Thread Victor Stinner
Hi, I pushed the most basic implementation of _PyObject_FastCall(), it doesn't support keyword parameters yet: https://hg.python.org/cpython/rev/a1a29d20f52d https://bugs.python.org/issue27128 Then I patched a lot of call sites calling PyObject_Call(), PyObject_CallObject(), PyEval_CallObject(),

Re: [Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-24 Thread Victor Stinner
2016-08-22 10:01 GMT+02:00 Victor Stinner : > The next step is to support keyword parameters. In fact, it's already > supported in all cases except of Python functions: > https://bugs.python.org/issue27809 Serhiy Storchaka proposed to use a single C array for positional and key

Re: [Python-Dev] New calling convention to avoid temporarily tuples when calling functions

2016-08-24 Thread Victor Stinner
Oh, I found a nice pice of CPython history in Modules/_pickle.c. Extract of Python 3.3: - /* A temporary cleaner API for fast single argument function call. XXX: Does caching the argument tuple provides any real performance benefits? A quick benchmark, on a 2.0GHz Athlon64 3

Re: [Python-Dev] File system path encoding on Windows

2016-08-29 Thread Victor Stinner
Hi, tl; dr: just drop byte support and help developers to use Unicode in their application! As you may already know, I dislike your whole project. But first of all, IMHO you should open a separated thread to discuss changes related to the Windows console. I consider that they are unrelated, well

Re: [Python-Dev] File system path encoding on Windows

2016-08-29 Thread Victor Stinner
2016-08-20 21:31 GMT+02:00 Nick Coghlan : > Reading your summary meant this finally clicked with something Victor > has been considering for a while: a "Force UTF-8" switch that told > Python to ignore the locale encoding on Linux, and instead assume > UTF-8 everywhere (command line parameter parsi

Re: [Python-Dev] File system path encoding on Windows

2016-08-29 Thread Victor Stinner
2016-08-24 17:44 GMT+02:00 Steve Dower : > I know Nick and Victor like the idea of a -X flag (or a direct -utf8 flag), > but I prefer more specific environment variables: > > - PYTHONWINDOWSLEGACYSTDIO (for the console changes) > - PYTHONWINDOWSLEGACYPATHENCODING (assuming getfilesystemencoding() i

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 Thread Victor Stinner
Le 30 août 2016 03:10, "Nick Coghlan" a écrit : > However, this view is also why I don't agree with being aggressive in > making this behaviour the default on Windows - I think we should make > it readily available as a provisional feature through a single > cross-platform command line switch and

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 Thread Victor Stinner
2016-08-30 16:31 GMT+02:00 Steve Dower : > It's the > random user on Windows who installed their library that has the problem. > They don't know the fix, and may not know how to apply it (e.g. if it's > their Jupyter notebook that won't find one of their files - no obvious > command line options he

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 Thread Victor Stinner
Le 30 août 2016 8:05 PM, "Nick Coghlan" a écrit : > This seems to be the crux of the disagreement: our perceptions of the > relative risks to native Windows Python applications that currently > work properly on Python 3.5 vs the potential compatibility benefits to > primarily *nix applications tha

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 Thread Victor Stinner
2016-08-30 23:51 GMT+02:00 Victor Stinner : > As I already wrote once, my problem is also tjat I simply have no idea how > much Python 3 code uses bytes filename. For example, does it concern more > than 25% of py3 modules on PyPi, or less than 5%? I made a very quick test on Window

Re: [Python-Dev] File system path encoding on Windows

2016-08-30 Thread Victor Stinner
I made another quick&dirty test on Django 1.10 (I ran Django test suite on my modified Python raising exception on bytes path): I didn't notice any exception related to bytes path. Django seems to only use Unicode for paths. I can try to run more tests if you know some other major Python applicat

Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects

2016-08-30 Thread Victor Stinner
The PEP 445, C API for malloc, allows to plug multiple wrappers and each wrapper has its own "void* context" data. When you register a new wrapper, you store the current context and function to later chain it. See the hooks example: https://www.python.org/dev/peps/pep-0445/#use-case-3-setup-hooks-

Re: [Python-Dev] PEP 467: last round (?)

2016-09-01 Thread Victor Stinner
2016-09-01 21:36 GMT+02:00 Ethan Furman : > Abstract > > > This PEP proposes five small adjustments to the APIs of the ``bytes`` and > ``bytearray`` types to make it easier to operate entirely in the binary > domain: You should add bchr() in the Abstract. > * Deprecate passing single in

Re: [Python-Dev] Patch reviews

2016-09-01 Thread Victor Stinner
2016-08-31 22:31 GMT+02:00 Christian Heimes : > https://bugs.python.org/issue27744 > Add AF_ALG (Linux Kernel crypto) to socket module This patch adds a new socket.sendmsg_afalg() method on Linux. "afalg" comes from AF_ALG which means "Address Family Algorithm". It's documented as "af_alg: User-s

Re: [Python-Dev] PEP 467: last round (?)

2016-09-01 Thread Victor Stinner
2016-09-02 0:04 GMT+02:00 Ethan Furman : > - `fromord` to replace the mistaken purpose of the default constructor To replace a bogus bytes(obj)? If someone writes bytes(obj) but expect to create a byte string from an integer, why not using bchr() to fix the code? Victor __

Re: [Python-Dev] Need help in debugging the python core

2016-09-02 Thread Victor Stinner
2016-09-02 8:49 GMT+02:00 Sajjanshetty, Amresh : > I’m using asyncio and paramiko to multiplex different channels into a single > SSH connection. Hum, asyncio found bugs in CPython. Please try with a more recent version of CPython than 3.4.3 :-/ > Program terminated with signal 11, Segmentation f

Re: [Python-Dev] Need help in debugging the python core

2016-09-02 Thread Victor Stinner
Oh, I forgot to mention that it would help to get the Python traceback on the crash. Try faulthandler: add faulthandler.enable() at the beginning of your program. https://docs.python.org/dev/library/faulthandler.html Maybe I should write once tools to debug such bug :-) Victor ___

Re: [Python-Dev] PEP 467: last round (?)

2016-09-03 Thread Victor Stinner
Yes, this was my point: I don't think that we need a bytearray method to create a mutable string from a single byte. Victor Le samedi 3 septembre 2016, Random832 a écrit : > On Fri, Sep 2, 2016, at 19:44, Ethan Furman wrote: > > The problem with only having `bchr` is that it doesn't help with >

[Python-Dev] Push PEP 528 (use utf8 on Windows) right now, but revert before 3.6 if needed

2016-09-05 Thread Victor Stinner
Hi, I just spoke with Steve Dower (thanks for the current sprint!) about the PEP 528. We somehow agreed that we need to push his implementation of the PEP right now to get enough time to test as much applications as possible on Windows to have a wide view of possible all regressions. The hope is

Re: [Python-Dev] (some) C99 added to PEP 7

2016-09-07 Thread Victor Stinner
2016-09-07 10:56 GMT-07:00 Benjamin Peterson : > To conclude our discussion about using C99 features, I've updated PEP 7 > to allow the following features: > - Standard integer types in and > - ``static inline`` functions > - designated initializers > - intermingled declarations > - boole

[Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-08 Thread Victor Stinner
Hi, I pushed INADA Naoki's implementation of the "compact dict". The hash table now stores indices pointing to a new second table which contains keys and values: it adds one new level of indirection. The table of indices is "compact": use 1, 2, 4 or 8 bytes per indice depending on the size of the

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-08 Thread Victor Stinner
2016-09-08 13:36 GMT-07:00 Guido van Rossum : > IIUC there's one small thing we might still want to change somewhere > after 3.6b1 but before 3.6rc1: the order is not preserved when you > delete some keys and then add some other keys. Apparently PyPy has > come up with a clever solution for this, a

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-09 Thread Victor Stinner
2016-09-09 10:17 GMT-07:00 Guido van Rossum : > - keyword args are ordered > - the namespace passed to a metaclass is ordered by definition order > - ditto for the class __dict__ Maybe we should define exactly "ordered" somewhere the language reference: https://docs.python.org/dev/reference/index.

[Python-Dev] Python 3.7: remove all private C functions from the Python C API?

2016-09-11 Thread Victor Stinner
Hi, Currently, Python has 3 C API: * python core API * regular API: subset of the core API * stable API (ABI?), the Py_LIMITED_API thing: subset of the regular API For practical purpose, all functions are declared in Include/*.h. Basically, Python exposes "everything". There are private function

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-11 Thread Victor Stinner
2016-09-10 23:24 GMT-04:00 Nick Coghlan : > To conform with the updated language spec, implementations just need > to use collections.OrderedDict in 3 places: > > (...) > - storage type for passing kwargs to functions I'm not sure about the "just need" for this one, especially if you care of perfo

Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-11 Thread Victor Stinner
2016-09-10 3:49 GMT-04:00 Ethan Furman : > With __definition_order__ Enum can display the actual creation order of enum > members and methods, while relying on Enum.__dict__.keys() presents a > jumbled mess with many attributes the user never wrote, the enum members > either > appearing /after/ al

Re: [Python-Dev] Python 3.7: remove all private C functions from the Python C API?

2016-09-12 Thread Victor Stinner
2016-09-12 8:23 GMT+02:00 Benjamin Peterson : > That seems like a good idea in abstract. However, the boundaries will > have to be delineated. Many functions beginning _Py are effectively part > of the public API even for "well-behaved" 3rd-party extensions Oh ok, that's also what I expected. So

<    11   12   13   14   15   16   17   18   19   20   >