Re: [Python-Dev] How to watch buildbots?

2018-05-30 Thread Victor Stinner
2018-05-30 14:30 GMT+02:00 Ivan Pozdeev via Python-Dev : > What's the big idea of separate buildbots anyway? I thought the purpose of > CI is to test everything _before_ > it breaks the main codebase. Then it's the job of the contributor rather > than maintainer to fix any breakages. I will answer

[Python-Dev] Withdraw PEP 546? Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2018-05-30 Thread Victor Stinner
Hi, tl; dr I will withdraw the PEP 546 in one week if noboy shows up to finish the implementation. Last year,I wrote the PEP 546 with Cory Benfield: "Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7" https://www.python.org/dev/peps/pep-0546/ The plan was to get a Python 2.7 implementation

Re: [Python-Dev] Withdraw PEP 546? Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2018-05-30 Thread Victor Stinner
2018-05-30 18:02 GMT+02:00 Nathaniel Smith : > On Wed, May 30, 2018, 07:30 Victor Stinner wrote: >> >> Does anyone would benefit of MemoryBIO in Python 2.7? Twisted, >> asyncio, trio, urllib3, anyone else? > > Asyncio and trio are strongly py3-only. Twisted's TLS

[Python-Dev] Microsoft to acquire GitHub for $7.5 billion

2018-06-04 Thread Victor Stinner
Hi, It's now official: Microsoft is acquiring GitHub. https://blog.github.com/2018-06-04-github-microsoft/ https://news.microsoft.com/2018/06/04/microsoft-to-acquire-github-for-7-5-billion/ I copy the news here since Python rely more and more on GitHub. At this point, I have no opinion about th

[Python-Dev] Why not using "except: (...) raise" to cleanup on error?

2018-06-04 Thread Victor Stinner
Hi, I just read a recent bugfix in asyncio: https://github.com/python/cpython/commit/9602643120a509858d0bee4215d7f150e6125468 + try: + await waiter + except Exception: + transport.close() + raise Why only catching "except Exception:"? Why not also catching KeyboardInterrupt or Memor

Re: [Python-Dev] Keeping an eye on Travis CI, AppVeyor and buildbots: revert on regression

2018-06-04 Thread Victor Stinner
2018-05-30 11:33 GMT+02:00 Victor Stinner : > I fixed a few tests which failed randomly. There are still a few, but > the most annoying have been fixed. Quick update a few days later. For an unknown reason, test_multiprocessing_forkserver.TestIgnoreEINTR.test_ignore() started to fai

Re: [Python-Dev] Why not using "except: (...) raise" to cleanup on error?

2018-06-04 Thread Victor Stinner
2018-06-04 18:45 GMT+02:00 Guido van Rossum : > It is currently a general convention in asyncio to only catch Exception, not > BaseException. I consider this a flaw and we should fix it, but it's > unfortunately not so easy -- the tests will fail if you replace all > occurrences of Exception with B

Re: [Python-Dev] Docstrings on builtins

2018-06-04 Thread Victor Stinner
Hi, For Argument Clinic, have a look at https://docs.python.org/dev/howto/clinic.html You can also try to copy/paste code from other files using Argument Clinic and then run "make clinic" to regenerate the generated files. Victor 2018-06-04 23:45 GMT+02:00 Chris Barker via Python-Dev : > Over o

Re: [Python-Dev] Withdraw PEP 546? Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2018-06-05 Thread Victor Stinner
So sorry for Python 2.7, I just rejected my PEP 546, no ssl.MemoryBIO for you! https://www.python.org/dev/peps/pep-0546/#rejection-notice The workaround is to use PyOpenSSL on Python 2.7. Victor 2018-05-30 16:28 GMT+02:00 Victor Stinner : > Hi, > > tl; dr I will withdraw the PEP 546 in

Re: [Python-Dev] Keeping an eye on Travis CI, AppVeyor and buildbots: revert on regression

2018-06-06 Thread Victor Stinner
2018-06-04 21:37 GMT+02:00 Ivan Pozdeev : > https://docs.travis-ci.com/user/running-build-in-debug-mode/ is the official > doc on how to debug a Travis CI build via ssh. Did you already try it? The doc mentions a "[Debug]" button, but I cannot see it whereas I'm logged in in the Python organizatio

[Python-Dev] Idea: reduce GC threshold in development mode (-X dev)

2018-06-08 Thread Victor Stinner
Hi, Yury Selivanov pushed his implementation of the PEP 567 -- Context Variables at January 23, 2018. Yesterday, 4 months after the commit and only 3 weeks before 3.7.0 final release, a crash has been found in the implementation: https://bugs.python.org/issue33803 (it's now fixed, don't worry Ned!

Re: [Python-Dev] Keeping an eye on Travis CI, AppVeyor and buildbots: revert on regression

2018-06-08 Thread Victor Stinner
2018-06-07 4:45 GMT+02:00 Mariatta Wijaya : > Are there APIs we can use to check the status of builbots? Buildbots offer different ways to send notifications: emails and IRC bot for example. If you want to *poll* for recent builds, I don't know. I would suggest to use notifications (push) rather

Re: [Python-Dev] Keeping an eye on Travis CI, AppVeyor and buildbots: revert on regression

2018-06-08 Thread Victor Stinner
2018-06-04 18:31 GMT+02:00 Victor Stinner : > Quick update a few days later. > (...) > Except Windows 7 which has issues with test_asyncio and > multiprocessing tests because this buildbot is slow, it seems like > most CIs are now stable. The bug wasn't specific to this bui

Re: [Python-Dev] Idea: reduce GC threshold in development mode (-X dev)

2018-06-08 Thread Victor Stinner
2018-06-08 10:17 GMT+02:00 Serhiy Storchaka : > Reducing GC threshold can hide other bugs that will be reproducible only in > the release mode (because of earlier releasing of resources or changed order > of destroying objects). > > What is the cost of traversing all objects? Would it be too high i

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update

2018-06-18 Thread Victor Stinner
Hi, I tried two options to add support for FASTCALL on calling an object: add a flag in tp_flags and reuse tp_call, or add a new tp_fastcall slot. I failed to implement correctly any of these two options. There are multiple issues with tp_fastcall: * ABI issue: it's possible to load a C extensio

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update

2018-06-19 Thread Victor Stinner
2018-06-19 13:58 GMT+02:00 Jeroen Demeyer : > Personally, I think that you are exaggerating these issues. I'm not trying to convince you to abandon the idea. I would be happy to be able to use FASTCALL in more cases! I just tried to explain why I chose to abandon my idea. FASTCALL is cute on tiny

Re: [Python-Dev] Can we make METH_FASTCALL public, from Python 3.7? (ref: PEP 579

2018-06-20 Thread Victor Stinner
Hi, I chose to make it private because I wasn't sure about the API. I was right: Serhiy removed keyword arguments from METH_FASTCALL, you now have to use METH_FASTCALL | METH_KEYWORDS to also pass keyword arguments ;-) I don't recall if this change was done in 3.7 or also in 3.6. FASTCALL has been

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods

2018-06-21 Thread Victor Stinner
https://www.python.org/dev/peps/pep-0580/#the-c-call-protocol CCALL_VARARGS: cc_func(PyObject *self, PyObject *args) If we add a new calling convention, I would prefer to reuse the FASTCALL calling convention to pass arguments: pass a PyObject **args array with a size (Py_ssize_t nargs) rather th

Re: [Python-Dev] Policy on refactoring/clean up

2018-06-26 Thread Victor Stinner
Hi, I created call.c when I worked on optimizations. I saw that performances depend on the code locality and that compilers are more eager to inline code when it's in the same file. Moreover, call.c now contains some private functions like function_code_fastcall() or _PyObject_CallFunctionVa() whi

Re: [Python-Dev] [Python-checkins] cpython (3.4): Issue #23840: tokenize.open() now closes the temporary binary file on error to

2015-05-25 Thread Victor Stinner
What is wrong with "except:" in this specific case? Victor Le mardi 26 mai 2015, Yury Selivanov a écrit : > > > On 2015-05-26 12:26 AM, Terry Reedy wrote: > >> +try: >>> +encoding, lines = detect_encoding(buffer.readline) >>> +buffer.seek(0) >>> +text = TextIOWrapper

Re: [Python-Dev] [Python-checkins] cpython (3.4): Issue #23840: tokenize.open() now closes the temporary binary file on error to

2015-05-26 Thread Victor Stinner
2015-05-26 8:44 GMT+02:00 Serhiy Storchaka : > No, finally is not correct there. buffer shouldn't be closed if no exception > is raised. Yep. The binary file must only be closed in case of error, as written in the commit message. > And this is one of the cases in which bare 'except:' is absolutel

Re: [Python-Dev] Computed Goto dispatch for Python 2

2015-05-28 Thread Victor Stinner
2015-05-28 18:07 GMT+02:00 Guido van Rossum : > This patch could save companies like Dropbox a lot of money. We run a ton of > Python code in large datacenters, and while we are slow in moving to Python > 3, we're good at updating to the latest 2.7. I'm not sure that backporting optimizations woul

[Python-Dev] PEP 490: Chain exceptions at C level

2015-06-20 Thread Victor Stinner
python.org/issue23763 Victor PEP: 490 Title: Chain exceptions at C level Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 25-March-2015 Python-Version: 3.6 Abstract Chain exceptions at C level, as already done

Re: [Python-Dev] PEP 490: Chain exceptions at C level

2015-06-23 Thread Victor Stinner
Hi, 2015-06-23 0:10 GMT+02:00 Nick Coghlan : > Chiming in again since I wasn't clear on this aspect last time: I'd also be > +1 on parallel APIs that handle the chaining. > > Since the auto-chaining idea seems largely unpopular, that suggests to me > that a parallel set of APIs would be the most r

Re: [Python-Dev] Adding c-api async protocol support

2015-06-25 Thread Victor Stinner
It looks like the code is currently moving fast. I suggest to wait until Python 3.6 to stabilize the Python C API for async/await. It's a pain to maintain a public API. I hate having to add 2 or 3 versions of a single function :-( Victor 2015-06-25 17:43 GMT+02:00 Yury Selivanov : > Hi Arc, > > >

Re: [Python-Dev] Adding c-api async protocol support

2015-06-25 Thread Victor Stinner
Hi, 2015-06-25 19:25 GMT+02:00 Andrew Svetlov : > P.S. > Thank you Victor so much for your work on asyncio. > Your changes on keeping source tracebacks and raising warnings for > unclosed resources are very helpful. Ah! It's good to know. You're welcome. We can still enhance the source traceback

Re: [Python-Dev] Should asyncio ignore KeyboardInterrupt?

2015-07-04 Thread Victor Stinner
Signal handling is not implemented in asyncio on Windows. I have working patch for that somewhere, it's not merged yet. UDP and SSL are also missing on Windows. Good news: SSL support comes with Python 3.5! Victor Le 4 juil. 2015 09:55, "Terry Reedy" a écrit : > Should the loop.run... methods

Re: [Python-Dev] cpython: Tighten-up code in the set iterator to use an entry pointer rather than

2015-07-07 Thread Victor Stinner
Maybe such issue can be detected if Raymond uses the bug tracker for code review? I remember many cases where Serhiy and Raymond collaborated successfully and wrote better code thanks to the code review. Victor 2015-07-07 9:42 GMT+02:00 Serhiy Storchaka : > On 07.07.15 05:03, raymond.hettinger wr

[Python-Dev] How far to go with user-friendliness

2015-07-18 Thread Victor Stinner
Hi Antoine, I'm really sad to read your message. Antoine is one of the most active Python core developer and it would be a big loss if he really stops contributing. Antoine helped me to stop the drug called "micro optimization", he always has good advices on the Python development. I tried to sta

Re: [Python-Dev] Benchmark Results for Python Default 2015-07-24

2015-07-24 Thread Victor Stinner
Hi, I don't know if it's related but at EuroPython at saw a new website which can also help: http://pybenchmarks.org/ Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mai

Re: [Python-Dev] Datetime SIG created

2015-07-28 Thread Victor Stinner
Thanks Brett :-) I agree that a dedicated mailing list is a good idea. Please stop flooding python-dev and move open discussions to the new mailing list. By the way, I suggest to open multiple threads since multiple topics were discussed in the same thread. So it became difficult to follow the dis

Re: [Python-Dev] updating ensurepip to include wheel

2015-08-05 Thread Victor Stinner
Le 5 août 2015 17:12, "Nick Coghlan" a écrit : > A hard dependency on wheel wouldn't fit into the same category - when > folks are using a build pipeline to minimise the installation > footprint on production systems, the wheel package itself has no > business being installed anywhere other than d

Re: [Python-Dev] updating ensurepip to include wheel

2015-08-07 Thread Victor Stinner
Le 7 août 2015 00:51, "Robert Collins" a écrit : > So - I was in a talk at PyCon AU about conda[*], and the author > believed they were using the latest pip with all the latest caching > features, but their experience (16 minute installs) wasn't that. If an expert user is unaware of having to exp

[Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Victor Stinner
Hi, I read the PEP but I don't understand how it is implemented. For me, it should be a simple preprocessor: - f'x={x}' is replaced with 'x={0}'.format(x) by the compiler - f'x={1+1}' is replaced with 'x={0}'.format(1+1) - f'x={foo()!r}' is replaced with 'x={0!r}'.format(foo()) - ... That's all.

Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Victor Stinner
Le lundi 10 août 2015, Eric V. Smith a écrit : > On 08/10/2015 10:18 AM, Victor Stinner wrote: > > Hi, > > > > I read the PEP but I don't understand how it is implemented. For me, it > > should be a simple preprocessor: > > > > - f'x={x}&#x

[Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
PEP 498: """ Leading whitespace in expressions is skipped Because expressions may begin with a left brace ('{'), there is a problem when parsing such expressions. For example: >>> f'{{k:v for k, v in [(1, 2), (3, 4)]}}' '{k:v for k, v in [(1, 2), (

Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
By the way, I don't think that fu'...' syntax should be allowed. IMHO u'...' was only reintroduced to Python 3.3 to ease transition from Python 2 to Python 3 of the existing u'...' Syntax. Since f'...' is a new syntax, backward compatibility doesn't matter here. Victor

Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Victor Stinner
Le mardi 11 août 2015, Eric V. Smith a écrit : > > Oops, I was thinking of going the other way (str.format -> f''). Yes, I > think you're correct. Ah ok. But in any event, I don't see the distinction between calling > str.format(), and calling each object's __format__ method. Both are > complia

Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
Le mardi 11 août 2015, Eric V. Smith a écrit : > It sounds like you want to disallow leading spaces just to > disallow this one type of expression. > I would like to reduce the number of subtle differences between f-string and str.format(). Victor __

Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
Le mardi 11 août 2015, MRAB a écrit : > > I'm a little bit surprised at seeing this: (...) > We may modify str.format to ignore leading spaces, but IMHO it should not be motivated by the PEP. Victor ___ Python-Dev mailing list Python-Dev@python.org htt

Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-16 Thread Victor Stinner
2015-08-16 7:21 GMT-07:00 Paul Moore : > 2. By far and away the most common use for me would be things like > print(f"Iteration {n}: Took {end-start) seconds"). At the moment I use > str,format() for this, and it's annoyingly verbose. This would be a > big win, and I'm +1 on the PEP for this specif

Re: [Python-Dev] provisional status for asyncio

2015-08-28 Thread Victor Stinner
Hi, 2015-08-27 23:15 GMT+02:00 Yury Selivanov : > Recently, in an asyncio related issue [1], Guido said that new features > for asyncio have to wait till 3.6, since asyncio is no longer a provisional > package. (...) > For example, there is an issue [2] to add starttls support to asyncio. > (...)

Re: [Python-Dev] provisional status for asyncio

2015-08-28 Thread Victor Stinner
2015-08-28 17:44 GMT+02:00 Brett Cannon : > Ultimately it's Larry's call, but I don't see why we couldn't. If we were > talking about something as low-level as the urllib package then I would > agree, but beyond its own tests is there anything in the stdlib that depends > on asyncio? At the beginn

Re: [Python-Dev] provisional status for asyncio

2015-08-28 Thread Victor Stinner
2015-08-28 17:34 GMT+02:00 Yury Selivanov : > I too would enjoy more frequent release schedule of asyncio. The problem is also to allow users to upgrade easily asyncio to retrieve new features, or simply latest bug fixes. As explained in other emails, if asyncio is part of the stlidb, it's *not* p

Re: [Python-Dev] provisional status for asyncio

2015-08-28 Thread Victor Stinner
2015-08-28 18:02 GMT+02:00 Donald Stufft : > Unless we fix the sys.path ordering to make it possible. The problem is the deadline: Python 3.5 final is scheduled for the September, 13. We have 2 weeks to decide what to do with asyncio. I don't think that it's a good idea to modify how sys.path is

[Python-Dev] Is hg.python.org out of memory?

2015-09-03 Thread Victor Stinner
Hi, I got HTTP 503 errors when trying to access http://hg.python.org/ and I got errors when trying to push changes with "hg push" (to ssh://h...@hg.python.org/cpython). I even got this error from "hg push": -- remote: abort: /srv/hg/lib/python/mercurial/mpatch.so: failed to map segment from share

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

2015-09-05 Thread Victor Stinner
> If you really want to allow *any* Python inside a f-string, can you >> please at least explain in the PEP why you consider that it's a good >> thing? >> > > Sigh. We've gone over this on python-ideas. Your objection is not new. > I'm sure that later others will have exactly the same question tha

[Python-Dev] What happens of the Python 3.4 branch?

2015-09-14 Thread Victor Stinner
Hi, Python 3.5.0 was released. What happens to the 3.4 branch in Mercurial? Does it still accept bugfixes, or is it only for security fixes now? Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev

[Python-Dev] My collection of Python 3.5.0 regressions

2015-09-18 Thread Victor Stinner
Hi, Sadly, Python 3.5.0 comes with regressions. FYI I fixed the following regressions: "OSError in os.waitpid() on Windows" http://bugs.python.org/issue25118 "Windows: datetime.datetime.now() raises an OverflowError for date after year 2038" http://bugs.python.org/issue25155 Victor ___

[Python-Dev] My collection of Python 3.5.0 regressions

2015-09-18 Thread Victor Stinner
(Oh hey, I don't understand how I sent the previous email. Mistake with keyboard shortcut in Gmail?) Hi, Sadly, Python 3.5.0 comes with regressions. FYI I fixed the following regressions: "OSError in os.waitpid() on Windows" http://bugs.python.org/issue25118 "Windows: datetime.datetime.now() ra

Re: [Python-Dev] Make stacklevel=2 by default in warnings.warn()

2015-09-21 Thread Victor Stinner
Would it be too verbose to display two frames or more by default? Maybe depending on the action (ex: only if the warning is emitted only once). Victor 2015-09-20 8:44 GMT+02:00 Serhiy Storchaka : > For now the default value of the stacklevel parameter in warnings.warn() is > 1. But in most cases

Re: [Python-Dev] Make stacklevel=2 by default in warnings.warn()

2015-09-21 Thread Victor Stinner
2015-09-20 8:44 GMT+02:00 Serhiy Storchaka : > I propose to make the default value of stacklevel to be 2. > I think that unlikely this will break existing code. Consider this simple script: --- import warnings warnings.warn("here") --- Currrent output: --- x.py:3: UserWarning: here warnings.war

Re: [Python-Dev] PEP 495 accepted

2015-09-21 Thread Victor Stinner
More context ;-) PEP 0495 -- Local Time Disambiguation Abstract This PEP adds a new attribute fold to instances of the datetime.time and datetime.datetime classes that can be used to differentiate between two moments in time for which local times are the same. The allowed values for the fold att

[Python-Dev] Issue #25256: Add sys.debug_build?

2015-10-02 Thread Victor Stinner
Hi, I created the issue "Add sys.debug_build public variable to check if Python was compiled in debug mode": http://bugs.python.org/issue25256 I would like to add an obvious way to check if Python was compiled in debug mode, instead of having hacks/tips to check it. On the Internet, I found vario

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

2015-10-02 Thread Victor Stinner
2015-10-02 9:37 GMT+02:00 Nick Coghlan : > Spell it as "sys.implementation.debug_build" and I'm in favour. Oh, in fact, I don't have no preference between sys.debug_flag and sys.implementation.debug_flag. If I understood correctly, Serhiy would prefer sys.implementation.debug_flag because he doesn

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

2015-10-02 Thread Victor Stinner
2015-10-02 13:16 GMT+02:00 Nir Soffer : > Whats wrong with: > sysconfig.get_config_var('Py_DEBUG') > 0 Again, refer to my first message "On the Internet, I found various recipes to check if Python is compiled is debug mode. Sadly, some of them are not portable." I don't think that sysconfig.

Re: [Python-Dev] Migrating to Python 3: the python 3 install issue

2015-10-02 Thread Victor Stinner
Fedora 23 (scheduled for the end of this month) will only come with python3 (/usr/bin/python3), no python2 (nor python), *in the base system*. Obviously, it will be possible to install Python 2 to install applications not compatible with Python 3 yet. Note: the current development version is Fedo

Re: [Python-Dev] Migrating to Python 3: the python 3 install issue

2015-10-02 Thread Victor Stinner
(grr, again i sent a draft by mistake, sorry about that) Fedora 23 (scheduled for the end of this month) will only come with python3 (/usr/bin/python3), no python2 (nor python), *in the base system*. Obviously, it will be possible to install Python 2 to install applications not compatible with Pyt

Re: [Python-Dev] under what circumstances can python still exhibit "high water mark" memory usage?

2015-10-14 Thread Victor Stinner
Hi, You may also try tracemalloc to get stats of the Python memory usage ;-) The Python memory allocator was optimized in Python 3.3: it now uses mmap() when available (on UNIX), it helps to reduce the fragmentation of the heap memory. Since Python 3.4, VirtualAlloc() is used for the same purpose

Re: [Python-Dev] PEP 506 secrets module

2015-10-16 Thread Victor Stinner
Hi, I like the PEP. IMHO it's a better solution than using a CPRNG for random by default. I suggest to raise an error if token_bytes(n) if calls with n < 16 bytes (128 bits). Well, I'm not sure that 16 is the good compromise between performance and security, but we must enforce users to use a min

Re: [Python-Dev] PEP 506 secrets module

2015-10-20 Thread Victor Stinner
2015-10-20 11:11 GMT+02:00 Nick Coghlan : > Folks wanting to simulate die rolls should be using the random module > rather than the secrets module anyway, Hum, why? Dices are used in Casino where security matters because it costs money. A bad API can be more likely misused and introduce security

Re: [Python-Dev] Generated Bytecode ...

2015-10-23 Thread Victor Stinner
Hi, 2015-10-22 19:02 GMT+02:00 Brett Cannon : > It's not specified anywhere; it's just what the peepholer decides to remove. > The exact code can be found at > https://hg.python.org/cpython/file/default/Python/peephole.c . There has > been talk in the past for adding a -X flag to disable the peeph

Re: [Python-Dev] Generated Bytecode ...

2015-10-26 Thread Victor Stinner
2015-10-24 4:34 GMT+09:00 Terry Reedy : > How about -x nopeep to specifically skip the peephole optimizer? Raymond wrote "IIRC, the code was never generated in the first place (before the peephole pass)." So "nopeep" would have a different purpose. Victor

[Python-Dev] Second milestone of FAT Python

2015-11-04 Thread Victor Stinner
Hi, I'm writing a new "FAT Python" project to try to implement optimizations in CPython (inlining, constant folding, move invariants out of loops, etc.) using a "static" optimizer (not a JIT). For the background, see the thread on python-ideas: https://mail.python.org/pipermail/python-ideas/2015-O

Re: [Python-Dev] Second milestone of FAT Python

2015-11-04 Thread Victor Stinner
2015-11-04 21:14 GMT+01:00 Sven R. Kunze : > Do you think there will be any change of merging this back into CPython? If I understood correctly the first feedback on python-ideas, there were mostly positive, so I believe that it would be possible to merge the FAT mode into CPython. The changes req

[Python-Dev] PYC filename and optimization

2015-11-05 Thread Victor Stinner
Hi, While working on the FAT Python project to try to optimize CPython, I have an issue with the cached .pyc files. I would like to have one .pyc without optimization when the FAT mode is disabled (default) and one .pyc with optimization when the FAT mode is enabled (-F command line option). Curre

Re: [Python-Dev] Second milestone of FAT Python

2015-11-05 Thread Victor Stinner
Hi, 2015-11-04 23:20 GMT+01:00 Terry Reedy : > On 11/4/2015 3:50 AM, Victor Stinner wrote: >> (...) I reached the second milestone: >> it's now possible to run the full Python test suite with these >> optimizations enabled. It confirms that the optimizations don'

Re: [Python-Dev] Support of UTF-16 and UTF-32 source encodings

2015-11-14 Thread Victor Stinner
These encodings are rarely used. I don't think that any text editor use them. Editors use ascii, latin1, utf8 and... all locale encoding. But I don't know any OS using UTF-16 as a locale encoding. UTF-32 wastes disk space. Ok, even if it exists, Python already accepts a very wide range of encoding

Re: [Python-Dev] [Python-checkins] Daily reference leaks (97e2a6810f7f): sum=10

2015-11-17 Thread Victor Stinner
Hi, I opened the http://bugs.python.org/issue25645 It looks like a simple regression in _pickle. I wrote a patch. Victor 2015-11-16 19:49 GMT+01:00 Brett Cannon : > Just an FYI there seems to be a consistent, minor refcount leak found by > test_capi that has been there for what seems like a cou

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

2015-11-17 Thread Victor Stinner
2015-11-17 1:00 GMT+01:00 Guido van Rossum : > Hm, making Christian the BDFL-delegate would mean two out of three > authors *and* the BDFL-delegate all working for Red Hat, which clearly > has a stake (and IIUC has already committed to this approach ahead of > PEP approval). SO then it would look l

Re: [Python-Dev] The AttributeError/__getattr__ mechanism

2015-11-27 Thread Victor Stinner
Hi, As R. David Murray wrote on the issue, the python-ideas mailing list is better suited to start such discussion ;-) Victor 2015-11-27 9:40 GMT+01:00 王珺 : > Hello everyone: > > I'm suggesting a modification to the AttributeError/__getattr__ mechanism, > see issue25634: > http://bugs.python.org

Re: [Python-Dev] Avoiding CPython performance regressions

2015-12-01 Thread Victor Stinner
2015-12-01 10:49 GMT+01:00 Fabio Zadrozny : > As for the graph, it should be easy to customize (and I'm open to > suggestions). In the case, as it is, red is slower and blue is faster (so, > for instance in > https://www.speedtin.com/reports/1_CPython27x_Performance_Over_Time For me, -10% means "f

Re: [Python-Dev] Deleting with setting C API functions

2015-12-02 Thread Victor Stinner
2015-12-02 9:42 GMT+01:00 Serhiy Storchaka : > You have enough time to update your projects, and you can update them > uniformly for all versions. And may be you will found few weird bugs related > to misuse of Set* API. Did you check popular projects using C extensions to check if they call Set*(

Re: [Python-Dev] Deleting with setting C API functions

2015-12-02 Thread Victor Stinner
2015-12-02 15:40 GMT+01:00 Random832 : > Are there plans for a Python 4? No. Don't. Don't schedule any "removal" or *any* kind of "break backward compatibility" anymore, or you will definetly kill the Python community. It will probably take 10 years or more to convert *all* Python 2 code around t

[Python-Dev] Third milestone of FAT Python

2015-12-04 Thread Victor Stinner
Hi, I implemented 3 new optimizations in FAT Python: loop unrolling, constant folding and copy builtin functions to constants. In the previous thread, Terry Reedy asked me if the test suite is complete enough to ensure that FAT Python doesn't break the Python semantic. I can now say that the secon

Re: [Python-Dev] Third milestone of FAT Python

2015-12-04 Thread Victor Stinner
2015-12-04 20:16 GMT+01:00 MRAB : > On 2015-12-04 12:49, Victor Stinner wrote: > [snip] > > I don't think that's constant folding, but constant _propagation_. > > Constant folding is when, say, "1 + 2" replaced by "2". Oh, you're right. I

[Python-Dev] Python semantic: Is it ok to replace not x == y with x != y? (no)

2015-12-15 Thread Victor Stinner
Hi, I implemented more constant folding optimizations in my FAT Python project, but it looks like I made a subtle change in the Python semantic. Replacing "not x == y" with "x != y" changes the behaviour of Python. For example, this optimization breaks test_unittest because unittest.mock._Call im

Re: [Python-Dev] Python semantic: Is it ok to replace not x == y with x != y? (no)

2015-12-15 Thread Victor Stinner
t be removed to not break the Python semantic. Python has also some funny objects like math.nan: >>> math.nan != math.nan True >>> math.nan == math.nan False >>> math.nan < math.nan False >>> math.nan > math.nan False >>> math.nan <= math.nan

Re: [Python-Dev] Third milestone of FAT Python

2015-12-15 Thread Victor Stinner
2015-12-15 12:23 GMT+01:00 Franklin? Lee : > I was thinking (as an alternative to versioning dicts) about a > dictionary which would be able to return name/value pairs, which would > also be internally used by the dictionary. This would be way less > sensitive to irrelevant changes in the scope dic

Re: [Python-Dev] Third milestone of FAT Python

2015-12-15 Thread Victor Stinner
2015-12-15 22:10 GMT+01:00 Franklin? Lee : > (Stealing your style of headers.) I'm using reStructured Text, it's not really a new style :-) > Overhead > > > If inner functions are being created a lot, that's extra work. But I > guess you should expect a lot of overhead if you're doing s

Re: [Python-Dev] Third milestone of FAT Python

2015-12-15 Thread Victor Stinner
Le mercredi 16 décembre 2015, Franklin? Lee a écrit : > > I am confident that the time overhead and the savings will beat the > versioning dict. The versioning dict method has to save a reference to > the variable value and a reference to the name, and regularly test > whether the dict has changed

Re: [Python-Dev] New poll about a macro for safe reference replacing

2015-12-16 Thread Victor Stinner
2015-12-16 15:12 GMT+01:00 Serhiy Storchaka : > Here are names gained the largest numbers of votes plus names proposed > during polling. > > 1. Py_SETREF +1: obvious name > 2. Py_DECREF_REPLACE -1: too long > 3. Py_REPLACE 0: less explicit than but: not mention of reference > 4. Py_SET_POINTE

Re: [Python-Dev] New poll about a macro for safe reference replacing

2015-12-16 Thread Victor Stinner
2015-12-16 16:12 GMT+01:00 Serhiy Storchaka : > Originally I proposed pairs of functions with and withot X in the name (as > Py_DECREF/Py_XDECREF). In this poll this detail is omitted for clearness. > Later we can create a new poll if needed. I would prefer a single macro to avoid bugs, I don't th

Re: [Python-Dev] Idea: Dictionary references

2015-12-17 Thread Victor Stinner
2015-12-17 11:54 GMT+01:00 Franklin? Lee : > My suggestion should improve *all* function calls which refer to > outside names. Ok, I now think that you should stop hijacking the FAT Python thread. I start a new thread. IMHO your dictionary reference idea is completly unrelated to FAT Python. FAT

Re: [Python-Dev] Idea: Dictionary references

2015-12-17 Thread Victor Stinner
2015-12-17 16:38 GMT+01:00 Franklin? Lee : > Yeah, maybe it's out of the scope of bytecode optimization. But I > think this would make guards unnecessary, since once a name is found, > there's a quick way to refer to it. FAT Python requires and supports various kinds of guards: * type of a functio

Re: [Python-Dev] Idea: Dictionary references

2015-12-17 Thread Victor Stinner
2015-12-17 23:17 GMT+01:00 Andrew Barnert via Python-Dev : > Builtins do two dict lookups. > > So, the only thing you can optimize there is builtins. But maybe that's worth > it. FYI I implemented an optimization in FAT Python to avoid lookups for builtin functions, builtin functions are copied t

Re: [Python-Dev] Idea: Dictionary references

2015-12-17 Thread Victor Stinner
Le vendredi 18 décembre 2015, Andrew Barnert via Python-Dev < python-dev@python.org> a écrit : > > >> Builtins do two dict lookups. > > > > Two? > > Actually, I guess three: first you fail to find the name in globals, then > you find __builtins__ in globals, then you find the name in __builtins__ o

Re: [Python-Dev] Change the repr for datetime.timedelta (was Re: Asynchronous context manager in a typical network server)

2015-12-22 Thread Victor Stinner
repr() with keywords is called a method, no? Like isoformat() Victor Le lundi 21 décembre 2015, Guido van Rossum a écrit : > I still think the repr change to use keywords has a good chance for 3.6. > > --Guido (mobile) > On Dec 21, 2015 2:09 PM, "Chris Barker" > wrote: > >> On Sun, Dec 20, 201

Re: [Python-Dev] Branches in which to fix the SSL tests

2016-01-07 Thread Victor Stinner
Hi, 2016-01-07 17:32 GMT+01:00 Larry Hastings : > On 01/06/2016 10:06 PM, Martin Panter wrote: > > According to Larry > , > 3.4.4 was the last bug fix release for 3.4, so I assumed the 3.4 > branch should now be in security-fi

[Python-Dev] PEP 509: Add a private version to dict

2016-01-11 Thread Victor Stinner
s/pep-0509/ PEP: 509 Title: Add a private version to dict Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-January-2016 Python-Version: 3.6 Abstract Add a new private version to builtin ``dict``

Re: [Python-Dev] PEP 509: Add a private version to dict

2016-01-11 Thread Victor Stinner
the user? > > On Mon, Jan 11, 2016 at 6:49 PM, Victor Stinner > wrote: > > Hi, > > > > After a first round on python-ideas, here is the second version of my > > PEP. The main changes since the first version are that the dictionary > > version is no more expos

Re: [Python-Dev] PEP 509: Add a private version to dict

2016-01-11 Thread Victor Stinner
2016-01-12 0:07 GMT+01:00 Gregory P. Smith : >> Changes >> === >> >> (...) > > Please be more explicit about what tests you are performing on the values. > setitem's "if the value is different" really should mean "if value is not > dict['key']". similarly for update, there should never be equa

Re: [Python-Dev] PEP 509: Add a private version to dict

2016-01-12 Thread Victor Stinner
Well, it was just a remark. 2016-01-12 0:35 GMT+01:00 Andrew Barnert : > Are you saying that d[key] = d[key] may or may not increment the version, so > any optimizer can't rely on the fact that it doesn't? Optimizers don't have to rely on this exactly behaviour. Not incrementing the version on s

Re: [Python-Dev] PEP 509

2016-01-12 Thread Victor Stinner
2016-01-12 19:52 GMT+01:00 Ethan Furman : > [1] We're not going to call it __version__ are we? Seems like > __cache_token__ is a much better name. See the online version to the most recent version of the PEP: https://www.python.org/dev/peps/pep-0509/ In the first version I proposed to expose the

Re: [Python-Dev] PEP 509

2016-01-12 Thread Victor Stinner
2016-01-12 19:34 GMT+01:00 Jim J. Jewett : > (1) Please make it clear within the abstract what counts as a change. I don't think that an abstract must give the long list of cases when the version is modified or not. It's explained in detail at: https://www.python.org/dev/peps/pep-0509/#changes >

[Python-Dev] PEP 510: Specialize functions with guards

2016-01-12 Thread Victor Stinner
-Modified: $Date$ Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-January-2016 Python-Version: 3.6 Abstract Add functions to the Python C API to specialize pure Python functions: add specialized codes with guards. It allows to implement static

Re: [Python-Dev] PEP 509

2016-01-12 Thread Victor Stinner
2016-01-12 23:24 GMT+01:00 Ethan Furman : > Even if not exposed at the Python layer, it's still exposed when working at > the C layer. Is __version__ any less confusing there? (I only work in C > when working on Python, and only occasionally, so my question is real.) Fields of the PyDictObject m

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Victor Stinner
Hi, 2016-01-13 20:32 GMT+01:00 Matthew Paulson : > I've spent some time performing memory leak analysis while using Python in an > embedded configuration. Hum, did you try tracemalloc? https://docs.python.org/dev/library/tracemalloc.html https://pytracemalloc.readthedocs.org/ > Is there someon

Re: [Python-Dev] PEP 510: Specialize functions with guards

2016-01-13 Thread Victor Stinner
I extracted a patch from my FAT Python project to implement the PEP 510: https://bugs.python.org/issue26098 FYI I also extracted the runtime part of the FAT Python optimizer and put it on GitHub: https://github.com/haypo/fat The fat module provides specialize(), get_specialized() and replace_cons

Re: [Python-Dev] Update PEP 7 to require curly braces in C

2016-01-17 Thread Victor Stinner
I like if without braces when the body is only one line, especially when there is no else block. Victor Le dimanche 17 janvier 2016, Brett Cannon a écrit : > While doing a review of http://bugs.python.org/review/26129/ I asked to > have curly braces put around all `if` statement bodies. Serhiy

<    1   2   3   4   5   6   7   8   9   10   >