Re: [Python-Dev] Standard library vs Standard distribution?
On Fri, 30 Nov 2018 at 00:17, Steven D'Aprano wrote: > Especially not *Paul's* problems, as I understand he personally is > reasonably satisfied with the stdlib and doesn't use any of those > third-party distros. (Paul, did I get that right?) That's correct. And not just "reasonably satisfied" - I strongly prefer the python.org distribution (with "just" the stdlib) over those others. But we should be careful making comparisons like this. The benefit of the stdlib is as a "common denominator" core functionality. It's not that I don't need other modules. I use data analysis packages a lot, but I still prefer the core distribution (plus PyPI) over Anaconda, because when I'm *not* using data analysis packages (which I use in a virtualenv) my "base" environment is something that exists in *every* Python installation, no matter what distribution. And the core is available in places where I *can't* use extra modules. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
On Fri, Nov 30, 2018 at 2:12 AM Brett Cannon wrote: > > > On Thu, 29 Nov 2018 at 14:12, Andrew Svetlov > wrote: > >> Neither http.client nor http.server doesn't support compression >> (gzip/compress/deflate) at all. >> I doubt if we want to add this feature: for client better to use requests >> or, well, aiohttp. >> The same for servers: almost any production ready web server from PyPI >> supports compression. >> > > There was actually a PR to add compressions support to http.server but I > closed it in the name of maintainability since http.server, as you said, > isn't for production use so compression isn't critical. > > I remember this PR and agree with your decision. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard library vs Standard distribution?
On Fri, 30 Nov 2018 11:14:47 +1100 Steven D'Aprano wrote: > > There are plenty of other languages that come with a tiny stdlib and > leave everything else to third parties. Outside of those like > Javascript, which has a privileged position due to it being the standard > browser scripting language (and is backed by an ISO standard and at > least one major companies vigourously driving it), how is that working > out for them? And even for Javascript, that seems to be a problem, with the myriad of dependencies JS apps seem to have for almost trivial matters, and the security issues that come with relying on so many (sometimes ill-maintained) third-party libraries. Actually, PyPI is also been targeted these days, even though hopefully it didn't (yet?) have the ramifications such attacks have had in the JS world (see e.g. the recent "event-stream" incident: https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident ) I agree with you that the stdlib's "batteries included" is a major feature of Python. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard library vs Standard distribution?
On Fri, Nov 30, 2018 at 01:45:17AM +, Oscar Benjamin wrote: > They do include much more than we need so I'd agree that Anaconda is a > nuclear reactor. The things I want from it are not though. Numpy is > not a nuclear reactor: at it's core it's just providing a > multidimensional array. And a metric tonne of functions in linear algebra, root finding, special functions, polynomials, statistics, generation of random numbers, datetime calculations, FFTs, interfacing with C, and including *eight* different specialist variations of sorting alone. Just because some people don't use the entire output of the nuclear reactor, doesn't mean it isn't one. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2018-11-23 - 2018-11-30)
Python tracker at https://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open6880 (+15)
closed 40234 (+47)
total 47114 (+62)
Open issues with patches: 2736
Issues opened (41)
==
#33015: Fix function cast warning in thread_pthread.h
https://bugs.python.org/issue33015 reopened by vstinner
#35302: create_connection with local_addr misses valid socket bindings
https://bugs.python.org/issue35302 opened by kyuupichan
#35305: subprocess.Popen(['/sbin/ldconfig', '-p'], stdin=PIPE) itself
https://bugs.python.org/issue35305 opened by Henrik Bengtsson
#35306: OSError [WinError 123] when testing if pathlib.Path('*') (aste
https://bugs.python.org/issue35306 opened by jimbo1qaz_
#35307: Command line help example is missing "--prompt" option
https://bugs.python.org/issue35307 opened by daftwullie
#35310: select which was interrupted by EINTR isn't re-run if the time
https://bugs.python.org/issue35310 opened by Brian Maissy
#35311: exception unpickling error causes `multiprocessing.Pool` to ha
https://bugs.python.org/issue35311 opened by Anthony Sottile
#35312: lib2to3.pgen2.parse.ParseError is not roundtrip pickleable
https://bugs.python.org/issue35312 opened by Anthony Sottile
#35316: test_eintr fails randomly on macOS
https://bugs.python.org/issue35316 opened by vstinner
#35318: Check accuracy of str() doc string for its encoding argument
https://bugs.python.org/issue35318 opened by rhettinger
#35319: pkgutil.get_data() is a wrapper for a deprecated class
https://bugs.python.org/issue35319 opened by Kevin.Norris
#35321: None _frozen_importlib.__spec__.origin attribute
https://bugs.python.org/issue35321 opened by maggyero
#35324: ssl: FileNotFoundError when do handshake
https://bugs.python.org/issue35324 opened by joseph...@yahoo.com
#35325: imp.find_module() return value documentation discrepancy
https://bugs.python.org/issue35325 opened by StefanBauerTT
#35327: Using skipTest with subTest gives a misleading UI.
https://bugs.python.org/issue35327 opened by p-ganssle
#35328: Set a environment variable for venv prompt
https://bugs.python.org/issue35328 opened by batisteo
#35329: Documentation - capitalization issue
https://bugs.python.org/issue35329 opened by Strijker
#35330: When using mock to wrap an existing object, side_effect requir
https://bugs.python.org/issue35330 opened by noamraph
#35331: Incorrect __module__ attribute for _struct.Struct and perhaps
https://bugs.python.org/issue35331 opened by bup
#35332: shutil.rmtree(..., ignore_errors=True) doesn't ignore errors f
https://bugs.python.org/issue35332 opened by rabraham
#35333: Rename private structs to use names closer to types
https://bugs.python.org/issue35333 opened by vstinner
#35335: msgfmt should be able to merge more than one po file
https://bugs.python.org/issue35335 opened by s-ball
#35337: Check index in PyTuple_GET_ITEM/PyTuple_SET_ITEM in debug mode
https://bugs.python.org/issue35337 opened by vstinner
#35338: set union/intersection/difference could accept zero arguments
https://bugs.python.org/issue35338 opened by carandraug
#35341: Add generic version of OrderedDict to typing module
https://bugs.python.org/issue35341 opened by itoijala
#35342: email "default" policy raises exception iterating over unparse
https://bugs.python.org/issue35342 opened by rptb1
#35344: platform: get macOS version rather than darwin version?
https://bugs.python.org/issue35344 opened by vstinner
#35346: Modernize Lib/platform.py code
https://bugs.python.org/issue35346 opened by vstinner
#35348: Problems with handling the file command output in platform.arc
https://bugs.python.org/issue35348 opened by serhiy.storchaka
#35350: importing "ctypes" immediately causes a segmentation fault
https://bugs.python.org/issue35350 opened by n0s69z
#35351: LTOFLAGS are passed to BASECFLAGS when using LTO
https://bugs.python.org/issue35351 opened by cstratak
#35352: test_asyncio fails on RHEL8
https://bugs.python.org/issue35352 opened by cstratak
#35353: Add frame command to pdb
https://bugs.python.org/issue35353 opened by cnklein
#35354: Generator functions stack overflow
https://bugs.python.org/issue35354 opened by asdwqii
#35357: unittest.mock.call can't represent calls to a method called 'p
https://bugs.python.org/issue35357 opened by cjw296
#35358: avoid '-' in importlib.import_module and builtins.__import__
https://bugs.python.org/issue35358 opened by matrixise
#35359: [2.7][Windows] Define _CRT_SECURE_NO_WARNINGS to build Modules
https://bugs.python.org/issue35359 opened by vstinner
#35360: [Windows] Update SQLite dependency
https://bugs.python.org/issue35360 opened by vstinner
#35361: Update libffi dependency to 3.2.1?
https://bugs.python.org/issue35361 opened by vstinner
#35362: list inheritor with abstract content does n
Re: [Python-Dev] C API changes
On 29Nov2018 2206, Armin Rigo wrote: On Thu, 29 Nov 2018 at 18:19, Steve Dower wrote: quo. We continue to not be able to change CPython internals at all, since that will break people using option B. No? That will only break users if they only have an option-B ``foo.cpython-318m-x86_64-linux-gnu.so``, no option-A .so and no source code, and want to run it elsewhere than CPython 3.18. That's the same as today. If you want option-B .so for N versions of CPython, recompile the source code N times. Just to be clear, if done correctly there should be no need for #ifdefs in the source code of the extension module. The problem is that if option B remains as compatible as it is today, we can't make option A faster enough to be attractive. The marketing pitch for this looks like: "rewrite all your existing code to be slower but works with PyPy, or don't rewrite your existing code and it'll be fastest with CPython and won't break in the future". This is status quo (where option A today is something like CFFI or Cython), and we can already see how many people have made the switch (FWIW, I totally prefer Cython over pure C for my own projects :) ). My proposed marketing pitch is: "rewrite your existing code to be forward-compatible today and faster in the future without more work, or be prepared to rewrite/update your source code for each CPython release to remain compatible with the low level API". The promise of "faster in the future" needs to be justified (and I think there's plenty of precedent in PyPy, Larry's Gilectomy and the various JavaScript VMs to assume that we can do it). We've already done enough investigation to know that making the runtime faster requires changing the low level APIs, and otherwise we're stuck in a local optima. Offering a stable, loosely coupled option A and then *planning* to break the low level APIs each version in the name of performance is the only realistic way to change what we're currently doing. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API changes
On Fri, 30 Nov 2018 09:22:30 -0800 Steve Dower wrote: > > My proposed marketing pitch is: "rewrite your existing code to be > forward-compatible today and faster in the future without more work, or > be prepared to rewrite/update your source code for each CPython release > to remain compatible with the low level API". The promise of "faster in > the future" needs to be justified (and I think there's plenty of > precedent in PyPy, Larry's Gilectomy and the various JavaScript VMs to > assume that we can do it). I think that should be qualified. Technically it's certainly possible to have a faster CPython with different internals. Socially and organisationally I'm not sure we're equipped to achieve it. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] C API changes
Steve Dower wrote: > My proposed marketing pitch is: "rewrite your existing code to be > forward-compatible today and faster in the future without more work, or > be prepared to rewrite/update your source code for each CPython release > to remain compatible with the low level API". The promise of "faster in > the future" needs to be justified (and I think there's plenty of > precedent in PyPy, Larry's Gilectomy and the various JavaScript VMs to > assume that we can do it). It's hard to discuss this in the abstract without knowing how big the breakage between each version is going to be. But for the scientific ecosystem this sounds a bit like a potential Python-4.0 breakage, which was universally rejected (so far). In the extreme case I can imagine people staying on 3.7. But it really depends on the nature of the changes. Stefan Krah ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API changes
On 2018-11-29, Armin Rigo wrote: > ...Also, although I'm discussing it here, I think the whole approach > would be better if done as a third-party extension for now, without > requiring changes to CPython---just use the existing C API to > implement the CPython version. Hello Armin, Thank you for providing your input on this subject. I too like the idea of an API "shim layer" as a separate project. What do you think of writing the shim layer in C++? I'm not a C++ programmer but my understanding is that modern C++ compilers are much better than years ago. Using C++ would allow us to provide a higher level API with smaller runtime costs. However, it would require that any project using the shim layer would have to be compiled with a C++ compiler (CPython and PyPy could still expose a C compatible API). Perhaps it is a bad idea. If someone does create such a shim layer, it will already be challenging to convince extension authors to move to it. If it requires them to switch to using a C++ compiler rather than a C compiler, maybe that's too much effort. OTOH, with C++ I think you could do things like use smart pointers to automatically handle refcounts on the handles. Or maybe we should just skip C++ and implement the layer in Rust. Then the Rust borrow checker can handle the refcounts. ;-) Regards, Neil ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API changes
Hi Steve, On 30/11/2018, Steve Dower wrote: > On 29Nov2018 2206, Armin Rigo wrote: >> On Thu, 29 Nov 2018 at 18:19, Steve Dower wrote: >>> quo. We continue to not be able to change CPython internals at all, >>> since that will break people using option B. >> >> No? That will only break users if they only have an option-B >> ``foo.cpython-318m-x86_64-linux-gnu.so``, no option-A .so and no >> source code, and want to run it elsewhere than CPython 3.18. That's >> the same as today. If you want option-B .so for N versions of >> CPython, recompile the source code N times. >> >> Just to be clear, if done correctly there should be no need for >> #ifdefs in the source code of the extension module. > > The problem is that if option B remains as compatible as it is today, we > can't make option A faster enough to be attractive. The marketing pitch > for this looks like: "rewrite all your existing code to be slower but > works with PyPy, or don't rewrite your existing code and it'll be > fastest with CPython and won't break in the future". This is status quo > (where option A today is something like CFFI or Cython), and we can > already see how many people have made the switch (FWIW, I totally prefer > Cython over pure C for my own projects :) ). > > My proposed marketing pitch is: "rewrite your existing code to be > forward-compatible today and faster in the future without more work, or > be prepared to rewrite/update your source code for each CPython release > to remain compatible with the low level API". > (...) Discussing marketing pitches on python-dev is not one of my favorite past-times, so I'll excuse myself out of this conversation. Instead, I might try to implement the basics, check out the performance on CPython and on PyPy, and seek out interest---I'm thinking about Cython, for example, which might relatively easily be adapted to generate that kind of code. This might be a solution for the poor performance of Cython on PyPy... If everything works out, maybe I'll come back here at some point, with the argument "the CPython C API is blocking CPython from evolution more and more? Here's one possible path forward." A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API changes
On Fri, 30 Nov 2018 13:06:11 -0600 Neil Schemenauer wrote: > On 2018-11-29, Armin Rigo wrote: > > ...Also, although I'm discussing it here, I think the whole approach > > would be better if done as a third-party extension for now, without > > requiring changes to CPython---just use the existing C API to > > implement the CPython version. > > Hello Armin, > > Thank you for providing your input on this subject. I too like the > idea of an API "shim layer" as a separate project. > > What do you think of writing the shim layer in C++? I'm not a C++ > programmer but my understanding is that modern C++ compilers are > much better than years ago. Using C++ would allow us to provide a > higher level API with smaller runtime costs. The main problem with exposing a C++ *API* is that all people implementing that API suddenly must understand and implement the C++ *ABI* (with itself varies from platform to platform :-)). That's trivially easy if your implementation is itself written in C++, but not if it's written in something else such as RPython, Java, Rust, etc. C is really the lingua franca when exposing an interface that can be understood, implemented and/or interfaced with from many different languages. So I'd turn the proposal on its head: you can implement the internals of your interpreter or object layer in C++ (and indeed I think it would be crazy to start writing a new Python VM in raw C), but you should still expose a C-compatible API for third-party providers and consumers. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API changes
On 30Nov2018 1133, Antoine Pitrou wrote: On Fri, 30 Nov 2018 13:06:11 -0600 Neil Schemenauer wrote: On 2018-11-29, Armin Rigo wrote: ...Also, although I'm discussing it here, I think the whole approach would be better if done as a third-party extension for now, without requiring changes to CPython---just use the existing C API to implement the CPython version. Hello Armin, Thank you for providing your input on this subject. I too like the idea of an API "shim layer" as a separate project. What do you think of writing the shim layer in C++? I'm not a C++ programmer but my understanding is that modern C++ compilers are much better than years ago. Using C++ would allow us to provide a higher level API with smaller runtime costs. The main problem with exposing a C++ *API* is that all people implementing that API suddenly must understand and implement the C++ *ABI* (with itself varies from platform to platform :-)). That's trivially easy if your implementation is itself written in C++, but not if it's written in something else such as RPython, Java, Rust, etc. C is really the lingua franca when exposing an interface that can be understood, implemented and/or interfaced with from many different languages. So I'd turn the proposal on its head: you can implement the internals of your interpreter or object layer in C++ (and indeed I think it would be crazy to start writing a new Python VM in raw C), but you should still expose a C-compatible API for third-party providers and consumers. I totally agree with Antoine here. C++ is great for internals, but not the public interfaces. The one additional point I'd add is that there are other ABIs that C++ can use (such as xlang, Corba and COM), which can provide stability in ways the plain-old C++ ABI does not. So we wouldn't necessarily have to design a new C-based ABI for this, we could adopt an existing one that is already proven and already has supporting tools. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
On 11/29/2018 2:10 PM, Andrew Svetlov wrote: Neither http.client nor http.server doesn't support compression (gzip/compress/deflate) at all. I doubt if we want to add this feature: for client better to use requests or, well, aiohttp. The same for servers: almost any production ready web server from PyPI supports compression. What production ready web servers exist on PyPi? Are there any that don't bring lots of baggage, their own enhanced way of doing things? The nice thing about the http.server is that it does things in a standard-conforming way, the bad thing about it is that it doesn't implement all the standards, and isn't maintained very well. From just reading PyPi, it is hard to discover whether a particular package is production-ready or not. I had used CherryPy for a while, but at the time it didn't support Python 3, and to use the same scripts behind CherryPy or Apache CGI (my deployment target, because that was what web hosts provided) became difficult for complex scripts so I reverted to http.server with a few private extensions (private because no one merged the bugs I reported some 3 versions of Python-development-process ago; back then I submitted patches, but I haven't had time to keep up with the churn of technologies Pythondev has used since Python 3 came out, which is when I started using Python, and I'm sure the submitted patches have bit-rotted by now). When I google "python web server" the first hit is the doc page for http.server, the second is a wiki page that mentions CherryPy and a bunch of others, but the descriptions, while terse, mostly point out some special capabilities of the server, making it seem like you not only get a web server, but a philosophy. I just want a web server. The last one, Waitress, is the only one that doesn't seem to have a philosophy in its description. So it would be nice if http.server and http.client could get some basic improvements to be complete, or if the docs could point to a replacement that is a complete server, but without a philosophy or framework (bloatware) to have to learn and/or work around. Glenn ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?
On Fri, 30 Nov 2018 13:00:37 -0800 Glenn Linderman wrote: > > So it would be nice if http.server and http.client could get some basic > improvements to be complete, or if the docs could point to a replacement > that is a complete server, but without a philosophy or framework > (bloatware) to have to learn and/or work around. Why do you think http.server is any different? If you want to implement your own Web service with http.server you must implement your own handler class, which is not very different from writing a handler for a third-party HTTP server. Maybe you're so used to this way of doing that you find it "natural", but in reality http.server is as opinionated as any other HTTP server implementation. By the way, there is a framework in the stdlib, it's called asyncio. And I'm sure you'll find production-ready HTTP servers written for it :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API changes
I just would want to say that I'm very happy to read the discussions about the C API finally happening on python-dev :-) The discussion is very interesting! > C is really the lingua franca Sorry, but why not writing the API directly in french? Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard library vs Standard distribution?
On Thu, 29 Nov 2018 10:46:23 -0800 Steve Dower wrote: > > I've previously volunteered to move certain modules to their own PyPI > packages and bundle them (omitting package names to avoid upsetting > people again), and I've done various analyses of which modules can be > moved out. I've also deliberately designed functionality into the > Windows installer to be able to bundle and install arbitrary packages > whenever we like (there's an example in > https://github.com/python/cpython/blob/master/Tools/msi/bundle/packagegroups/packageinstall.wxs). > > Plus I've been involved in packaging longer than I've been involved in > core development. I find it highly embarrassing, but there are people > out there who publicly credit me with "making it possible to use any > packages at all on Windows". Please don't accuse me of throwing out > ideas in this area without doing any work. Sorry. I've been unfair and unduly antagonistic. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard library vs Standard distribution?
On 30Nov2018 1435, Antoine Pitrou wrote: Sorry. I've been unfair and unduly antagonistic. Apology is totally accepted and all is forgiven. Thank you. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
