Re: [Python-Dev] updating ensurepip to include wheel
On 6 August 2015 at 15:04, Nick Coghlan wrote: > On 6 August 2015 at 09:29, Victor Stinner wrote: >> Le 5 août 2015 17:12, "Nick Coghlan" a écrit : >>> A hard dependency on wheel wouldn't fit into the same category - when >>> folks are using a build pipeline to minimise the installation >>> footprint on production systems, the wheel package itself has no >>> business being installed anywhere other than developer systems and >>> build servers. >> >> I'm quite sure that virtualenv is used to deploy python on production. >> >> Pip 7 automatically creates wheel packages when no build wheel package is >> available on PyPI. Examples numpy and any pure python package only providing >> a tarball. >> >> For me it makes sense to embed wheel in ensurepip and to install wheel on >> production systems (to install pacakes, not to build them). > > pip can install from wheels just fine without the wheel package being > present - that's how ensurepip already works. pip can also do this without setuptools being installed; yet we bundle setuptools with pip in ensurepip. I am thus confused :). When I consider the harm to a production pipeline that using setuptools can cause (in that it triggers easy_install, and easy_install has AFAIK none of the security improvements pip has added over the last couple years), I find the acceptance of setuptools, but non-acceptance of wheel flummoxing. > The wheel package itself is only needed in order to support the > setuptools "bdist_wheel" command, which then allows pip to implicitly > cache wheel files when installing from an sdist. > > Installing from sdist in production is a *fundamentally bad idea*, > because it means you have to have a build toolchain on your production > servers. One of the benefits of the wheel format and projects like > devpi is that it makes it easier to discourage people from doing that. > Even without getting into Linux containers and tools like pyp2rpm, > it's also possible to create an entire virtualenv on a build server, > bundle that up as an RPM or DEB file, and use the system package > manager to do the production deployment. Yes: but the logic chain from 'its a bad idea' to 'we don't include wheel but we do include setuptools' is the bit I'm having a hard time with. > However, production Linux servers aren't the only case we need to care > about, and there's a strong user experience argument to be made for > providing wheel by default upstream, and telling downstream > redistributors that care about the distinction to do the necessary > disentangling to make it easy to have "build dependency free" > production images. > > We've learned from experience that things go far more smoothly if we > thrash out those kinds of platform dependent behavioural differences > *before* we inflict them on end users, rather than having downstream > redistributors tackle foreseeable problems independently of both each > other and upstream :) > > Hence my request for a PEP - I can see why adding wheel to the > ensurepip bundle would be a good idea for upstream, but I can also see > why it's a near certainty downstream Linux distros (including Fedora) > would take it out again in at least some situations to better meet the Does Fedora also take out setuptools? If not, why not? > needs of *our* user base. (Since RPM has weak dependency support now, > we'd likely make python-wheel a "Recommends:" dependency, rather than > a "Requires:" dependency - still installed by default, but easy to > omit if not wanted or needed) So, a new PEP? -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] updating ensurepip to include wheel
On 6 August 2015 at 19:04, Robert Collins wrote: > On 6 August 2015 at 15:04, Nick Coghlan wrote: > When I consider the harm to a production pipeline that using > setuptools can cause (in that it triggers easy_install, and > easy_install has AFAIK none of the security improvements pip has added > over the last couple years), I find the acceptance of setuptools, > but non-acceptance of wheel flummoxing. When ensurepip was implemented, pip couldn't install from wheel files without setuptools yet, and the level of adoption of wheel files in general was lower than it is today. >> The wheel package itself is only needed in order to support the >> setuptools "bdist_wheel" command, which then allows pip to implicitly >> cache wheel files when installing from an sdist. >> >> Installing from sdist in production is a *fundamentally bad idea*, >> because it means you have to have a build toolchain on your production >> servers. One of the benefits of the wheel format and projects like >> devpi is that it makes it easier to discourage people from doing that. >> Even without getting into Linux containers and tools like pyp2rpm, >> it's also possible to create an entire virtualenv on a build server, >> bundle that up as an RPM or DEB file, and use the system package >> manager to do the production deployment. > > Yes: but the logic chain from 'its a bad idea' to 'we don't include > wheel but we do include setuptools' is the bit I'm having a hard time > with. Just an accident of history due to the relative timing of ensurepip's introduction, pip gaining the ability to install wheel files without setuptools, and high levels of adoption of the wheel format on PyPI. If PEP 453 was redone today, it's entirely possible setuptools wouldn't have been bundled, but it wasn't a viable option at the time. Accepting the bundling was a nice piece of technical debt that bought several additional months of feature availability :) >> Hence my request for a PEP - I can see why adding wheel to the >> ensurepip bundle would be a good idea for upstream, but I can also see >> why it's a near certainty downstream Linux distros (including Fedora) >> would take it out again in at least some situations to better meet the > > Does Fedora also take out setuptools? If not, why not? Not at the moment - while I'd like to see the dependency go away eventually, there are plenty of other things in the world that bother me more, especially since it comes back the moment someone has an "import pkg_resources" anywhere in their application. >> needs of *our* user base. (Since RPM has weak dependency support now, >> we'd likely make python-wheel a "Recommends:" dependency, rather than >> a "Requires:" dependency - still installed by default, but easy to >> omit if not wanted or needed) > > So, a new PEP? Yeah. I don't think it needs to be too fancy, just provide a way to indicate whether or not ensurepip should install the wheel package, and make it clear that if folks want to ensure pip can build wheels, they should install it explicitly (at the command line or as a dependency), rather than assuming it will always be there by default. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] updating ensurepip to include wheel
> On Aug 6, 2015, at 5:04 AM, Robert Collins wrote: > > Yes: but the logic chain from 'its a bad idea' to 'we don't include > wheel but we do include setuptools' is the bit I'm having a hard time > with. In my opinion, it’s the severity of how crippled their experience is without that particular thing installed. In the case of wheel not being installed they lose the ability to have an implicit wheel cache and to run ``pip wheel``. This makes pip less good but, unless they are running ``pip wheel`` everything is still fully functioning. In the case of setuptools they lose the ability to ``pip install`` when there isn’t a wheel available and the ability to run ``pip wheel``. This is making pip completely unusable for a lot of people, and if we did not pre-install setup tools the number one thing people would do is to ``pip install setuptools``, most likely while bitching under their breath about the command that just failed because they tried to install from sdist. So it’s really just “how bad are we going to break people’s expectations”. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] updating ensurepip to include wheel
On 7 August 2015 at 03:28, Donald Stufft wrote: > >> On Aug 6, 2015, at 5:04 AM, Robert Collins wrote: >> >> Yes: but the logic chain from 'its a bad idea' to 'we don't include >> wheel but we do include setuptools' is the bit I'm having a hard time >> with. > > > In my opinion, it’s the severity of how crippled their experience is without > that particular thing installed. > > In the case of wheel not being installed they lose the ability to have an > implicit wheel cache and to run ``pip wheel``. This makes pip less good but, > unless they are running ``pip wheel`` everything is still fully functioning. > > In the case of setuptools they lose the ability to ``pip install`` when there > isn’t a wheel available and the ability to run ``pip wheel``. This is making > pip completely unusable for a lot of people, and if we did not pre-install > setup tools the number one thing people would do is to ``pip install > setuptools``, most likely while bitching under their breath about the command > that just failed because they tried to install from sdist. > > So it’s really just “how bad are we going to break people’s expectations”. So - I was in a talk at PyCon AU about conda[*], and the author believed they were using the latest pip with all the latest caching features, but their experience (16 minute installs) wasn't that. I dug into that with them after the talk, and it was due to Conda not installing wheel by default. Certainly the framing of ensurepip as 'this installs pip' is going to be confusing and misleading if it doesn't install pip the way get-pip.py (or virtualenv) install pip, leading to confusion such as that. Given the inconsequential impact of installing wheel, I see only harm in holding it back, and only benefits in adding it. All the harm from having source builds comes in with setuptools ;). -Rob *) https://www.youtube.com/watch?v=Fqknoni5aX0 -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Bitbucket mirror is out-of-date
Bitbucket has a mirror of cpython, here: https://bitbucket.org/mirror/cpython It was last updated on May 7 and still says it's Python 3.5.0a4. It's not clear to me who owns the "mirror" account--is it Atlassian themselves? Anyway it'd be nice if it were, y'know, fresher. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
