Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Distribution class (through Setuptools) that > doesn't behave like Distutils' one anymore. > > For Distutils, I'll add some documentation explaining this, and a > section that will list community-driven commands. > > - Work left to do : write the documentation > - Goal: 2.7 / 3.2 > > = Distribute = > > I won't explain here again why we have forked, I think it's obvious to > anyone here now. I'll rather explain what > we are planning in Distribute and how it will interact with Distutils. > > Distribute has two branches: > > - 0.6.x : provides a Setuptools-0.6c9 compatible version > - 0.7.x : will provide a refactoring > > == 0.6.x == > > Not "much" is going to happen here, we want this branch to be helpful > to the community *today* by addressing the 40-or-so bugs > that were found in Setuptools and never fixed. This is eventually > happen soon because its development is > fast : there are up to 5 commiters that are working on it very often > (and the number grows weekly.) > > The biggest issue with this branch is that it is providing the same > packages and modules setuptools does, and this > requires some bootstrapping work where we make sure once Distribute is > installed, all Distribution that requires Setuptools > will continue to work. This is done by faking the metadata of > Setuptools 0.6c9. That's the only way we found to do this. > > There's one major thing though: thanks to the work of Lennart, Alex, > Martin, this branch supports Python 3, > which is great to have to speed up Py3 adoption. > > The goal of the 0.6.x is to remove as much bugs as we can, and try if > possible to remove the patches done > on Distutils. We will support 0.6.x maintenance for years and we will > promote its usage everywhere instead of > Setuptools. > > Some new commands are added there, when they are helpful and don't > interact with the rest. I am thinking > about "upload_docs" that let you upload documentation to PyPI. The > goal is to move it to Distutils > at some point, if the documentation feature of PyPI stays and starts to be > used. > > == 0.7.x == > > We've started to refactor Distribute with this roadmap in mind (and > no, as someone said, it's not vaporware, > we've done a lot already) > > - 0.7.x can be installed and used with 0.6.x > > - easy_install is going to be deprecated ! use Pip ! > > - the version system will be deprecated, in favor of the one in Distutils > > - no more Distutils monkey-patch that happens once you use the code > (things like 'from distutils import cmd; cmd.Command = CustomCommand') > > - no more custom site.py (that is: if something misses in Python's > site.py we'll add it there instead of patching it) > > - no more namespaced packages system, if PEP 381 (namespaces package > support) makes it to 2.7 > > - The code is splitted in many packages and might be distributed under > several distributions. > >- distribute.resources: that's the old pkg_resources, but > reorganized in clean, pep-8 modules. This package will > only contain the query APIs and will focus on being PEP 376 > compatible. We will promote its usage and see if Pip wants > to use it as a basis. And maybe PyPM once it's open source ? > ( ). > It will probably shrink a lot though, once the stdlib provides PEP 376 > support. > >- distribute.entrypoints: that's the old pkg_resources entry points > system, but on its own. it uses distribute.resources > >- distribute.index: that's package_index and a few other things. > everything required to interact with PyPI. We will promote > its usage and see if Pip wants to use it as a basis. > >- distribute.core (might be renamed to main): that's everything > else, and uses the other packages. > > > Goal: A first release before (or when) Python 2.7 / 3.2 is out. > > > = Virtualenv and the multiple version support in Distribute = > > (I am not saying "We" here because this part was not discussed yet > with everyone) > > Virtualenv allows you to create an isolated environment to install > some distribution without polluting the > main site-packages, a bit like a user site-packages. > > My opinion is that this tool exists only because Python doesn't > support the installation of multiple versions for the same > distributions. Not only, for me it exists also to provide a way to isolate/chroot an application from the host system. > But if PEP 376 and PEP 386 support are added in Python, we're not far > from being able to provide multiple version support with > the help of importlib. > > Setuptools provided a multiple version support but I don't like its > implementation and the way its works. > I would like to create a new site-packages format that can contains > several versions of the same distribution, and : > > - a special import system using importlib that would automatically > pick the latest version, thanks to PEP 376. > - an API to force at runtime a specific version (that would be located > at the beginning of all imports, like __future__) > - a layout that is compatible with the way OS packagers works with > python packages > > Goal: a prototype asap (one was started under the "VSP" name (virtual > site-packages) but not finished yet) > > Regards > Tarek > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
hould be resolved by them. But for the class > of user that is an end-user, version conflicts are a totally foreign > concept. They should be dealt with by the person who is coding the > application for them. > > Also note, the ability to have multiple versions makes things easier for > system packagers and provides an easy alternative to a virtualenv for > end-users. > > * System packagers: virtualenv does not provide a method suitable for system Yes, there is no doubt virtualenv is useless for system packagers but: * System and applications deployment have not to be tied. It s up to the user to install things system wide or to use locally isolation technics. Virtualenv is one of those. As a conclusion, there are not very much problem for system packagers as if their users have specific needs, they will do something Outside the system. If not, they use their global python with packages installed in that global one. > packagers. The nearest adaptation would be for the system packager to > install python packages into their own hierarchy not in the PYTHONPATH. > Then they would need to create a virtualenv-like directory that symlinks > to the packages in that directory. Then they would need to write a > wrapper script for the application that put that virtualenv-like directory > into the PYTHONPATH before any other site package directories and have > that wrapper call the real binary. This is needless complication for the > typical virtualenv install so the work is not likely to be done there and > it's incredibly hacky for the system packager so the work is not likely to > be done there. > * End users: virtualenv creates a whole environment for the application to > live in. If python understood how to use multiple versions then we'd only > need to install the versions of packages that didn't match up with what's > already on the system into the user's personal site-package directory. > Then the module loader would take care of loading the module from the > personal site-packages since it met the version requirements instead of > the system copy. > I dont want to have interaction between my different hosted projects, isolation is the only thing i want. As many other people there already. I do not want at all to have to deal with multiple version picking system and debug an application at 2:00am because of that. > -Toshio > > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
hould be resolved by them. But for the class > of user that is an end-user, version conflicts are a totally foreign > concept. They should be dealt with by the person who is coding the > application for them. > > Also note, the ability to have multiple versions makes things easier for > system packagers and provides an easy alternative to a virtualenv for > end-users. > > * System packagers: virtualenv does not provide a method suitable for system Yes, there is no doubt virtualenv is useless for system packagers but: * System and applications deployment have not to be tied. It s up to the user to install things system wide or to use locally isolation technics. Virtualenv is one of those. As a conclusion, there are not very much problem for system packagers as if their users have specific needs, they will do something Outside the system. If not, they use their global python with packages installed in that global one. > packagers. The nearest adaptation would be for the system packager to > install python packages into their own hierarchy not in the PYTHONPATH. > Then they would need to create a virtualenv-like directory that symlinks > to the packages in that directory. Then they would need to write a > wrapper script for the application that put that virtualenv-like directory > into the PYTHONPATH before any other site package directories and have > that wrapper call the real binary. This is needless complication for the > typical virtualenv install so the work is not likely to be done there and > it's incredibly hacky for the system packager so the work is not likely to > be done there. > * End users: virtualenv creates a whole environment for the application to > live in. If python understood how to use multiple versions then we'd only > need to install the versions of packages that didn't match up with what's > already on the system into the user's personal site-package directory. > Then the module loader would take care of loading the module from the > personal site-packages since it met the version requirements instead of > the system copy. > I dont want to have interaction between my different hosted projects, isolation is the only thing i want. As many other people there already. I do not want at all to have to deal with multiple version picking system and debug an application at 2:00am because of that. > -Toshio > > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: PGP signature signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Tarek Ziadé a écrit : > The choice to deprecate easy_install in 0.7 is done because the Pip project > is not far to meet all uses cases easy_install users have, and we're betting > on the fact that Pip is active and will be much more advanced that what > we could do with a 'new' refactored easy_install by the time 0.7 is ready. But how about retro-compatibility? Like with all those buildout recipes which relies on setuptool APIs (Requirement, PackageIndex and so on), it's a risk to break same at some point. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Ian Bicking a écrit : > I'm coming in late and breaking threading, but wanted to reply to > Tarek's original email: > > This is verifiable, stable, and to varying degrees concrete > (virtualenv being more concrete than buildout, which tends more > towards the declarative). Is that a friday troll ? declarative over control ? > > What virtualenv does could certainly be in the Python interpreter (and > much more compact as a result, I am sure). PYTHONHOME does it to a > degree, though binding a script to a environment through the > interpreter listed in #! is more stable than the implicit environment > of PYTHONHOME. workingenv used an environmental variable (PYTHONPATH, > before PYTHONHOME existed) and it caused problems. Also virtualenv > offers more system isolation. > > If I had my way, buildout would use virtualenv and throw away its > funny script generation. If virtualenv had existed before buildout Which one, the one provided to generate scripts from entry points with the *.egg recipes or the bin/buildout auto regeneration? > began development, probably things would have gone this way. I think > it would make the environment more pleasant for buildout users. Also * I don't think so, buildout is the only tool atm that permit to have really reproducible and isolated environments. Even, if you use the pip freezing machinery, it is not equivalent to buildout, Control! * Buildout can have single part to construct required eggs, at a specific version and let you control that. Pip will just search for this version, see that it's not available and fail. You have even recipes (like minitage.recipe.egg that permit to construct eggs with special version when you apply patches onto, thus, you can have the same egg in different flavors in the same eggs cache available for different projects. Those projects will just have to pin the right version to use, Control!. * Another thing is the "funny script generation", you have not one global site-packages for your project, but one global cache. But from this global cache, your scripts will only have available the eggs you declared, see Control! * Moreover buildout is not only a python packages manager, it's some of its recipes that permit to use it as. Buildout is just a great deployment tool that allow to script and manage your project in a "funny" and "flexible" way, Control! > I wish it used pip instead of its own installation procedure (based on > easy_install). I don't think the philosophical differences are that > great, and that it's more a matter of history -- because the code is And retro-compatibility. > written, there's not much incentive for buildout to remove that code > and rely on other libraries (virtualenv and pip). * Virtualenv is not really needed as buildout uses "eggs caches" and do the isolation from there. Scripts generate the adequate sys.path relying on the application requirements (which, once more, you are in total control). I prefer throught to use a virtualenv for each project (or a bunch of projects), and to bootstrap my buildouts from there. For me, they are complementary to achieve your deployment and isolation needs. > > -- > Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
I'm crossposting to continue on distutils. Ian Bicking a écrit : > On Fri, Oct 9, 2009 at 3:54 AM, kiorky wrote: > Well, if multi-versioned installs were deprecated, it would not be > necessary to use Setuptools' style of script generation. Instead you > could simply dereference the entry point, calling the underlying > function directly in the script. This detail is probably more of a > distutils-sig question, and I don't have a strong opinion. > > But I was thinking specifically of the egg activation buildout puts at > the top of scripts. The egg activation is just the result of the constructed working set, which aggregates all the python packages that fulfill the options inputed by the user. The only needed packages and nothing else or more. >>> began development, probably things would have gone this way. I think >>> it would make the environment more pleasant for buildout users. Also >> * I don't think so, buildout is the only tool atm that permit to have really >> reproducible and isolated environments. Even, if you use the pip freezing >> machinery, it is not equivalent to buildout, Control! > > I believe that to fully insulate buildout you need still virtualenv > --no-site-packages. But I'm not arguing that virtualenv/pip makes Right, I do so, but i also drop nothing in that virtualenv site-packages. > buildout obsolete, only that they have overlapping functionality, and > I think buildout would benefit from making use of that overlap. recipes have more features and are another way to do, complementary or not. (patching, compiling options, pinning version). You have also more control on what is installed/used/shared with buildout than with the others. >> * Buildout can have single part to construct required eggs, at a specific >> version and let you control that. Pip will just search for this version, see >> that it's not available and fail. You have even recipes (like >> minitage.recipe.egg that permit to construct eggs with special version when >> you >> apply patches onto, thus, you can have the same egg in different flavors in >> the >> same eggs cache available for different projects. Those projects will just >> have >> to pin the right version to use, Control!. > > In my own work I use multiple virtualenv environments for this use > case, to similar effect. pip of course is not a generalized build > tool, but then minitage.recipe.egg is not the main egg installer > either. *.recipe.egg can use caches to avoid duplications among many projects environments. You setup seems to duplicate stuff around. >> * Another thing is the "funny script generation", you have not one global >> site-packages for your project, but one global cache. But from this global >> cache, your scripts will only have available the eggs you declared, see >> Control! >> * Moreover buildout is not only a python packages manager, it's some of its >> recipes that permit to use it as. Buildout is just a great deployment tool >> that >> allow to script and manage your project in a "funny" and "flexible" way, >> Control! > > Sure; I'm just advocating that buildout more explicitly use some of > the functionality of virtualenv/pip (which may require some more > features in those tools, but I'm open to that). But specific > discussion of this would probably be more appropriate on > distutils-sig. > -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
