Re: [Python-Dev] eggs now mandatory for pypi?
David Lyon wrote: > On Tue, 06 Oct 2009 16:45:29 +0100, Chris Withers > wrote: > >> Well yeah, and the only sane way I can think to handle this is to have a >> metadata file that gets uploaded with each distribution that covers all >> these things (and the other things that other people need) and then have >> the index (which would hopefully be PyPI on the whole) be queryable >> along the lines of "give me a download url for a distribution named X >> that is for Python 2.6, Win32, UCS4, PostGreSQL 8.5, BlagBlah"... > > Exactly. > > I'd like to see that and it sounds like a very simple and reasonable > proposal. > > One could say that much of the setuptools cloud came about because of > the lack of the queryable download url. Setuptools does a lot of work > here to 'work-around' the ommission on pypi of a package download url. I think you two have missed my email to PJE. In summary: PyPI already does have mechanism for querying download URLs. http://wiki.python.org/moin/PyPiXmlRpc It only lacks some more meta-data that would be required to match up the user's configuration with the one used in the uploaded URLs. And it would be handy to have a mechanism to just upload the URL data rather than the whole package. Looking at distutils' upload command, PyPI currently appears to only support uploads of the package file itself. Basically, most of the mechnisms are already there, they just need some extra care. BTW: Who would I need to contact for the PyPI side to work out an upload_url distutils command ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] please consider changing --enable-unicode default to ucs4
Adam Olsen wrote: > On Sun, Sep 20, 2009 at 10:17, Zooko O'Whielacronx wrote: >> On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou wrote: >>> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 >>> setting. >> >> That would be an improvement! Unfortunately we instead get mysterious >> misbehavior of the module, e.g.: >> >> http://bugs.python.org/setuptools/msg309 >> http://allmydata.org/trac/tahoe/ticket/704#comment:5 I agree that a better error message would help. I'm just not sure how to achieve that. The error message you currently see gets generated by the dynamic linker trying to resolve a Python Unicode API symbol: the API names are mangled to assure that you cannot mix UCS2 interpreters and UCS4 extensions (and vice-versa). We could try to scan the linker error message for 'Py.*UCS.' and then replace the message with a more helpful one (in importdl.c), but I'm not sure how portable that is. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Here's a quick summary of the main things that are going to happen in
Distutils, and Distribute, and a few words on virtualenv and pip.
(there is much much more work going on, but I don't want to drown
people with details)
= Distutils =
Distutils is a package manager and competes with OS package managers.
This is a good thing because, unless you are developing a library or
an application that will only run one specific system that has its own
packaging system like Debian, you will be able to reach much more
people. Of course the goal is to avoid making the work of a Debian
packager (or any other OS that has a package manager) too hard. In
other words, re-packaging a Distutils-based project should be easy and
Distutils should not get in their way (or as less as possible).
But right now Distutils is incomplete in many ways and we are trying to fix'em.
== What's installed ? what's the installation format ? how to uninstall ? ==
First, it's an incomplete package manager : you can install a
distribution using it, but there's no way to list installed
distributions. Worst, you can't uninstall a distribution.
PEP 376 resolves this, and once it's finished, the goal is to include
the APIs described there into Distutils itself and into the pkgutil
module in stdlib. Notice that there's an implementation at
http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376
so people can see what we are talking about.
Another problem that popped during the last years is the fact that, in
the same site-packages, depending on the tool that was used to install
a Distribution, and depending if this distribution uses Distutils or
Setuptools, you can have different installation formats.
End-users end up with zipped eggs (one file), unzipped eggs (one
self-contained format in a directory) and regular Distutils (packages
and modules in site-packages). And the Metadata are also located in
many different places depending on the installation format used.
That can't be. there's no point to keep various installation format in
the *same* site-packages directory.
PEP 376 also resolves this by describing a *unique* format that works
for all. Once this is finished, Distutils will implement it by
changing the install command accordingly.
- Work left to do in PEP 376 : restrict its scope to a disk-based,
file-based site-packages.
- Goal: 2.7 / 3.2
== Dependencies ==
The other feature that makes a packaging system nice is dependencies.
e.g. a way to list in a distribution, the distributions it requires to
run. As a matter of fact, PEP 314 has introduced in the Metadata new
fields for this purpose ("Requires", "Provides and "Obsoletes"). So,
you can write things like "Requires: lxml >= 2.2.1", meaning that your
distribution requires lxml 2.2.1 or a newer version to run. But this
was just description fields and Distutils was not providing any
feature based on these new fields.
In fact, no third-party tool either provided a feature based on those
fields. Setuptools provided "easy_install" a script that looks for the
dependencies and install them, by querying the Python Package Index
(PyPI). But this feature was implemented with its own metadata: you
can add an "install_requires" option in the setup() call in setup.py,
and it will end up in a "requires.txt" file at installation time that
is located alongside the Metadata for you distribution.
So the goal is to review PEP 314 and update the Metadata w.r.t. the
setuptools feedback and community usage. Once it's done, Distutils
will implement this new metadata version and promote its usage.
Promoting its usage means that Distutils will provide some APIs to
work with these APIs, like a version comparison algorithm.
And while we're at it, we need to work out some inconsistency with the
"Author" and "Maintainer" fields. (The latter doesn't exists in the
Metadata but exists on setup.py side).
- Work left to do in PEP 314 : finish PEP 386, finish the discussion
on the "maintainer" field.
- Goal: 2.7 / 3.2
== Version comparison ==
Once you provide dependency fields in the metadata, you need to
provide a version scheme: a way to compare two versions. Distutils has
two version comparison algorithms that are not used in its code and in
only one place in the stdlib where it could be removed with no pain.
One version scheme is "strict" and one is "loose". And Setuptools has
another one, which is more heuristic (it will deal with any version
string and compare it, wether it's wrong or not).
PEP 386 goal is to describe a version scheme that can be used by all
and if we can meet a consensus there, we can move on and add
it as a reference in the update done in PEP 314, besides the
dependencies fields. Then, in Distutils we can deprecate the existing
version
comparison algorithms and provide a new one based on PEP 386 and
promote its usage.
One very important point: we will not force the community to use the
scheme described in PEP 386, but *there is* already a
de-facto convention on ve
Re: [Python-Dev] eggs now mandatory for pypi?
Tarek Ziadé wrote: > On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg wrote: >> BTW: Who would I need to contact for the PyPI side to work out >> an upload_url distutils command ? > > I am not sure to understand. How upload_url will differ from the > register command > that let you upload metadata at PyPI ? The register command only uploads the package's meta-data. The upload command uploads a file and registers the file's details with PyPI as well. I'd like to add an upload_url command which only registers a file's details (including the URL, but without the actual file) with PyPI and perhaps extend the number of attributes sent to PyPI with more file configuration information (upload currently only sends the Python version). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] eggs now mandatory for pypi?
On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg wrote: > BTW: Who would I need to contact for the PyPI side to work out > an upload_url distutils command ? I am not sure to understand. How upload_url will differ from the register command that let you upload metadata at PyPI ? Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] eggs now mandatory for pypi?
On Thu, Oct 8, 2009 at 10:38 AM, M.-A. Lemburg wrote: > Tarek Ziadé wrote: >> On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg wrote: >>> BTW: Who would I need to contact for the PyPI side to work out >>> an upload_url distutils command ? >> >> I am not sure to understand. How upload_url will differ from the >> register command >> that let you upload metadata at PyPI ? > > The register command only uploads the package's meta-data. > > The upload command uploads a file and registers the file's > details with PyPI as well. > > I'd like to add an upload_url command which only registers a > file's details (including the URL, but without the actual file) > with PyPI and perhaps extend the number of attributes sent to > PyPI with more file configuration information (upload currently > only sends the Python version). Sounds like an option to be added in upload. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Oct 08 2009) Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > > > ::: Try our new mxODBC.Connect Python Database Interface for free ! > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
2009/10/8 Tarek Ziadé : > Here's a quick summary of the main things that are going to happen in > Distutils, and Distribute, and a few words on virtualenv and pip. > (there is much much more work going on, but I don't want to drown > people with details) Thanks for this summary. The overview was getting lost in all the details. > == What's installed ? what's the installation format ? how to uninstall ? == > > First, it's an incomplete package manager : you can install a > distribution using it, but there's no way to list installed > distributions. Worst, you can't uninstall a distribution. > > PEP 376 resolves this, and once it's finished, the goal is to include > the APIs described there into Distutils itself and into the pkgutil > module in stdlib. Notice that there's an implementation at > http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376 > so people can see what we are talking about. > > Another problem that popped during the last years is the fact that, in > the same site-packages, depending on the tool that was used to install > a Distribution, and depending if this distribution uses Distutils or > Setuptools, you can have different installation formats. > > End-users end up with zipped eggs (one file), unzipped eggs (one > self-contained format in a directory) and regular Distutils (packages > and modules in site-packages). And the Metadata are also located in > many different places depending on the installation format used. > > That can't be. there's no point to keep various installation format in > the *same* site-packages directory. > > PEP 376 also resolves this by describing a *unique* format that works > for all. Once this is finished, Distutils will implement it by > changing the install command accordingly. > > - Work left to do in PEP 376 : restrict its scope to a disk-based, > file-based site-packages. > - Goal: 2.7 / 3.2 +1 This addresses my biggest concern with the way PEP 376 was going (largely prompted by my suggestions, so I'll take the blame for this!) which was that PEP 376 was trying to define list/uninstall methods for package formats (i.e. zip files/eggs) that it couldn't install itself. I presume that when you refer to the restriction in scope, you do mean that PEP 375 will lose all support for zip files, as well as the generalised PEP 302 support I proposed? I still believe that special-casing zip files is wrong - PEP 302 deliberately abstracted the import mechanisms and built zipfile support on top of that, and I still believe that's a good thing. But expanding PEP 302 to cover the requirements of PEP 376 looked more and more inappropriate the more I wrote code to do it - what would be needed would be a *new* protocol on top of which zip-based formats could be layered. I'm quite happy for that to be outside the scope of PEP 376, though, as distutils itself doesn't support any such formats. The egg format One thing missing from your roadmap (unless I missed it) is the fate of the egg (zipfile) format. If it's to remain a valid option (bdist_egg and the like) then I'd assume that Distribute needs to be the place to develop it. One thing it will need is an equivalent to PEP 376 introspection and uninstall features. And *that* is where I believe a standard protocol belongs - developed based on the real use requirements for egg files, with a suitable file-based layer which can be added into distutils (somewhat like importlib does for PEP 302) once the protocol is agreed and standardised. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 10:56 AM, Paul Moore wrote: > > +1 > > This addresses my biggest concern with the way PEP 376 was going > (largely prompted by my suggestions, so I'll take the blame for this!) > which was that PEP 376 was trying to define list/uninstall methods for > package formats (i.e. zip files/eggs) that it couldn't install itself. > > I presume that when you refer to the restriction in scope, you do mean > that PEP 375 will lose all support for zip files, as well as the > generalised PEP 302 support I proposed? I still believe that > special-casing zip files is wrong - PEP 302 deliberately abstracted > the import mechanisms and built zipfile support on top of that, and I > still believe that's a good thing. But expanding PEP 302 to cover the > requirements of PEP 376 looked more and more inappropriate the more I > wrote code to do it - what would be needed would be a *new* protocol > on top of which zip-based formats could be layered. I'm quite happy > for that to be outside the scope of PEP 376, though, as distutils > itself doesn't support any such formats. Yes exactly so. > > The egg format > > One thing missing from your roadmap (unless I missed it) is the fate > of the egg (zipfile) format. If it's to remain a valid option > (bdist_egg and the like) then I'd assume that Distribute needs to be > the place to develop it. > One thing it will need is an equivalent to > PEP 376 introspection and uninstall features. And *that* is where I > believe a standard protocol belongs - developed based on the real use > requirements for egg files, with a suitable file-based layer which can > be added into distutils (somewhat like importlib does for PEP 302) > once the protocol is agreed and standardised. Distribute is planning to continue supporting the egg format, but will probably drop the zipped version. This is still under discussions though so don't take this as the final word. In any case, distribute.resources will probably add the missing bits on the top of a PEP 376 -compliant Distutils to provide support for any file querying Distutils will not provide (for installation and uninstallation) Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 5:31 PM, Tarek Ziadé wrote: > = Virtualenv and the multiple version support in Distribute = > > (I am not saying "We" here because this part was not discussed yet > with everyone) > > Virtualenv allows you to create an isolated environment to install > some distribution without polluting the > main site-packages, a bit like a user site-packages. > > My opinion is that this tool exists only because Python doesn't > support the installation of multiple versions for the same > distributions. I am really worried about this, because it may encourage people to use multiple versions as a bandaid to maintaining backward compatibility. At least with virtual-env, the problem is restricted to the user. Generalized multiple, side by side installation has been tried in many different contexts, and I have never seen a single one working and not bringing more problems that it solved. One core problem being the exponential number of combination (package A depends on B and C, B depends on one version of D, C on another version of D). Being able to install *some* libraries in multiple versions is OK, but generalizing is very dangerous IMHO. David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.6.4rc1
On 7 Oct, 2009, at 23:09, Barry Warsaw wrote: On Oct 7, 2009, at 3:46 PM, Brett Cannon wrote: On Wed, Oct 7, 2009 at 12:42, Ronald Oussoren wrote: On 7 Oct, 2009, at 20:53, Brett Cannon wrote: I just tried building out of svn and a ton of tests that rely on urllib failed because the _scproxy module wasn't built and it unconditionally imports it under darwin. Turns out that it requires the Mac toolbox glue to be built which I always skip since I don't care about it. I am fairly certain this toolbox glue dependency for urllib is a regression as I used to always be able to get networking working w/ o the Mac-specific modules. Bummer. _scproxy was introduced in 2.6.3 to fix a crash in the ctypes based code it replaces, as well as very vague failures in the same code on OSX 10.5 Server on PPC machines. Luckily the fix is easy enough: move some code in setup.py from the block that checks for "not --disable-toolbox-glue", the _scproxy module has no dependencies on the toolbox glue. The attached patch should fix the issue, Patch fixed it. Barry, can Ronald apply the patch? Since this is a 2.6.3 regression, yes. I've commited the patch (r75282) Ronald smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 11:42 AM, David Cournapeau wrote: > On Thu, Oct 8, 2009 at 5:31 PM, Tarek Ziadé wrote: > >> = Virtualenv and the multiple version support in Distribute = >> >> (I am not saying "We" here because this part was not discussed yet >> with everyone) >> >> Virtualenv allows you to create an isolated environment to install >> some distribution without polluting the >> main site-packages, a bit like a user site-packages. >> >> My opinion is that this tool exists only because Python doesn't >> support the installation of multiple versions for the same >> distributions. > > I am really worried about this, because it may encourage people to use > multiple versions as a bandaid to maintaining backward compatibility. > At least with virtual-env, the problem is restricted to the user. > > Generalized multiple, side by side installation has been tried in > many different contexts, and I have never seen a single one working > and not bringing more problems that it solved. One core problem being > the exponential number of combination (package A depends on B and C, B > depends on one version of D, C on another version of D). Being able to > install *some* libraries in multiple versions is OK, but generalizing > is very dangerous IMHO. The goal here is to be able to have a *single* instance of "Foo 1.2" on your system, managed by the admin sys. that can be shared and used for several python apps. And yes, I don't expect to have so many different versions of many distributions on a system, but just to provide a mechanism to allow it. But that's just an idea we had with some people, and it still requires a lot of work and brainstroming (even if the prototype kinda work) I should've mentioned that I've added it at the end of the roadmap because I had the feeling that something has to be done to reunite zc.buildout, virtualenv and Python one day. But consider this as the topic for the "fun sprint" we would have at Pycon, after we will have issued other points. If you are interested in this particular topic, I propose that we continue this discussion on distutils-SIG. Regards Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Tarek Ziadé a écrit :
> Here's a quick summary of the main things that are going to happen in
> Distutils, and Distribute, and a few words on virtualenv and pip.
> (there is much much more work going on, but I don't want to drown
> people with details)
>
> = Distutils =
>
> Distutils is a package manager and competes with OS package managers.
> This is a good thing because, unless you are developing a library or
> an application that will only run one specific system that has its own
> packaging system like Debian, you will be able to reach much more
> people. Of course the goal is to avoid making the work of a Debian
> packager (or any other OS that has a package manager) too hard. In
> other words, re-packaging a Distutils-based project should be easy and
> Distutils should not get in their way (or as less as possible).
>
> But right now Distutils is incomplete in many ways and we are trying to
> fix'em.
>
> == What's installed ? what's the installation format ? how to uninstall ? ==
>
> First, it's an incomplete package manager : you can install a
> distribution using it, but there's no way to list installed
> distributions. Worst, you can't uninstall a distribution.
>
> PEP 376 resolves this, and once it's finished, the goal is to include
> the APIs described there into Distutils itself and into the pkgutil
> module in stdlib. Notice that there's an implementation at
> http://bitbucket.org/tarek/pep376 that is kept up to date with PEP 376
> so people can see what we are talking about.
>
> Another problem that popped during the last years is the fact that, in
> the same site-packages, depending on the tool that was used to install
> a Distribution, and depending if this distribution uses Distutils or
> Setuptools, you can have different installation formats.
>
> End-users end up with zipped eggs (one file), unzipped eggs (one
> self-contained format in a directory) and regular Distutils (packages
> and modules in site-packages). And the Metadata are also located in
> many different places depending on the installation format used.
>
> That can't be. there's no point to keep various installation format in
> the *same* site-packages directory.
>
> PEP 376 also resolves this by describing a *unique* format that works
> for all. Once this is finished, Distutils will implement it by
> changing the install command accordingly.
>
> - Work left to do in PEP 376 : restrict its scope to a disk-based,
> file-based site-packages.
> - Goal: 2.7 / 3.2
>
> == Dependencies ==
>
> The other feature that makes a packaging system nice is dependencies.
> e.g. a way to list in a distribution, the distributions it requires to
> run. As a matter of fact, PEP 314 has introduced in the Metadata new
> fields for this purpose ("Requires", "Provides and "Obsoletes"). So,
> you can write things like "Requires: lxml >= 2.2.1", meaning that your
> distribution requires lxml 2.2.1 or a newer version to run. But this
> was just description fields and Distutils was not providing any
> feature based on these new fields.
>
> In fact, no third-party tool either provided a feature based on those
> fields. Setuptools provided "easy_install" a script that looks for the
> dependencies and install them, by querying the Python Package Index
> (PyPI). But this feature was implemented with its own metadata: you
> can add an "install_requires" option in the setup() call in setup.py,
> and it will end up in a "requires.txt" file at installation time that
> is located alongside the Metadata for you distribution.
>
> So the goal is to review PEP 314 and update the Metadata w.r.t. the
> setuptools feedback and community usage. Once it's done, Distutils
> will implement this new metadata version and promote its usage.
> Promoting its usage means that Distutils will provide some APIs to
> work with these APIs, like a version comparison algorithm.
>
> And while we're at it, we need to work out some inconsistency with the
> "Author" and "Maintainer" fields. (The latter doesn't exists in the
> Metadata but exists on setup.py side).
>
> - Work left to do in PEP 314 : finish PEP 386, finish the discussion
> on the "maintainer" field.
> - Goal: 2.7 / 3.2
>
> == Version comparison ==
>
> Once you provide dependency fields in the metadata, you need to
> provide a version scheme: a way to compare two versions. Distutils has
> two version comparison algorithms that are not used in its code and in
> only one place in the stdlib where it could be removed with no pain.
> One version scheme is "strict" and one is "loose". And Setuptools has
> another one, which is more heuristic (it will deal with any version
> string and compare it, wether it's wrong or not).
>
> PEP 386 goal is to describe a version scheme that can be used by all
> and if we can meet a consensus there, we can move on and add
> it as a reference in the update done in PEP 314, besides the
> dependencies fields. Then, in Distutils we can deprecate the existing
> version
> compar
[Python-Dev] PEP 370 and IronPython
Dear Pythonistas! Michael Foord has written a blog posting [1] regarding IronPython, site packages and my PEP 370. I'm referring to the section that talks about an issue with PEP 370. --- If you install CPython 2.6 on Windows it creates a folder in the location: %APPDATA%\Python\Python26\site-packages. On my machine the full path is: C:\Users\michael\AppData\Roaming\Python\Python26\site-packages. This is put on the path by site.py when you run Python and as it is a user directory it doesn't require elevation to install there. You install into the user directory rather than the 'usual' location with: python setup.py --user install Herein lies another problem though. IronPython uses the standard Python site.py, meaning that currently it shares the user site-packages folder with CPython. This is not ideal as typically you will want separate libraries install for IronPython. When Jython ports to Python 2.6 it will have the same issue. --- I didn't think about IronPython during the design phase of the PEP. I didn't come into my mind that another implementation may require a different site packages directory. After spinning my head around the issue a simple solution came to me. The solution requires a new attribute in the sys module that contains the name of the implementation. As an alternative we could use the first field of sys.subversion but I prefer an explicit attribute. I'm proposing "sys.name" with a value of "CPython", "IronPython", "Jython" or "PyPy". The site module uses the information of the attribute to modify the path to the user site directory. The altered user site directories are: CPython: ~/.local/lib/python2.6/site-packages %APPDATA%/Python/Python26 IronPython: ~/.local/lib/ironpython2.6/site-packages %APPDATA%/Python/IronPython26 Jython: ~/.local/lib/jython2.6/site-packages %APPDATA%/Python/Jython26 Proposed patch: Index: Lib/site.py === --- Lib/site.py (revision 75282) +++ Lib/site.py (working copy) @@ -74,7 +74,12 @@ USER_SITE = None USER_BASE = None +interpreter_name = getattr(sys, "name", "CPython") +if interpreter_name == "CPython": +# b/w compatible name +interpreter_name = "python" + def makepath(*paths): dir = os.path.abspath(os.path.join(*paths)) return dir, os.path.normcase(dir) @@ -253,10 +258,12 @@ return USER_SITE if os.name == "nt": -USER_SITE = os.path.join(user_base, "Python" + sys.version[0] + - sys.version[2], "site-packages") +USER_SITE = os.path.join(user_base, interpreter_name.capitalize() + + sys.version[0] + sys.version[2], + "site-packages") else: -USER_SITE = os.path.join(user_base, "lib", "python" + sys.version[:3], +USER_SITE = os.path.join(user_base, "lib", + interpreter_name.lower() + sys.version[:3], "site-packages") return USER_SITE The patch doesn't modify the search path for global directories. These can be changed by altering sys.prefix and sys.exec_prefix. Christian [1] http://ironpython-urls.blogspot.com/2009/10/distributing-ironpython-packages.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
I have just a few comments to make, so I'm trimming the long quote a bit... > Tarek Ziadé a écrit : >> == Version comparison == >> >> One very important point: we will not force the community to use the >> scheme described in PEP 386, but *there is* already a >> de-facto convention on version schemes at PyPI if you use Pip or >> easy_install, so let's have a documented standard for this, >> and a reference implementation in Distutils. >> >> There's an implementation at >> http://bitbucket.org/tarek/distutilsversion that is kept up-to-date >> with PEP 386. >> >> - Work left to do in PEP 386 : another round with the community >> - Goal: 2.7 / 3.2 Please make sure that the version scheme is compatible with other tools we use in the distutils chain: RPM and the MSI compiler. Some data points: RPM doesn't like hyphens in version strings. MSI requires a strict N.N.N format. >> == The fate of setup.py, and static metadata == >> >> So we are going to separate the metadata description from setup.py, in >> a static configuration file, that can be open and read by anyone >> without >> running any code. The only problem with this is the fact that some >> metadata fields might depend on the execution environment. For >> instance, once "Requires" is re-defined and re-introduced via PEP 314, >> we will have cases where "pywin32" will be a dependency to have only >> on win32 systems. How are you planning to deal with dynamic version strings, e.g. say you want to build a snapshot release the current date and revision number included in the version string ? I suppose this file is going to be uploaded to PyPI, so the file would have to be formatted at build time and include markers to be replaced with values taken from the distribution meta-data. >> - Work left to do : publish the final syntax, and do the implementation Is there a PEP for this ? >> == The fate of bdist_* commands == >> >> So the best way to handle this is to ask these communities to build >> their own tool and to encourage them to use Distutils as a basis for >> that. Rather than having external communities build their own tools and then basically fork distutils in order to get these implemented, I think it'd be better to make the set of distutils commands extensible via namespace packages. >> So what is going to happen is a status-quo: no bdist_* command will be >> removed but no new bdist_* command will be added. I hope that comment means "no new bdist_* command will be added *right now*". It's well possible to create new bdist_* commands that don't rely on external tool chains or only on ones that don't change often. I'm thinking of bdist_inno (Windows installer using InnoSetup) and bdist_nsis (Windows installer using NSIS) here. >> = Virtualenv and the multiple version support in Distribute = >> >> (I am not saying "We" here because this part was not discussed yet >> with everyone) >> >> Virtualenv allows you to create an isolated environment to install >> some distribution without polluting the >> main site-packages, a bit like a user site-packages. >> >> My opinion is that this tool exists only because Python doesn't >> support the installation of multiple versions for the same >> distributions. > > Not only, for me it exists also to provide a way to isolate/chroot an > application from the host system. I agree: IMHO, that's the main use case for things like virtualenv. Note that Python has had a PYTHONHOME environment variable for ages. This pretty much serves the same purpose and together with setting PYTHONPATH can easily be used to create your own private Python universe. >> But if PEP 376 and PEP 386 support are added in Python, we're not far >> from being able to provide multiple version support with >> the help of importlib. Before putting much work into this: do you really think that having multiple versions of the same package in the same Python installation is a good idea ? Examples: What if you have an application that uses two modules which each require two different versions of the same package ? Would that load the same package twice, causing globals used by these package to no work (e.g. isinstance(x, class_name) or global caches) ? This sounds a lot like DLL- or RPM-hell to me. I think it's much better to keep things simple and under user control, e.g. by encouraging use of virtualenv-like setups and perhaps adding better native support for these to Python. If the user finds that there's a version conflict this can then be resolved during environment setup instead of hoping for the best and waiting for application failures at run-time. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ _
Re: [Python-Dev] eggs now mandatory for pypi?
Tarek Ziadé wrote: > On Thu, Oct 8, 2009 at 10:38 AM, M.-A. Lemburg wrote: >> Tarek Ziadé wrote: >>> On Thu, Oct 8, 2009 at 9:35 AM, M.-A. Lemburg wrote: BTW: Who would I need to contact for the PyPI side to work out an upload_url distutils command ? >>> >>> I am not sure to understand. How upload_url will differ from the >>> register command >>> that let you upload metadata at PyPI ? >> >> The register command only uploads the package's meta-data. >> >> The upload command uploads a file and registers the file's >> details with PyPI as well. >> >> I'd like to add an upload_url command which only registers a >> file's details (including the URL, but without the actual file) >> with PyPI and perhaps extend the number of attributes sent to >> PyPI with more file configuration information (upload currently >> only sends the Python version). > > Sounds like an option to be added in upload. With a little code refactoring that would be possible as well, yes. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Hello, Thanks for the summary :) [Distribute 0.7] > - easy_install is going to be deprecated ! use Pip ! Apparently there are a couple of things Pip doesn't support: « It cannot install from eggs. It only installs from source. (Maybe this will be changed sometime, but it’s low priority.) » « It is incompatible with some packages that customize distutils or setuptools in their setup.py files. » « Maybe it doesn’t work on Windows. At least, the author doesn’t test on Windows often. » (from http://pip.openplans.org/#differences-from-easy-install) > Virtualenv allows you to create an isolated environment to install > some distribution without polluting the > main site-packages, a bit like a user site-packages. > > My opinion is that this tool exists only because Python doesn't > support the installation of multiple versions for the same > distributions. virtualenv has the virtue of keeping your main installation clean and manageable. Besides, as someone said, it also allows you to create completely isolated environments if you want full control over your deployment (especially when you're not the sysadmin). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Le Thu, 08 Oct 2009 13:27:57 +0200, M.-A. Lemburg a écrit : > > This sounds a lot like DLL- or RPM-hell to me. > > I think it's much better to keep things simple and under user control, > e.g. by encouraging use of virtualenv-like setups and perhaps adding > better native support for these to Python. Agreed. virtualenv enables a feature while keeping things simple and clear. The proposed scheme (multi-versioned stuff) is supposed to enable the same feature, but by making things more complicated. ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Antoine Pitrou wrote:
> Vinay Sajip yahoo.co.uk> writes:
> "%0#8x" % 0x1234
>> '0x001234'
> "{0:0>#8x}".format(0x1234)
>> '000x1234'
>
> Apart from the sheer unreadability of the {}-style format string, the result
> looks rather unexpected from a human being's point of view.
The percent format string is pretty unreadable too - you're just more
used to it, so it doesn't look as weird :)
Vinay's problem above is due to using the wrong alignment flag: ">",
which says to right align everything, instead of "=", which says to left
align the sign character and the numeric prefix with the fill character
inserted in the middle. In this particular case he could also use the
zero-padding shortcut which leaves out the alignment flag altogether
(and implies a "0=" alignment format).
That is (using 2.6/3.1):
>>> "%#08x" % 0x1234
'0x001234'
>>> "%0#8x" % 0x1234
'0x001234'
>>> "{0:#08x}".format(0x1234)
'0x001234'
>>> "{0:0>#8x}".format(0x1234)
'000x1234'
>>> "{0:0=#8x}".format(0x1234)
'0x001234'
Adding in the sign bit gives the following translations:
>>> "%+#08x" % 0x1234
'+0x01234'
>>> "%+0#8x" % 0x1234
'+0x01234'
>>> "{0:+#08x}".format(0x1234)
'+0x01234'
>>> "{0:0>+#8x}".format(0x1234)
'0+0x1234'
>>> "{0:0=+#8x}".format(0x1234)
'+0x01234'
Note that ">" alignment is actually now *broken* on trunk and py3k,
since ">" and "=" are now behaving exactly the same instead of the
former right aligning the entire number including the sign bit and prefix:
>>> "{:0>+8x}".format(0x1234)
'+0001234'
>>> "{:0=+8x}".format(0x1234)
'+0001234'
(bug assigned to Eric: http://bugs.python.org/issue7081)
Note that, since percent formatting doesn't allow specification of the
fill characters or the field alignment, translations should probably
rely on the simple field width specifier, optionally selecting zero
padding by preceding it with a zero. It should never be necessary to use
the full alignment spec for translated formats.
The other thing to keep in mind is that brace formatting is fussier
about the order of things - items *must* appear in the order they are
listed in PEP 3101 (i.e. if wanting a zero padded field with leading
sign and numeric prefix, you must write "+#0"). Percent format, on the
other hand, allows the "#", "+" and "0" to be placed in any order you
like (although they must appear before the field width definition,
precision specifier and type code).
As far as I can see, that leaves the prefixing of octal numbers ("0o" vs
"0") as the only true incompatibility between percent formatting and
brace formatting, and even for those the incompatibility is limited to
cases where a field width is specified without leading zeroes or a sign
character is specified. In other cases, the translation can just stick a
leading literal "0" in front of the field in the brace formatting string.
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 and IronPython
Christian Heimes wrote: > The solution requires a new attribute in the sys module that contains > the name of the implementation. As an alternative we could use the first > field of sys.subversion but I prefer an explicit attribute. I'm > proposing "sys.name" with a value of "CPython", "IronPython", "Jython" > or "PyPy". My Google skills almost failed me, but searching for "sys.vm" found me what I was after: http://bugs.python.org/issue4242 (a discussion relating to a similar need in the context of flagging implementation specific tests). As mentioned in that discussion, as of Python 2.6, you can do the following: >>> import platform >>> platform.python_implementation() 'CPython' (Although according to the function docstring, PyPy is currently missing from the list of known implementations) Importing yet-another-module for use in site.py doesn't sound like a great idea, so it may make sense to migrate that information into the sys module is this approach is taken. "sys.name" is a little generic though - something more explicit like "sys.vm" would be better. > The site module uses the information of the attribute to > modify the path to the user site directory. Implementation specific user directories sound like a good idea indeed. An alternative to a lookup table approach might be to be a little more direct and just retrieve the final part of the user specific directory name directly from a new function in the sys module. Then different VM authors can choose any directory name they want without CPython's site.py needing to know anything else about them. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.6.4rc1
On Oct 7, 2009, at 2:09 PM, Barry Warsaw wrote:
I want us to be very careful about 2.6.4. This isn't a normal bug
fix release, it's specifically there to remove the brown paper bag
of 2.6.3 from our heads. So let's be conservative and fix this one
in 2.6.5.
Can we get the readline patch reviewed as well. It was applied to
trunk already: http://svn.python.org/view?view=rev&revision=74970
It's marked for backport to 2.6 as "needs review":
http://bugs.python.org/issue6877
It fixes a BusError crash that was just swiped under a rug by
disabling the build in setup.py. :-)
FWIW, the part of the patch that changes setup.py will be useful for
another comparison of Mac OS X release numbers. The line 1362 on the
trunk (1364 on the 2.6 branch) compares strings and, unfortunately,
'10.' < '8.' for Darwin 10 (Mac OS X 10.6 Snow Leopard):
if os.uname()[2] > '8.':
So, it should be changed to:
if os_release > 8:
where ``os_release`` is introduced by the patch mentioned above as:
os_release = int(os.uname()[2].split('.')[0])
on the line 550 on the trunk.
Regards,
Zvezdan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziadé wrote: > = Virtualenv and the multiple version support in Distribute = ... > My opinion is that this tool exists only because Python doesn't > support the installation of multiple versions for the same > distributions. This is not at all how I use virtualenv. For me virtualenv is a sandbox so that I don't have to become root whenever I need to install a Python package for testing purposes and to allow me to hop between sets of installed Python packages while developing on multiple Python projects. I also use it as a sandbox for build bots so that multiple bots on the same machine can each build their own projects with just the known dependencies installed. Schiavo Simon ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 1:27 PM, M.-A. Lemburg wrote: > >>> == The fate of setup.py, and static metadata == >>> >>> So we are going to separate the metadata description from setup.py, in >>> a static configuration file, that can be open and read by anyone >>> without >>> running any code. The only problem with this is the fact that some >>> metadata fields might depend on the execution environment. For >>> instance, once "Requires" is re-defined and re-introduced via PEP 314, >>> we will have cases where "pywin32" will be a dependency to have only >>> on win32 systems. > > How are you planning to deal with dynamic version strings, e.g. > say you want to build a snapshot release the current date and > revision number included in the version string ? > > I suppose this file is going to be uploaded to PyPI, so the > file would have to be formatted at build time and include > markers to be replaced with values taken from the distribution > meta-data. The metadata will always be customizable from setup.py somehow. We won't treat those cases : if the metadata is provided in the static file, you can get it without any further processing, if not you will get an "UNKOWN" value for it and you will have to grab the whole archive and run setup.py to get it. Now the example you've shown is in my opinion not a problem : you can create the static value for "version" that you put in your static file with some pre-processing that occurs when you build your release. (ala autoconf) > >>> - Work left to do : publish the final syntax, and do the implementation > > Is there a PEP for this ? No, but if you ask for it I can write it. > >>> == The fate of bdist_* commands == >>> >>> So the best way to handle this is to ask these communities to build >>> their own tool and to encourage them to use Distutils as a basis for >>> that. > > Rather than having external communities build their own > tools and then basically fork distutils in order to get > these implemented, I think it'd be better to make the > set of distutils commands extensible via namespace packages. I am not sure, why they would fork distutils. Distutils is already extensible, you can configure in distutils.cfg a list of namespace packages, that contains commands and they'll get loaded by dist.py. > >>> So what is going to happen is a status-quo: no bdist_* command will be >>> removed but no new bdist_* command will be added. > > I hope that comment means "no new bdist_* command will be added > *right now*". It's well possible to create new bdist_* commands > that don't rely on external tool chains or only on ones that > don't change often. > > I'm thinking of bdist_inno (Windows installer using InnoSetup) > and bdist_nsis (Windows installer using NSIS) here. As I said, windows don't have any packaging system so I don't think it's a problem to have more installers for this platform. Regards Tarek -- Tarek Ziadé | http://ziade.org | オープンソースはすごい! | 开源传万世，因有你参与 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 and IronPython
On Thu, Oct 8, 2009 at 1:17 PM, Christian Heimes wrote: [...] > The altered user site directories are: > > CPython: > ~/.local/lib/python2.6/site-packages > %APPDATA%/Python/Python26 > > IronPython: > ~/.local/lib/ironpython2.6/site-packages > %APPDATA%/Python/IronPython26 > > Jython: > ~/.local/lib/jython2.6/site-packages > %APPDATA%/Python/Jython26 > Hi Christian, +1 I have a suggestion though, that completes Nick's answer. distutils/command/install.py already contains install schemes and imports USER_SITE and USER_BASE to builds user-specific install schemes. I think it wouldn't hurt for clarity to reunite all these schemes in a single place in the stdlib (so in sys maybe?) Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Simon Cross wrote: On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziadé wrote: = Virtualenv and the multiple version support in Distribute = ... My opinion is that this tool exists only because Python doesn't support the installation of multiple versions for the same distributions. This is not at all how I use virtualenv. For me virtualenv is a sandbox so that I don't have to become root whenever I need to install a Python package for testing purposes and to allow me to hop between sets of installed Python packages while developing on multiple Python projects. I also use it as a sandbox for build bots so that multiple bots on the same machine can each build their own projects with just the known dependencies installed. This is exactly why I use virtualenv as well. I don't recall ever having wanted / needed to install multiple versions of the same library - whilst I can appreciate that it *can* be a real issue it has never been a problem for me. Michael Schiavo Simon ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] locals() different behaviour when tracing?
Is it intentional that locals() returns a copy/snapshot of the local
variables sometimes, and sometimes a reference? Like when enabling
tracing, such as in the code pasted below. The documentation ("Update
and return a dictionary containing the current scope's local
variables.") is pretty unclear.
import sys
def X():
l = locals()
i = "foo"
print "Is 'i' in stored locals()? ", ('i' in l)
print "Running normally"
X()
print "Enabling tracing"
def t(*a):
return t
sys.settrace(t)
X()
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
On approximately 10/7/2009 10:45 PM, came the following characters from the keyboard of Vinay Sajip: Glenn Linderman g.nevcal.com> writes: But DictConfigurator the name seems misleading... like you are configuring how dicts work, rather than how logs work. Maybe with more context this is not a problem, but if it is a standalone class, it is confusing what it does, by name alone. Of course the fully qualified name would be logging.config.DictConfigurator which does provide the requisite context, but if I think of/someone suggests a better name I'll certainly use it. Yes, I suspected it might be something like that, and that does provide the context, so I'm content. Thanks for verifying it, and the rest is bikeshedding... In thinking more about this, I guess I would have the same comment about the existing fileConfig() API also... it isn't file that is being configured, but the logs. readConfigFromFile would have been more descriptive (and lots longer :( ). Had fileConfig been called "bulkConfig" or "configAll" (these names attempt to contrast the function of this API versus the individual APIs that configure one thing at a time) taking a file parameter, then it could just simply/suddenly also allow a dict parameter, and no new API would have to be created. Too late for this, however, and with the new API, it would be possible to deprecate the fileConfig API, and tell the user to use the ConfigParser (or JSON or YAML or whatever) to create the dict, and then pass that to DictConfigurator (or bulkConfig or whatever). -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Nick Coghlan gmail.com> writes:
> Vinay's problem above is due to using the wrong alignment flag: ">",
> which says to right align everything, instead of "=", which says to left
> align the sign character and the numeric prefix with the fill character
> inserted in the middle. In this particular case he could also use the
> zero-padding shortcut which leaves out the alignment flag altogether
> (and implies a "0=" alignment format).
>
[snip]
> Note that, since percent formatting doesn't allow specification of the
> fill characters or the field alignment, translations should probably
> rely on the simple field width specifier, optionally selecting zero
> padding by preceding it with a zero. It should never be necessary to use
> the full alignment spec for translated formats.
>
> The other thing to keep in mind is that brace formatting is fussier
> about the order of things - items *must* appear in the order they are
> listed in PEP 3101 (i.e. if wanting a zero padded field with leading
> sign and numeric prefix, you must write "+#0"). Percent format, on the
> other hand, allows the "#", "+" and "0" to be placed in any order you
> like (although they must appear before the field width definition,
> precision specifier and type code).
>
> As far as I can see, that leaves the prefixing of octal numbers ("0o" vs
> "0") as the only true incompatibility between percent formatting and
> brace formatting, and even for those the incompatibility is limited to
> cases where a field width is specified without leading zeroes or a sign
> character is specified. In other cases, the translation can just stick a
> leading literal "0" in front of the field in the brace formatting string.
Helpful analysis there, Nick, thanks. Bonzer ;-)
There's also the corner case of things like
%#.0f
which, when asked to format 3e100, will print
3.e+100
whereas the translated format {0:.0f}, will print
3e+100
for the same value.
BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
worth raising an enhancement request on the bug tracker using "O" to generate
compatible rendering for octals.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Vinay Sajip wrote:
BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
worth raising an enhancement request on the bug tracker using "O" to generate
compatible rendering for octals.
I didn't get your message, could you resend?.
I was thinking the same thing, but it seems like a transition step. I'd
rather not keep such backward compatibility hacks (if you will) around
for the long haul. How about a flag (maybe '*') at the start of the
format specification which says "operate in backward compatibility
mode"? We could document it as being only useful for the % to {}
translator, and promise to remove it at some point in the future. Either
actually deprecate it or just promise to deprecate it in the future.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
On Thu, Oct 8, 2009 at 2:44 AM, Glenn Linderman wrote: > On approximately 10/7/2009 10:45 PM, came the following characters from the > keyboard of Vinay Sajip: >> >> Glenn Linderman g.nevcal.com> writes: >> >>> But DictConfigurator the name seems misleading... like you are >>> configuring how dicts work, rather than how logs work. Maybe with more >>> context this is not a problem, but if it is a standalone class, it is >>> confusing what it does, by name alone. >> >> Of course the fully qualified name would be >> logging.config.DictConfigurator >> which does provide the requisite context, but if I think of/someone >> suggests a >> better name I'll certainly use it. > [...] > > In thinking more about this, I guess I would have the same comment about the > existing fileConfig() API also... it isn't file that is being configured, > but the logs. readConfigFromFile would have been more descriptive (and lots > longer :( ). > cfgFromIni | cfgFromFile ? I prefer the former because YAML, shelves, JSON & Co can also be stored in a file , so at least I get a better understanding of the format in use. Otherwise I'd had to read the docs or figure out which one > Had fileConfig been called "bulkConfig" or "configAll" (these names attempt > to contrast the function of this API versus the individual APIs that > configure one thing at a time) taking a file parameter, then it could just > simply/suddenly also allow a dict parameter, and no new API would have to be > created. Too late for this, however, and with the new API, it would be > possible to deprecate the fileConfig API, and tell the user to use the > ConfigParser (or JSON or YAML or whatever) to create the dict, and then pass > that to DictConfigurator (or bulkConfig or whatever). > similar and consistent with what I mentioned before but IMO fileConfig should remain for backwards compatibility Objections ? -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Looking for a technique to create flexible, graphical dashboards ... - http://feedproxy.google.com/~r/TracGViz-full/~3/71kRhT34BgU/43789 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Oct 8, 2009, at 4:31 AM, Tarek Ziadé wrote: - no more namespaced packages system, if PEP 381 (namespaces package support) makes it to 2.7 Make that PEP 382: http://www.python.org/dev/peps/pep-0382/ -Barry PGP.sig Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
> I've had bad experiences in the past with dictionary-based APIs. They seem > "simpler" in the short run, because the user "only needs to create some > dictionaries". Once the complexity of that nested dictionary grows to a > certain > point, though, one has to refer back to documentation constantly to make sure Fair point, and agreed that the schema needs some care, here's roughly what I'm thinking of as an example configuration (YAML): formatters: brief: format: '%(levelname)-8s: %(name)-15s: %(message)s' precise: format: '%(asctime)s %(name)-15s %(levelname)-8s %(message)s' filters: allow_foo: name: foo handlers: console: class : logging.StreamHandler formatter: brief level : INFO stream : sys.stdout filters: [allow_foo] file: class : logging.handlers.RotatingFileHandler formatter: precise filename: logconfig.log maxBytes: 1024 backupCount: 3 debugfile: class : logging.FileHandler formatter: precise filename: logconfig-detail.log mode: a loggers: foo: level : ERROR handlers: [debugfile] spam: level : CRITICAL handlers: [debugfile] propagate: no bar.baz: level: WARNING root: level : DEBUG handlers : [console, file] It's not too deeply nested, and I can't see any need for it being more deeply nested. It more or less mirrors the way you'd have to write the corresponding ConfigParser-compatible configuration, but if you use a dict as a common format, you have the benefits which I mentioned of supporting JSON, YAML or Python code in a Django settings.py. > the structure conforms to the "schema". Building a simple config tree using > light-weight classes with documented APIs tends to be more sustainable in the > long run. When you say 'config tree using light-weight classes', I presume you mean a parallel set of classes to the actual logging classes which will be instantiated? ISTM logging classes are already reasonably usable for programmatic configuration, but this doesn't address e.g. updating configurations easily without program code changes, or the ability to configure from say a YAML node, where YAML may be being used for application configuration as a whole. (Unless of course I've misunderstood what you're getting at.) Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
> I didn't get your message, could you resend?.
Resent, it may have been stopped by your spam filters since it came from my
vinay-underscore-sajip-at-red-hyphen-dove-dot-com address. The subject was
"Python str.format() and octal formatting compatibility".
> I was thinking the same thing, but it seems like a transition step. I'd
> rather
> not keep such backward compatibility hacks (if you will) around for the long
> haul. How about a flag (maybe '*') at the start of the format specification
> which says "operate in backward compatibility mode"? We could document it as
> being only useful for the % to {} translator, and promise to remove it at
> some
> point in the future. Either actually deprecate it or just promise to
> deprecate
> it in the future.
I don't much mind matter exactly which mechanism we use to distinguish between
0o and 0 prefixes, as long as it's one most people are happy with :-)
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Tarek Ziadé wrote: > On Thu, Oct 8, 2009 at 1:27 PM, M.-A. Lemburg wrote: >> == The fate of setup.py, and static metadata == So we are going to separate the metadata description from setup.py, in a static configuration file, that can be open and read by anyone without running any code. The only problem with this is the fact that some metadata fields might depend on the execution environment. For instance, once "Requires" is re-defined and re-introduced via PEP 314, we will have cases where "pywin32" will be a dependency to have only on win32 systems. >> >> How are you planning to deal with dynamic version strings, e.g. >> say you want to build a snapshot release the current date and >> revision number included in the version string ? >> >> I suppose this file is going to be uploaded to PyPI, so the >> file would have to be formatted at build time and include >> markers to be replaced with values taken from the distribution >> meta-data. > > The metadata will always be customizable from setup.py somehow. > > We won't treat those cases : if the metadata is provided in the static file, > you can get it without any further processing, if not you will get an > "UNKOWN" value > for it and you will have to grab the whole archive and run setup.py to get it. > > Now the example you've shown is in my opinion not a problem : you can > create the static value for "version" that you put in your static file > with some pre-processing that > occurs when you build your release. (ala autoconf) Hmm, then I'm not really sure I understand the proposal for a new static meta-data file... distutils already creates the static meta-data file PKG-INFO when building sdist distributions and that's done based on the data read from the setup() call. It would be easy to use that file as static meta-data file in other distribution formats as well (e.g. like you proposed in PEP 376). OTOH, the register command sends the meta-data directly to the PyPI database, so it doesn't even need another file for storing away the meta data. Unless I'm missing something important (which I probably am), the only added value of the new format over PKG-INFO is the micro-language. Couldn't we just stick that into the values of the various meta-data fields, e.g. to have the Requires field be different depending on platform ? We'd then only have one meta-data format in distutils - the PKG-INFO one. - Work left to do : publish the final syntax, and do the implementation >> >> Is there a PEP for this ? > > No, but if you ask for it I can write it. Please do. The format will need documentation anyway and the motivation should be made clear as well. == The fate of bdist_* commands == So the best way to handle this is to ask these communities to build their own tool and to encourage them to use Distutils as a basis for that. >> >> Rather than having external communities build their own >> tools and then basically fork distutils in order to get >> these implemented, I think it'd be better to make the >> set of distutils commands extensible via namespace packages. > > I am not sure, why they would fork distutils. Not really a fork, but a copy of distutils in order to give the user the complete tool set in one download, rather than having to configure a system installed distutils to load the new command. > Distutils is already extensible, you can configure in distutils.cfg a > list of namespace packages, > that contains commands and they'll get loaded by dist.py. Right, but dist.py could load all distutils.command modules it can find to make the installation of extensions more user friendly (like in a plugin system). Just a minor nit, though. So what is going to happen is a status-quo: no bdist_* command will be removed but no new bdist_* command will be added. >> >> I hope that comment means "no new bdist_* command will be added >> *right now*". It's well possible to create new bdist_* commands >> that don't rely on external tool chains or only on ones that >> don't change often. >> >> I'm thinking of bdist_inno (Windows installer using InnoSetup) >> and bdist_nsis (Windows installer using NSIS) here. > > As I said, windows don't have any packaging system so I don't think > it's a problem to have more installers for this platform. Ok. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registe
Re: [Python-Dev] transitioning from % to {} formatting
Christian Tanzer wrote:
Eric Smith wrote at Thu, 08 Oct 2009 10:24:33 -0400:
Vinay Sajip wrote:
BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it was
worth raising an enhancement request on the bug tracker using "O" to generate
compatible rendering for octals.
I didn't get your message, could you resend?.
I was thinking the same thing, but it seems like a transition step. I'd
rather not keep such backward compatibility hacks (if you will) around
for the long haul. How about a flag (maybe '*') at the start of the
format specification which says "operate in backward compatibility
mode"? We could document it as being only useful for the % to {}
translator, and promise to remove it at some point in the future. Either
actually deprecate it or just promise to deprecate it in the future.
That doesn't seem very useful to me. IIUC, the point of the translator
is to allow porting of the millions of existing %-formating operations
to the new-style .format.
If the result of that is deprecated or removed a few years from now,
all maintainers of long existing code have exactly the same problem.
I was thinking of it as a transition step until all application code
switched to {} formatting. In which case the application has to deal
with it.
IMHO, either the translation is done once and gives identical output or
it isn't worth doing at all.
I disagree. I doubt even 0.001% of all format strings involve octal
formatting. Is it really worth not providing a transition path if it
can't cover this case?
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 and IronPython
2009/10/8 Nick Coghlan : > As mentioned in that discussion, as of Python 2.6, you can do the following: import platform platform.python_implementation() > 'CPython' > > (Although according to the function docstring, PyPy is currently missing > from the list of known implementations) That docstring should be updated then because I added support for PyPy in 2.7 and 3.1. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Eric Smith wrote at Thu, 08 Oct 2009 10:55:21 -0400:
> >>> BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if
> >>> it was
> >>> worth raising an enhancement request on the bug tracker using "O" to
> >>> generate
> >>> compatible rendering for octals.
> >> I didn't get your message, could you resend?.
> >>
> >> I was thinking the same thing, but it seems like a transition step. I'd
> >> rather not keep such backward compatibility hacks (if you will) around
> >> for the long haul. How about a flag (maybe '*') at the start of the
> >> format specification which says "operate in backward compatibility
> >> mode"? We could document it as being only useful for the % to {}
> >> translator, and promise to remove it at some point in the future. Either
> >> actually deprecate it or just promise to deprecate it in the future.
> >
> > That doesn't seem very useful to me. IIUC, the point of the translator
> > is to allow porting of the millions of existing %-formating operations
> > to the new-style .format.
> >
> > If the result of that is deprecated or removed a few years from now,
> > all maintainers of long existing code have exactly the same problem.
>
> I was thinking of it as a transition step until all application code
> switched to {} formatting. In which case the application has to deal
> with it.
You lost me here.
All that talk of deprecating %-formatting makes me really nervous.
%-formatting is pervasive in all existing Python code.
Without an automatic translator that is 100% accurate, porting all
that code to {}-formatting is not possible. Heck, it's not even
possible to grep for all instances of %-formatting.
How do you suppose that maintainers could ever do the transition from
%- to {}-formatting manually?
> > IMHO, either the translation is done once and gives identical output or
> > it isn't worth doing at all.
>
> I disagree. I doubt even 0.001% of all format strings involve octal
> formatting. Is it really worth not providing a transition path if it
> can't cover this case?
If %-formatting is first deprecated then removed from Python and there
is no automatic transition path that effectively means that existing
code using %-formatting is forced to stay at whatever Python version
was the last one supporting %-formatting.
I surely hope nobody is seriously considering such a scenario. Perl 6
seems harmless in comparison.
--
Christian Tanzerhttp://www.c-tanzer.at/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Eric Smith wrote at Thu, 08 Oct 2009 10:24:33 -0400:
> Vinay Sajip wrote:
> > BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see if it
> > was
> > worth raising an enhancement request on the bug tracker using "O" to
> > generate
> > compatible rendering for octals.
>
> I didn't get your message, could you resend?.
>
> I was thinking the same thing, but it seems like a transition step. I'd
> rather not keep such backward compatibility hacks (if you will) around
> for the long haul. How about a flag (maybe '*') at the start of the
> format specification which says "operate in backward compatibility
> mode"? We could document it as being only useful for the % to {}
> translator, and promise to remove it at some point in the future. Either
> actually deprecate it or just promise to deprecate it in the future.
That doesn't seem very useful to me. IIUC, the point of the translator
is to allow porting of the millions of existing %-formating operations
to the new-style .format.
If the result of that is deprecated or removed a few years from now,
all maintainers of long existing code have exactly the same problem.
IMHO, either the translation is done once and gives identical output or
it isn't worth doing at all.
--
Christian Tanzerhttp://www.c-tanzer.at/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Benjamin Peterson wrote:
2009/10/8 Eric Smith :
Christian Tanzer wrote:
IMHO, either the translation is done once and gives identical output or
it isn't worth doing at all.
I disagree. I doubt even 0.001% of all format strings involve octal
formatting. Is it really worth not providing a transition path if it can't
cover this case?
It's also really easy to just write 0{:o} like my translator does.
I apologize for not having looked at anyone's implementation yet. But I
suspect you'll have a problem with this case unless int.__format__ has
special logic for backward compatible octal formatting:
>>> '%# 5o' % 8
' 010'
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locals() different behaviour when tracing?
Anders Waldenborg 0x63.nu> writes: > > Is it intentional that locals() returns a copy/snapshot of the local > variables sometimes, and sometimes a reference? Like when enabling > tracing, such as in the code pasted below. Since someone else opened a bug, I answered there. Anyone, feel free to correct me if my answer is wrong. http://bugs.python.org/issue7083 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Eric Smith wrote:
Vinay Sajip wrote:
BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see
if it was
worth raising an enhancement request on the bug tracker using "O" to
generate
compatible rendering for octals.
I didn't get your message, could you resend?.
I was thinking the same thing, but it seems like a transition step. I'd
rather not keep such backward compatibility hacks (if you will) around
for the long haul. How about a flag (maybe '*') at the start of the
format specification which says "operate in backward compatibility
mode"? We could document it as being only useful for the % to {}
translator, and promise to remove it at some point in the future. Either
actually deprecate it or just promise to deprecate it in the future.
I think the flag should be '%'.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Christian Tanzer wrote: [snip...] IMHO, either the translation is done once and gives identical output or it isn't worth doing at all. I disagree. I doubt even 0.001% of all format strings involve octal formatting. Is it really worth not providing a transition path if it can't cover this case? If %-formatting is first deprecated then removed from Python and there is no automatic transition path that effectively means that existing code using %-formatting is forced to stay at whatever Python version was the last one supporting %-formatting. I surely hope nobody is seriously considering such a scenario. Perl 6 seems harmless in comparison. That is vastly overstating it. Making 'with' and 'as' keywords and removing string exceptions (which have already happened) will affect far more programs than a minor incompatibility in transitioning string formatting. Michael -- Christian Tanzerhttp://www.c-tanzer.at/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging, Unicode and sockets
Vinay Sajip wrote: Thanks to http://bugs.python.org/issue7077 I've noticed that the socket-based logging handlers - SocketHandler, DatagramHandler and SysLogHandler - aren't Unicode-aware and can break in the presence of Unicode messages. I'd like to fix this by giving these handlers an optional (encoding=None) parameter in their __init__, and then using this to encode on output. If no encoding is specified, is it best to use locale.getpreferredencoding(), sys.getdefaultencoding(), sys.getfilesystemencoding(), 'utf-8' or something else? On my system: sys.getdefaultencoding() 'ascii' sys.getfilesystemencoding() 'mbcs' locale.getpreferredencoding() 'cp1252' which suggests to me that the locale.getpreferredencoding() should be the default. However, as I'm not a Unicode maven, any suggestions would be welcome. Well, encodings can vary from machine to machine, and if the encoding doesn't cover all the Unicode codepoints then you could get an encoding exception. For these reasons I'd vote for UTF-8. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
On Oct 8, 2009, at 10:47 AM, Vinay Sajip wrote: I've had bad experiences in the past with dictionary-based APIs. They seem "simpler" in the short run, because the user "only needs to create some dictionaries". Once the complexity of that nested dictionary grows to a certain point, though, one has to refer back to documentation constantly to make sure Fair point, and agreed that the schema needs some care, here's roughly what I'm thinking of as an example configuration (YAML): formatters: brief: format: '%(levelname)-8s: %(name)-15s: %(message)s' precise: format: '%(asctime)s %(name)-15s %(levelname)-8s %(message)s' filters: allow_foo: name: foo handlers: console: class : logging.StreamHandler formatter: brief level : INFO stream : sys.stdout filters: [allow_foo] file: class : logging.handlers.RotatingFileHandler formatter: precise filename: logconfig.log maxBytes: 1024 backupCount: 3 debugfile: class : logging.FileHandler formatter: precise filename: logconfig-detail.log mode: a loggers: foo: level : ERROR handlers: [debugfile] spam: level : CRITICAL handlers: [debugfile] propagate: no bar.baz: level: WARNING root: level : DEBUG handlers : [console, file] It's not too deeply nested, and I can't see any need for it being more deeply nested. It more or less mirrors the way you'd have to write the corresponding ConfigParser-compatible configuration, but if you use a dict as a common format, you have the benefits which I mentioned of supporting JSON, YAML or Python code in a Django settings.py. Yes, that makes sense. the structure conforms to the "schema". Building a simple config tree using light-weight classes with documented APIs tends to be more sustainable in the long run. When you say 'config tree using light-weight classes', I presume you mean a parallel set of classes to the actual logging classes which will be instantiated? ISTM logging classes are already reasonably usable for programmatic configuration, but this doesn't address e.g. updating configurations easily without program code changes, or the ability to configure from say a YAML node, where YAML may be being used for application configuration as a whole. (Unless of course I've misunderstood what you're getting at.) The probably are light-weight enough, but I didn't want to assume those classes would be used directly. I suspect they could be, though. I guess there aren't *so* many levels or keys that the data structure will become unwieldy. Doug ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 9:52 AM, Michael Foord wrote: > Simon Cross wrote: >> >> On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziadé >> wrote: >> >>> >>> = Virtualenv and the multiple version support in Distribute = >>> >> >> ... >> >>> >>> My opinion is that this tool exists only because Python doesn't >>> support the installation of multiple versions for the same >>> distributions. >>> >> >> This is not at all how I use virtualenv. For me virtualenv is a >> sandbox so that I don't have to become root whenever I need to install >> a Python package for testing purposes and to allow me to hop between >> sets of installed Python packages while developing on multiple Python >> projects. I also use it as a sandbox for build bots so that multiple >> bots on the same machine can each build their own projects with just >> the known dependencies installed. >> >> > > This is exactly why I use virtualenv as well. I don't recall ever having > wanted / needed to install multiple versions of the same library - whilst I > can appreciate that it *can* be a real issue it has never been a problem for > me. > > Michael +1 - virtualenv, AFAIK is used for sandboxing/isolation of various environments, not dealing with multiple versions within the *same* environment. Of course, it does solve the "being dependent on a specific version" of a dependency because it *is* sandboxed from everything else. Adding multiple version support doesn't remove the need for sandboxing. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Michael Foord wrote at Thu, 08 Oct 2009 16:56:35 +0100: > > If %-formatting is first deprecated then removed from Python and there > > is no automatic transition path that effectively means that existing > > code using %-formatting is forced to stay at whatever Python version > > was the last one supporting %-formatting. > > > > I surely hope nobody is seriously considering such a scenario. Perl 6 > > seems harmless in comparison. > > > That is vastly overstating it. Making 'with' and 'as' keywords and > removing string exceptions (which have already happened) will affect far > more programs than a minor incompatibility in transitioning string > formatting. `with` and `as` are trivial to fix and certainly not pervasive in existing code. String exceptions have been deprecated for years. -- Christian Tanzerhttp://www.c-tanzer.at/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 370 and IronPython
Christian Heimes wrote: [snip...] The altered user site directories are: CPython: ~/.local/lib/python2.6/site-packages %APPDATA%/Python/Python26 IronPython: ~/.local/lib/ironpython2.6/site-packages %APPDATA%/Python/IronPython26 Jython: ~/.local/lib/jython2.6/site-packages %APPDATA%/Python/Jython26 +1 I really like this scheme. The important thing for IronPython is that we can get it into Python 2.6 (along with other fixes to make distutils compatible with IronPython - like not attempting to bytecode-compile when sys.dont_write_bytecode is True). All the best, Michael Foord -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
2009/10/8 Eric Smith :
> Christian Tanzer wrote:
>> IMHO, either the translation is done once and gives identical output or
>> it isn't worth doing at all.
>
> I disagree. I doubt even 0.001% of all format strings involve octal
> formatting. Is it really worth not providing a transition path if it can't
> cover this case?
It's also really easy to just write 0{:o} like my translator does.
--
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote: > > Tarek Ziadé a écrit : > >> But if PEP 376 and PEP 386 support are added in Python, we're not far > >> from being able to provide multiple version support with > >> the help of importlib. > > Before putting much work into this: do you really think that having > multiple versions of the same package in the same Python installation > is a good idea ? > I think it is a good idea. > Examples: > What if you have an application that uses two modules which each > require two different versions of the same package ? Would that > load the same package twice, causing globals used by these package > to no work (e.g. isinstance(x, class_name) or global caches) ? > That's not how it should work. Look at other systems that allow for installing multiple versions of a library -- for instance, loading dynamic shared objects in C * You can install multiple versions of a library in parallel * The dynamic loader will pick the version of the library that is appropriate from the list of available options (the program specifies the SONAME it needs -- library name plus API version. The loader then chooses the most recent revision that matches that API version.) * When one binary needs multiple API versions of the library, the application cannot start. The last point addresses your concern -- depending on multiple, incompatible versions of a library is prohibited. The programmer of the application needs to make the code run with a single version of the code. > This sounds a lot like DLL- or RPM-hell to me. > RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression that it is the lack of enough version specification rather than too much but I don't know for sure). is similar but it takes place on the end-user's system. This should take place on the programmer's system instead. End-users are not in a position to fix things like RPM-hell. Programmers are. Example RPM-hell: Application Foo requires libbar-1.x Application Baz requires libbar-2.x The user may either have Foo or Baz installed on their system with the appropriate libbar but not both. They depend on the packagers and developers of Foo and Bar to do one of the following to resolve the situation: * Port Foo and Baz to use the same version of libbar. * Package libbar in such a way that libbar-1 and libbar-2 are parallel installable on the system. Then they can install two separate packages, libbar1-1.0 and libbar2-2.0. Example of similar Distutils multiple version problem: The programmer creates an application Foo that depends on python-bar-1.x. He has recently started work on a file that imports python-baz-1.0. python-baz depends on python-bar-2.x. The first time he tries to run his new code, python gives him an error message that it is impossible to satisfy the version requirements for python-bar. Depending on how the versions are specified, the error message could be very specific and helpful: Impossible version requirements: bar Requires: python-baz>=2.0, < 3.0 foo.py Requires: python-baz >=1.0, < 2.0 The programmer can then discard their new code, port foo.py to python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to the upstream of that module. Note two things about this scenario: 1) The programmer is the person who is responsible for creating the conflict and for resolving it. They are the proper authority for making the decision to port to python-baz-2.x or not using python-bar. The end-user who is not responsible is not impacted by this at all. 2) The programmer would have had to deal with this issue whether we allow multiple versions to be installed or not. With multiple version support we may be able to get them better error messages (depending on how the dependency information is formatted and how completely it was specified in the app and modules). > I think it's much better to keep things simple and under user > control, e.g. by encouraging use of virtualenv-like setups > and perhaps adding better native support for these to Python. > > If the user finds that there's a version conflict this can > then be resolved during environment setup instead of hoping > for the best and waiting for application failures at run-time. > For the class of user that is actually a developer, it might be somewhat true that version conflicts should be resolved by them. But for the class of user that is an end-user, version conflicts are a totally foreign concept. They should be dealt with by the person who is coding the application for them. Also note, the ability to have multiple versions makes things easier for system packagers and provides an easy alternative to a virtualenv for end-users. * System packagers: virtualenv does not provide a method suitable for system packagers. The nearest adaptation would be for the system packager to install python packages into their own hierarchy not in the PYTHONPATH. Then they w
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 08, 2009 at 02:52:24PM +0200, Simon Cross wrote: > On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziadé wrote: > > = Virtualenv and the multiple version support in Distribute = > ... > > My opinion is that this tool exists only because Python doesn't > > support the installation of multiple versions for the same > > distributions. > Let's actually look at these reasons: > This is not at all how I use virtualenv. For me virtualenv is a > sandbox so that I don't have to become root whenever I need to install > a Python package for testing purposes This is needing to install multiple versions and use the newly installed version for testing. > and to allow me to hop between > sets of installed Python packages while developing on multiple Python > projects. This is the ability to install multiple versions and specify different versions for different projects you're working on. > I also use it as a sandbox for build bots so that multiple > bots on the same machine can each build their own projects with just > the known dependencies installed. > This is the only use in the list that is virtualenv specific. The rest are cases of needing to install multiple versions on the system. -Toshio pgpFewnC5XGPp.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new way to configure logging
On Oct 7, 2009, at 10:49 AM, Vinay Sajip wrote: All three of the contenders for the title of "commonly found configuration mechanism" - JSON, YAML and Python code - will be expressible, in Python, as Python dicts. So it seems to make sense to add, to logging.config, a new callable bound to "dictConfig" which will take a single dictionary argument and configure logging from that dictionary. I've had bad experiences in the past with dictionary-based APIs. They seem "simpler" in the short run, because the user "only needs to create some dictionaries". Once the complexity of that nested dictionary grows to a certain point, though, one has to refer back to documentation constantly to make sure the structure conforms to the "schema". Building a simple config tree using light-weight classes with documented APIs tends to be more sustainable in the long run. Doug ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] locals() different behaviour when tracing?
2009/10/8 Anders Waldenborg :
>
> Is it intentional that locals() returns a copy/snapshot of the local
> variables sometimes, and sometimes a reference? Like when enabling tracing,
> such as in the code pasted below. The documentation ("Update and return a
> dictionary containing the current scope's local variables.") is pretty
> unclear.
Yes, it does, and that's why the documentation says changing it is undefined. :)
--
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and so me words on Virtualenv, Pip)
Toshio Kuratomi gmail.com> writes: > > This is needing to install multiple versions and use the newly installed > version for testing. [...] What you're missing is that having separate environments has a virtue of cleanliness, understandability and robustness that a multiple-versioned solution doesn't have. While the technical merits are debatable I'm sure some people definitely prefer to manage a virtualenv-based version. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Toshio Kuratomi wrote: On Thu, Oct 08, 2009 at 02:52:24PM +0200, Simon Cross wrote: On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziadé wrote: = Virtualenv and the multiple version support in Distribute = ... My opinion is that this tool exists only because Python doesn't support the installation of multiple versions for the same distributions. Let's actually look at these reasons: This is not at all how I use virtualenv. For me virtualenv is a sandbox so that I don't have to become root whenever I need to install a Python package for testing purposes This is needing to install multiple versions and use the newly installed version for testing. Not really - it is wanting to install a single version of a library that you don't want to install into your 'main' (whether that be user or system) Python install. It is sandboxing and orthogonal to multiple versions. and to allow me to hop between sets of installed Python packages while developing on multiple Python projects. This is the ability to install multiple versions and specify different versions for different projects you're working on. No, it is working on multiple projects that have *different* dependencies and being able to work in an environment that *only* has direct dependencies installed - again sandboxed from your main Python environment. The fact that virtualenv allows you to have multiple different versions of the same library installed in the different environments you create is completely separate from the motivation that causes many people to use it. What virtualenv *doesn't* do (I believe) is allow you to have multiple versions of libraries installed within a single environment and switch between them (at least it doesn't offer anything beyond what setuptools or pip provides). Michael I also use it as a sandbox for build bots so that multiple bots on the same machine can each build their own projects with just the known dependencies installed. This is the only use in the list that is virtualenv specific. The rest are cases of needing to install multiple versions on the system. -Toshio ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On 8 Oct 2009, at 18:17 , Toshio Kuratomi wrote: This is not at all how I use virtualenv. For me virtualenv is a sandbox so that I don't have to become root whenever I need to install a Python package for testing purposes This is needing to install multiple versions and use the newly installed version for testing. No it's not. It's keeping the python package *being tested* out of the system's or user's site-package because it's potentially unstable or unneeded. It provides a trivial way of *removing* the package to get rid of it: delete the virtualenv. No trace anywhere that the package was ever installed, no impact on the system (apart from the potential side-effects of executing the system). The issue here isn't "multiple installed packages", it will more than likely be the only version of itself: note that it's a package being tested, not an *upgrade* being tested. The issues solved are: * not having to become root (solved by PEP 370 if it ever lands) * minimizing as much as possible the impact of testing the package on the system (not solved by any other solution) and to allow me to hop between sets of installed Python packages while developing on multiple Python projects. This is the ability to install multiple versions and specify different versions for different projects you're working on. No, this is the ability to not needlessly clutter site-packages, keep them clean, tight, focused; and the ability to keep a project's environment (such as its dependencies) clear and repeatable. Nowhere was it indicated that multiple versions were involved. Both of those *might* imply issues of multiple versions concurrently installed on the system, and virtualenv would incidentally solve these issues, but these issues are *not* the core of either use case. They're at best peripheral and potential. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On 8 Oct 2009, at 18:17 , Toshio Kuratomi wrote: This is not at all how I use virtualenv. For me virtualenv is a sandbox so that I don't have to become root whenever I need to install a Python package for testing purposes This is needing to install multiple versions and use the newly installed version for testing. No it's not. It's keeping the python package *being tested* out of the system's or user's site-package because it's potentially unstable or unneeded. It provides a trivial way of *removing* the package to get rid of it: delete the virtualenv. No trace anywhere that the package was ever installed, no impact on the system (apart from the potential side-effects of executing the system). The issue here isn't "multiple installed packages", it will more than likely be the only version of itself: note that it's a package being tested, not an *upgrade* being tested. The issues solved are: * not having to become root (solved by PEP 370 if it ever lands) * minimizing as much as possible the impact of testing the package on the system (not solved by any other solution) and to allow me to hop between sets of installed Python packages while developing on multiple Python projects. This is the ability to install multiple versions and specify different versions for different projects you're working on. No, this is the ability to not needlessly clutter site-packages, keep them clean, tight, focused; and the ability to keep a project's environment (such as its dependencies) clear and repeatable. Nowhere was it indicated that multiple versions were involved. Both of those *might* imply issues of multiple versions concurrently installed on the system, and virtualenv would incidentally solve these issues, but these issues are *not* the core of either use case. They're at best peripheral and potential ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 4:55 PM, M.-A. Lemburg wrote: > OTOH, the register command sends the meta-data directly to > the PyPI database, so it doesn't even need another file > for storing away the meta data. > > Unless I'm missing something important (which I probably am), > the only added value of the new format over PKG-INFO is the > micro-language. That's one. The other one is to avoid to deal with yet another configuration file alongside setup.py, and just deal with a [metadata] section in setup.cfg. And PKG-INFO stays a static, platform-specific file that exists only when the distribution is installed on the target platform. >> No, but if you ask for it I can write it. > > Please do. The format will need documentation anyway and the > motivation should be made clear as well. Ok I will Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 389: argparse - new command line parsing module
2009/10/7 Antoine Pitrou : > Python 3 complains at startup and is a bit more explicit: > > $ ./python -c '1' >&- > Fatal Python error: Py_Initialize: can't initialize sys standard streams > OSError: [Errno 9] Bad file descriptor > Abandon > > Of course, if it is stderr that you explicitly close, you lose all reporting: > > $ ./python -c '1' 2>&- > Abandon Hmm, how does Python 3's pythonw work on Windows? (I don't have a Windows installation of Python 3 to hand at the moment) Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
Benjamin Peterson python.org> writes:
> It's also really easy to just write 0{:o} like my translator does.
How does that cope when handed a negative number to format?
>>> "%#0o" % -1234
'-02322'
>>> "0{0:o}".format(-1234)
'0-2322'
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Thu, Oct 8, 2009 at 8:08 AM, Christian Tanzer wrote:
> All that talk of deprecating %-formatting makes me really nervous.
> %-formatting is pervasive in all existing Python code.
>
> Without an automatic translator that is 100% accurate, porting all
> that code to {}-formatting is not possible. Heck, it's not even
> possible to grep for all instances of %-formatting.
>
> How do you suppose that maintainers could ever do the transition from
> %- to {}-formatting manually?
This is pretty much the situation with integer division (you can only
recognize it by running the code), and yet we figured a way to change
that in 3.x. Or take classic classes vs. new-style classes. They
cannot be translated 100% automatically either. The solution is to
support the old and new style in parallel for a really long time -- we
did this with int division (read PEP 238), we did it with classes, and
we can do it again with formatting.
Unless I missed something, we're not planning to remove %-formatting
until Python 4.0 comes along, which we won't even start until a long
time after everyone has switched to some version of 3.x. So the same
approach will apply: support both forms, nudge people to start using
the new form, wait, nudge some more, etc.
So, yes, we will continue to make noise about this. And yes you should
opportunistically migrate your code to {}-formatting, like when you're
rewriting some code anyway. One of the nice things about {}-formatting
is that in most cases (things like the logging API excluded) you can
change it one format string at a time.
And no, the sky isn't falling.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Toshio Kuratomi a écrit : > On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote: >>> Tarek Ziadé a écrit : But if PEP 376 and PEP 386 support are added in Python, we're not far from being able to provide multiple version support with the help of importlib. >> Before putting much work into this: do you really think that having >> multiple versions of the same package in the same Python installation >> is a good idea ? >> > I think it is a good idea. > >> Examples: >> What if you have an application that uses two modules which each >> require two different versions of the same package ? Would that >> load the same package twice, causing globals used by these package >> to no work (e.g. isinstance(x, class_name) or global caches) ? >> > That's not how it should work. Look at other systems that allow for > installing multiple versions of a library -- for instance, loading dynamic > shared objects in C > * You can install multiple versions of a library in parallel > * The dynamic loader will pick the version of the library that is > appropriate from the list of available options (the program specifies the > SONAME it needs -- library name plus API version. The loader then > chooses the most recent revision that matches that API version.) > * When one binary needs multiple API versions of the library, the > application cannot start. > > The last point addresses your concern -- depending on multiple, incompatible > versions of a library is prohibited. The programmer of the application > needs to make the code run with a single version of the code. > >> This sounds a lot like DLL- or RPM-hell to me. >> > RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression > that it is the lack of enough version specification rather than too much but > I don't know for sure). is similar but it takes place on the end-user's > system. This should take place on the programmer's system instead. > End-users are not in a position to fix things like RPM-hell. Programmers > are. > > Example RPM-hell: > Application Foo requires libbar-1.x > Application Baz requires libbar-2.x > > The user may either have Foo or Baz installed on their system with the > appropriate libbar but not both. They depend on the packagers and > developers of Foo and Bar to do one of the following to resolve the > situation: > > * Port Foo and Baz to use the same version of libbar. > * Package libbar in such a way that libbar-1 and libbar-2 are parallel > installable on the system. Then they can install two separate packages, > libbar1-1.0 and libbar2-2.0. > > Example of similar Distutils multiple version problem: > The programmer creates an application Foo that depends on python-bar-1.x. He > has recently started work on a file that imports python-baz-1.0. python-baz > depends on python-bar-2.x. The first time he tries to run his new code, > python gives him an error message that it is impossible to satisfy the > version requirements for python-bar. Depending on how the versions are > specified, the error message could be very specific and helpful: > > Impossible version requirements: > bar Requires: python-baz>=2.0, < 3.0 > foo.py Requires: python-baz >=1.0, < 2.0 > > The programmer can then discard their new code, port foo.py to > python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to > the upstream of that module. Note two things about this scenario: > > 1) The programmer is the person who is responsible for creating the conflict > and for resolving it. They are the proper authority for making the decision > to port to python-baz-2.x or not using python-bar. The end-user who is not > responsible is not impacted by this at all. > 2) The programmer would have had to deal with this issue whether we allow > multiple versions to be installed or not. With multiple version support we > may be able to get them better error messages (depending on how the > dependency information is formatted and how completely it was specified in > the app and modules). > >> I think it's much better to keep things simple and under user >> control, e.g. by encouraging use of virtualenv-like setups >> and perhaps adding better native support for these to Python. >> >> If the user finds that there's a version conflict this can >> then be resolved during environment setup instead of hoping >> for the best and waiting for application failures at run-time. >> > For the class of user that is actually a developer, it might be somewhat > true that version conflicts should be resolved by them. But for the class > of user that is an end-user, version conflicts are a totally foreign > concept. They should be dealt with by the person who is coding the > application for them. > > Also note, the ability to have multiple versions makes things easier for > system packagers and provides an easy alternative to a virtualenv for > end-users. > > * System packagers: virtualenv does not provi
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Toshio Kuratomi a écrit : > On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote: >>> Tarek Ziadé a écrit : But if PEP 376 and PEP 386 support are added in Python, we're not far from being able to provide multiple version support with the help of importlib. >> Before putting much work into this: do you really think that having >> multiple versions of the same package in the same Python installation >> is a good idea ? >> > I think it is a good idea. > >> Examples: >> What if you have an application that uses two modules which each >> require two different versions of the same package ? Would that >> load the same package twice, causing globals used by these package >> to no work (e.g. isinstance(x, class_name) or global caches) ? >> > That's not how it should work. Look at other systems that allow for > installing multiple versions of a library -- for instance, loading dynamic > shared objects in C > * You can install multiple versions of a library in parallel > * The dynamic loader will pick the version of the library that is > appropriate from the list of available options (the program specifies the > SONAME it needs -- library name plus API version. The loader then > chooses the most recent revision that matches that API version.) > * When one binary needs multiple API versions of the library, the > application cannot start. > > The last point addresses your concern -- depending on multiple, incompatible > versions of a library is prohibited. The programmer of the application > needs to make the code run with a single version of the code. > >> This sounds a lot like DLL- or RPM-hell to me. >> > RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression > that it is the lack of enough version specification rather than too much but > I don't know for sure). is similar but it takes place on the end-user's > system. This should take place on the programmer's system instead. > End-users are not in a position to fix things like RPM-hell. Programmers > are. > > Example RPM-hell: > Application Foo requires libbar-1.x > Application Baz requires libbar-2.x > > The user may either have Foo or Baz installed on their system with the > appropriate libbar but not both. They depend on the packagers and > developers of Foo and Bar to do one of the following to resolve the > situation: > > * Port Foo and Baz to use the same version of libbar. > * Package libbar in such a way that libbar-1 and libbar-2 are parallel > installable on the system. Then they can install two separate packages, > libbar1-1.0 and libbar2-2.0. > > Example of similar Distutils multiple version problem: > The programmer creates an application Foo that depends on python-bar-1.x. He > has recently started work on a file that imports python-baz-1.0. python-baz > depends on python-bar-2.x. The first time he tries to run his new code, > python gives him an error message that it is impossible to satisfy the > version requirements for python-bar. Depending on how the versions are > specified, the error message could be very specific and helpful: > > Impossible version requirements: > bar Requires: python-baz>=2.0, < 3.0 > foo.py Requires: python-baz >=1.0, < 2.0 > > The programmer can then discard their new code, port foo.py to > python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to > the upstream of that module. Note two things about this scenario: > > 1) The programmer is the person who is responsible for creating the conflict > and for resolving it. They are the proper authority for making the decision > to port to python-baz-2.x or not using python-bar. The end-user who is not > responsible is not impacted by this at all. > 2) The programmer would have had to deal with this issue whether we allow > multiple versions to be installed or not. With multiple version support we > may be able to get them better error messages (depending on how the > dependency information is formatted and how completely it was specified in > the app and modules). > >> I think it's much better to keep things simple and under user >> control, e.g. by encouraging use of virtualenv-like setups >> and perhaps adding better native support for these to Python. >> >> If the user finds that there's a version conflict this can >> then be resolved during environment setup instead of hoping >> for the best and waiting for application failures at run-time. >> > For the class of user that is actually a developer, it might be somewhat > true that version conflicts should be resolved by them. But for the class > of user that is an end-user, version conflicts are a totally foreign > concept. They should be dealt with by the person who is coding the > application for them. > > Also note, the ability to have multiple versions makes things easier for > system packagers and provides an easy alternative to a virtualenv for > end-users. > > * System packagers: virtualenv does not provi
Re: [Python-Dev] transitioning from % to {} formatting
[Christian Tanzer]
How do you suppose that maintainers could ever do the transition from
%- to {}-formatting manually?
[Guido van Rossum]
This is pretty much the situation with integer division (you can only
recognize it by running the code),
Do you think there may be some possible parallel to the -3 option
to flag cases of %-formatting? If so, that could be helpful.
So, yes, we will continue to make noise about this. And yes you should
opportunistically migrate your code to {}-formatting, like when you're
rewriting some code anyway. One of the nice things about {}-formatting
is that in most cases (things like the logging API excluded) you can
change it one format string at a time.
I've already have some code that mixes the styles (using {} for new stuff).
Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 389: argparse - new command line parsing module
2009/10/8 Paul Moore : > 2009/10/7 Antoine Pitrou : >> Python 3 complains at startup and is a bit more explicit: >> >> $ ./python -c '1' >&- >> Fatal Python error: Py_Initialize: can't initialize sys standard streams >> OSError: [Errno 9] Bad file descriptor >> Abandon >> >> Of course, if it is stderr that you explicitly close, you lose all reporting: >> >> $ ./python -c '1' 2>&- >> Abandon > > Hmm, how does Python 3's pythonw work on Windows? (I don't have a > Windows installation of Python 3 to hand at the moment) When running pythonw, fileno(stdout) is negative, so sys.stdout is set to None, and print() silently returns. But the situation is not perfect, see http://bugs.python.org/issue6501 where Apache provides a stdout, but python cannot determine the encoding because there is no console. -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Fri, Oct 9, 2009 at 1:35 AM, Masklinn wrote: > On 8 Oct 2009, at 18:17 , Toshio Kuratomi wrote: >> >>> This is not at all how I use virtualenv. For me virtualenv is a >>> sandbox so that I don't have to become root whenever I need to install >>> a Python package for testing purposes >> >> This is needing to install multiple versions and use the newly installed >> version for testing. >> > No it's not. It's keeping the python package *being tested* out of the > system's or user's site-package because it's potentially unstable or > unneeded. It provides a trivial way of *removing* the package to get rid of > it: delete the virtualenv. No trace anywhere that the package was ever > installed, no impact on the system (apart from the potential side-effects > of executing the system). > > The issue here isn't "multiple installed packages", it will more than likely > be the only version of itself: note that it's a package being tested, not an > *upgrade* being tested. > > The issues solved are: > * not having to become root (solved by PEP 370 if it ever lands) > * minimizing as much as possible the impact of testing the package on the > system (not solved by any other solution) This is not true - stow solves the problem in a more general way (in the sense that it is not restricted to python), at least on platforms which support softlink. The only inconvenience of stow compared to virtual env is namespace packages, but that's because of a design flaw in namespace package (as implemented in setuptools, and hopefully fixed in the upcoming namespace package PEP). Virtualenv provides a possible solution to some deployment problems, and is useful in those cases, but it is too specialized to be included in python itself IMO. cheers, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Thu, Oct 8, 2009 at 10:14 AM, Raymond Hettinger wrote: > Do you think there may be some possible parallel to the -3 option > to flag cases of %-formatting? If so, that could be helpful. Absolutely. This should be simple, since there's just one or two places where to place the warning. We might also automatically turn it on when Python 2.7 is run with -3. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On 8 Oct 2009, at 19:22 , David Cournapeau wrote: This is not true - stow solves the problem in a more general way (in the sense that it is not restricted to python), at least on platforms which support softlink. I was, of course, talking about "pure" Python solutions (but I should indeed have qualified my statement thus), in the general cases there are several solutions to handle that including but not limited to stow you already mentioned, BSD jails or full-blown virtual machines. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Tarek Ziadé wrote: > On Thu, Oct 8, 2009 at 4:55 PM, M.-A. Lemburg wrote: >> OTOH, the register command sends the meta-data directly to >> the PyPI database, so it doesn't even need another file >> for storing away the meta data. >> >> Unless I'm missing something important (which I probably am), >> the only added value of the new format over PKG-INFO is the >> micro-language. > > That's one. The other one is to avoid to deal with yet another > configuration file > alongside setup.py, and just deal with a [metadata] section in setup.cfg. > > And PKG-INFO stays a static, platform-specific file that exists only when the > distribution is installed on the target platform. Hmm, PKG-INFO is generated from the parameters you pass to setup() in setup.py, so the package author will not have to deal with another config file and only has to write down the meta-data in setup.py. Being able to add some meta-data via setup.cfg would probably be an extra feature which some may find handy. However, your summary made also made it look like the meta-data would be required in setup.cfg and that the whole file would get uploaded (including all the options and settings for other distutils commands). Was that intended ? >>> No, but if you ask for it I can write it. >> >> Please do. The format will need documentation anyway and the >> motivation should be made clear as well. > > Ok I will Thanks, I'll wait for the PEP. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 08, 2009 at 06:56:19PM +0200, kiorky wrote: > > > Toshio Kuratomi a écrit : > > > > Also note, the ability to have multiple versions makes things easier for > > system packagers and provides an easy alternative to a virtualenv for > > end-users. > > > > * System packagers: virtualenv does not provide a method suitable for system > > Yes, there is no doubt virtualenv is useless for system packagers but: > > * System and applications deployment have not to be tied. > It s up to the user to install things system wide or to use locally isolation > technics. Virtualenv is one of those. > As a conclusion, there are not very much problem for system packagers as if > their users have specific needs, they will do something Outside the system. > If not, they use their global python with packages installed in that global > one. > This misses the point. If there's two pieces of software to be deployed via system packages and they use two different versions of a module, it's currently not very easy to do this. I do it with setuptools curently even with all its warts. Having a way to do this from within the stdlib that tied directly into the import mechanism would make for a much cleaner situation. In other words, the suggestion that there is no need for a method to install multiple versions of a module because virtualenv is a better solution is bogus. virtualenv is a better solution for creating isolated environments. It is not a solution for all of the cases that being able to install multiple versions of a library would be. -Toshio pgpP09FaEZeVL.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some?words?on Virtualenv, Pip)
On Thu, Oct 08, 2009 at 04:28:52PM +, Antoine Pitrou wrote: > Toshio Kuratomi gmail.com> writes: > > > > This is needing to install multiple versions and use the newly installed > > version for testing. > [...] > > What you're missing is that having separate environments has a virtue of > cleanliness, understandability and robustness that a multiple-versioned > solution > doesn't have. While the technical merits are debatable I'm sure some people > definitely prefer to manage a virtualenv-based version. > I'm not missing it. I'm only saying that the precise requirement that is being stated is not sandboxing (that was listed later). It's being able to use a newly installed library for testing. The essential element of that is being able to install a new version of the library and use that instead of the sytem installed version. sandboxing may be how someone wants to do this but it isn't essential to be able to do this. -Toshio pgp8e1xXN576p.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 08, 2009 at 05:30:00PM +0100, Michael Foord wrote: > Toshio Kuratomi wrote: >> On Thu, Oct 08, 2009 at 02:52:24PM +0200, Simon Cross wrote: >> >>> On Thu, Oct 8, 2009 at 10:31 AM, Tarek Ziadé wrote: >>> = Virtualenv and the multiple version support in Distribute = >>> ... >>> My opinion is that this tool exists only because Python doesn't support the installation of multiple versions for the same distributions. >> Let's actually look at these reasons: >> >> >>> This is not at all how I use virtualenv. For me virtualenv is a >>> sandbox so that I don't have to become root whenever I need to install >>> a Python package for testing purposes >>> >> >> This is needing to install multiple versions and use the newly installed >> version for testing. >> >> > > Not really - it is wanting to install a single version of a library that > you don't want to install into your 'main' (whether that be user or > system) Python install. It is sandboxing and orthogonal to multiple > versions. > What I'm replying to is specifically the need to: "become root whenever I need to install a Python package for testing purposes" That doesn't require sandboxing at all. Can you use sandboxing to do this? Yes. But that is separate to being able to install a non-system version of a library and be able to test it. My quoting of that phrase could have been better -- I missed the reference to sandboxing since it is in a separate clause of the sentence from what I was responding to. >>> >>> and to allow me to hop between >>> sets of installed Python packages while developing on multiple Python >>> projects. >>> >> >> This is the ability to install multiple versions and specify different >> versions for different projects you're working on. >> > > No, it is working on multiple projects that have *different* > dependencies and being able to work in an environment that *only* has > direct dependencies installed - again sandboxed from your main Python > environment. > No. Here what is written is: "allow me to hop between sets of installed Python packages while developing on multiple Python projects." There's nothing about *only* having direct dependencies installed. That's a separate requirement than what was written. > The fact that virtualenv allows you to have multiple different versions > of the same library installed in the different environments you create > is completely separate from the motivation that causes many people to > use it. > Precisely! We see 100% eye-to-eye here. My reply is just trying to say that the ideas of * testing a locally installed, conflicting version of a library * running multiple projects with different, conflicting version requirements are completely satisfiable without sandboxing. Virtualenv is a sandboxing tool. It can be used to perform these tasks. But it isn't necessary. Having sandboxing is an additional feature on top of the base requirements to perform the task. > What virtualenv *doesn't* do (I believe) is allow you to have multiple > versions of libraries installed within a single environment and switch > between them (at least it doesn't offer anything beyond what setuptools > or pip provides). > Yep. Which makes virtualenv unsuitable for certain other problem spaces where sandboxing is inappropriate. -Toshio pgpzP7TagU4bO.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging, Unicode and sockets
> I've noticed that the socket-based logging handlers - SocketHandler, > DatagramHandler and SysLogHandler - aren't Unicode-aware and can break in the > presence of Unicode messages. I can't understand what the problem with SocketHandler/DatagramHandler is. As they use pickle, they should surely be able to send records with Unicode strings in them, no? OTOH, why is SMTPHandler not in your list? > I'd like to fix this by giving these handlers an > optional (encoding=None) parameter in their __init__, and then using this to > encode on output. For syslog, I don't think that's appropriate. I presume this is meant to follow RFC 5424? If so, it SHOULD send the data in UTF-8, in which case it MUST include a BOM also. A.8 then says that if you are not certain that it is UTF-8 (which you wouldn't be if the application passes a byte string), you MAY omit the BOM. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, 08 Oct 2009 06:52:57 -0700, Michael Foord wrote: I don't recall ever having wanted / needed to install multiple versions of the same library - whilst I can appreciate that it *can* be a real issue it has never been a problem for me. Multiple versions is going to be a mess. It is a pathetic workaround for packages that do not care about backward compatibility (eg: CherryPy-2.x vs CherryPy-3.x). Drop support for multiple versions to force package authors to deal with it. I applaud the Jinja team for doing this: ARMIN: The final version of the Jinja2 Django-inspired template engine was just released. It comes with tons of improvements over the older Jinja1 engine and breaks backwards compatibility to some point over the old Jinja1 engine. It's packaged as a separate package so that you can use both right next to each other for an easier transition. http://lucumr.pocoo.org/2008/7/17/jinja2-final-aka-jinjavitus-released This is something the Python community can also learn from the Perl world. -srid ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
2009/10/8 Tarek Ziadé : >> The egg format >> >> One thing missing from your roadmap (unless I missed it) is the fate >> of the egg (zipfile) format. If it's to remain a valid option >> (bdist_egg and the like) then I'd assume that Distribute needs to be >> the place to develop it. >> One thing it will need is an equivalent to >> PEP 376 introspection and uninstall features. And *that* is where I >> believe a standard protocol belongs - developed based on the real use >> requirements for egg files, with a suitable file-based layer which can >> be added into distutils (somewhat like importlib does for PEP 302) >> once the protocol is agreed and standardised. > > Distribute is planning to continue supporting the egg format, but will > probably drop the zipped version. > This is still under discussions though so don't take this as the final word. > > In any case, distribute.resources will probably add the missing bits > on the top of a PEP 376 -compliant Distutils > to provide support for any file querying Distutils will not provide > (for installation and uninstallation) [Note: For the purposes of below I use "egg format" to only refer to zipped eggs. I see no value AT ALL in having an unzipped "egg format" which is almost the same as, but subtly different from, the distutils-supported standard filesystem format. If the distutils standard doesn't provide for all the requirements, fix that, don't add another variation!] I'm not sure I follow this. I think it's important to provide for non-filesystem based packages (that's the whole point of things like PEP 302 and importlib!). The fact that core distutils doesn't support installing, querying or removing such packages is fine. They can be supplied by 3rd party systems like Distribute. However, it is *not* fine (in my view) for each independent format to have to implement its own set of interfaces. PEP 302 is precisely the solution for this in terms of import - every importer has a standardised set of protocols to follow, and if it does so, then it will (should :-)) work seamlessly with any Python code. PEP 302 is not perfect, in part because it aims to be minimal rather than comprehensive, hence the fact that not all packages are "zip safe". But that's a failing in PEP 302 rather than in the concept. Multiple package formats should be supported in the same way - with a set of well-defined (via a PEP) interfaces that all package formats must provide, which enable core and user code to be written in a format-independent way. Once such a PEP is written, distutils' PEP 376 support should be rewritten to use those interfaces, and to implement the necessary filesystem-format support layer. (In other words, whereas I support reducing PEP 376's scope to filesystem only for now, I still strongly advocate that the longer-term solution should be based on a user extensible format-independent approach). In this context, eggs are "merely" the first (and most important) example of a format extension, and so should drive the development of a standard. To summarise: I believe that we need a statement of direction on the (zipped) egg format. I see a number of options: 1. Retain it as-is in Distribute, but deprecated and no further development. 2. Deprecate egg support in Distribute, and ultimately drop it. 3. Develop into a reference example of an extensible architecture for adding to PEP 376 support. 4. Extend PEP 376 support to eggs by means of egg-specific approaches hooking into distutils. For the record, I dislike (1), I see (2) as a lost opportunity, I see (3) as the right way to go, but appreciate that it's the most work, and I strongly oppose (4) (not least because it effectively makes a latrge part of PEP 302 and importlib wasted effort). It looks like you're proposing (2), but aren't entirely clear because you're still allowing for non-zipped "egg" formats (which I see no value in, as I noted above). I'm willing to help work on option (3). Once it's decoupled from questions around OS package managers (which are more the province of PEP 376) then I'd hope that my lack of knowledge about package managers is less of a problem. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Logging, Unicode and sockets
Martin v. Löwis v.loewis.de> writes: > I can't understand what the problem with SocketHandler/DatagramHandler > is. As they use pickle, they should surely be able to send records with > Unicode strings in them, no? Of course you are right. When I posted that it was a knee-jerk reaction to the issue that was raised for SysLogHandler configured to use UDP. I did realise a bit later that the issue didn't apply to the other two handlers but I was hoping nobody would notice ;-) > OTOH, why is SMTPHandler not in your list? I assumed smtp.sendmail() would deal with it, as it deals with the wire protocol, but perhaps I was wrong to do so. I noticed that Issue 521270 (SMTP does not handle Unicode) was closed, but I didn't look at it closely. I now see it was perhaps only a partial solution. I did a bit of searching and found this post by Marius Gedminas: http://mg.pov.lt/blog/unicode-emails-in-python.html Now if that's the right approach, shouldn't it be catered for in a more general part of the stdlib than logging - perhaps in smtplib itself? Or, seeing that Marius' post is five years old, is there a better way of doing it using the stdlib as it is now? > For syslog, I don't think that's appropriate. I presume this is meant to > follow RFC 5424? If so, it SHOULD send the data in UTF-8, in which case > it MUST include a BOM also. A.8 then says that if you are not certain > that it is UTF-8 (which you wouldn't be if the application passes a byte > string), you MAY omit the BOM. So ISTM that the right thing to do on 2.x would be: if str to be sent, send as is; if unicode to be sent, encode using utf-8 and send with a BOM. For 3.x, just encode using utf-8 and send with a BOM. Does that seem right? Thanks and regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
In article <94bdd2610910080131j323b98d9i871bce43465f2...@mail.gmail.com>, Tarek Ziadé wrote: > Here's a quick summary of the main things that are going to happen in > Distutils, and Distribute, and a few words on virtualenv and pip. > (there is much much more work going on, but I don't want to drown > people with details) Thanks for the summary! Unfortunately, as the saying goes, the devil is in the details. Here are a few comments. > = Distribute = > > I won't explain here again why we have forked, I think it's obvious to > anyone here now. I'll rather explain what > we are planning in Distribute and how it will interact with Distutils. > > Distribute has two branches: > > - 0.6.x : provides a Setuptools-0.6c9 compatible version > - 0.7.x : will provide a refactoring > > == 0.6.x == > > Not "much" is going to happen here, we want this branch to be helpful > to the community *today* by addressing the 40-or-so bugs > that were found in Setuptools and never fixed. This is eventually > happen soon because its development is > fast : there are up to 5 commiters that are working on it very often > (and the number grows weekly.) > > The biggest issue with this branch is that it is providing the same > packages and modules setuptools does, and this > requires some bootstrapping work where we make sure once Distribute is > installed, all Distribution that requires Setuptools > will continue to work. This is done by faking the metadata of > Setuptools 0.6c9. That's the only way we found to do this. > > There's one major thing though: thanks to the work of Lennart, Alex, > Martin, this branch supports Python 3, > which is great to have to speed up Py3 adoption. > > The goal of the 0.6.x is to remove as much bugs as we can, and try if > possible to remove the patches done > on Distutils. We will support 0.6.x maintenance for years and we will > promote its usage everywhere instead of > Setuptools. This is a key and, I think, difficult issue that's going to need a lot of thought and work to be successful. setuptools / easy_install filled several unmet needs of both package developers and end users and became a de-facto standard because there was essentially no competition. So, in a sense, there was never a migration to setuptools from something. But now, especially given GvR's blessing of Distribute, there needs to be a migration from setuptools to Distribute, although neither are part of the standard library. There are no references to setuptools or easy_install in any of the standard Python documentation. So how do end users know about easy_install? Their first encounter is/was probably through some recipe on a web site, most likely the installation instructions for some package they want to use. And I suspect pretty much all of those installation instructions direct the user to one of two places, either the PyPi page for setuptools or the peakcommunity wiki page: http://pypi.python.org/pypi/setuptools http://peak.telecommunity.com/DevCenter/EasyInstall Right now those are essentially the ultimate contact points for users. A general Google seach for easy_install gets 357,000 hits; one restricted to pypi.python.org gets 2,500. I don't know how many of the latter are active packages on PyPi but certainly there must be hundreds that currently use setuptools and document the use of easy_install for their users. Those packages' documentation often directs users to download the ez_install script to bootstrap setuptools; I believe there are some packages that download it automatically, if necessary. So, without some other intervention, users will continue to follow the instructions with those hundreds of packages (or on those thousands of websites) and manually or automatically try to install setuptools. Which may or may not work - as witnessed with the release of 2.6.3. Brett and others suggested including some sort of warnings in the python release documentation. Is there consensus that should be done? Starting with 2.6.4? Seems like that would be up to Barry and Guido to pronounce. Ultimately, to successfully migrate to Distribute is going to require a good understanding of the use cases for easy_install (primarily those of end users) and for the setuptools API (those of package developers). Those use cases should be documented somehow, either in the roadmap or a reference elsewhere. This is especially true if your intentions are to deprecate the easy_install command in Distribute 0.7 and if APIs are going to change for package developers. As Antoine points out, pip is not equivalent to easy_install. BTW, keep in mind that today people also use easy_install to manage the installation of packages that do not use setuptools. It will also be important to understand and address the major distribution channels for Python installations, many of which already include setuptools in their distributions. There are the multi-platform distribut
Re: [Python-Dev] Distutils and Distribute roadmap (and so me words on Virtualenv, Pip)
Ned Deily acm.org> writes: > > How to decide on and then effectively > communicate that direction is not at all trivial, I think. I think it's quite trivial actually. Since everybody agrees (except perhaps PJE) that Distribute should replace setuptools, the word will spread and trickle quite quickly in the various layers of the community. If you look at the XFree86 -> XOrg transition, I don't think there have been a lot of problems to bring people to understand the change. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
In article , Antoine Pitrou wrote: > Ned Deily acm.org> writes: > > How to decide on and then effectively > > communicate that direction is not at all trivial, I think. > > I think it's quite trivial actually. Since everybody agrees (except perhaps > PJE) > that Distribute should replace setuptools, the word will spread and trickle > quite quickly in the various layers of the community. > > If you look at the XFree86 -> XOrg transition, I don't think there have been > a > lot of problems to bring people to understand the change. I'm not sure that's a such a good example: some of the impacts of that transition are still affecting users. The unspoken assumption here is that it is a good idea to make this transition as painless as possible within some unspecified limits of practicality. Sure, at one end of the spectrum, you can just throw things out in the world and hope for the best. I think the Python community has a well-deserved reputation for doing a lot better than that. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Thu, Oct 8, 2009 at 11:28 PM, Ned Deily wrote: [..] > So, without some other intervention, users will continue to follow the > instructions with those hundreds of packages (or on those thousands of > websites) and manually or automatically try to install setuptools. I think this will be in the hands of the projects developers : if they start to use Distribute, they will start to document the way to install and use it, and the word will spread eventually. [..] > Ultimately, to successfully migrate to Distribute is going to require a > good understanding of the use cases for easy_install (primarily those of > end users) and for the setuptools API (those of package developers). > Those use cases should be documented somehow, either in the roadmap or a > reference elsewhere. This is especially true if your intentions are to > deprecate the easy_install command in Distribute 0.7 and if APIs are > going to change for package developers. As Antoine points out, pip is > not equivalent to easy_install. BTW, keep in mind that today people > also use easy_install to manage the installation of packages that do not > use setuptools. The choice to deprecate easy_install in 0.7 is done because the Pip project is not far to meet all uses cases easy_install users have, and we're betting on the fact that Pip is active and will be much more advanced that what we could do with a 'new' refactored easy_install by the time 0.7 is ready. meaning : why doing this work twice ? > Thanks again for your hard work on this, Tarek! Thanks for the advices Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
Tarek Ziadé a écrit : > The choice to deprecate easy_install in 0.7 is done because the Pip project > is not far to meet all uses cases easy_install users have, and we're betting > on the fact that Pip is active and will be much more advanced that what > we could do with a 'new' refactored easy_install by the time 0.7 is ready. But how about retro-compatibility? Like with all those buildout recipes which relies on setuptool APIs (Requirement, PackageIndex and so on), it's a risk to break same at some point. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net -- -- Cordialement, KiOrKY GPG Key FingerPrint: 0x1A1194B7681112AF signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On approximately 10/8/2009 7:24 AM, came the following characters from
the keyboard of Eric Smith:
Vinay Sajip wrote:
BTW I sent Eric a private mail re. the "0o" versus "0" issue, to see
if it was
worth raising an enhancement request on the bug tracker using "O" to
generate
compatible rendering for octals.
I didn't get your message, could you resend?.
I was thinking the same thing, but it seems like a transition step. I'd
rather not keep such backward compatibility hacks (if you will) around
for the long haul. How about a flag (maybe '*') at the start of the
format specification which says "operate in backward compatibility
mode"? We could document it as being only useful for the % to {}
translator, and promise to remove it at some point in the future. Either
actually deprecate it or just promise to deprecate it in the future.
Seems like the ability for Python {} formatting to be able to match not
only old Python % formatting output, but also output created by C's
sprintf, and other numeric formatting systems, make this particular
feature useful in more scenarios than a "backward compatibility hack".
If you want to replace a C program that produces parsed output in a
given format, and that given format includes leading-0-octal numbers,
then it would be good to have the capability in Python .format, even
though Python itself uses 0o prefix.
Similar arguments may apply anywhere else that sprintf produces
something that .format cannot currently produce.
--
Glenn -- http://nevcal.com/
===
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
On Fri, Oct 9, 2009 at 1:11 AM, kiorky wrote: > > > Tarek Ziadé a écrit : > >> The choice to deprecate easy_install in 0.7 is done because the Pip project >> is not far to meet all uses cases easy_install users have, and we're betting >> on the fact that Pip is active and will be much more advanced that what >> we could do with a 'new' refactored easy_install by the time 0.7 is ready. > > But how about retro-compatibility? > Like with all those buildout recipes which relies on setuptool APIs > (Requirement, PackageIndex and so on), it's a risk to break same at some > point. Keep in mind that Distribute 0.6.x will still be maintained and will still provide easy_install and al, and that you will be able to rely on it for your projects. Distribute 0.7.x is to be considered as a new project in some ways : all namespaces will be different and the code will not interfer with a 0.6 installation. So a buildout recipe will have the choice to use the old packages you are mentioning and at some point evolve and use the new tools 0.7 will provide. Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap
On Thu, 8 Oct 2009 21:41:21 + (UTC), Antoine Pitrou wrote: > I think it's quite trivial actually. Since everybody agrees (except perhaps > PJE) > that Distribute should replace setuptools, the word will spread and trickle > quite quickly in the various layers of the community. I don't think that it's that black and white. PJE put a lot of work (and creativity) into setuptools. There's some degree of tragedy if his project gets forked off simply because of failures in the testing and quality assurance department. Personally, I find the setuptools code somewhat hard to follow. The style goes back to early programming by 'masters' where the only person that could follow it was the master himself. Sadly, that is not what we need in open source. The newer trend is in highly readily readable and easy to understand code. Tarek is excellent at producing high quality code of the type that is required. But his programming world (on the mac) isn't full of the challenges that we poor other unfortunates face on other platforms face. Coming back to the point, setuptools is a pretty poor package to have to fork. It's very difficult to follow and the structure and implementation is cryptic. Perphaps it's a thankless job that Tarek has taken on in that regard - but he seems to be handling the distribute fork quite well. Being new to python, it's taken me a little while to figure out the process. But I've decided to write a static metadata pep which is underway now as an alternate proposal. Being primarily a hacker, writing nice design documents doesn't come easy. If there is any distutils roadmap, I'd like to see improvements that work on Windows. Yes.. python does work on windows and we can't all get our bosses to change to os-x just yet. There's many things in distutils that I could best describe as being unfinished or partially done. Let me make a few rambling direction suggestions... - command line package installation for distutils We need a simple script, not based on setuptools to allow installation, listing and deinstallation of packages. Currently *nothing* exists in distutils to do this. It's a **major** failing. - Improvements to operation on platform Windows Package/Application installation on windows sucks... For example, there's no documented way to get your programs installed to "\Program Files\myApplication" For non windows users, that's where programs are supposed to go. Any other place is "evil" - like where they are going now. If it is only a documentation issue, well can we fix that. All I'm asking for is a matching "elif sys.platform = 'win32'" for every "if sys.platform = 'darwin'" or less. It's not unreasonable... It's probably a sensible idea for Tarek to keep going with Distribute. My user experience with setuptools was that it wasted a lot of my time unnecessarily. I'm planning to put Distribute support in the Python Package Manager project (pythonpkgmgr) for it's next release. David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] eggs now mandatory for pypi?
On Thu, 08 Oct 2009 09:35:57 +0200, "M.-A. Lemburg" wrote: >> One could say that much of the setuptools cloud came about because of >> the lack of the queryable download url. Setuptools does a lot of work >> here to 'work-around' the ommission on pypi of a package download url. > > I think you two have missed my email to PJE. > > In summary: > > PyPI already does have mechanism for querying download URLs. > > http://wiki.python.org/moin/PyPiXmlRpc > > It only lacks some more meta-data that would be required to > match up the user's configuration with the one used in the > uploaded URLs. > > And it would be handy to have a mechanism to just upload the > URL data rather than the whole package. Looking at distutils' > upload command, PyPI currently appears to only support uploads > of the package file itself. > > Basically, most of the mechnisms are already there, they just need > some extra care. Maybe you're right. I'll look into it. > BTW: Who would I need to contact for the PyPI side to work out > an upload_url distutils command ? pypi is on sourceforge... ask to become a developer I guess.. David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)
I'm coming in late and breaking threading, but wanted to reply to Tarek's original email: > - easy_install is going to be deprecated ! use Pip ! Cool! I wouldn't have written pip if I didn't think it would improve substantially on easy_install. Incidentally (because I know people get really enthused about this) Carl Meyer just contributed a feature to pip to do atomic uninstallation. Someone mentioned that easy_install provided some things pip didn't; outside of multi-versioned installs (which I'm not very enthusiastic about) I'm not sure what this is? > - distribute.resources: that's the old pkg_resources, but reorganized in > clean, pep-8 modules. This package will > only contain the query APIs and will focus on being PEP 376 compatible. > We will promote its usage and see if Pip wants > to use it as a basis. And maybe PyPM once it's open source ? ( > ). > It will probably shrink a lot though, once the stdlib provides PEP 376 > support. This seems easy enough to use in pip. >- distribute.index: that's package_index and a few other things. > everything required to interact with PyPI. We will promote > its usage and see if Pip wants to use it as a basis. This is a little tricky. Primarily because there's a fair amount of logic involved in the indexing (going around to different URLs, parsing links, finding stuff). So long as there is logic, something can go wrong -- often not in the package itself, but simple user error (e.g., it doesn't look where the user thinks it should, or a link is malformed, etc). Because of this, and as a general design goal of pip, I want to show as much as I can about what it is doing and why. This is primarily tied into pip's logging system (which is oriented towards command-line output, and isn't the standard logging system). Also it tracks *why* it got to a certain links. These are the two things I can think of where the index code in pip is tied to pip, and why it would be hard to use an external system. > = Virtualenv and the multiple version support in Distribute = > > (I am not saying "We" here because this part was not discussed yet > with everyone) > > Virtualenv allows you to create an isolated environment to install some > distribution without polluting the > main site-packages, a bit like a user site-packages. > > My opinion is that this tool exists only because Python doesn't > support the installation of multiple versions for the same > distributions. > But if PEP 376 and PEP 386 support are added in Python, we're not far > from being able to provide multiple version support with > the help of importlib. Before making workingenv (virtualenv's predecessor) I actively tried to use Setuptools' multi-version support. I found it very unsuccessful. I don't think it was due to any problems with Setuptools -- maybe a slight problem was the conflict between "active" eggs and "multiversion" eggs (where active eggs would be more likely to cause conflicts, while multiversion eggs aren't available until you specifically require them). But that was just awkward, I don't think it was the real problem. The real problem is that a set of packages that makes up a working application is something separate from any library. And you can only know that an application works with an exact set of libraries. Every update can break a working application (and with fairly high probability). You can't know what updates are safe. And it's really a bit tricky to even be sure you know what libraries a package really requires -- lots of libraries might be incidentally available but no longer formally required. (Someone mentioned a coworker that only installed packages with easy_install -m, because he said it kept him honest -- only packages that are explicitly required would be available. But most people don't do this, and it really only solves the one problem of undeclared dependencies) The way both virtualenv and buildout handle this is that libraries will have a single, static version until you explicitly do something to update that version. Both are somewhat focused on a functional unit -- like one virtualenv environment for one task, or one buildout config for one application. Buildout allows for a globally shared set of versioned eggs, but that's really just a little optimization (for disk space or installation speed) -- each egg is brought in only explicitly, at build time, and not as an option during the program's runtime. This is verifiable, stable, and to varying degrees concrete (virtualenv being more concrete than buildout, which tends more towards the declarative). What virtualenv does could certainly be in the Python interpreter (and much more compact as a result, I am sure). PYTHONHOME does it to a degree, though binding a script to a environment through the interpreter listed in #! is more stable than the implicit environment of PYTHONHOME. workingenv used an environmental variable (PYTHONPATH, before PYTHONHOME existed
