Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
On Wed, Dec 23, 2009 at 3:31 AM, Tres Seaver wrote: [..] >> >> The deprecation of the existing Requires/Provides/Obsoletes fields >> should be more prominent - tucked away below the examples, I missed >> these notices on the first read through (I only noticed that they >> actually had been formally deprecated when I got to the summary of >> differences at the end). I suggest placing the deprecation notice >> immediately after the relevant field headers. > > Good point. I thought I had done so in the initial editing pass. I've done it yesterday. > >> There also needs to be an explanation in the PEP as to whether or not it >> is legal to use both Requires and Requires-Dist (etc) in the same >> PKG-INFO file. (i.e. what is the use case for allowing the old fields to >> be used in a metadata v1.2 PKG-INFO file? Should PEP 345 aware packaging >> tools just ignore the old fields, while v1.1 tools ignore the new ones? >> Or should new tools attempt to handle both?) > > No tools that I know of currently use 'Requires' / 'Provides' / > 'Obsoletes' at all: their contents have never been informative enough > to allow for useful automation. For completeness sake, we can document > that tools should ignore any 'Requires', 'Provides', or 'Obsoletes' > fields when any of the '-Dist' versions are present. Although some (a small number) distributions use these, so one way to handle it is to do a little bit like what was done when 1.1 came out: if a "1.2" field is found and no "1.1" field is found: metadata 1.2 is used if a "1.1" field is found and no "1.2" field is found: metadata 1.1 is used + a warning is displayed if a "1.1" field is found and a "1.2" field is found: a warning is displayed and 1.2 is used, 1.1 fields are ignored if no 1.1 field or 1.2 fields are found: metadata 1.2 is used >> The various lines about there being no standards or canonical >> definitions for particular fields also seem to run counter to the spirit >> of the detailed guidelines in the description of each field (which imply >> that some standards have already been adopted by convention). Perhaps >> these comments could be softened to say that although the metadata >> specification formally allows arbitrary strings in these fields, the >> descriptions are recommended guidelines for creating field entries that >> automated tools will handle correctly? > > That language is left over from PEP 314, which introduced those > "advisory" fields. The expectation of PEP 345 is that developers who > want their packages to be easily consumable by automated tools will > avoid the deprecated fields and use the more usefully-specifiied new ones. > Notice that I now provide in 2.7/3.2 a way to read *and* write PKG-INFO from an API: http://docs.python.org/dev/distutils/examples.html#reading-the-metadata meaning that whatever fields a developer use, this API will let the installers and other automated tool gets the metadata. It might be a good thing to inform about that API in the PEP I guess, >> Finally, as a general formatting request - some blank space between the >> end of the previous example and the header for the next field >> description would make the field descriptions much easier to read. > > Hmm, I thought we were following stock ReST formats: perhaps the CSS > should be adjusted to give a larger leading space to headings? I've changed the layout yesterday, so its easier to read. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
On Wed, Dec 23, 2009 at 11:18 AM, Tarek Ziadé wrote: [..] > > if a "1.2" field is found and no "1.1" field is found: > metadata 1.2 is used > if a "1.1" field is found and no "1.2" field is found: > metadata 1.1 is used + a warning is displayed > if a "1.1" field is found and a "1.2" field is found: > a warning is displayed and 1.2 is used, 1.1 fields are ignored > if no 1.1 field or 1.2 fields are found: > metadata 1.2 is used That would be internal to Distutils btw. tools should consume PKG-INFO using the DistributionMetadata reader, looking for the various attributes. "requires", etc. would be kept, most of the time set to "None", then removed at the next Python version. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
> The deprecation of the existing Requires/Provides/Obsoletes fields > should be more prominent - tucked away below the examples, I missed > these notices on the first read through (I only noticed that they > actually had been formally deprecated when I got to the summary of > differences at the end). I suggest placing the deprecation notice > immediately after the relevant field headers. I would go beyond deprecation, and propose to remove them altogether. In the metadata format, there is no point in deprecating something. If you have a valid use for these fields, use version 1.1. > There also needs to be an explanation in the PEP as to whether or not it > is legal to use both Requires and Requires-Dist (etc) in the same > PKG-INFO file. (i.e. what is the use case for allowing the old fields to > be used in a metadata v1.2 PKG-INFO file? Should PEP 345 aware packaging > tools just ignore the old fields, while v1.1 tools ignore the new ones? > Or should new tools attempt to handle both?) I think that depends much on the tool. Distutils will just write it into the metadata. PyPI will just store it in the database (and create new columns for the new fields), and render them to the user. The only issue would be with tools that actually do resolve dependencies. I believe that, for the old fields, no such tools actually exist. I may be wrong, in which case authors of such tools should speak up when the PEP proposes removal (or propose re-introduction in v1.3 of the format). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] First draft of "sysconfig"
On Mon, Dec 14, 2009 at 5:58 PM, Tarek Ziadé wrote: > and for Linux and al, I am not sure but maybe a prefix for > Jython/etc.. could be used > for all paths. > > ~/.locale/lib/python/2.6/site-packages/... > ~/.locale/jython/lib/python/2.6/site-packages/... > > (I didn't digg on how Jython organizes things yet, any hint would be > appreciated) Jython does not yet support user site-packages, but I think the above looks like a fine structure for us when we get to implementing it. -Frank ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] First draft of "sysconfig"
On Dec 23, 2009, at 10:00 AM, Frank Wierzbicki wrote: > On Mon, Dec 14, 2009 at 5:58 PM, Tarek Ziadé wrote: >> and for Linux and al, I am not sure but maybe a prefix for >> Jython/etc.. could be used >> for all paths. >> >> ~/.locale/lib/python/2.6/site-packages/... >> ~/.locale/jython/lib/python/2.6/site-packages/... >> >> (I didn't digg on how Jython organizes things yet, any hint would be >> appreciated) > Jython does not yet support user site-packages, but I think the above > looks like a fine structure for us when we get to implementing it. Two minor points: 1. It's "~/.local", not "~/.locale" ;-) 2. I think it would be a better idea to do "~/.local/lib/jython/2.6/site-packages". The idea with ~/.local is that it's a mirror of the directory structure convention in /, /usr/, /opt/ or /usr/local/. In other words it's got "bin", "lib", "share", "etc", etc.. ~/.local/jython/lib suggests that the parallel scripts directory would be ~/.local/jython/bin, which means I need 2 entries on my $PATH instead of one, which means yet more setup for people who use both Python and Jython per-user installation. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
2009/12/23 "Martin v. Löwis" : >> The deprecation of the existing Requires/Provides/Obsoletes fields >> should be more prominent - tucked away below the examples, I missed >> these notices on the first read through (I only noticed that they >> actually had been formally deprecated when I got to the summary of >> differences at the end). I suggest placing the deprecation notice >> immediately after the relevant field headers. > > I would go beyond deprecation, and propose to remove them altogether. > > In the metadata format, there is no point in deprecating something. > If you have a valid use for these fields, use version 1.1. I think we want something stronger than that since they were not really used by the community and removed and replaced by something better. Using them should raise a warning so developers abandon them, so it would be "don't use 1.1 anymore" PEP 314 (PEP 345 predecessor) was implemented in a very particular manner: If you do use options like 'requires' in Distutils' setup.py, your package ends up with a PKG-INFO Metadata v1.1. Otherwise it stays v1.0. In other words, v1.0 was not really superseded by v1.1. The latter was just an alternative version and the current Distutils can produce both depending on the options you use in your setup.py. Once 1.2 is defined, we could deprecate the options that are corresponding to 1.1 at Distutils' level and make Distutils only produces metadata v1.2 PKG-INFO files (even if the options used by the developer could be written in a valid 1.0 format). This will be clearer I think. So, I am +1 for removing deprecated fields from PEP 345 and work on the deprecation issues at the code level in Distutils. Meaning Distutils' DistributionMetadata class will provide a way to read 1.0/1.1/1.2 but will only write 1.2 starting at 2.7/3.2, If a deprecated option is used, Distutils will produce a 1.1 with a warning. If a deprecated option is used together with a new option, like Require and Require-Dist, it will raise an error. Regards, Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
> I think we want something stronger than that since they were not really used > by > the community and removed and replaced by something better. Using them > should raise a warning so developers abandon them, so it would be > "don't use 1.1 anymore" I think you are mixing the distutils implementation (and parameters to the setup function) with PEP 345. This PEP *only* describes a data format. As such, it is not capable of raising exceptions. Whether you deprecate parameters to the setup function is an entirely different issue. Alternatives are removing them entirely, ignoring them, deprecating them, and conditionally accepting them (on the condition that they don't get mixed with parameters that are meant for 1.2 of the metadata). > PEP 314 (PEP 345 predecessor) was implemented in a very particular manner: PEP 314 was implemented multiple times - not only in distutils, but also in PyPI (for example). > If you do use options like 'requires' in Distutils' setup.py, your > package ends up with a PKG-INFO Metadata v1.1. Otherwise it stays > v1.0. Please do keep distutils out of PEP 345. The remaining occurrences (such as what the "interpret_marker" function does) should be removed. > In other words, v1.0 was not really superseded by v1.1. The latter was > just an alternative version and the current Distutils can produce > both depending on the options you use in your setup.py. That is all well, and should probably be extended into the implementation of PEP 345 in distutils. However, what specifically distutils does is really not the subject here. > Once 1.2 is defined, we could deprecate the options that are > corresponding to 1.1 at Distutils' level and make Distutils only > produces metadata v1.2 PKG-INFO files (even if the options used by the > developer could be written in a valid 1.0 format). > > This will be clearer I think. It would be also incompatible with existing consumers that expect a Python package to have an earlier version of the metadata. Dropping 1.0 may be fine though - but again, this is out of scope here. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
2009/12/23 "Martin v. Löwis" : >> I think we want something stronger than that since they were not really used >> by >> the community and removed and replaced by something better. Using them >> should raise a warning so developers abandon them, so it would be >> "don't use 1.1 anymore" > > I think you are mixing the distutils implementation (and parameters > to the setup function) with PEP 345. This PEP *only* describes a > data format. As such, it is not capable of raising exceptions. I am just describing the needs and the end user PoV with the reference implementation that happens to be used by *all* tools out there. So that will happen in the code of course, but we need the PEP to state clearly wether metadata 1.0 and 1.1 should be dropped by implementations or not. >> PEP 314 (PEP 345 predecessor) was implemented in a very particular manner: > > PEP 314 was implemented multiple times - not only in distutils, but also > in PyPI (for example). PyPI doesn't produce PKG-INFO files AFAIK, it just consumes them, no ? I am referring to the implementation in Distutils that produces 1.0 *or* 1.1 PKG-INFO files. > >> If you do use options like 'requires' in Distutils' setup.py, your >> package ends up with a PKG-INFO Metadata v1.1. Otherwise it stays >> v1.0. > > Please do keep distutils out of PEP 345. The remaining occurrences > (such as what the "interpret_marker" function does) should be removed. That's the reference implementation of a PEP 345 reader/writer that happens to be in the stdlin. For what reason should we remove it from the PEP ? [..] > >> Once 1.2 is defined, we could deprecate the options that are >> corresponding to 1.1 at Distutils' level and make Distutils only >> produces metadata v1.2 PKG-INFO files (even if the options used by the >> developer could be written in a valid 1.0 format). >> >> This will be clearer I think. > > It would be also incompatible with existing consumers that expect > a Python package to have an earlier version of the metadata. > Dropping 1.0 may be fine though - but again, this is out of scope > here. I don't understand why you are saying this is out of scope. Shouldn't we state clearly in the PEP that 1.0 and 1.1 should not be used in the future ? Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
> So that will happen in the code of course, but we need the PEP to state > clearly > wether metadata 1.0 and 1.1 should be dropped by implementations or not. Ok. We should recommend that implementations support these versions indefinitely. I see no point in dropping them. But then, this is really up to the implementations. >>> PEP 314 (PEP 345 predecessor) was implemented in a very particular manner: >> PEP 314 was implemented multiple times - not only in distutils, but also >> in PyPI (for example). > > PyPI doesn't produce PKG-INFO files AFAIK, it just consumes them, no ? Correct - but that's also an implementation of the PEP. > I am referring to the implementation in Distutils that produces 1.0 > *or* 1.1 PKG-INFO files. But it works both ways. Applications that consume then need to decide what versions they want to consume. >> Please do keep distutils out of PEP 345. The remaining occurrences >> (such as what the "interpret_marker" function does) should be removed. > > That's the reference implementation of a PEP 345 reader/writer that > happens to be in the stdlin. For what reason should we remove it from > the PEP ? Because there shouldn't be a reference implementation. If we have both a spec and an a reference implementation, then we need to define what happens in case they conflict. If the reference implementation is right, implementers of the PEP would *also* need to study the reference implementation to find out what conforming behaviour is. This is bad; the PEP should be the only definition of the metadata format. >>> This will be clearer I think. >> It would be also incompatible with existing consumers that expect >> a Python package to have an earlier version of the metadata. >> Dropping 1.0 may be fine though - but again, this is out of scope >> here. > > I don't understand why you are saying this is out of scope. Shouldn't we > state clearly in the PEP that 1.0 and 1.1 should not be used in the future ? In terms of conformance, what would that mean? If I implement 1.0 (in addition to also implementing 1.2), would I then be non-conforming (because the PEP says I should not support 1.0)? For PyPI, that would be fairly bad, as it will need to support earlier versions for many years to come (at a minimum, 10 years). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
Martin, As an application developer, I really stand with Tarek here. On Wed, 23 Dec 2009 20:07:30 +0100, Tarek Ziadé wrote: > 2009/12/23 "Martin v. Löwis" : >>> I think we want something stronger than that since they were not really >>> used by >>> the community and removed and replaced by something better. Using them >>> should raise a warning so developers abandon them, so it would be >>> "don't use 1.1 anymore" Yes. But that is a software warning message to be implemented within the installation software. The important thing is what is in the metadata. > I am just describing the needs and the end user PoV with the reference > implementation that happens to be used by *all* tools out there. That's good. That's what we need right now. > So that will happen in the code of course, but we need the PEP to state > clearly > wether metadata 1.0 and 1.1 should be dropped by implementations or not. +1 >> It would be also incompatible with existing consumers that expect >> a Python package to have an earlier version of the metadata. >> Dropping 1.0 may be fine though - but again, this is out of scope >> here. That's a software implementation issue. Not a metadata issue. > I don't understand why you are saying this is out of scope. Shouldn't we > state clearly in the PEP that 1.0 and 1.1 should not be used in the future I agree with you Tarek. Whilst we can suggest that the implementation be done in certain ways. The PEP in it's current form seems good enough an I am hoping it will go through soon. Three more years of deliberation and niggling on this PEP will have more of an adverse affect than a positive effect. It was started in 2005 and that seems like a long time to hold things up. There's always time for a 1.3 version in two years time if there is an unstoppable problem here - but I can't see any. As an application developer, I have to side with Tarek. Lets get this finalised. Thank you David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
David Lyon writes: > > I am just describing the needs and the end user PoV with the reference > > implementation that happens to be used by *all* tools out there. > > That's good. That's what we need right now. Martin's point is that the PEP process doesn't *have* "reference" implementations. It has *sample* implementations. It may be useful to refer to a sample implementation as an example, but the PEP must *define* the behaviors without such references, and compare the behavior of the sample to the defined behavior. And references to internal details like function names is inappropriate in most cases; I assume that is Martin's point here. > > So that will happen in the code of course, but we need the PEP to > > state clearly wether metadata 1.0 and 1.1 should be dropped by > > implementations or not. > > +1 You can state what you want. What implementations do is another matter. An implementation with lots of state like PyPI is not likely to change quickly. As a matter of user relations (including but not limited to developers like you), Python doesn't want to deprecate practices that are expensive to change too soon. (That's not my opinion about what is appropriate, that is my assessment of the historical policy of Python, which I don't think will change.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposing PEP 345 : Metadata for Python Software Packages 1.2
On Thu, 24 Dec 2009 10:31:09 +0900, "Stephen J. Turnbull" wrote: > Martin's point is that the PEP process doesn't *have* "reference" > implementations. It has *sample* implementations. It may be useful > to refer to a sample implementation as an example.. Fair enough. But otoh, asking for sample implementations on this type of project can skew the PEP towards a particular implementation or product. A definition for metadata should be something quite abstract and self contained. So imo I am happier where it is currently. > What implementations do is another matter. Right. We all agree on that. > An implementation with lots of state like PyPI is not likely > to change quickly. As a matter of user relations (including but not > limited to developers like you), Python doesn't want to deprecate > practices that are expensive to change too soon. (That's not my > opinion about what is appropriate, that is my assessment of the > historical policy of Python, which I don't think will change.) Well I would need more convincing that it is better to do one PEP every 4-5 years as a user relations exercise than one PEP every year or two years. Whilst I agree that the core language is really great and the rate of progress can happily slow. It would be nice to see the rate of progress of other areas, such as the metadata side, increase a little. That wouldn't break policy David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
