Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Toshio Kuratomi]

On Wed, Dec 05, 2012 at 02:46:11AM -0500, Donald Stufft wrote:
> On Wednesday, December 5, 2012 at 2:13 AM, PJ Eby wrote:
> 
> On Mon, Dec 3, 2012 at 2:43 PM, Daniel Holth  wrote:
> 
> How to use Obsoletes:
> 
> The author of B decides A is obsolete.
> 
> A releases an empty version of itself that Requires: B
> 
> B Obsoletes: A
> 
> The package manager says "These packages are obsolete: A". Would you
> like to
> remove them?
> 
> User says "OK".
> 
> 
> Um, no. Even if the the author of A and B are the same person, you
> can't remove A if there are other things on the user's system using
> it. The above scenario does not work *at all*, ever, except in the
> case where B is simply an updated version of A (i.e. identical API) --
> in which case, why bother? To change the project name? (Then it
> should be "Formerly-named" or something like that, not "Obsoletes".)
> 
> You can automatically uninstall A from B in an automatic dependency
> management system.  I *think* RPM does this, at the very least

This is correct.

> I believe it refuses to install B if A is already there (and the reverse
> as well).*

I'd have to test this but I believe you are correct about the first.  Not
sure about the reverse.

> There's nothing preventing an installer from, during it's attempt to
> install B, see it Obsoletes A, looking at what depends on A and
> warning the user what is going to happen and prompt it.
> 
In rpm-land, if something depended on A and nothing besides the actual
A package provided A, rpm will refuse to install B.  But rpm is meant to be
used unattended so different package managers could certainly choose to
prompt.  For package renames, package B would have both an Obsoletes: A <=
$OLD_VERSION and a Provides: A = NEW_VERSION

-Toshio


pgptWKkjHQPiB.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] slightly misleading Popen.poll() docs

[Chris Withers]

Hi All,

Would anyone object to me making a change to the docs for 2.6, 2.7 and 
3.x to clarify the following:


http://docs.python.org/3/library/subprocess.html#subprocess.Popen.poll

A couple of my colleagues have ended up writing code like this:

proc = Popen(['some', 'thing'])
code = proc.poll()
if code:
raise Exception('An error happened: %s' % code)

...on the back of the fact that if your process terminates *really* 
quickly, *and* the docs say that the returncode is set by poll() (*sigh*).


I'd like to change the docs for poll() to say:

"""
Check if child process has terminated.
If it has, the returncode attribute will be set and that value will be 
returned.
If it has not, None will be returned and the returncode attribute will 
remain None.

"""

Any objections?

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] slightly misleading Popen.poll() docs

[Antoine Pitrou]

Le Wed, 05 Dec 2012 16:08:46 +,
Chris Withers  a écrit :
> Hi All,
> 
> Would anyone object to me making a change to the docs for 2.6, 2.7
> and 3.x to clarify the following:
> 
> http://docs.python.org/3/library/subprocess.html#subprocess.Popen.poll

The doc looks clear to me. poll() returns the returncode attribute which
is described thusly:

"A None value indicates that the process hasn’t terminated yet."

Therefore, I don't understand the confusion.  poll() is explicitly
non-blocking, and it is silly to expect it to return a process return
code when the process hasn't returned yet (!).  The correct answer is
to use the wait() method (or communicate()), which is described two
lines below poll().

May I suggest your colleagues didn't read the doc at all?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] slightly misleading Popen.poll() docs

[Chris Withers]

On 05/12/2012 16:34, Antoine Pitrou wrote:

http://docs.python.org/3/library/subprocess.html#subprocess.Popen.poll


The doc looks clear to me. poll() returns the returncode attribute which
is described thusly:

"A None value indicates that the process hasn’t terminated yet."

Therefore, I don't understand the confusion.


Because lazy/busy people don't bother reading the links underneath docs...


poll() is explicitly
non-blocking, and it is silly to expect it to return a process return
code when the process hasn't returned yet (!).  The correct answer is
to use the wait() method (or communicate()), which is described two
lines below poll().


I agree, however, I also:
- don't see any harm in the change I propose
- do see a slight improvement for the comprehending impaired ;-)


May I suggest your colleagues didn't read the doc at all?


One of them quoted the docs at me at proof that his code must be correct ;-)

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] slightly misleading Popen.poll() docs

[Steven D'Aprano]

On 06/12/12 03:08, Chris Withers wrote:


I'd like to change the docs for poll() to say:

"""
Check if child process has terminated.
If it has, the returncode attribute will be set and that value will be returned.
If it has not, None will be returned and the returncode attribute will remain 
None.
"""

Any objections?


Possibly because it is 4am here, I had to read this three times to understand 
it.
How is this instead?

"""
Check if child process has terminated. Returns None while the child is still 
running,
any non-None value means that the child has terminated. In either case, the 
return
value is also available from the instance's returncode attribute.
"""



--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] slightly misleading Popen.poll() docs

[Antoine Pitrou]

On Thu, 06 Dec 2012 04:15:08 +1100
Steven D'Aprano  wrote:
> On 06/12/12 03:08, Chris Withers wrote:
> 
> > I'd like to change the docs for poll() to say:
> >
> > """
> > Check if child process has terminated.
> > If it has, the returncode attribute will be set and that value will be 
> > returned.
> > If it has not, None will be returned and the returncode attribute will 
> > remain None.
> > """
> >
> > Any objections?
> 
> Possibly because it is 4am here, I had to read this three times to understand 
> it.
> How is this instead?
> 
> """
> Check if child process has terminated. Returns None while the child is still 
> running,
> any non-None value means that the child has terminated. In either case, the 
> return
> value is also available from the instance's returncode attribute.
> """

I like this wording.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[PJ Eby]

On Wed, Dec 5, 2012 at 2:46 AM, Donald Stufft  wrote:
> There's nothing preventing an installer from, during it's attempt to
> install B, see it Obsoletes A, looking at what depends on A and
> warning the user what is going to happen and prompt it.

Unless the user wrote those things that depend on A, they aren't going
to be in a position to do anything about it.  (Contrast with a distro,
where dependencies are indirect - the other package will depend on an
abstraction provided by both A and B, rather than directly depending
on A *or* B.)

(Also note that all the user knows at this point is that the author of
B *claims* to obsolete A, not that the authority managing the
repository as a whole has decreed B to obsolete A.)


> You can automatically uninstall A from B in an automatic dependency
management system

My point is that this can only work if the "obsoleting" is effectively
just a rename, in which case the field should be "renames", or better
still, "renamed-to" on the originating package.

As I've mentioned repeatedly, Obsoleted-By handles more use cases than
Obsoletes, and has at least one practical automated use case
(notifying a developer that their project is depending on something
that's obsolete).

Also, the example given as a use case in the PEP (Gorgon to Torqued)
is not just wrong, it's *actively misleading*.  Gorgon and Torqued are
transparent renames of Medusa and Twisted, which do not share a common
API and thus cannot be used as the subject of any automated processing
(in the case of Obsoletes) without doing some kind of PyPI metadata
search for every package installed every time a package is installed.


> I think Obsoletes as is an alright bit of information.

1. It cannot be used to prevent the installation of an obsolete
package without a PyPI metadata search, since you must examine every
*other* package on PyPI to find out whether some package obsoletes the
one you're trying to install.

2. Unlike RPM, where metadata is provided by a trusted third party,
Obsoletes can be specified by any random forker (no pun intended),
which makes this information a mere advertisement... and an
advertisement to the wrong audience at that, because they must have
*already* found B in order to discover that it replaces A!

3. Nobody has yet supplied a use case where Obsoletes would not be
strictly improved upon by Obsoleted-By.  (Note that "the author of
package X no longer maintains it" does not equal "package Y is
entitled to name itself the successor and enforce this upon all users"
-- this can work in RPM only because it is a third party Z who
declares Y the successor to X, and there is no such party Z in the
Python world.)


> I don't see this in this thread, could you link it again?

http://mail.python.org/pipermail/catalog-sig/2010-October/003368.html
http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html

These posts also address why a "Conflicts" field is *also* unlikely to
be particularly useful in practice, in part for reasons that relate to
differences between RPM-land and Python-land.  (For example, RPMs can
conflict over things besides files, due to runtime and configuration
issues that are out-of-scope for a Python installer tool.)

While it's certainly desirable to not invent wheels, it's important to
understand that the Python community does not work the same way as a
Linux distribution.  We are not a single organization shipping a
fully-functional and configured machine, we are hundreds of individual
authors shipping our own stuff.  Conflict resolution and package
replacement (and even deciding what it is that things "provide" or
"require") are primarily *human* processes, not technical ones.
Relationship and support "contracts", IOW, rather than software
contracts.

That's why, in the distro world, a package manager can use simple
fields to carry out the will of the human organization that made those
support and compatibility decisions.  For Python, the situation is a
bit more complicated, which is why clear thinking is needed.  Simply
copying fields blindly from other packaging systems just isn't going
to cut it.

Now, if the will of the community is to turn PyPI into a distro-style
repository, that's fine... but even if you completely ignore the human
issues, there are still technical ones.  Generally, distro-style
repositories work by downloading the full metadata set (or at least an
index) to a user's machine.  And that's the sort of architecture you'd
need in order for these type of fields to be technically feasible
(e.g., doing an index search for Obsoletes), without grinding the PyPI
servers into dust.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Daniel Holth]

On Wed, Dec 5, 2012 at 4:10 PM, PJ Eby  wrote:

> On Wed, Dec 5, 2012 at 2:46 AM, Donald Stufft 
> wrote:
> > There's nothing preventing an installer from, during it's attempt to
> > install B, see it Obsoletes A, looking at what depends on A and
> > warning the user what is going to happen and prompt it.
>
> Unless the user wrote those things that depend on A, they aren't going
> to be in a position to do anything about it.  (Contrast with a distro,
> where dependencies are indirect - the other package will depend on an
> abstraction provided by both A and B, rather than directly depending
> on A *or* B.)
>
> (Also note that all the user knows at this point is that the author of
> B *claims* to obsolete A, not that the authority managing the
> repository as a whole has decreed B to obsolete A.)
>
>
> > You can automatically uninstall A from B in an automatic dependency
> management system
>
> My point is that this can only work if the "obsoleting" is effectively
> just a rename, in which case the field should be "renames", or better
> still, "renamed-to" on the originating package.
>
> As I've mentioned repeatedly, Obsoleted-By handles more use cases than
> Obsoletes, and has at least one practical automated use case
> (notifying a developer that their project is depending on something
> that's obsolete).
>
> Also, the example given as a use case in the PEP (Gorgon to Torqued)
> is not just wrong, it's *actively misleading*.  Gorgon and Torqued are
> transparent renames of Medusa and Twisted, which do not share a common
> API and thus cannot be used as the subject of any automated processing
> (in the case of Obsoletes) without doing some kind of PyPI metadata
> search for every package installed every time a package is installed.
>
>
> > I think Obsoletes as is an alright bit of information.
>
> 1. It cannot be used to prevent the installation of an obsolete
> package without a PyPI metadata search, since you must examine every
> *other* package on PyPI to find out whether some package obsoletes the
> one you're trying to install.
>
> 2. Unlike RPM, where metadata is provided by a trusted third party,
> Obsoletes can be specified by any random forker (no pun intended),
> which makes this information a mere advertisement... and an
> advertisement to the wrong audience at that, because they must have
> *already* found B in order to discover that it replaces A!
>
> 3. Nobody has yet supplied a use case where Obsoletes would not be
> strictly improved upon by Obsoleted-By.  (Note that "the author of
> package X no longer maintains it" does not equal "package Y is
> entitled to name itself the successor and enforce this upon all users"
> -- this can work in RPM only because it is a third party Z who
> declares Y the successor to X, and there is no such party Z in the
> Python world.)
>
>
> > I don't see this in this thread, could you link it again?
>
> http://mail.python.org/pipermail/catalog-sig/2010-October/003368.html
> http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html
>
> These posts also address why a "Conflicts" field is *also* unlikely to
> be particularly useful in practice, in part for reasons that relate to
> differences between RPM-land and Python-land.  (For example, RPMs can
> conflict over things besides files, due to runtime and configuration
> issues that are out-of-scope for a Python installer tool.)
>
> While it's certainly desirable to not invent wheels, it's important to
>

My desire is to invent the useful "wheel" binary package format in a
reasonable and limited amount of time by making changes to Metadata 1.2 and
implementing the new metadata format and wheel in distribute and pip. Help
me out by allowing useless but un-changed fields to remain in this version
of the PEP. I am done with the PEP and submit that it is not worse than its
predecessor.

I can participate in a discussion about any of the following:
Summary of Differences From PEP 345

   - Metadata-Version is now 1.3.
   - Values are now expected to be UTF-8.
   - A payload (containing the description) may appear after the headers.
   - Added extra to environment markers.
   - Most fields are now optional.
   - Changed fields:
  - Description
  - Project-URL
  - Requires-Dist
   - Added fields:
  - Extension
  - Provides-Extra
  - Setup-Requires-Dist
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Donald Stufft]

On Wednesday, December 5, 2012 at 4:10 PM, PJ Eby wrote:
> My point is that this can only work if the "obsoleting" is effectively
> just a rename, in which case the field should be "renames", or better
> still, "renamed-to" on the originating package.

Arguing over Obsoletes vs Renames is a massive bikeshedding argument. 
> As I've mentioned repeatedly, Obsoleted-By handles more use cases than
> Obsoletes, and has at least one practical automated use case
> (notifying a developer that their project is depending on something
> that's obsolete).
> 
> Also, the example given as a use case in the PEP (Gorgon to Torqued)
> is not just wrong, it's *actively misleading*. Gorgon and Torqued are
> transparent renames of Medusa and Twisted, which do not share a common
> API and thus cannot be used as the subject of any automated processing
> (in the case of Obsoletes) without doing some kind of PyPI metadata
> search for every package installed every time a package is installed.
> 
> 

So it's a bad example. Hardly an argument against it.
> 1. It cannot be used to prevent the installation of an obsolete
> package without a PyPI metadata search, since you must examine every
> *other* package on PyPI to find out whether some package obsoletes the
> one you're trying to install.
> 
> 

Will require support from PyPI but this ultimately isn't a big deal. 
> 
> 2. Unlike RPM, where metadata is provided by a trusted third party,
> Obsoletes can be specified by any random forker (no pun intended),
> which makes this information a mere advertisement... and an
> advertisement to the wrong audience at that, because they must have
> *already* found B in order to discover that it replaces A!
> 
> 

If you're installing B you've prescribed trust to that author. If you don't
trust the author then why are you installing (and then executing) code
they wrote. 
> 
> 3. Nobody has yet supplied a use case where Obsoletes would not be
> strictly improved upon by Obsoleted-By. (Note that "the author of
> package X no longer maintains it" does not equal "package Y is
> entitled to name itself the successor and enforce this upon all users"
> -- this can work in RPM only because it is a third party Z who
> declares Y the successor to X, and there is no such party Z in the
> Python world.)
> 
> 

Very convenient to declare that one of the major use cases for
Obsoletes over Obsoleted-By is not valid because of your own
personal opinions. Like I said above, if you're installing a package
that someone has uploaded you've implicitly granted them trust. There
is far worse things that a bad Python citizen can do during, and after
and install that what is allowed by Obsoletes.
> 
> 
> > I don't see this in this thread, could you link it again?
> 
> http://mail.python.org/pipermail/catalog-sig/2010-October/003368.html
> http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html
> 
> These posts also address why a "Conflicts" field is *also* unlikely to
> be particularly useful in practice, in part for reasons that relate to
> differences between RPM-land and Python-land. (For example, RPMs can
> conflict over things besides files, due to runtime and configuration
> issues that are out-of-scope for a Python installer tool.)
> 
> 

I don't think Conflicts is something that every single package is going
to require. As you said the tools themselves are going to handle the
obvious cases for the bulk of situations. Unless you think there are
no cases where two packages can conflict in more than what files
are going to be installed then there are cases where it would be helpful
and merely having the ability to use it when it is the best tool for the job
isn't going to cause any great issue. 
> 
> While it's certainly desirable to not invent wheels, it's important to
> understand that the Python community does not work the same way as a
> Linux distribution. We are not a single organization shipping a
> fully-functional and configured machine, we are hundreds of individual
> authors shipping our own stuff. Conflict resolution and package
> replacement (and even deciding what it is that things "provide" or
> "require") are primarily *human* processes, not technical ones.
> Relationship and support "contracts", IOW, rather than software
> contracts.
> 
> 

End systems often times do not have a singular organization controlling
every package in their system. The best example is Ubuntu and their PPA's. 
> 
> Now, if the will of the community is to turn PyPI into a distro-style
> repository, that's fine... but even if you completely ignore the human
> issues, there are still technical ones. Generally, distro-style
> repositories work by downloading the full metadata set (or at least an
> index) to a user's machine. And that's the sort of architecture you'd
> need in order for these type of fields to be technically feasible
> (e.g., doing an index search for Obsoletes), without grinding the PyPI
> servers into dust.

This is insane. A fairly simple dat

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Barry Warsaw]

On Dec 05, 2012, at 04:10 PM, PJ Eby wrote:

>While it's certainly desirable to not invent wheels, it's important to
>understand that the Python community does not work the same way as a
>Linux distribution.  We are not a single organization shipping a
>fully-functional and configured machine, we are hundreds of individual
>authors shipping our own stuff.  Conflict resolution and package
>replacement (and even deciding what it is that things "provide" or
>"require") are primarily *human* processes, not technical ones.
>Relationship and support "contracts", IOW, rather than software
>contracts.
>
>That's why, in the distro world, a package manager can use simple
>fields to carry out the will of the human organization that made those
>support and compatibility decisions.  For Python, the situation is a
>bit more complicated, which is why clear thinking is needed.  Simply
>copying fields blindly from other packaging systems just isn't going
>to cut it.

+1

>Now, if the will of the community is to turn PyPI into a distro-style
>repository, that's fine...

Please no!  -1 :)

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Barry Warsaw]

On Dec 05, 2012, at 06:07 PM, Donald Stufft wrote:

>If you're installing B you've prescribed trust to that author. If you don't
>trust the author then why are you installing (and then executing) code
>they wrote. 

What you installed Z, but B got installed because it was a dependency three
levels down?

>Very convenient to declare that one of the major use cases for
>Obsoletes over Obsoleted-By is not valid because of your own
>personal opinions. Like I said above, if you're installing a package
>that someone has uploaded you've implicitly granted them trust. There
>is far worse things that a bad Python citizen can do during, and after
>and install that what is allowed by Obsoletes.

Well, basically never installing anything from PyPI except into a virtualenv
is probably a good recommendation (maybe even now).

>End systems often times do not have a singular organization controlling
>every package in their system. The best example is Ubuntu and their PPA's. 

Well, PPAs are awesome, but have known and well-publicized trust issues.  I
wouldn't enable a PPA into my running system without really knowing who the
owner is and why I'm using their PPA.  Or doing a lot of testing in a chroot
first, and probably pinning the package set to just the one(s) from the PPA I
care about.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Donald Stufft]

On Wednesday, December 5, 2012 at 6:18 PM, Barry Warsaw wrote:
> On Dec 05, 2012, at 06:07 PM, Donald Stufft wrote:
> 
> > If you're installing B you've prescribed trust to that author. If you don't
> > trust the author then why are you installing (and then executing) code
> > they wrote. 
> > 
> 
> 
> What you installed Z, but B got installed because it was a dependency three
> levels down?
> 
> 

Sure, you granted trust to Z, Z granted trust to Y, and Y granted trust to B. 
Like
in SSL certificates there was a chain of trust. If you don't trust Z then don't 
install
their package. 
> 
> > Very convenient to declare that one of the major use cases for
> > Obsoletes over Obsoleted-By is not valid because of your own
> > personal opinions. Like I said above, if you're installing a package
> > that someone has uploaded you've implicitly granted them trust. There
> > is far worse things that a bad Python citizen can do during, and after
> > and install that what is allowed by Obsoletes.
> > 
> 
> 
> Well, basically never installing anything from PyPI except into a virtualenv
> is probably a good recommendation (maybe even now).
> 
> 

A virtualenv only protects you from well behaved packages. There is no way
to prevent a package author from doing very nasty things to you if they wish.
Providing more power in the metadata doesn't make this situation better or
worse, it just makes more standard paths in the cases where you do need
to do it.
> 
> > End systems often times do not have a singular organization controlling
> > every package in their system. The best example is Ubuntu and their PPA's. 
> > 
> 
> 
> Well, PPAs are awesome, but have known and well-publicized trust issues. I
> wouldn't enable a PPA into my running system without really knowing who the
> owner is and why I'm using their PPA. Or doing a lot of testing in a chroot
> first, and probably pinning the package set to just the one(s) from the PPA I
> care about.
> 
> 

Basically the same thing can be said about packages on PyPI. All the same
trust issues exist there. Simply installing a Python package is already granting
far more trust than Obsoletes requires since installing a package is executed
someone else's python code on your system. Even if you remove setup.py you're
still going to be executing their code on your system. If you do not trust the
author of the packages you are installing, you do not install their packages.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[PJ Eby]

On Wed, Dec 5, 2012 at 5:30 PM, Daniel Holth  wrote:
> My desire is to invent the useful "wheel" binary package format in a
> reasonable and limited amount of time by making changes to Metadata 1.2 and
> implementing the new metadata format and wheel in distribute and pip. Help
> me out by allowing useless but un-changed fields to remain in this version
> of the PEP. I am done with the PEP and submit that it is not worse than its
> predecessor.

You could just mark those fields as deprecated and that they should
not be used to delete packages or block packages from installation.

Justification: nobody has managed to make them work in an automated
tool yet, and their use in same is controversial, so they are
downgraded to human-informational only.

Please, let's not have yet *another* metadata spec that advertises
these attractive nuisance[1] fields.  I do not want us to be having
this same conversation AGAIN the next time any metadata changes are
being considered.  We've already had it too many times already.  PEPs
are supposed to summarize these discussions for that very reason.

---
[1] For non-native speakers, an attractive nuisance is a dangerous
thing that entices unsuspecting persons to play with it;
http://en.wikipedia.org/wiki/Attractive_nuisance_doctrine has more
details.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[PJ Eby]

On Wed, Dec 5, 2012 at 6:07 PM, Donald Stufft  wrote:
> Arguing over Obsoletes vs Renames is a massive bikeshedding argument.

And is entirely beside the point.  The substantive question is whether
it's Obsoletes or Obsoleted-By - i.e., which side is it declared on.

> So it's a bad example. Hardly an argument against it.

Nobody has actually proposed a better one, outside of package renaming
-- and that example featured an author who could just as easily have
used an obsoleted-by field.


> Will require support from PyPI but this ultimately isn't a big deal.

...and every PyPI clone.  And of course the performance issues.


> If you're installing B you've prescribed trust to that author. If you don't
> trust the author then why are you installing (and then executing) code
> they wrote.

Trusting their code is one thing; trusting whether they understood a
PEP (and its interactions with various installation tools) well enough
to not accidentally delete *somebody else's code* out of my system is
another thing altogether.

OTOH, trusting an author to tell me (in an automated fashion), "hey,
you should switch to this other thing as soon as you can" is a FAR
smaller amount of required trust.

Arguing that because I have to trust one thing, means I must trust
another, is a "Fallacy of Gray" argument.


> Very convenient to declare that one of the major use cases for
> Obsoletes over Obsoleted-By is not valid because of your own
> personal opinions.

I didn't say it was invalid, I said:

"""Note that "the author of package X no longer maintains it" does not
equal "package Y is entitled to name itself the successor and enforce
this upon all users"""

These things are not equal.  AFAIK, well-managed Linux distros do not
allow random forkers to declare themselves the official successor to a
defunct package, so any analogy between this use case in the Python
world and the distro world is strained at *best*.


> Unless you think there are
> no cases where two packages can conflict in more than what files
> are going to be installed

The rationale for that is laid out in the posts I linked.

> then there are cases where it would be helpful

Please, present a *real-life instance* where it would have been helpful to you.

> and merely having the ability to use it when it is the best tool for the job
> isn't going to cause any great issue.

One of the posts I linked presents an instance where it would have
actually *harmed* things to specify it, and it's quite easy to see how
the same problem would arise if used for non-file-related conflicts...

And the problem present is *directly* tied to the lack of a
third-party Z who decides whether X and Y, as configured for release Q
of distro P, "conflict".

This is not a problem that is solvable even in *principle* for an
automated tool in the absence of party Z, which means that any such
field's actual function is limited to a heads-up to a human user.


> This is insane. A fairly simple database query is going to "grind the PyPI
> servers into dust"?  You're going to need to back up this FUD or please
> refrain from spouting it.

I take it you're not familiar with PyPI's history of performance and
scaling problems over the last several years, then.  The statically
cached "/simple" index was developed precisely to stop *today's* class
of installation tools from killing the servers...  and then mirroring
PyPI was still required to scale.  Any proposal that calls for
encouraging tools to query a metadata field *every time* a package is
installed (or even just downloaded) almost certainly needs to be
vetted with the PyPI admin team.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Stephen J. Turnbull]

I understand the PEP author's frustration with continued discussion,
but I think this subthread on Obsoletes vs. Obsoleted-By is not mere
bikeshedding on names.  It matters *which package* presents the
information.


Donald Stufft writes:
 > On Wednesday, December 5, 2012 at 6:18 PM, Barry Warsaw wrote:
 > > On Dec 05, 2012, at 06:07 PM, Donald Stufft wrote:
 > > 
 > > > If you're installing B you've prescribed trust to that
 > > > author. If you don't trust the author then why are you
 > > > installing (and then executing) code they wrote.

The author may be a genius when it comes to writing code, and an idiot
when it comes to distributing it.  Distribution is much harder than it
looks, as you know.  Trusting the author's *content* and trusting the
author's *metadata* are not equivalent!

As far as I can see, the semantics of putting "Obsoletes: A" into B
without changing A are the same as the semantics of putting "Provides:
A" into B (without changing A).[1]  Only if A includes "Obsoleted-By: B"
can a user be confident that B is a true successor to A.

Furthermore, as has been pointed out, the presence of "Obsoleted-By"
in A has the huge advantage of informing users and developers of
dependent packages alike that A is obsolete when they try to update A.
If A is not changed, then an attempted update will tell them exactly
that, and they may never find out about B.  But if A is modified in
this trivial way, the package system can automatically inform them.
This is also trivial, requiring no database queries.

"Simple is better than complex."




Footnotes: 
[1]  A trustworthy author of B wouldn't use "Provides" unless he
thought B was indeed a drop-in, and presumbly superior, replacement
for A.  And that's all that "Obsoletes" can tell you!


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[MRAB]

On 2012-12-06 02:12, Stephen J. Turnbull wrote:

I understand the PEP author's frustration with continued discussion,
but I think this subthread on Obsoletes vs. Obsoleted-By is not mere
bikeshedding on names.  It matters *which package* presents the
information.


Donald Stufft writes:
  > On Wednesday, December 5, 2012 at 6:18 PM, Barry Warsaw wrote:
  > > On Dec 05, 2012, at 06:07 PM, Donald Stufft wrote:
  > >
  > > > If you're installing B you've prescribed trust to that
  > > > author. If you don't trust the author then why are you
  > > > installing (and then executing) code they wrote.

The author may be a genius when it comes to writing code, and an idiot
when it comes to distributing it.  Distribution is much harder than it
looks, as you know.  Trusting the author's *content* and trusting the
author's *metadata* are not equivalent!

As far as I can see, the semantics of putting "Obsoletes: A" into B
without changing A are the same as the semantics of putting "Provides:
A" into B (without changing A).[1]  Only if A includes "Obsoleted-By: B"
can a user be confident that B is a true successor to A.

Furthermore, as has been pointed out, the presence of "Obsoleted-By"
in A has the huge advantage of informing users and developers of
dependent packages alike that A is obsolete when they try to update A.
If A is not changed, then an attempted update will tell them exactly
that, and they may never find out about B.  But if A is modified in
this trivial way, the package system can automatically inform them.
This is also trivial, requiring no database queries.

"Simple is better than complex."


That makes sense.

In summary, someone using B won't care that it has replaced A, but
someone using A needs to be told that it has been replaced by B.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Daniel Holth]

Makes sense. How about calling it Replacement. 0 or 1?

Replacement (optional)
::

Indicates that this project is no longer being developed.  The named
project provides a drop-in replacement.

A version declaration may be supplied and must follow the rules described
in `Version Specifiers`_.

The most common use of this field will be in case a project name changes.

Examples::

Name: BadName
Replacement: AcceptableName

Replacement: AcceptableName (>=4.0.0)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] slightly misleading Popen.poll() docs

[Nick Coghlan]

On Thu, Dec 6, 2012 at 4:55 AM, Antoine Pitrou  wrote:

> On Thu, 06 Dec 2012 04:15:08 +1100
> Steven D'Aprano  wrote:
> > Possibly because it is 4am here, I had to read this three times to
> understand it.
> > How is this instead?
> >
> > """
> > Check if child process has terminated. Returns None while the child is
> still running,
> > any non-None value means that the child has terminated. In either case,
> the return
> > value is also available from the instance's returncode attribute.
> > """
>
> I like this wording.
>

Steven's proposed wording sounds good to me, too.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Nick Coghlan]

On Thu, Dec 6, 2012 at 8:30 AM, Daniel Holth  wrote:

> My desire is to invent the useful "wheel" binary package format in a
> reasonable and limited amount of time by making changes to Metadata 1.2 and
> implementing the new metadata format and wheel in distribute and pip. Help
> me out by allowing useless but un-changed fields to remain in this version
> of the PEP. I am done with the PEP and submit that it is not worse than its
> predecessor.
>

Agreed. PJE's arguments sound reasonable (especially since Obsoletes
doesn't get used much in RPM-land either - Provides & Conflicts are both
far more common), but they're orthogonal to the current aims of the
metadata 1.3 update. If another author wanted to create a subsequent 1.4
update that was focused on replacing Obsoletes with Obsoleted-By, that
would be fine (alternatively, a patch to the current PEP draft may be
acceptable, but accepting such a change would be up to Daniel as the PEP
author).


>
> I can participate in a discussion about any of the following:
> Summary of Differences From PEP 345
>
>- Metadata-Version is now 1.3.
>- Values are now expected to be UTF-8.
>- A payload (containing the description) may appear after the headers.
>- Added extra to environment markers.
>- Most fields are now optional.
>- Changed fields:
>   - Description
>   - Project-URL
>   - Requires-Dist
>- Added fields:
>   - Extension
>   - Provides-Extra
>   - Setup-Requires-Dist
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
>


-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Terry Reedy]

On 12/5/2012 10:12 PM, Daniel Holth wrote:

Makes sense. How about calling it Replacement. 0 or 1?

Replacement (optional)
::

Indicates that this project is no longer being developed.  The named
project provides a drop-in replacement.

A version declaration may be supplied and must follow the rules described
in `Version Specifiers`_.

The most common use of this field will be in case a project name changes.

Examples::

 Name: BadName
 Replacement: AcceptableName

 Replacement: AcceptableName (>=4.0.0)


I like it. 'Replacement' is broader in meaning, more neutral, and less 
awkward than 'Obsoleted-by'. And I agree that A users have much more 
need to know about B the vice-versa. It is much the same situation with 
Py 2 and Py 3 (although the latter is *not* a drop-in replacement).


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Nick Coghlan]

On Thu, Dec 6, 2012 at 1:12 PM, Daniel Holth  wrote:

> Makes sense. How about calling it Replacement. 0 or 1?
>

Hah, you'd think I'd have learned by now to finish reading a thread before
replying. It will be nice to get this addressed along with the other
changes :)

(FWIW, Conflicts and Obsoletes are messy in RPM as well, and especially
troublesome as soon as you start enabling multiple upstream repos from
different providers. The metadata problem is handled by prebuilding indices
when the repo changes, but that's still more work for the server, and more
work for clients)


> Replacement (optional)
> ::
>


I like verb forms like Obsoleted-By or Replaced-By, as the noun form is
ambiguous about the direction of the change. Since the field being replaced
is Obsoletes, Obsoleted-By makes sense.


>
> Indicates that this project is no longer being developed.  The named
> project provides a drop-in replacement.
>

Typically, the new version *won't* be a drop-in replacement (e.g. you'll
likely at least have to import from a different top level package).
Instead, the field would more often be used as an explicit indicator that
the project is no longer receiving updates, as the *development team* has
moved on, so users may want to consider either migrating, taking over
development (if the former developers are amenable) or forking.

If the replacing project *is* a drop-in replacement for the old project,
then it should also advertise a Provides-Dist for the original project.
Automated tools can then easily detect the two cases:

A Obsoleted-By-Dist B and B Provides-Dist A = A is defunct, and B should be
a drop-in replacement for A
A Obsoleted-By-Dist B (without a Provides-Dist on B) = A is defunct, B is a
replacement for A, but some porting will be needed

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Nick Coghlan]

On Thu, Dec 6, 2012 at 2:54 PM, Nick Coghlan  wrote:

> On Thu, Dec 6, 2012 at 1:12 PM, Daniel Holth  wrote:
>
>> Makes sense. How about calling it Replacement. 0 or 1?
>>
>
> Hah, you'd think I'd have learned by now to finish reading a thread before
> replying. It will be nice to get this addressed along with the other
> changes :)
>
> (FWIW, Conflicts and Obsoletes are messy in RPM as well, and especially
> troublesome as soon as you start enabling multiple upstream repos from
> different providers. The metadata problem is handled by prebuilding indices
> when the repo changes, but that's still more work for the server, and more
> work for clients)
>
>
>> Replacement (optional)
>> ::
>>
>
>
> I like verb forms like Obsoleted-By or Replaced-By, as the noun form is
> ambiguous about the direction of the change. Since the field being replaced
> is Obsoletes, Obsoleted-By makes sense.
>

Although Replaced-By would be fine as well - it's certainly much easier to
say than the mouthful that is Obsoleted-By.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]

[Toshio Kuratomi]

On Wed, Dec 05, 2012 at 07:34:41PM -0500, PJ Eby wrote:
> On Wed, Dec 5, 2012 at 6:07 PM, Donald Stufft  wrote:
> 
> Nobody has actually proposed a better one, outside of package renaming
> -- and that example featured an author who could just as easily have
> used an obsoleted-by field.
> 
How about pexpect and pextpect-u as a better example?

> 
> > Very convenient to declare that one of the major use cases for
> > Obsoletes over Obsoleted-By is not valid because of your own
> > personal opinions.
> 
> I didn't say it was invalid, I said:
> 
> """Note that "the author of package X no longer maintains it" does not
> equal "package Y is entitled to name itself the successor and enforce
> this upon all users"""
> 
> These things are not equal.  AFAIK, well-managed Linux distros do not
> allow random forkers to declare themselves the official successor to a
> defunct package, so any analogy between this use case in the Python
> world and the distro world is strained at *best*.
> 
Note that although well-managed Linux distros attempt to control random
forking internally, the distro package managers don't prevent people from
installing from third parties.  So Ubuntu PPAs, upstreams that provide their
own rpms/debs, and major third party repos (for instance, rpmfusion as
an add-on repo to Fedora) all have and sometimes (mis)use the ability to
Obsolete packages in the base repository.

So Donald isn't stretching the relationship quite as far as you make it out.
The ecosystem of packages for a distro carries uncontrolled packages just as
much as pypi.

> 
> > and merely having the ability to use it when it is the best tool for the job
> > isn't going to cause any great issue.
> 
> One of the posts I linked presents an instance where it would have
> actually *harmed* things to specify it, and it's quite easy to see how
> the same problem would arise if used for non-file-related conflicts...
> 
> And the problem present is *directly* tied to the lack of a
> third-party Z who decides whether X and Y, as configured for release Q
> of distro P, "conflict".
> 
> This is not a problem that is solvable even in *principle* for an
> automated tool in the absence of party Z, which means that any such
> field's actual function is limited to a heads-up to a human user.
> 
And the same for Provides. (ie: latest foo is 0.6c; bar Provides: foo-0.6d.
an automated tool that finds both foo and bar in its dep tree can choose to
install bar and not foo.)

The ability for this class of fields to cause harm is not, to me,
a compelling argument not to include them.  It could be an argument to
explicitly tell implementers of install tools that they all have caveats
when used with pypi and similar unpoliced community package repositories.
The install tools can then choose how they wish to deal with those caveats.
Some example strategies: choose to prompt the user as to which to install,
choose to always treat the fields as human-informational only, mark some
repositories as being trusted to contain packages where these fields are
active and other repositories where the fields are ignored.

-Toshio



pgpIdSsnfDzFy.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com