On Fri, 2012-11-30 at 13:22 -0500, Barry Warsaw wrote:
> FWIW, let's reference the Python documentation for --prefix:
>
> http://docs.python.org/2.7/install/index.html#alternate-installation-unix-the-prefix-scheme
>
> On Nov 30, 2012, at 04:37 PM, Ian Campbell wrote:
>
> >What this bug is about is what happens when I build a 3rd party module
> >for use with the system Python using --prefix=/usr.
> >
> >In this case the modules are installed
> >to /usr/lib/python2.7/site-packages/ even though I built against the
> >system Python.
> >
> >This means that the system Python doesn't see the module which I built
> >for it.
>
> I can interpret the referenced documentation in two ways.
>
> On first reading of Python's prefix scheme, it seems that using --prefix=/usr
> only should indeed install into site-packages.
This document doesn't consider the possibility of the dist-/site-
package split at all (that being a Debian specific thing) so I don't
think it is surprising that it doesn't say anything about how this
should work.
> But the documentation also says this:
>
> Incidentally, the real reason the prefix scheme is important is simply
> that a standard Unix installation uses the prefix scheme, but with
> --prefix and --exec-prefix supplied by Python itself as sys.prefix and
> sys.exec_prefix. Thus, you might think you’ll never use the prefix scheme,
> but every time you run python setup.py install without any other options,
> you’re using it.
>
> Given that the system Python's sys.prefix and sys.exec_prefix are both /usr,
> then you could reasonably expect these two commands to be the same:
>
> $ python setup.py install
> $ python setup.py install --prefix=/usr
>
> and of course, they are not. It's not just {dist,site}-packages that are
> different but that the former uses /usr/local by default.
It could be considered reasonable to expect that they only differ in the
prefix and not other path elements, but I think that is by the by.
I think what is more pertinent here is the expectation that they should
both install into some directory which is in the default sys.path for
the /usr/bin/python binary. This is what this Debian bug report is
ultimately about.
> >> [2] Which I personally think is a bit suspect since IMHO only Debian
> >> packages should be used with the system Python
> >
> >I wholeheartedly disagree with this. It is entirely reasonable for users
> >to want to use a simple 3rd party module with the system Python without
> >having to rebuild the whole Python system (including all the modules
> >they use which *are* in Debian) from source.
>
> They don't have to rebuild the whole Python system of course. You can use
> `virtualenv --system-site-packages` and then just build the special stuff into
> the virtualenv. Everything else, you get from the system.
This seems like an awfully big hammer for this particular nut. Also the
availability of virtualenv doesn't change the fact that "setup.py
--prefix=/usr" is doing something inconsistent and nonsensical.
> The reason I don't favor installing 3rd party packages into /usr outside of
> the packaging system is that it becomes much more difficult to manage your
> Python system when you do this. What if you find that the newer Xen (or one
> of its automatically installed dependencies) breaks your system? How do you
> uninstall things and get back to a clean state?
There is nothing particular to Python in this argument, it could just as
easily apply to anything from C libraries upwards. The answer is that
administrators who install stuff from source understand the risks and
accept the consequences.
As George said earlier it's fine for the software to do something
sensible by default but when the admin asks for something specific then
they should have the power to shoot themselves in the foot.
> Sadly distutils doesn't really provide package management support so without
> the Debian packaging system, you can't really be certain you've cleaned things
> up properly.
This is true of the majority of build systems IME.
> In a very real sense then, if you do this, you're own your own, and you
> probably have more to worry about than adding --install-layout=deb or removing
> --prefix and letting things get installed into
> /usr/local/lib/pythonX.Y/dist-packages. As you note, the latter would still
> be importable by your system Python, but is definitely much easier to clean
> up after.
>
> >> - and why not use virtualenv or a
> >> from-source build of Python?
> >
> >It is unreasonable to expect users to have to build Python from source
> >just to do a build of a newer version of Xen (or any other python
> >module) than is packaged in Debian.
>
> As I've mentioned, there are other workaround that I would *personally*
> recommend (e.g. virtualenv), but one thing is also not clear to me. From your
> original bug report:
>
> Removing the --prefix=$(PREFIX) results in installation into
> /usr/local/lib/python2.7/dist-packages/ which is in the sys.path (so
> correct in that sense) but doing this in the Xen tree removes the
> option for people to install Xen to an alternate prefix (e.g. I think
> NetBSD uses --prefix=/usr/pkg or some such).
>
> Why couldn't you change the Xen build system to look something like this:
>
> $(PYTHON) setup.py install $(PREFIX)
>
> and leave $PREFIX empty by default? Then you'd get acceptable behavior on
> Debian, and leave open the possibility for NetBSD users to provide
> `PREFIX=--prefix=/usr/pkg`.
PREFIX is a raw path in the Xen build system, it is used all over and I
don't think we can realistically change it. I think the normal
expectation which users have in general is that PREFIX is a simple path
not something more complex so I don't think we would want to change it
anyhow.
As a user I wouldn't expect to have to set both PREFIX=/usr and
PYTHON_PREFIX=<somethingelse>.
> Also, why doesn't NetBSD do the right thing with no --prefix?
I'm not really a NetBSD person and I expect this is mostly a red-herring
WRT the actual issue on Debian here. But anyway...
My reference to (not) specifying PREFIX here was about user giving
PREFIX=/usr/pkg to the Xen build system as a whole, not to the Python
bits in particular (which are just one part of Xen). I worded it quite
badly though, sorry. If a user specifies PREFIX=/something to Xen then I
think they could reasonably expect that all of Xen, including the python
modules, will end up under /something, which is why we turn this into
--prefix=$(PREFIX) when we call setup.py.
Anyway, NetBSD doesn't have the dist-packages patch and if they are
having issues with how things work WRT Python in Xen then they aren't
reporting them to the Xen lists.
Ian.
--
Ian Campbell
Current Noise: Annihilator - The Trend
There are no manifestos like cannon and musketry.
-- The Duke of Wellington
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
