On Thu, 18 Oct 2012 16:12:55 -0700 Brian Harring <ferri...@gmail.com> wrote:
> On Thu, Oct 18, 2012 at 11:54:21PM +0200, Micha?? G??rny wrote:
> > On Thu, 18 Oct 2012 02:15:43 -0700
> > Brian Harring <ferri...@gmail.com> wrote:
> >
> > > There's a trick to this; currently, those generated scripts hardcode
> > > the allowed/known python versions for that package. We obviously have
> > > to preserve that; I propose we shove it into the symlink path.
> > >
> > > Basically, we add a /usr/libexec/python directory; within it, we have
> > > a wrapper binary (explained below), and a set of symlinks pointing at
> > > the root of that directory. To cover our current python versions, the
> > > following would suffice:
> > >
> > > for x in {2.{4,5,6,7},3.{0,1,2,3,4}}-cpy 2.5-jython 2.7-pypy-1.{7,8}
> > > \2.7-pypy-1.9; do
> > > ln -s ./ /usr/libexec/python/$x
> > > done
> > >
> > > While that seems insane, there is a reason; via that, we can encode
> > > the allowed versions into the symlink. Using pkgcore's pquery for
> > > example (which should support cpy: 2.5, 2.6, 2.7, 3.1, 3.2, 3.3)
> > > instead of a wrapper script at /usr/bin/pquery, we'd have thus:
> > >
> > > targets=( 2.{5,6,7}-cpy 3.{1,2,3}-cpy )
> > > targets=$(IFS=/;echo -n "${targets[*]}")
> > > # This results in
> > > # targets=2.5-cpy/2.6-cpy/2.7-cpy/3.1-cpy/3.2-cpy/3.3-cpy
> > > ln -s "/usr/libexec/python/${targets}/wrapper" \
> > > /usr/bin/pquery
> > >
> > > /usr/libexec/python/wrapper upon invocation, takes a look at argv[0];
> > > sees how it was invoked basically. This will be the /usr/bin/whatever
> > > pathway. It reads the symlink, in the process getting the allowed
> > > versions and preferred order of the versions.
> > >
> > > Few notes; vast majority of filesystems will store the symlink target
> > > into the core inode if at all possible- in doing so, this avoids
> > > wasting an inode and is only limited by the length of the target.
> > > That length is capped by PATH_MAX- which can range from 256 to 4k (or
> > > higher).
> > >
> > > For the pquery example above, that comes out to ~73 bytes for the
> > > symlink pathway; well under PATH_MAX.
> > >
> > > For the scenarios where PATH_MAX caps the symlink pathway, or for
> > > whatever reason we don't want to use that trick, a tree of files
> > > contained within /usr/libexec/python/ holding the allowed versions for
> > > the matching pathway would suffice.
> >
> > While I agree that it's a clever trick, I doubt it's worth the effort.
> > Did you got any numbers proving it being superior over, say, trying to
> > exec() scripts like I do in python-exec?
> >
> > While I can imagine, that in an worst case that bunch of exec()s is
> > going to be definitely slower than storing the list anyway, I doubt
> > such a bad case is often.
>
> The difference in performance there is going to be negligable; I'm not
> particularly concerned about that, and it shouldn't be a debate point
> between my notion and yours (it's only a debate point when one is
> talking about a c binary vs a python script).
>
> One thing you're ignoring here is that the route(s) I mentioned all
> allow for control/specifying what the order of preference is for
> lookup/fallback (each link/shebang can encode that order how ever the
> hell it wants).
Where's the use-case for that? Do we really prefer having the order of
preference hard-wired when installing a particular package instead of
having a control over it in the eclass/a single package?
> > Considering that the most common Python version used now is Python 2,
> > how often doesn't the script support that Python version? That's a very
> > rare case, and often just executing "foo-${EPYTHON}" works. In your
> > case, that common case involves readlink() + parsing + exec().
> >
> > Even in case of Python 3 being selected, I doubt the overhead
> > of multiple exec()s on the scripts not supporting Python 3 is really
> > relevant. Please measure it if you believe so.
> >
> > To be honest, I don't see any real advantage in this solution. It is
> > complex; understanding it requires explanation or some thinking.
> > The code will be fragile, and I'm not even sure if I'm not missing
> > something important here.
>
> Not particularly sure how you claim this is fragile, but whatever,
> your view.
The fragile part is using a symlink to store *data*.
> > > Either proposal here would be far faster than what we've got now; also
> > > will use less space (ancillary benefit).
> > >
> > > One subtle aspect here is that if we did this, it makes it possible to
> > > avoid losing the invocation information- currently if you did
> > > `/usr/bin/python3.2 $(which sphinx-build) blah`, because of how things
> > > are implemented now (specifically the two layers of wrappers)- you'll
> > > get python2.7 running that sphinx-build invocation.
> > >
> > > This is wrong (it's directly discarding what the invocation
> > > requested), although you're only going to see it for scripts that
> > > do python introspection.
> > >
> > > Via doing the restructuring I'm mentioning above, that issue can be
> > > fixed, while making things faster/saner.
> >
> > I don't see how this is relevant to the wrapper. As Mike pointed out,
> > python3.2 is the actual Python executable, and the wrapper you're
> > suggesting is a C executable -- something that does not really work
> > together like that.
> >
> > So please elaborate on how you are actually going to solve this. Hope
> > it's not through patching Python...
>
> Same thing, read my responses to floppym; not going to reiterate those
> points over here.
>
> I will note your python-exec bit still doesn't fly for python3.2
> sphinx-build- which frankly, is the wrong direction to go in my views.
Yes, indeed it is a problem. I will be happy to fix it when a good
solution comes out.
--
Best regards,
Michał Górny
signature.asc
Description: PGP signature
