On 16 Apr, 03:36 pm, p...@telecommunity.com wrote:
At 03:46 AM 4/16/2009 +0000, gl...@divmod.com wrote:
On 15 Apr, 09:11 pm, p...@telecommunity.com wrote:
Twisted has its own system for "namespace" packages, and I'm not
really sure where we fall in this discussion. I haven't been able to
follow the whole thread, but my original understanding was that the
PEP supports "defining packages", which we now seem to be calling
"base packages", just fine.
Yes, it does. The discussion since the original proposal, however, has
been dominated by MAL's counterproposal, which *requires* a defining
package.
[snip clarifications]
Does that all make sense now?
Yes. Thank you very much for the detailed explanation. It was more
than I was due :-).
MAL's proposal requires a defining package, which is counterproductive
if you have a pure package with no base, since it now requires you to
create an additional project on PyPI just to hold your defining
package.
Just as a use-case: would the Java "com.*" namespace be an example of a
"pure package with no base"? i.e. lots of projects are in it, but no
project owns it?
I'd appreciate it if the PEP could also be extended cover Twisted's
very similar mechanism for namespace packages,
"twisted.plugin.pluginPackagePaths". I know this is not quite as
widely used as setuptools' namespace package support, but its
existence belies a need for standardization.
The PEP also seems a bit vague with regard to the treatment of other
directories containing __init__.py and *.pkg files.
Do you have a clarification to suggest? My understanding (probably a
projection) is that to be a nested namespace package, you have to have
a parent namespace package.
Just to clarify things on my end: "namespace package" to *me* means
"package with modules provided from multiple distributions (the
distutils term)". The definition provided by the PEP, that a package is
spread over multiple directories on disk, seems like an implementation
detail.
Entries on __path__ slow down import, so my understanding of the
platonic ideal of a system python installation is one which has a single
directory where all packages reside, and a set of metadata off to the
side explaining which files belong to which distributions so they can be
uninstalled by a package manager.
Of course, for a development installation, easy uninstallation and quick
swapping between different versions of relevant dependencies is more
important than good import performance. So in that case, you would want
to optimize differently by having all of your distributions installed
into separate directories, with a long PYTHONPATH or lots of .pth files
to point at them.
And of course you may want a hybrid of the two.
So another clarification I'd like in the PEP is an explanation of
motivation. For example, it comes as a complete surprise to me that the
expectation of namespace packages was to provide only single-source
namespaces like zope.*, peak.*, twisted.*. As I mentioned above, I
implicitly thought this was more for com.*, twisted.plugins.*.
Right now it just says that it's a package which resides in multiple
directories, and it's not made clear why that's a desirable feature.
The concept of a "defining package" seems important to avoid
conflicts like this one:
http://twistedmatrix.com/trac/ticket/2339
[snip some stuff about plugins and package layout]
Namespaces are not plugins and vice versa. The purpose of a namespace
package is to allow projects managed by the same entity to share a
namespace (ala Java "package" names) and avoid naming conflicts with
other authors.
I think this is missing a key word: *separate* projects managed by the
same entity.
Hmm. I thought I could illustrate that the same problem actually occurs
without using a plugin system, but I can actually only come up with an
example if an application implements multi-library-version compatibility
by doing
try:
from bad_old_name import bad_old_feature as feature
except ImportError:
from good_new_name import good_new_feature as feature
rather than the other way around; and that's a terrible idea for other
reasons. Other than that, you'd have to use
pkg_resources.resource_listdir or somesuch, at which point you pretty
much are implementing a plugin system.
So I started this reply disagreeing but I think I've convinced myself
that you're right.
Precisely. Note, however, that neither is twisted.plugins a namespace
package, and it should not contain any .pkg files. I don't think it's
reasonable to abuse PEP 382 namespace packages as a plugin system. In
setuptools' case, a different mechanism is provided for locating plugin
code, and of course Twisted already has its own system for the same
thing. It would be nice to have a standardized way of locating plugins
in the stdlib, but that will need to be a different PEP.
Okay. So what I'm hearing is that Twisted should happily continue using
our own wacky __path__-calculation logic for twisted.plugins, but that
*twisted* should be a namespace package so that our separate
distributions (TwistedCore, TwistedWeb, TwistedConch, et. al.) can be
installed into separate directories.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
