Re: [Python-Dev] [Distutils] request for comments - standardization of python's purelib and platlib

[Matthias Klose]

On 14.08.2009 10:02, Tarek Ziadé wrote:

On Thu, Aug 13, 2009 at 9:22 PM, Brett Cannon  wrote:



On Thu, Aug 13, 2009 at 11:23, Jan Matejek  wrote:


Hello,

I'm cross-posting this to distributi...@freedesktop and python-dev,
because the topic is relevant to both groups and should be solved in
cooperation.

The issue:

In Python's default configuration (on linux), both purelib (location for
pure python modules) and platlib (location for platform-dependent binary
extensions) point to $prefix/lib/pythonX.Y/site-packages.
That is no good for two main reasons.

One, python depends on the "lib" directory. (from distro's point of
view, prefix is /usr, so let's talk /usr/lib) Due to this, it's
impossible to install python under /usr/lib64 without heavy patching.
Repeated attempts to bring python developers to acknowledge and rectify
the situation have all failed (common argument here is "that would mean
redesign of distutils and huge parts of whatnot").


This is now Tarek's call, so this may or may not have changed in terms of
what the (now) distutils maintainer thinks.



I don't recall those repeated attempts , but I've been around for less
than two years.

You are very welcome to come in the Distutils-SIG ML to discuss these matters.
I'm moving the discussion there.

Among the proposals you have detailed, the sharedir way seems like the
most simple/interesting
one (depending on you answer to Brett's question )


The approach of splitting the installation into two different locations seems to 
be wrong, it changes the semantics for imports of python packages which are not 
installed in the same location. Simplest counter example is the use of relative 
imports, which will fail if the imported module/extension is not found in the 
same paths.


Other languages like Perl or Java don't have relative imports, or they map all 
components on the "path" into one logical path so you don't have this kind of 
problem.


I don't see an explict statement that code really has to live inside /usr/share, 
and even generated .py files differ depending on the architecture you build for 
(e.g. sip, qt bindings).


  Matthias
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Two laments about CPython's AST Nodes

[Frank Wierzbicki]

On Wed, Aug 19, 2009 at 5:00 PM, "Martin v. Löwis" wrote:
>> Now on to the complaints: Though I recently added support for this in
>> Jython, I don't like that nodes can be defined without required
>> attributes, for example:
>>
>> node = ast.Assign()
>
> I think we disagree in two points in our evaluation of this behavior:
>
> a) ast.Assign is *not* a node of the CPython AST. The CPython AST is
>   a set of C structures in Include/Python-ast.h. ast.Assign is merely
>   a mirror structure of that.
Ah -- that is different from Jython's current design (The nodes that
are constructed by ast.Assign() in Jython actually are the exact nodes
that are used in real parsing)

> b) it is, IMO, not reasonable to require users who create AST trees
>   out of nothing to have correct trees at all times. I.e. it must be
>   possible to represent incorrect trees.
That does seem reasonable.  Luckily it was easy to implement for me :)

> c) the AST is *not* part of the Python language or library. It may
>   change at any time without notice, and Jython is not required to
>   duplicate its behavior exactly.
Sure, I'm really just talking about ast.py (though for Jython ATM they
are the same thing).  So that I understand better: when this call is
made in Python:

x = compile("def foo():pass", "foo", "exec", _ast.PyCF_ONLY_AST)

Does x contain real AST nodes or does it contain mirror structures
(feel free to just tell me: don't be lazy, go read the code). If it
contains real nodes, this is where I have some implementation trouble.
 If a tree of real nodes is then manipulated so that you end up with a
mix, I don't want to walk the entire thing over again to find the
mirror objects (that might be incomplete) and replace them with real
nodes.  If this creates a tree of mirror nodes, then I may want to
consider doing the same thing on the Jython side (it makes sense, now
that I understand CPython better I realize that the cost I am
incurring is probably due to having the real and mirror AST as the
same beast).

> [so that's three items - as there should be in any good list of
> two items :-]
:)

> What precisely is it that makes this difficult to implement. If you
> would follow CPython's implementation strategy (i.e. generate glue
> code out of ASDL), I feel that it should be straight-forward to provide
> exactly the same behavior in Jython.
I do use the ASDL to generate this stuff, but again, the real and the
mirror nodes are not separated ATM, and that is what makes it
difficult.

>> BTW -- I *am* volunteering to attempt to implement these things in
>> CPython if there is support :)
>
> I'm not sure I can support such a change. Giving the child nodes at
> creation time, optionally, would be fine with me. Requiring the
> tree to conform to the grammar at all times is unreasonable, IMO
> (you can't simultaneously create all nodes in the tree and glue
> them together, so you have to create the tree in steps - which means
> that it will be intermittently incorrect).
That is quite reasonable, I'll withdraw gripe #1 -- in fact the reason
I have already implemented this in Jython is that there is already
real world use out there.  I still need to understand gripe #2 a
little better before I back down on that one.

-Frank
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library

[Nick Coghlan]

Peter Moody wrote:
> while not the the prettiest, you can already (ignoring the set_prefix)
> do something like:
> 
 newobject = ipaddr.IP(str(o.network) + "/new prefix")
> 
> Is this sufficient?

At this point, that is probably fine. If it comes up often enough to be
worth providing a cleaner interface then it is easier to add that later.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library

[Jeroen Ruigrok van der Werven]

-On [20090818 22:15], Peter Moody (pe...@hda3.com) wrote:
>I have a first draft of a PEP for including an IP address manipulation
>library in the python stdlib. It seems like there are a lot of really
>smart folks with some, ahem, strong ideas about what an IP address
>module should and shouldn't be so I wanted to solicit your input on
>this pep.
>
>the pep can be found here:
>
>  http://www.python.org/dev/peps/pep-3144/

No chance at the moment to test/look through the code, so please excuse any
obvious ones, I'm basing my comments on the PEP.

Some elaboration on handling ipv4 mapped addresses would be nice, e.g.
:::c000:280 and/or :::192.168.0.128

Some IPv6 examples would also help the PEP I think. Especially on how 0
compression is handled in addresses.

Maybe show ipv4 examples on non-class boundaries, e.g. /23 instead of /24,
so people are more convinced it handles CIDR properly.

Clarification on whether this library will support converting a sequence of
networks into another sequence where the networks which comprise consecutive
netblocks will be collapsed in a new entry. E.g. 2 /24s that are neighbours
will be represented as one /23.

I realise some might be answered by the last paragraph of your PEP, but it
would be nice to know what you consider essential and what not.

-- 
Jeroen Ruigrok van der Werven  / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
They have learned nothing, and forgotten nothing...
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Microsoft MSDN

[Paul Moore]

2009/8/13 Christian Heimes :
> Steve Holden wrote:
>>
>> I sent fourteen requests for licenses in to Microsoft. I've asked them
>> to let me know which they grant (since they may choose to limit the
>> number) and will inform you all personally when I hear their decision.
>
> I've received my MSDN subscription today. Everybody watch out for a message
> from MSDN! I almost confused the email with spam.
>
> Thanks for your work and please forward my gratitude to James Rice.

I've received mine, too, and my thanks to all as well.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] standard library mimetypes module pathologically broken?

[Paul Moore]

2009/8/14 Nick Coghlan :
> Georg Brandl wrote:
>> Nick Coghlan schrieb:
>>> P.S. For anyone else that is slow like me, take a close look at PEP 387...
>>
>> What should we see, other than that we have two PEPs on the same topic that
>> should be merged?
>
> Benjamin wrote the second one, so he obviously knows there's a written
> deprecation policy in place, and hence his mini-rant probably wasn't
> meant to be taken literally - a point I completely missed on first reading.

IIRC, the point is probably the fact that PEP 387 has status "Draft"
rather than "Accepted" - Benjamin proposed the PEP, met with a fair
bit of discussion but no consensus, and everything fizzled out before
a conclusion was reached.

> I agree the two PEPs should probably be consolidated into one, but
> absent a volunteer for that task, leaving them as is doesn't really hurt
> anything.

Agreed, if both were accepted...

Paul.

PS I personally have no firm opinion on PEP 387. The idea seems good,
but I don't feel qualified to say whether the proposed approach is
sound.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library

[Peter Moody]

On Thu, Aug 20, 2009 at 6:46 AM, Jeroen Ruigrok van der
Werven wrote:
> -On [20090818 22:15], Peter Moody (pe...@hda3.com) wrote:
>>I have a first draft of a PEP for including an IP address manipulation
>>library in the python stdlib. It seems like there are a lot of really
>>smart folks with some, ahem, strong ideas about what an IP address
>>module should and shouldn't be so I wanted to solicit your input on
>>this pep.
>>
>>the pep can be found here:
>>
>>  http://www.python.org/dev/peps/pep-3144/
>
> No chance at the moment to test/look through the code, so please excuse any
> obvious ones, I'm basing my comments on the PEP.
>
> Some elaboration on handling ipv4 mapped addresses would be nice, e.g.
> :::c000:280 and/or :::192.168.0.128
>
> Some IPv6 examples would also help the PEP I think. Especially on how 0
> compression is handled in addresses.
>
> Maybe show ipv4 examples on non-class boundaries, e.g. /23 instead of /24,
> so people are more convinced it handles CIDR properly.
>
> Clarification on whether this library will support converting a sequence of
> networks into another sequence where the networks which comprise consecutive
> netblocks will be collapsed in a new entry. E.g. 2 /24s that are neighbours
> will be represented as one /23.
>
> I realise some might be answered by the last paragraph of your PEP, but it
> would be nice to know what you consider essential and what not.

I've updated the pep with lots of examples; most of the stuff you're
asking for is already supported, I just didn't do a good job
explaining it. A few things are pending review.

Cheers,
/peter

> --
> Jeroen Ruigrok van der Werven  / asmodai
> イェルーン ラウフロック ヴァン デル ウェルヴェン
> http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
> They have learned nothing, and forgotten nothing...
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Two laments about CPython's AST Nodes

[Brett Cannon]

On Thu, Aug 20, 2009 at 06:20, Frank Wierzbicki wrote:
> On Wed, Aug 19, 2009 at 5:00 PM, "Martin v. Löwis" wrote:
>>> Now on to the complaints: Though I recently added support for this in
>>> Jython, I don't like that nodes can be defined without required
>>> attributes, for example:
>>>
>>> node = ast.Assign()
>>
>> I think we disagree in two points in our evaluation of this behavior:
>>
>> a) ast.Assign is *not* a node of the CPython AST. The CPython AST is
>>   a set of C structures in Include/Python-ast.h. ast.Assign is merely
>>   a mirror structure of that.
> Ah -- that is different from Jython's current design (The nodes that
> are constructed by ast.Assign() in Jython actually are the exact nodes
> that are used in real parsing)
>
>> b) it is, IMO, not reasonable to require users who create AST trees
>>   out of nothing to have correct trees at all times. I.e. it must be
>>   possible to represent incorrect trees.
> That does seem reasonable.  Luckily it was easy to implement for me :)
>
>> c) the AST is *not* part of the Python language or library. It may
>>   change at any time without notice, and Jython is not required to
>>   duplicate its behavior exactly.
> Sure, I'm really just talking about ast.py (though for Jython ATM they
> are the same thing).  So that I understand better: when this call is
> made in Python:
>
> x = compile("def foo():pass", "foo", "exec", _ast.PyCF_ONLY_AST)
>
> Does x contain real AST nodes or does it contain mirror structures
> (feel free to just tell me: don't be lazy, go read the code). If it
> contains real nodes, this is where I have some implementation trouble.
>  If a tree of real nodes is then manipulated so that you end up with a
> mix, I don't want to walk the entire thing over again to find the
> mirror objects (that might be incomplete) and replace them with real
> nodes.  If this creates a tree of mirror nodes, then I may want to
> consider doing the same thing on the Jython side (it makes sense, now
> that I understand CPython better I realize that the cost I am
> incurring is probably due to having the real and mirror AST as the
> same beast).
>
I am using PEP 339 to help me with this.

Looking at Python/bltinmodule.c:builtin_compile() you will notice that
when you take an AST and compile it the call goes to
Python/Python-ast.c:PyAST_obj2mod(). That function calls obj2ast_mod()
which turns a ast.Module object into an mod_ty (defined in
Python/Python-ast.c). The reverse is done with ast2obj_mod() when you
get the AST out. And looking at those conversion functions it seems
that the PyObject values convert as needed or reuse constants like
ints and strings (see obj2ast_unaryop() to see a conversion from
object AST to internal AST).

But I would double-check me on all of this. =)

-Brett

>> [so that's three items - as there should be in any good list of
>> two items :-]
> :)
>
>> What precisely is it that makes this difficult to implement. If you
>> would follow CPython's implementation strategy (i.e. generate glue
>> code out of ASDL), I feel that it should be straight-forward to provide
>> exactly the same behavior in Jython.
> I do use the ASDL to generate this stuff, but again, the real and the
> mirror nodes are not separated ATM, and that is what makes it
> difficult.
>
>>> BTW -- I *am* volunteering to attempt to implement these things in
>>> CPython if there is support :)
>>
>> I'm not sure I can support such a change. Giving the child nodes at
>> creation time, optionally, would be fine with me. Requiring the
>> tree to conform to the grammar at all times is unreasonable, IMO
>> (you can't simultaneously create all nodes in the tree and glue
>> them together, so you have to create the tree in steps - which means
>> that it will be intermittently incorrect).
> That is quite reasonable, I'll withdraw gripe #1 -- in fact the reason
> I have already implemented this in Jython is that there is already
> real world use out there.  I still need to understand gripe #2 a
> little better before I back down on that one.
>
> -Frank
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library

[Peter Moody]

The pep has been updated with the excellent suggestions thus far.

Are there any more?

Cheers,
/peter

On Tue, Aug 18, 2009 at 1:00 PM, Peter Moody wrote:
> Howdy folks,
>
> I have a first draft of a PEP for including an IP address manipulation
> library in the python stdlib. It seems like there are a lot of really
> smart folks with some, ahem, strong ideas about what an IP address
> module should and shouldn't be so I wanted to solicit your input on
> this pep.
>
> the pep can be found here:
>
>  http://www.python.org/dev/peps/pep-3144/
>
> the code can be found here:
>
>  http://ipaddr-py.googlecode.com/svn/branches/2.0.x/
>
> Please let me know if you have any comments (some already coming :)
>
> Cheers,
> /peter
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library

[Joel Bender]

Nick Coghlan wrote:


Maybe this is something that differs by country, but I have *never*
heard the first address in an IP network (i.e. every bit not covered by
the netmask set to zero) referred to as anything other than the "network
address".


Ah!  A change to interject a mostly pointless comment...

Prior to IEN-212 [1] it wasn't standardized, the 'zero' was used and 
supported by the Berkeley socket library.  This was a number of years 
ago, however (!), and I dare say the sample code is lost to antiquity.



And if someone does need to deal with that, then they create an
appropriate subclass or use a less lightweight IP addressing library.


Indeed.


Joel
[1] 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Two laments about CPython's AST Nodes

[Martin v. Löwis]

> x = compile("def foo():pass", "foo", "exec", _ast.PyCF_ONLY_AST)
> 
> Does x contain real AST nodes or does it contain mirror structures
> (feel free to just tell me: don't be lazy, go read the code).

It only contains a mirror structure. See
pythonrun.c:Py_CompileStringFlags, and the (generated) PyAST_mod2obj
function. There is no way for a Python script to get hold of the
real AST.

> I do use the ASDL to generate this stuff, but again, the real and the
> mirror nodes are not separated ATM, and that is what makes it
> difficult.

Couldn't you just generate a check function for your tree that
would be invoked before you try to process a tree that a
script got access to?

> That is quite reasonable, I'll withdraw gripe #1 -- in fact the reason
> I have already implemented this in Jython is that there is already
> real world use out there.  I still need to understand gripe #2 a
> little better before I back down on that one.

If you are asking that a type check is made on assigning a value to
these fields - I'm not quite sure whether you could implement that
check reliably. Wouldn't it be possible to bypass it by filling a
value directly into __dict__?

If you can come up with a patch that checks in a reliable manner,
I would be in favor of adding that (in 2.7 and 3.2), taking out
the corresponding checks when converting to the internal AST.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library

[Case Vanhorsen]

>On Thu, Aug 20, 2009 at 2:00 PM, Peter Moody wrote:
> The pep has been updated with the excellent suggestions thus far.
>
> Are there any more?

Thanks for writing the PEP.

I tried a few of the common scenarios that I use at work. Disclaimer:
my comments are based on my work environment.

I was surprised that IP('172.16.1.1') returned
IPv4Address('172.16.1.1/32') instead of IPv4Address('172.16.1.1'). I
know I can change the behavior by using host=True, but then
IP('172.16.1.1/24', host=True) will raise an error. It makes more
sense, at least to me, that if I input just an IP address, I get an IP
address back. I would prefer that IP('172.16.1.1/32') return an
IPv4Network and IP('172.16.1.1') return an IPv4Address.

Would it be possible to provide an iterator that returns just the
valid host IP addresses (minus the network and broadcast addresses)?
"for i in IPv4Network('172.16.1.0/28')" and "for in in
IPv4Network('172.16.1.0/28').iterhosts()" both return all 16 IP
addresses. I normally describe 172.16.1.0/28 as consisting of one
network address, 14 host addresses, and one broadcast address. I would
prefer that "for i in IPv4Network('172.16.1.0/28')" return all IP
addresses and that "for in in
IPv4Network('172.16.1.0/28').iterhosts()" exclude the network and
broadcast addresses. I think creating a list of IP addresses that can
be assigned to devices on a network is a common task.

Can .subnet() be enhanced to accept masks? For example,
IPv4Network('172.16.0.0/16').subnet('/19') would return the eight /19
subnets.

What about supporting multiple parameters to subnet? I frequently need
to create complex subnet layouts. The following subnet layout is NOT
made up!

172.16.0.0/22
>172.16.0.0/23
>>>172.16.2.0/25
172.16.2.128/26
172.16.2.192/26
>>172.16.3.0/28
>>172.16.3.16/28
172.16.3.32/30
172.16.3.36/30
172.16.3.40/30
172.16.3.44/30
172.16.3.48/30
172.16.3.52/30
172.16.3.56/30
172.16.3.60/30
>>172.16.3.64/32
   
>>172.16.3.79/32
>>172.16.3.80/28
>>172.16.3.96/28
>>172.16.3.112/28
>172.16.3.128/27
>172.16.3.160/27
172.16.3.192/26

A possible syntax would be:
.subnet((1,'/23'),(1,'/25'),(2,'/26'),(2,'/28'),(8,'/30'),(16,'/32'),(3,'/28'),(2,'/27'),(1,'/26'))

Note: I am willing to provide patches to implement my suggestions. I
just won't have much time over the next couple weeks.

casevh


> Cheers,
> /peter
>
> On Tue, Aug 18, 2009 at 1:00 PM, Peter Moody wrote:
>> Howdy folks,
>>
>> I have a first draft of a PEP for including an IP address manipulation
>> library in the python stdlib. It seems like there are a lot of really
>> smart folks with some, ahem, strong ideas about what an IP address
>> module should and shouldn't be so I wanted to solicit your input on
>> this pep.
>>
>> the pep can be found here:
>>
>>  http://www.python.org/dev/peps/pep-3144/
>>
>> the code can be found here:
>>
>>  http://ipaddr-py.googlecode.com/svn/branches/2.0.x/
>>
>> Please let me know if you have any comments (some already coming :)
>>
>> Cheers,
>> /peter
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com