[Python-Dev] Issues with Py3.1's new ipaddr
On Mon, Jun 1, 2009 at 11:32 AM, Guido van Rossum wrote: > I haven't read the bug, but given the extensive real-life use that > ipaddr.py has seen at Google before inclusion into the stdlib, I think > "deep conceptual flaws" must be an overstatement. When the users of the library are also the authors of the library, it is not surprising that the library enjoys "extensive real-life use." The real test of a library is not how well it succeeds at one installation, but how well it meets the needs of the larger user base. Having read the code and the author's comments, it seems to me that networking concepts not employed at Google have been neglected. While some of these features are easily added to ipaddr, their omission exposes a narrow view of the problem domain that has resulted in more fundamental problems in the library, such as the conflation of addresses and networks. > I think we should just stick to "sorry, too late, try again for 3.2". > We've done that with plenty of more important flaws that were > discovered on the verge of a release, and I don't recall ever > regretting it. We can always add more features to the module in 3.2. My concerns are not so much about adding features as they are about changing the API. Addressing the concerns that I and others have raised requires making backwards-incompatible changes. Doing that now, before ipaddr enjoys widespread deployment as part of the stdlib, seems prudent. Removing ipaddr from 3.1 appears to me less painful than fixing apps when ipaddr's API changes in 3.2. If this were a core feature that many developers were anxiously awaiting, I could understand the desire to release and iterate. But is there really a pressing need for an IP library in the stdlib? Until a satisfactory library is available for inclusion in the stdlib, the few developers that do require such functionality can easily enough download a library that meets their needs. Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. Löwis" wrote: > For the net-vs-host issue, I think a backwards-compatible solution > is possible: just give the IP() function an option parameter that > makes it reject a netmask during parsing. That doesn't solve much. IPv4 objects still always use CIDR notation when coerced to strings, meaning that IP addresses will always be rendered with a trailing "/32". Such notation is unacceptable in real-world applications that (correctly) distinguish between address and network. > That was my feeling as well when ipaddr was first offered. It's just > not an important library, and people will continue to roll their own > for some time. OTOH, with ipaddr in the standard library, people will > also start contributing extensions that make it support their use cases, > so it should grow a wider application area than it currently supports. That being the case, why not delay its inclusion until we can be sure that it in fact represents a good base upon which to build? Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 1:47 AM, "Martin v. Löwis" wrote: >> If this were a core feature that many developers were anxiously >> awaiting, I could understand the desire to release and iterate. But is >> there really a pressing need for an IP library in the stdlib? > > You should have asked this question a few month ago. Now, release > mechanics make it difficult to remove the library, except if a severe > problem was uncovered - which you haven't been able to demonstrate. This argument makes no sense. The feasibility of removing the library does not depend on the severity of the issues found within it. Either it is hard to remove the library, or it is easy. If it's hard to remove, it doesn't get any easier because I discover a fatal flaw. I've actually provided several examples of where the library fails when used in common scenarios, yet your solution involves incremental hacks that don't fix the underlying problems. You write as if you have a vested interest in releasing the library -- why? > I don't believe that your issue "hosts and nets are represented with the > same class" is severe: it is very well possible to use the IP function > to represent individual hosts (including having a string representation > that doesn't print a netmask). The only possibly-missing feature is > support for rejecting host strings that include a mask on parsing; that > can be added in a backwards-compatible way as an optional parameter to > the IP function (as discussed on the tracker). There are other missing features, but again, my concern is not about missing features: it's about a quirky API that conflates concepts in the problem domain, leading to subtle bugs and breaking the principle of least surprise. Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 9:22 AM, R. David Murray wrote: > ip[0] >> >> '10.33.11.17' > > Actually I find that very intuitive if I understand that the objects > are always networks. I suspect the authors would disagree with you on this point, since they insist that host routes and host addresses are the same thing, but assuming you are right, two flaws are immediately apparent: * The ipaddr classes are poorly named. Since they model networks, they should be named appropriately: IPNet or some such. * The ipaddr authors have overlooked IP addresses, which also deserve first-class treatment in any meaningful IP address library. It is called ipaddr, after all, and not ipnet. > ip.ip_ext >> >> '10.33.11.17' > > That's not intuitive :) No, nor is the rest of the library intuitive. This is one of the reasons I've called it "quirky". Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 2:15 AM, "Martin v. Löwis" wrote: > We could remove it, but then what we have wouldn't really be a release > candidate anymore, so the release would get delayed. How long do release candidates soak in the field before being accepted? I'm curious if an exception could be made in this case, given that you have admitted that ipaddr is not an important library. The chances of a problem being introduced due to its removal are vanishingly small. No other components of the stdlib depend on ipaddr, and the few (approximately zero?) developers who will have started writing applications depending on ipaddr due to its inclusion in the release candidate could simply download the library from Google. > I believe the API appears more quirky to you than it actually is, > probably because you don't have understood it fully (but I might > be wrong with that, and there might be a different reason). I believe the API is quirky because: * It tries to represent distinct domain model concepts in a single class * Its methods and properties are strangely named * Methods and properties that should return domain model objects return native types instead * It cannot correctly parse output from netstat, nor can it correctly pass values to ifconfig You seem comfortable with these quirks, but then you're not planning to write software with this library. Developers who do intend to write meaningful network applications do seem concerned, yet we're ignored. If you consult the issue notes, you'll see objections of the type I just mentioned were raised months ago, yet no work has been done to correct them. The only changes that I can see were related to pedantic style issues: camel case and indentation. Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. Löwis" wrote:
>> That doesn't solve much. IPv4 objects still always use CIDR notation
>> when coerced to strings, meaning that IP addresses will always be
>> rendered with a trailing "/32".
>
> That's not true:
>
> py> x = ipaddr.IP("30.40.50.60")
> py> print(x.ip_ext_full)
> 30.40.50.60
Thankfully the authors have provided this obscure and strangely-named
method to get at the correct string representation of an IP address,
but sadly their __str__ method -- which is the Pythonic way to get
string representations of objects -- fails in this regard because they
have only one class representing two distinct concepts.
I could probably make ipaddr work in my application; that is not the
issue (at least in my mind). The issue is that I shouldn't have to
work around design flaws in the library when they're simple enough to
fix before the library is bundled with the stdlib. I do not see the
utility of including ipaddr when my choice as a user is to either use
its strangely-named methods instead of obvious Pythonic conventions,
or to write my own library.
>> Such notation is unacceptable in
>> real-world applications that (correctly) distinguish between address
>> and network.
>
> So use a different notation that is also supported by the library.
I'm not referring to my software here -- I'm referring to applications
like ifconfig that expect addresses to be formatted properly. If the
default string representation produced by the ipaddr library does not
match the canonical representation expected by software that has
existed as long as IP itself, that indicates to me the library is
doing something wrong.
> Because we *are* sure that it in fact represents a good base upon which
> to build.
You certainly seem convinced. I could not disagree more.
Clay
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 4:53 PM, R. David Murray wrote:
> Having thought more about this, I will agree with you that it would
> be useful to have an address-without-netmask class.
Not only useful, but necessary. It seems there are few people on this
list who understand IP well enough to realize how distorted ipaddr
actually is.
> (Note, however, that even a DNS entry can conceptually be considered to
> be a host route.)
Not at all. A host route and a host address (as represented in DNS)
are fundamentally different concepts. Please see my recent post to
ipaddr-py-dev for a refresher on these concepts:
http://groups.google.com/group/ipaddr-py-dev/t/94d54fe581d24e72
> So I'm not in favor of pulling ipaddr from 3.1, and it's too late in
> the release cycle to change anything.
I'm not sure why you say that when others have said that another
release candidate is planned, and that removing ipaddr is essentially
trivial to do.
> I wish you had brought this energy to bear earlier, when changes could
> have been made. Reality is what it is, though, and now we should work
> on making improvements for the next release. I see in the ticket that
> the netaddr folks were going to propose improvements so that they could
> build netaddr on top of ipaddr, but I guess that didn't happen (yet?).
I don't think that can happen, actually. If I was a netaddr committer
(which I'm not), I would find it hard to salvage anything reusable
from ipaddr. It is certainly simpler and clearer to start over with an
object model that actually makes sense.
> I have no association with Google, by the way, and I do intend to use
> ipaddr in upcoming code, and have hacked my own address manipulation
> stuff in previous code.
Sorry, I wasn't aware of that. My mistake. Regardless, I find that
your understanding of IP is similar to that of ipaddr's authors, which
is to say imprecise.
> I don't understand why you are trying to use ifconfig as an example.
Because it's an obvious real world example that explains why these two
strings are not equivalent:
192.168.1.1
and
192.168.1.1/32
You and others continue to suggest that those strings are equivalent,
yet ifconfig is a tool that has been around for thirty years that
clearly demonstrates that those strings are not equivalent. If what
you say is true, I should be able to pass either string to ifconfig
and get the same result. That is not the case, because the strings are
not equivalent, because a host route is not the same thing as a host
address.
> So this does the Right Thing:
>
> myip = ipaddr.IP('192.168.1.1/26')
> system('ifconfig eth0 {}'.format(myip))
Sure, but shouldn't this also do the right thing?
address = ipaddr.IP('192.168.1.1')
netmask = ipaddr.IP('255.255.255.192')
system("ifconfig eth0 %s/%s" % (address, netmask))
It doesn't.
> Hmm. I think there is a conceptual divide here. You have said you
> think about IP addresses and networks as separate objects, so I wonder
> if you would be pulling the netmask for ifconfig out of a separate
> network object?
Of course, because addresses don't have masks; networks do. This command:
ifconfig en0 192.168.1.1/24
is shorthand for operator convenience. What's going on behind the
scenes is quite a lot different than it looks. First, ifconfig
computes a network address by masking the supplied interface address
with the supplied network mask. It is then able to configure a route
for the proper network: 192.168.1.0/24.
The fact that "192.168.1.1/24" appears in the command does *not* mean
that the address 192.168.1.1 has a mask of /24. That is absurd.
Addresses don't have masks; networks do. That's why they're called
netmasks.
> On the other hand I, a network professional, think about an IP address
> paired with a netmask as a fundamental object.
The IT industry is unique among engineering disciplines in that formal
training and licensing are typically not required for IT
professionals. Whereas concepts like resistance, current, and voltage
have very specific meanings to electrical engineers, the IT vernacular
is not so precise. Since formal training is rare, and what little is
available is often high-priced and vendor-specific, IT professionals
tend to learn their trade from trade books, word of mouth, and
hands-on experience. As a result, IT professionals tend to have a good
working knowledge of how technology applies to their particular job,
but may not have an appreciation of the more theoretical aspects of
the discipline.
What this means in practice is that your experience as a network
professional may not resemble the experiences of other network
professionals. That you tend to think of addresses as having masks is
probably not universal, or even particularly common, among network
professionals. Some electrical engineers probably think of voltage as
pressure, and that may be a useful abstraction, but I would be
surprised to see a voltmeter calibrated in pascals.
What are we to do? How do we arrive at a common un
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Tue, Jun 2, 2009 at 10:41 PM, Gregory P. Smith wrote: > ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when > iterating over it instead of strings and this example would work > properly. I have opened an issue in the ipaddr bug tracker that I think addresses this issue. There are a few methods and properties in the ipaddr.IP classes that return native types but should return IP objects: http://code.google.com/p/ipaddr-py/issues/detail?id=21 > It sounds like we have a 3.1rc2 scheduled for June 13th. At this > point based on the multitude of other developer comments in these > threads and barring strong arguments in favor of including it as it > stands I will remove it because there seem to be quite a number of > people with objections to it as an API in its current form. Thank you. > Would someone volunteer write up a proposal (PEP) for such with a goal > of having it completed within the next few months? (I know people > have suggested various other libraries, that counts; i have not taken > the time to look at them). I am happy to take a stab at that. Thanks, Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with Py3.1's new ipaddr
On Wed, Jun 3, 2009 at 11:16 AM, R. David Murray wrote: > I think this hits the nail on the head. Rather than network engineers > having a less precise understanding of IP, what we have is two different > sets of domain requirements. Network engineers deal with networks, with > IPs being a necessary special case. Others deal with host addresses, > with networks as an additional data type. It has been brought to my attention that my use of the word "imprecise" may have been construed as an insult. For that I apologize. That was not my intent. I am sure that your understanding of IP in your domain (network engineering) is as good or better than mine in my domain (UNIX administration). That is not the point I was arguing. Had you said that thinking of addresses as having netmasks is a useful mental model, I would have agreed with you wholeheartedly. To me, this is similar to thinking of voltage as pressure. Instead, you said (or at least implied) that addresses in fact do have netmasks, which I think is technically an imprecise way of describing how the technology works. To me, that would be like building a voltmeter calibrated in pascals. As another poster has commented, I think the name of the module is the source of some confusion. While I see the validity of your use case, that is not the use case I had in mind for a module named "ipaddr". In any case, I think with some enhanced documentation and maybe some class name changes, we can clarify ipaddr's API and strive to make it support both use cases. Your continued input will be invaluable as we move forward with the PEP. Clay ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
