[Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

[Thomas Lee]

Hi all,

I've been working on optimization of the AST, including the porting of 
the old bytecode-level optimizations to the AST level. A few questions 
have come up in the process of doing this, all of which are probably 
appropriate for discussion on this list. The code I'm referring to here 
can be found in the tlee-ast-optimize branch. Most of the relevant code 
is in Python/optimize.c and Python/peephole.c.


Nearly all of the bytecode-level optimizations have been moved to the 
AST optimizer with a few exceptions. Most of those waiting to be ported 
are stuck in limbo due to the fact we can't yet inject arbitrary 
PyObject constants into the AST. Examples are tuples of constants and 
the optimization of "LOAD_GLOBAL/LOAD_NAME None" as "LOAD_CONST None".


This leaves us with a few options:

1. A new AST expr node for constant values for types other than Str/Num

I imagine this to be something like Const(PyObject* v), which is 
effectively translated to a "LOAD_CONST v" by the compiler. This trades 
the purity of the AST for a little practicality. A "Const" node has no 
real source representation, it would exist solely for the purpose of 
injecting PyObject constants into the AST.


2. Allow arbitrary annotations to be applied to the AST as compiler hints.

For example, each AST node might have an optional dict that contains a 
set of annotation values. Then when traversing the AST, the compiler 
might do something along the lines of:


if (expr->annotations) {
 PyObject* constvalue = PyDict_GetItemString(expr->annotations, 
"constantvalue");

 if (constvalue)
   ADDOP_O(c, LOAD_CONST, constvalue, consts)
 else
   VISIT(c, expr, expr)
}

This is a more general solution if we want to keep other compiler hints 
down the track, but unless somebody can think of another use case this 
is probably overkill.


3. Keep these particular optimizations at the bytecode level.

It would be great to be able to perform the optimizations at a higher 
level, but this would require no effort at all. This would mean two 
passes over the same code at two different levels.


If anybody familiar with this stuff could weigh in on the matter, it 
would be much appreciated. I've got a list of other issues that I need 
to discuss here, but this would be a great start.


Thanks,
Tom
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] cStringIO buffer interface

[Farshid Lashkari]

Hello,

I'm not sure if this is the right place to ask, so I apologize ahead
of time if it is.

I was just curious as to why cStringIO objects don't implement the
buffer interface.  cStringIO objects seem similar to string and array
objects, and those support the buffer protocol. Is there a reason
against allowing cStringIO to support at least the read buffer
interface, or is just that nobody has considered it until now?

Thanks,
Farshid
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cStringIO buffer interface

[Guido van Rossum]

On Wed, Apr 30, 2008 at 9:36 AM, Farshid Lashkari <[EMAIL PROTECTED]> wrote:
>  I was just curious as to why cStringIO objects don't implement the
>  buffer interface.  cStringIO objects seem similar to string and array
>  objects, and those support the buffer protocol. Is there a reason
>  against allowing cStringIO to support at least the read buffer
>  interface, or is just that nobody has considered it until now?

Well, for one, it would mean you could no longer exchange a StringIO
instance for a cStringIO instance.

Also, what's the compelling use case you're thinking of?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] socket.try_reuse_address()

[Trent Nelson]

> > Giampaolo Rodola' wrote:
> > > Maybe it would be better considering Windows CE systems too:
> > >
> > > - if os.name == 'nt'
> > > + if os.name in ('nt', 'ce')
> > >
> > Cygwin? I don't know how Unix-like it is.
>
> Yeah, that's a fair point, it's the behaviour of the
> underlying Winsock API we're targeting, so it would apply to
> Cygwin as well.  (And CE and anything else on Windows.)

including Jython and IronPython -- which all exhibit the same undesirable 
behaviour on Windows when SO_REUSEADDR is set against a TCP/IP socket.  Updated 
patch below.  Assuming there are no objections, I'd like to clean this up and 
commit over the weekend, once I've updated the various parts of the stdlib 
currently using SO_REUSEADDR, as well as affected documentation.

Index: socket.py
===
--- socket.py   (revision 62509)
+++ socket.py   (working copy)
@@ -143,8 +143,18 @@
 'sendall', 'setblocking',
 'settimeout', 'gettimeout', 'shutdown')

+# Attempt to determine if we're running on Windows, irrespective of our Python
+# incarnation.  We need to know this so that we *don't* set the SO_REUSEADDR
+# against TCP/IP sockets in socket.try_reuse_addr().  Note that IronPython is
+# covered below as it sets os.name to 'nt'.
 if os.name == "nt":
 _socketmethods = _socketmethods + ('ioctl',)
+_is_windows = True
+elif os.name == 'java':
+from java.lang import System
+_is_windows = 'windows' in System.getProperty('os.name').lower()
+elif os.name == 'posix' and sys.platform == 'cygwin':
+_is_windows = True

 if sys.platform == "riscos":
 _socketmethods = _socketmethods + ('sleeptaskw',)
@@ -197,6 +207,13 @@
 Return a new socket object connected to the same system resource."""
 return _socketobject(_sock=self._sock)

+def try_reuse_address(self):
+if not (_is_windows and self._sock.type != SOCK_DGRAM):
+try:
+self._sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
+except socket.error:
+pass
+
 def makefile(self, mode='r', bufsize=-1):
 """makefile([mode[, bufsize]]) -> file object



Trent.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] socket.try_reuse_address()

[Trent Nelson]

>  if os.name == "nt":
>  _socketmethods = _socketmethods + ('ioctl',)
> +_is_windows = True
> +elif os.name == 'java':
> +from java.lang import System
> +_is_windows = 'windows' in System.getProperty('os.name').lower()
> +elif os.name == 'posix' and sys.platform == 'cygwin':
> +_is_windows = True

Oops, that last line should have been:

elif os.name == 'ce' or (os.name == 'posix' and sys.platform == 'cygwin'):
_is_windows = True
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

[Brett Cannon]

On Wed, Apr 30, 2008 at 3:15 AM, Thomas Lee <[EMAIL PROTECTED]> wrote:
> Hi all,
>
>  I've been working on optimization of the AST, including the porting of the
> old bytecode-level optimizations to the AST level. A few questions have come
> up in the process of doing this, all of which are probably appropriate for
> discussion on this list. The code I'm referring to here can be found in the
> tlee-ast-optimize branch. Most of the relevant code is in Python/optimize.c
> and Python/peephole.c.
>
>  Nearly all of the bytecode-level optimizations have been moved to the AST
> optimizer with a few exceptions. Most of those waiting to be ported are
> stuck in limbo due to the fact we can't yet inject arbitrary PyObject
> constants into the AST. Examples are tuples of constants and the
> optimization of "LOAD_GLOBAL/LOAD_NAME None" as "LOAD_CONST None".
>
>  This leaves us with a few options:
>
>  1. A new AST expr node for constant values for types other than Str/Num
>
>  I imagine this to be something like Const(PyObject* v), which is
> effectively translated to a "LOAD_CONST v" by the compiler. This trades the
> purity of the AST for a little practicality. A "Const" node has no real
> source representation, it would exist solely for the purpose of injecting
> PyObject constants into the AST.
>

Slight issue with this is that people like Jython are standardizing on
our AST, so adding to it does up their burden if they have no use for
it.

>  2. Allow arbitrary annotations to be applied to the AST as compiler hints.
>
>  For example, each AST node might have an optional dict that contains a set
> of annotation values. Then when traversing the AST, the compiler might do
> something along the lines of:
>
>  if (expr->annotations) {
>   PyObject* constvalue = PyDict_GetItemString(expr->annotations,
> "constantvalue");
>   if (constvalue)
>ADDOP_O(c, LOAD_CONST, constvalue, consts)
>   else
>VISIT(c, expr, expr)
>  }
>
>  This is a more general solution if we want to keep other compiler hints
> down the track, but unless somebody can think of another use case this is
> probably overkill.
>

Possibly, but the AST stuff is so new who knows if it truly will be
overkill or not down the road. I personally prefer this approach.

>  3. Keep these particular optimizations at the bytecode level.
>
>  It would be great to be able to perform the optimizations at a higher
> level, but this would require no effort at all. This would mean two passes
> over the same code at two different levels.
>

This might end up being the case, but I would rather avoid the
overhead if possible. The question becomes how many other possible
optimizations might come up that would only work reasonably at the
bytecode level.

And you forgot option 4): ditching the optimizations that only work on
the bytecode. I am not advocating this, but it is an option.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] socket.try_reuse_address()

[Raghuram Devarakonda]

On Wed, Apr 30, 2008 at 1:42 PM, Trent Nelson <[EMAIL PROTECTED]> wrote:
> >  if os.name == "nt":
>  >  _socketmethods = _socketmethods + ('ioctl',)
>  > +_is_windows = True
>  > +elif os.name == 'java':
>  > +from java.lang import System
>  > +_is_windows = 'windows' in System.getProperty('os.name').lower()

This one will not work.

>>> 'windows' in System.getProperty('os.name').lower()
Traceback (innermost last):
  File "", line 1, in ?
TypeError: string member test needs char left operand
>>>

You may have to do something like
System.getProperty('os.name').lower().find('windows').
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] socket.try_reuse_address()

[Trent Nelson]

> This one will not work.
>
> >>> 'windows' in System.getProperty('os.name').lower()
> Traceback (innermost last):
>   File "", line 1, in ?
> TypeError: string member test needs char left operand
> >>>

Interesting, which version of Jython were you using?

> You may have to do something like
> System.getProperty('os.name').lower().find('windows').

That didn't work for me.  I assume the following works for you:

System.getProperty('os.name').lower().startswith('windows')

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] socket.try_reuse_address()

[Raghuram Devarakonda]

On Wed, Apr 30, 2008 at 2:07 PM, Trent Nelson <[EMAIL PROTECTED]> wrote:
>
>  > This one will not work.
>  >
>  > >>> 'windows' in System.getProperty('os.name').lower()
>  > Traceback (innermost last):
>  >   File "", line 1, in ?
>  > TypeError: string member test needs char left operand
>  > >>>
>
>  Interesting, which version of Jython were you using?

2.1. Now that you mentioned it, I tested with 2.2 and your code works there.

>  > You may have to do something like
>  > System.getProperty('os.name').lower().find('windows').
>
>  That didn't work for me.  I assume the following works for you:
>
> System.getProperty('os.name').lower().startswith('windows')

It does.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cStringIO buffer interface

[Jean-Paul Calderone]

On Wed, 30 Apr 2008 09:51:25 -0700, Guido van Rossum <[EMAIL PROTECTED]> wrote:

On Wed, Apr 30, 2008 at 9:36 AM, Farshid Lashkari <[EMAIL PROTECTED]> wrote:

 I was just curious as to why cStringIO objects don't implement the
 buffer interface.  cStringIO objects seem similar to string and array
 objects, and those support the buffer protocol. Is there a reason
 against allowing cStringIO to support at least the read buffer
 interface, or is just that nobody has considered it until now?


Well, for one, it would mean you could no longer exchange a StringIO
instance for a cStringIO instance.


It would probably only mean that there is one further incompatibility
between cStringIO and StringIO - you already can't exchange them in a
number of cases.  They handle unicode differently, they have different
methods, etc.

Maybe making them diverge even further is a step in the wrong direction,
though.


Also, what's the compelling use case you're thinking of?


I'm not sure what use-case Farshid Lashkari had.  For Twisted, it has
been considered as a way to reduce peak memory usage (by reducing the
need for memory copying, which also speeds things up).  I'm not sure
if anyone has benchmarked this yet, so I don't know if it's a real win
or not.  I think Thomas Hervé has a patch to cStringIO which implements
the feature, though.

For reference, .

This isn't high on my priority list, but I thought I'd point out the
potential use-case.

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

[Martin v. Löwis]

> This leaves us with a few options:

5. Reuse/Abuse Num(object) for arbitrary constants.
   AFAICT, this should work out of the box.

> 1. A new AST expr node for constant values for types other than Str/Num
> 
> I imagine this to be something like Const(PyObject* v), which is
> effectively translated to a "LOAD_CONST v" by the compiler. This trades
> the purity of the AST for a little practicality. A "Const" node has no
> real source representation, it would exist solely for the purpose of
> injecting PyObject constants into the AST.

I think this is the way to go. It doesn't violate purity: it is an
*abstract* syntax, meaning that there doesn't need to be a 1:1
relationship to source syntax. However, it is still possible to
reproduce source code from this Const node.

I also don't worry about Jython conflicts. The grammar has a version
number precisely so that you can refer to a specific version if you
need to.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Module properties for C modules

[Christian Heimes]

Hello Python Dev!

As you all know modules don't support properties. However several places
and modules could use properties on module level. For example the
sys.py3k_warnings flag could be implemented with a property. Other flags
in the sys module could benefit from read only properties, too.

How do you like the general idea of properties for builtin modules. That
is modules written in C like the sys module.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Module properties for C modules

[Benjamin Peterson]

On Wed, Apr 30, 2008 at 4:17 PM, Christian Heimes <[EMAIL PROTECTED]> wrote:
> Hello Python Dev!
>
>  As you all know modules don't support properties. However several places
>  and modules could use properties on module level. For example the
>  sys.py3k_warnings flag could be implemented with a property. Other flags
>  in the sys module could benefit from read only properties, too.

Big +1. Frankly, the get/set methods of sys are quite ugly!
>
>  How do you like the general idea of properties for builtin modules. That
>  is modules written in C like the sys module.

Good idea. Perhaps eventually they could be extended to Python, but
they are definitely useful in C now. How about passing a list of
getsets to PyImport_InitModule(5)?

>
>  Christian
>  ___
>  Python-Dev mailing list
>  Python-Dev@python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Another GSoC Student Introduction

[Heiko Weinen]

Hi!

My name is Heiko N. Weinen and I was accepted as GSoC Student, too. Hoorray!
The project i chose is about Filesystem Virtualisation for Python's Standard 
Library,
hopefully something which will prove quite useful. :)

Christian Heimes is my Mentor and I'd like to thank him right now for his help.

Feel free to contact me, if you have any further questions or related ideas!
I will set up a project site at http://heiko.weinen.org/gsoc08 in a few days.

Regards,
Heiko

-- 
Lubarsky's Law of Cybernetic Entomology: prov.
"There is always one more bug."
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Module properties for C modules

[Christian Heimes]

Benjamin Peterson schrieb:
> Good idea. Perhaps eventually they could be extended to Python, but
> they are definitely useful in C now. How about passing a list of
> getsets to PyImport_InitModule(5)?

Yeah, I've a similar idea with PyImport_InitModule5() and a list of
structs containing name, getter, setter, deleter, docstring. The module
struct has to gain an additional slot which may contain a dict of names
-> propertyobjects.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Debian/alpha test_math failures

[Mark Dickinson]

Hello all,
test_math is currently failing on the Debian/alpha buildbots (trunk and
py3k).
I've been trying, unsuccessfully, to figure out what's going wrong, and I'm
starting to run out of ideas, so I thought I'd ask the list for help to see
if
anyone has any useful suggestions.

Details of the failure:  running the following code:

from math import log
ar = 9.88e-324
x = log(ar)

produces:

ValueError: math domain error

Somehow, it looks like errno is getting set to something nonzero in
math_1_from_whatever in mathmodule.c, but I really can't figure out
how.

I've tried adding '-mieee' to the gcc compile flags, and I've added a bunch
of autoconf tests to verify that log(9.88e-324) succeeds, produces
roughly the right result, and doesn't set errno to anything nonzero.
All the autoconf tests that should pass do.  So I can't find anything
wrong with the libm implementation of log.

test_math is fine on Tru64/alpha.

Does anyone have access to a Linux/alpha machine, and a few minutes
to figure out exactly what's failing?  Or any suggestions about what might
be failing.  I'm open to wild ideas at this point... :-)

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Module properties for C modules

[Guido van Rossum]

On Wed, Apr 30, 2008 at 2:17 PM, Christian Heimes <[EMAIL PROTECTED]> wrote:
>  As you all know modules don't support properties. However several places
>  and modules could use properties on module level. For example the
>  sys.py3k_warnings flag could be implemented with a property. Other flags
>  in the sys module could benefit from read only properties, too.
>
>  How do you like the general idea of properties for builtin modules. That
>  is modules written in C like the sys module.

But wouldn't this mean that those properties would no longer be
available in the module's __dict__?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Problems with the new super()

[Armin Ronacher]

Hi all,

I blogged about that topic today which turned out to be a very bad idea,
so I summarize it for the mailinglist here to hopefully start a discussion
about the topic, which I think is rather important.

In the last weeks something remarkable happened in the Python3 sources:
self kinda became implicit.  Not in function definitions, but in super
calls.  But not only self: also the class passed to `super`.  That's
interesting because it means that the language shifts into a completely
different direction.

`super` was rarely used in the past, mainly because it was weird to use.
In the most common use case the current class and the current instance
where passed to it, and the super typed returned looked up the parent
methods on the MRO for you.  It was useful for multiple inheritance and
mixin classes that don't know their parent but confusing for many.  I can
see that a replacement is a good idea, but I'm not so sure if the current
implementation is the way to go.

The main problem with replacing `super(Foo, self).bar()` with something
like `super.bar()` is obviously that self is explicit and the class (in
that case Foo) can't be determined by the caller.  Furthermore the Python
principle was always against functions doing stack introspection to find
the caller.  There are few examples in the stdlib or builtins that do
some sort of caller introspection.  Those are the special functions
`vars`, `locals`, `gloabal`, `vars` and some functions in the inspect
module.  And all of them do nothing more than getting the current frame
and accessing the dict of locals or globals.  What super in current
Python 3 builds does goes way beyond that.

The implementation of the automatic super currently has two ugly details
that I think violate the Python Zen:  The bytecode generated is differently
if the name "super" is used in the function.  `__class__` is only added as
cell to the code if `super` or `__class__` is referenced.  That and the fact
that `f_localsplus` is completely unavailable from within python makes the
whole process appear magical.

This is way more magical than anything we've had in Python in the past and
just doesn't fit into the language in my opinion.  We do have an explicit
self in methods and methods are more or less just functions.  Python's
methods are functions, just that a descriptor puts a method object around
it to pass the self as first arguments.  That's an incredible cool thing
to have and makes things very simple and non-magical.  Breaking that
principle by introducing an automatic super seems to harm the concept.

Another odd thing is that Python 3 starts keeping information on the C
layer we can't access from within Python.  Super is one example, another
good one are methods.  They don't have a descriptor that wraps them if
they are accessed via their classes.  This as such is not a problem as you
can call them the same (just that you can call them with completely
different receivers now) but it becomes a problem if some of the functions
are marked as staticmethods.  Then they look completely the same when
looking at them from a classes perspective:

|   >>> class C:
|   ...  normal = lambda x: None
|   ...  static = staticmethod(lambda x: None)
|   ... 
|   >>> type(C.normal) is type(C.static)
|   True
|   >>> C.normal
|at 0x4da150>

As far as I can see a documentation tool has no chance to keep them apart
even though they are completely different on an instance:

|   >>> type(C().normal) is type(C().static)
|   False
|   >>> C().normal
|of <__main__.C object at 0x4dbcf0>>
|   >>> C().static
|at 0x4da198>

While I don't knwo about the method thing, I think an automatic super should
at least be implementable from within Python.  I could imagine that by adding
__class__ and __self__ to scopes automatically a lot of that magic could be
removed.

Regards,
Armin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problems with the new super()

[Guido van Rossum]

The staticmethod thing isn't new; that's also the case in 2.x.

The super() thing is a case of practicality beats purity. Note that
you pay a small but measurable cost for the implicit __class__ (it's
implemented as a "cell variable", the same mechanism used for nested
scopes) so we wouldn't want to introduce it unless it is used.

On Wed, Apr 30, 2008 at 2:46 PM, Armin Ronacher
<[EMAIL PROTECTED]> wrote:
> Hi all,
>
>  I blogged about that topic today which turned out to be a very bad idea,
>  so I summarize it for the mailinglist here to hopefully start a discussion
>  about the topic, which I think is rather important.
>
>  In the last weeks something remarkable happened in the Python3 sources:
>  self kinda became implicit.  Not in function definitions, but in super
>  calls.  But not only self: also the class passed to `super`.  That's
>  interesting because it means that the language shifts into a completely
>  different direction.
>
>  `super` was rarely used in the past, mainly because it was weird to use.
>  In the most common use case the current class and the current instance
>  where passed to it, and the super typed returned looked up the parent
>  methods on the MRO for you.  It was useful for multiple inheritance and
>  mixin classes that don't know their parent but confusing for many.  I can
>  see that a replacement is a good idea, but I'm not so sure if the current
>  implementation is the way to go.
>
>  The main problem with replacing `super(Foo, self).bar()` with something
>  like `super.bar()` is obviously that self is explicit and the class (in
>  that case Foo) can't be determined by the caller.  Furthermore the Python
>  principle was always against functions doing stack introspection to find
>  the caller.  There are few examples in the stdlib or builtins that do
>  some sort of caller introspection.  Those are the special functions
>  `vars`, `locals`, `gloabal`, `vars` and some functions in the inspect
>  module.  And all of them do nothing more than getting the current frame
>  and accessing the dict of locals or globals.  What super in current
>  Python 3 builds does goes way beyond that.
>
>  The implementation of the automatic super currently has two ugly details
>  that I think violate the Python Zen:  The bytecode generated is differently
>  if the name "super" is used in the function.  `__class__` is only added as
>  cell to the code if `super` or `__class__` is referenced.  That and the fact
>  that `f_localsplus` is completely unavailable from within python makes the
>  whole process appear magical.
>
>  This is way more magical than anything we've had in Python in the past and
>  just doesn't fit into the language in my opinion.  We do have an explicit
>  self in methods and methods are more or less just functions.  Python's
>  methods are functions, just that a descriptor puts a method object around
>  it to pass the self as first arguments.  That's an incredible cool thing
>  to have and makes things very simple and non-magical.  Breaking that
>  principle by introducing an automatic super seems to harm the concept.
>
>  Another odd thing is that Python 3 starts keeping information on the C
>  layer we can't access from within Python.  Super is one example, another
>  good one are methods.  They don't have a descriptor that wraps them if
>  they are accessed via their classes.  This as such is not a problem as you
>  can call them the same (just that you can call them with completely
>  different receivers now) but it becomes a problem if some of the functions
>  are marked as staticmethods.  Then they look completely the same when
>  looking at them from a classes perspective:
>
>  |   >>> class C:
>  |   ...  normal = lambda x: None
>  |   ...  static = staticmethod(lambda x: None)
>  |   ...
>  |   >>> type(C.normal) is type(C.static)
>  |   True
>  |   >>> C.normal
>  |at 0x4da150>
>
>  As far as I can see a documentation tool has no chance to keep them apart
>  even though they are completely different on an instance:
>
>  |   >>> type(C().normal) is type(C().static)
>  |   False
>  |   >>> C().normal
>  |of <__main__.C object at 0x4dbcf0>>
>  |   >>> C().static
>  |at 0x4da198>
>
>  While I don't knwo about the method thing, I think an automatic super should
>  at least be implementable from within Python.  I could imagine that by adding
>  __class__ and __self__ to scopes automatically a lot of that magic could be
>  removed.
>
>  Regards,
>  Armin
>
>  ___
>  Python-Dev mailing list
>  Python-Dev@python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/o

[Python-Dev] shal we redefine "module" and "package"?

[zooko]

Folks:

Here's an experiment you can perform.  Round up a Python programmer  
and ask him the following three questions:


Q1.  You type "import foo" and it works.  What kind of thing is foo?

Q2.  You go to the Python package index and download something named  
"bar-1.0.0.tar.gz".  What kind of thing is bar?


Q3.  What is a "distribution"?

I'm willing to bet that you will get the following answers:

A1.  foo is a module.

A2.  bar is a package.

A3.  A distribution is a version of Linux that comes with a lot of  
Free Software.



Unfortunately these answers aren't quite right.  A "package" is  
actually a directory containing an __init__.py file, and a  
distribution is actually what you think of when you say "package" --  
a reusable package of Python code that you can, for example, get from  
the Python package index.


Educational efforts such as the Python tutorial and the distutils  
docs have not succeeded in training Python programmers to understand  
the terminology for these things as used by the Python implementors,  
so perhaps instead the implementors should start using the  
terminology understood by the programmers:


1.  A "module" shall henceforth be the name for either a foo.py file  
(a single-file module), or a directory with an __init__.py in it (a  
directory module).


2.  A "package" shall henceforth be the name of the thing that is  
currently called a "distribution".



Regards,

Zooko

who doesn't mind stirring up trouble on occasion...

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] shal we redefine "module" and "package"?

[Oleg Broytmann]

On Wed, Apr 30, 2008 at 04:21:13PM -0600, zooko wrote:
> so perhaps instead the implementors should start using the  
> terminology understood by the programmers:
> 
> 1.  A "module" shall henceforth be the name for either a foo.py file  
> (a single-file module), or a directory with an __init__.py in it (a  
> directory module).
> 
> 2.  A "package" shall henceforth be the name of the thing that is  
> currently called a "distribution".

   And while we are at it, I'd like to propose to rename the very language
because everybody knows that python is a huge four-legged poisonous lizard.
(Sorry for being so poisonously [pythonic?] sarcastic. Just cannot resist.)

Oleg.
-- 
 Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED]
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] shal we redefine "module" and "package"?

[Steve Holden]

zooko wrote:

Folks:

Here's an experiment you can perform.  Round up a Python programmer and 
ask him the following three questions:


Q1.  You type "import foo" and it works.  What kind of thing is foo?

Q2.  You go to the Python package index and download something named 
"bar-1.0.0.tar.gz".  What kind of thing is bar?


Q3.  What is a "distribution"?

I'm willing to bet that you will get the following answers:

A1.  foo is a module.

A2.  bar is a package.

A3.  A distribution is a version of Linux that comes with a lot of Free 
Software.



Unfortunately these answers aren't quite right.  A "package" is actually 
a directory containing an __init__.py file, and a distribution is 
actually what you think of when you say "package" -- a reusable package 
of Python code that you can, for example, get from the Python package 
index.


Educational efforts such as the Python tutorial and the distutils docs 
have not succeeded in training Python programmers to understand the 
terminology for these things as used by the Python implementors, so 
perhaps instead the implementors should start using the terminology 
understood by the programmers:


1.  A "module" shall henceforth be the name for either a foo.py file (a 
single-file module), or a directory with an __init__.py in it (a 
directory module).


2.  A "package" shall henceforth be the name of the thing that is 
currently called a "distribution".



-1



Zooko

who doesn't mind stirring up trouble on occasion...


regards
 Steve

who doesn't mind trouble but would rather see communications improve
--
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] shal we redefine "module" and "package"?

[Nick Coghlan]

zooko wrote:

Folks:

Here's an experiment you can perform.  Round up a Python programmer and 
ask him the following three questions:


Q1.  You type "import foo" and it works.  What kind of thing is foo?


foo is a package or a module. Not enough information is provide here to 
say which.


Q2.  You go to the Python package index and download something named 
"bar-1.0.0.tar.gz".  What kind of thing is bar?


bar is a package, a module, or a collection of a number of these things. 
Again, not enough information is provided in the question to say which 
is actually the case.


Cheers,
Nick.

P.S. People, including programmers, are often sloppy with terminology. 
This is not a new problem.


--
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] shal we redefine "module" and "package"?

[glyph]

On 10:53 pm, [EMAIL PROTECTED] wrote:

zooko wrote:


Unfortunately these answers aren't quite right.  A "package" is 
actually a directory containing an __init__.py file, and a 
distribution is actually what you think of when you say "package" -- a 
reusable package of Python code that you can, for example, get from 
the Python package index.


The fact that the Python "package" index is a place where you get 
"distributions" does seem a bit weird to me, but this is not necessarily 
a problem that can be fixed.  Ambiguity is part of human language. 
Perhaps suggest transliterations of these terms for talking about python 
in lojban? :)
1.  A "module" shall henceforth be the name for either a foo.py file 
(a single-file module), or a directory with an __init__.py in it (a 
directory module).


I have a less disruptive counterproposal.  How about just starting to 
refer to directories (or "folders", or zip entries) with '__init__.py' 
in them as "package modules"?  A package is-a module anyway.
2.  A "package" shall henceforth be the name of the thing that is 
currently called a "distribution".


I belive a multi-word term here would be similarly more memorable and 
precise.  A "package distribution" would include the more familiar term 
while still being specific, consistent with the old terminology, and 
correct.  Using a qualifying word is probably a good idea in this 
context anyway.  I usually say "debian package", "RPM", "MSI", or 
"tarball" unless I'm specifically talking about "packages for your 
platform", almost always in the phrase, "please do not use distutils to 
do a system install of Twisted, use the specific package for your 
platform".

-1


I do, however, agree with Steve emphatically on your original proposal. 
Changing the terminology now will make billions upon billions of Python 
web pages, modules (c.f. 
twisted.python.modules.PythonModule.isPackage()) documents, and 
searchable message archives obsolete, not to mention that 90% of the 
community will probably ignore you and use the old terminology anyway, 
creating more confusion than it eliminates.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] shal we redefine "module" and "package"?

[Terry Reedy]

"zooko" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
| Folks:
| Unfortunately these answers aren't quite right.  A "package" is
| actually a directory containing an __init__.py file, and a
| distribution is actually what you think of when you say "package" --
| a reusable package of Python code that you can, for example, get from
| the Python package index.

If I go to a market, I may buy an individual food item (an apple, for 
instance), but most foods come in packages.  The items and packages are 
them put in distribution carriers (bags) for transport home, where they are 
unbagged.  A .tgz file is a bag, not a package.  Since I am buying food, 
not bags, I would not want the store called a distribution/bag store. 
Indeed, other types of stores use the same or similar distribution 
carriers, just as .tgz is used for things others than Python modules and 
packages. 



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] shal we redefine "module" and "package"?

[Ben Finney]

zooko <[EMAIL PROTECTED]> writes:

> I'm willing to bet that you will get the following answers:
> 
> A1.  foo [from 'import foo'] is a module.
> 
> A2.  bar [of 'bar-1.2.3.tar.gz'] is a package.
> 
> A3.  A distribution is a version of Linux that comes with a lot of
> Free Software.
> 
> 
> Unfortunately these answers aren't quite right.

More accurately, they're right in the context from which the speaker
learned them.

> A "package" is actually a directory containing an __init__.py file,
> and a distribution is actually what you think of when you say
> "package" -- a reusable package of Python code that you can, for
> example, get from the Python package index.

Only within Python's terminology. This, of course, conflicts with the
meanings that adhere to those words just about everywhere else on the
user's operating system.

> Educational efforts such as the Python tutorial and the distutils
> docs have not succeeded in training Python programmers to understand
> the terminology for these things as used by the Python implementors,

More accurately, the meanings you list in the hypothetical responses
above are entrenched and more useful for general use.

> so perhaps instead the implementors should start using the
> terminology understood by the programmers:

In principle this would be good. However:

> 1.  A "module" shall henceforth be the name for either a foo.py file
> (a single-file module), or a directory with an __init__.py in it (a
> directory module).

How then to distinguish between these? They're clearly separate
concepts, and I think they need distinct terms.

> 2.  A "package" shall henceforth be the name of the thing that is
> currently [in Python terminology] called a "distribution".

That one would be good.

You then have the converse problem of changing the use of terminology
that is currently entrenched in existing Python documentation and
minds :-)

-- 
 \"Always code as if the guy who ends up maintaining your code |
  `\ will be a violent psychopath who knows where you live." —John |
_o__) F. Woods |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com