[Python-Dev] Builtin functions are magically static methods?

[Paul Sokolovsky]

Hello,

I looked into porting Python3 codecs module to MicroPython and saw
rather strange behavior, which is best illustrated with following
testcase:

==
def foo(a):
print("func:", a)

import _codecs
fun = _codecs.utf_8_encode
#fun = hash
#fun = str.upper
#fun = foo


class Bar:
meth = fun

print(fun)
print(fun("foo"))
b = Bar()
print(b.meth("bar"))
==

Uncommenting either _codecs.utf_8_encode or hash (both builtin
functions) produces 2 similar output lines, which in particular means
that its possible to call a native function as (normal) object method,
which then behaves as if it was a staticmethod - self is not passed to
a native function.

Using native object method in this manner produces error of self type
mismatch (TypeError: descriptor 'upper' for 'str' objects doesn't apply
to 'Bar' object).

And using standard Python function expectedly produces error about
argument number mismatch, because used as a method, function gets extra
self argument.

So the questions are:

1. How so, the native functions exhibit such magic behavior? Is it
documented somewhere - I never read or heard about that (cannot say I
read each and every word in Python reference docs, but read enough. As
an example, https://docs.python.org/3/library/stdtypes.html#functions
is rather short and mentions difference in implementation, not in
meta-behavior).

2. The main question: how to easily and cleanly achieve the same
behavior for standard Python functions? I'd think it's staticmethod(),
but:

>>> staticmethod(lambda:1)()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'staticmethod' object is not callable

Surprise.

(By "easily and cleanly" I mean without meta-programming tricks, like
instead of real arguments accept "*args, **kwargs" and then munge args).


Thanks,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MemoryError and other bugs on AMD64 OpenIndiana 3.x

[Jesus Cea]

Sorry, I am currently traveling. Away from real computer until April 7th.


On 28/03/15 10:48, Victor Stinner wrote:
> Hi,
> 
> The buildbot AMD64 OpenIndiana 3.x is always red since at least 6
> months. Almost always, tests fail because the buildbot has not enough
> memory. Well, it's fun to see how Python handles MemoryError (it's
> pretty good, there is no hard crash! my work on failmalloc was maybe
> useful ;-)), but it doesn't help to detect regressions.
> 
> Would it be possible to enhance this buildbot?

Those buildbots are hosted in OpenIndiana infraestructure. I take care
of the Solaris Zone (similar to docker but better :-)) but I can't
interact with the "real" machine. Sometimes changes in the "real"
machine had impacted us and nobody realized until some time later, when
nobody remembers what was changed or who did it.

I have contacted the machine manager and he has said to me that 16GB
were taken in the "/tmp" directory by buildbot user. He has deleted them
and everything should be fine now. Lets see.

Anyway if buildbot is leaving garbage behind we should take care of it.
Doing a daily cron for manual cleaning up seems a bit hacky :).

I would be interested, too, in getting an email when one of my buildbots
is red for more than, lets say, 4 days in a row. An email per day would
be fine.

BTW, buildbot pages are painfully slow. The manager had to "insist" and
be patient to see
,
etc.

How can that be improved?.

> I copy this email to the owner of buildbot. But I also sent it to
> python-dev because MemoryError is not the only error, please take care
> of our buildbots!

Thanks for keeping me in the loop. I hope everything is going green for
a while now :)

-- 
Jesús Cea Avión _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
Twitter: @jcea_/_/_/_/  _/_/_/_/_/
jabber / xmpp:j...@jabber.org  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Builtin functions are magically static methods?

[Mark Shannon]

On 29/03/15 19:16, Paul Sokolovsky wrote:

Hello,

I looked into porting Python3 codecs module to MicroPython and saw
rather strange behavior, which is best illustrated with following
testcase:

==
def foo(a):
 print("func:", a)

import _codecs
fun = _codecs.utf_8_encode
#fun = hash
#fun = str.upper
#fun = foo


class Bar:
 meth = fun

print(fun)
print(fun("foo"))
b = Bar()
print(b.meth("bar"))
==

Uncommenting either _codecs.utf_8_encode or hash (both builtin
functions) produces 2 similar output lines, which in particular means
that its possible to call a native function as (normal) object method,
which then behaves as if it was a staticmethod - self is not passed to
a native function.

Using native object method in this manner produces error of self type
mismatch (TypeError: descriptor 'upper' for 'str' objects doesn't apply
to 'Bar' object).

And using standard Python function expectedly produces error about
argument number mismatch, because used as a method, function gets extra
self argument.

So the questions are:

1. How so, the native functions exhibit such magic behavior? Is it
documented somewhere - I never read or heard about that (cannot say I
read each and every word in Python reference docs, but read enough. As
an example, https://docs.python.org/3/library/stdtypes.html#functions
is rather short and mentions difference in implementation, not in
meta-behavior).


In fact the "magic" is exhibited by Python functions, not by builtin 
ones. Python functions are descriptors, builtin functions are not.




2. The main question: how to easily and cleanly achieve the same
behavior for standard Python functions? I'd think it's staticmethod(),
but:



Write your own "BuiltinFunction" class which has the desired properties, 
ie. it would be callable, but not a descriptor.
Then write a "builtin_function" decorator to produce such an object from 
a function. The class and decorator could be the same object.


Personally, I think that such a class (plus a builtin function type that 
behaved like a Python function) would be a useful addition to the 
standard library. Modules do get converted from Python to C and vice-versa.



staticmethod(lambda:1)()

Traceback (most recent call last):
   File "", line 1, in 
TypeError: 'staticmethod' object is not callable

Surprise.

(By "easily and cleanly" I mean without meta-programming tricks, like
instead of real arguments accept "*args, **kwargs" and then munge args).


Thanks,
  Paul  mailto:pmis...@gmail.com


Cheers,
Mark
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Sporadic failures of test_subprocess and test_multiprocessing_spawn

[Victor Stinner]

Hi,

2015-03-28 12:26 GMT+01:00 Chris Angelico :
> It seems to be stalling out. I'm not sure exactly what's happening
> here. Running just that one file doesn't do it, but running the full
> test suite results in a stall.

Ok, I reproduced the issue on David's buildbot. A (child) process was
stuck in _Py_open(), function called from _close_open_fds_safe() which
is called to run a child process.

Calling _Py_open() is not safe here because the GIL may or may not be
held. After fork, the status of the GIL is unclear.

I fixed the issue by replacing _Py_open() with _Py_open_noraise(),
this version doesn't use the GIL (it doesn't release the GIL to call
open() and it doesn't raise an exception).
https://hg.python.org/cpython/rev/2e1234208bab

Thanks David for the buildbot.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Sporadic failures of test_subprocess and test_multiprocessing_spawn

[Victor Stinner]

Hi Serhiy,

2015-03-28 17:40 GMT+01:00 Serhiy Storchaka :
> Just run tests with low memory limit.
>
> (ulimit -v 6; ./python -m test.regrtest -uall -v
> test_multiprocessing_spawn;)
>
> test_io also hangs.

I confirm that some tests using threads hang under very low memory
condition. At bootstrap, Python doesn't handle correctly all
exceptions when starting a new thread. The "parent thread" waits until
the "child thread" completes, which never occurs because the thread
already completed.

I don't think that it was my regression, it probably exists since
Python 2 and maybe before. I'm not interested to touch this fragile
part of Python. It's maybe easy to fix the issue, I don't know.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #23752: When built from an existing file descriptor, io.FileIO() now only

[Serhiy Storchaka]

On 30.03.15 04:22, victor.stinner wrote:

https://hg.python.org/cpython/rev/bc2a22eaa0af
changeset:   95269:bc2a22eaa0af
user:Victor Stinner 
date:Mon Mar 30 03:21:06 2015 +0200
summary:
   Issue #23752: When built from an existing file descriptor, io.FileIO() now 
only
calls fstat() once. Before fstat() was called twice, which was not necessary.

files:
   Misc/NEWS|  26 ++
   Modules/_io/fileio.c |  24 
   2 files changed, 26 insertions(+), 24 deletions(-)


diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -2,6 +2,32 @@
  Python News
  +++

+What's New in Python 3.5.0 alpha 4?
+===


Return time machine back Victor. Current version is 3.5.0a2+.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com