Re: [Cython] Strange cimport behaviour

2011-07-21 Thread Robert Bradshaw
On Wed, Jul 20, 2011 at 10:30 PM, Robert Bradshaw
 wrote:
> On Wed, Jul 20, 2011 at 9:40 PM, Vitja Makarov  
> wrote:
>> 2011/7/21 Robert Bradshaw :
>>> On Wed, Jul 20, 2011 at 9:27 PM, Vitja Makarov  
>>> wrote:
 2011/7/16 Lars Buitinck :
> 2011/7/15 Vitja Makarov :
>> I've found strange bug. In my example cimport misses "fcntl.h" include:
>>
>> ((c2f2e12...)) vitja@vitja-laptop:~/work/cython-vitek/zzz$ cat ./fff.pyx
>> cimport posix.unistd
>> cimport posix.fcntl
>>
>> print posix.fcntl.O_RDWR
>
> Fascinating; I can reproduce the error (on Scientific Linux 5.5), but
> it goes away when I reverse the order of the includes.
>

 What's about this issue? Is that easy to fix before release or it's
 better to create a ticket?
>>>
>>> I'm not sure what we can do here--we make sure to emit the #include
>>> statements in the same order as they are encountered in the Cython
>>> sources because C is sensitive to this kind of thing, but we can't
>>> really "fix" C. I suppose we could cimport posix.fcntl from within
>>> posix.unistd to force an ordering.
>>>
>>
>> That's not C bug. That's cython bug. Actualy fcntl.h isn't included at all.
>> That seems to be a problem with cimport and packages.
>>
>> As I said "from posix cimport unistd, fcntl" works just fine.
>
> I thought you meant that
>
>    cimport posix.fcntl
>    cimport posix.unistd
>
> worked but
>
>    cimport posix.unistd
>    cimport posix.fcntl
>
> didn't. I'm seeing the bug now. (There are of course cases where one
> does care about import order.)

https://github.com/cython/cython/commit/55d5e576935d83c6bdadc593e36793aecffe0bae
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Strange cimport behaviour

2011-07-21 Thread Vitja Makarov
2011/7/21 Robert Bradshaw :
> On Wed, Jul 20, 2011 at 10:30 PM, Robert Bradshaw
>  wrote:
>> On Wed, Jul 20, 2011 at 9:40 PM, Vitja Makarov  
>> wrote:
>>> 2011/7/21 Robert Bradshaw :
 On Wed, Jul 20, 2011 at 9:27 PM, Vitja Makarov  
 wrote:
> 2011/7/16 Lars Buitinck :
>> 2011/7/15 Vitja Makarov :
>>> I've found strange bug. In my example cimport misses "fcntl.h" include:
>>>
>>> ((c2f2e12...)) vitja@vitja-laptop:~/work/cython-vitek/zzz$ cat ./fff.pyx
>>> cimport posix.unistd
>>> cimport posix.fcntl
>>>
>>> print posix.fcntl.O_RDWR
>>
>> Fascinating; I can reproduce the error (on Scientific Linux 5.5), but
>> it goes away when I reverse the order of the includes.
>>
>
> What's about this issue? Is that easy to fix before release or it's
> better to create a ticket?

 I'm not sure what we can do here--we make sure to emit the #include
 statements in the same order as they are encountered in the Cython
 sources because C is sensitive to this kind of thing, but we can't
 really "fix" C. I suppose we could cimport posix.fcntl from within
 posix.unistd to force an ordering.

>>>
>>> That's not C bug. That's cython bug. Actualy fcntl.h isn't included at all.
>>> That seems to be a problem with cimport and packages.
>>>
>>> As I said "from posix cimport unistd, fcntl" works just fine.
>>
>> I thought you meant that
>>
>>    cimport posix.fcntl
>>    cimport posix.unistd
>>
>> worked but
>>
>>    cimport posix.unistd
>>    cimport posix.fcntl
>>
>> didn't. I'm seeing the bug now. (There are of course cases where one
>> does care about import order.)
>
> https://github.com/cython/cython/commit/55d5e576935d83c6bdadc593e36793aecffe0bae


Cool, thanks!


-- 
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] runtests libgomp fail

2011-07-21 Thread Robert Bradshaw
On Thu, Jul 14, 2011 at 4:24 AM, mark florisson
 wrote:
> I added a test for nested parallelism with exceptions (with OpenMP
> nesting explicitly enabled), however if libgomp cannot create more
> threads it exits the process with exit status 1 and the message
> "libgomp: Thread creation failed: Resource temporarily unavailable".
> This then results in a red Hudson and I think it exits the testrunner
> (are the tests not run as separate processes?). So what should I do?
> Should I fork manually in my test and watch the exit status?
>
> The result is here:
> https://sage.math.washington.edu:8091/hudson/view/cython-mark/job/cython-mark-tests-py27-c/72/console

Yes, I think it's worth forking and checking the exit code in this
case so we can continue on. That's not to say we shouldn't see if we
can't solve the underlying flakiness.

- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] PEP 3135 -- New Super

2011-07-21 Thread Robert Bradshaw
On Thu, Jul 7, 2011 at 2:13 PM, Vitja Makarov  wrote:
>
>
> 2011/7/8 mark florisson 
>>
>> On 7 July 2011 22:39, Vitja Makarov  wrote:
>> >
>> >
>> > 2011/7/8 mark florisson 
>> >>
>> >> On 7 July 2011 22:15, Vitja Makarov  wrote:
>> >> >
>> >> >
>> >> > 2011/7/7 mark florisson 
>> >> >>
>> >> >> On 6 July 2011 10:01, Vitja Makarov  wrote:
>> >> >> > 2011/7/6 Stefan Behnel :
>> >> >> >> Vitja Makarov, 06.07.2011 09:05:
>> >> >> >>>
>> >> >> >>> 2011/7/6 Stefan Behnel:
>> >> >> 
>> >> >>  Stefan Behnel, 05.07.2011 10:04:
>> >> >> >
>> >> >> > Vitja Makarov, 05.07.2011 09:17:
>> >> >> >>
>> >> >> >> 2011/7/5 Stefan Behnel:
>> >> >> >>>
>> >> >> >>> Vitja Makarov, 05.07.2011 08:21:
>> >> >> 
>> >> >>  I was thinking about implementing new super() with no
>> >> >>  arguments.
>> >> >> >>>
>> >> >> >>> http://trac.cython.org/cython_trac/ticket/696
>> >> >> >>>
>> >> >>  The problem is where to store __class__, I see two options
>> >> >>  here:
>> >> >> 
>> >> >>  1. Add func_class member to CyFunction, this way __class__
>> >> >>  will
>> >> >>  be
>> >> >>  private and not visible for inner functions:
>> >> >>  2. Put it into closure
>> >> >> >>>
>> >> >> >>> The second option has the advantage of requiring the field
>> >> >> >>> only
>> >> >> >>> when
>> >> >> >>> super()
>> >> >> >>> is used, whereas the first impacts all functions.
>> >> >> >>>
>> >> >> >>> I would expect that programs commonly have a lot more
>> >> >> >>> functions
>> >> >> >>> than
>> >> >> >>> specifically methods that use a no-argument call to super(),
>> >> >> >>> so
>> >> >> >>> this
>> >> >> >>> may
>> >> >> >>> make a difference.
>> >> >> >>>
>> >> >> >>
>> >> >> >> So, now classes are created the following way:
>> >> >> >>
>> >> >> >> class_dict = {}
>> >> >> >> class_dict.foo = foo_func
>> >> >> >> class = CreateClass(class_dict)
>> >> >> >>
>> >> >> >> So after class is created I should check its dict for
>> >> >> >> CyFunction
>> >> >> >> members (maybe only ones that actually require __class__)
>> >> >> >> and set __class__:
>> >> >> >>
>> >> >> >> for value in class.__dict__.itervalues():
>> >> >> >> if isinstance(value, CyFunction) and value.func_class is
>> >> >> >> WantClass:
>> >> >> >> value.func_class = class
>> >> >> >>
>> >> >> >> Btw, first way requires cyfunction signature change, it would
>> >> >> >> accept
>> >> >> >> cyfunction object as first argument.
>> >> >> >
>> >> >> > We currently pass the binding (i.e. owning) object, right?
>> >> >> 
>> >> >>  So, how would this work for methods? We need to pass the 'self'
>> >> >>  object
>> >> >>  there, which the CyFunction doesn't know. If anything, it only
>> >> >>  knows
>> >> >>  the
>> >> >>  class it was defined in, which doesn't help here.
>> >> >> >>>
>> >> >> >>> From PEP: "super() is equivalent to:
>> >> >> >>> super(__class__,)"
>> >> >> >>
>> >> >> >> I wasn't speaking of super(). What I meant, was: how do we pass
>> >> >> >> 'self'
>> >> >> >> when
>> >> >> >> we pass the CyFunction object as the first argument?
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > Oh, ok. Now we pass closure or nothing in self. So method's self
>> >> >> > is
>> >> >> > passed via tuple.
>> >> >> > Instancemethod do this for us. Now CyFucntion uses
>> >> >> > PyCFunction_Call
>> >> >> > we
>> >> >> > can override this and change signature of cyfunction to:
>> >> >> >
>> >> >> > PyObject func(CyFunction *func, PyObject *self, PyObject *args,
>> >> >> > PyObject *kwargs);
>> >> >> >
>> >> >> > This way we should implement new instancemethod type.
>> >> >>
>> >> >> Would it be easier to make scope objects attributes of functions?
>> >> >> Then
>> >> >> you could still utilize PyCFunction_Call without needing to check
>> >> >> the
>> >> >> argument flags and such right?
>> >> >>
>> >> >
>> >> > Sure, scope object is already functions attribute now it's stored
>> >> > inside
>> >> > self.
>> >> > But, instancemethods desc_get builds new args tuple with self as
>> >> > first
>> >> > element.
>> >> > We can implement cython version of instance method and change method
>> >> > signature a little bit.
>> >>
>> >> Right, and you pass it in as the first argument to the C function.
>> >> However, if you want to change the signature, you have to override
>> >> PyCFunction_Call, which, if I'm not mistaken, means you will have to
>> >> interpret the flags from the PyMethodDef and call the C function in
>> >> the right way (e.g. with or without the args tuple and kwargs) and
>> >> check if it's being called correctly. If you leave it as an attribute
>> >> of the function, you can pass in the function and access the scope
>> >> object from the function object in the closure C function. So I t

Re: [Cython] Strange cimport behaviour

2011-07-21 Thread Lars Buitinck
2011/7/21 Robert Bradshaw :
> On Wed, Jul 20, 2011 at 10:30 PM, Robert Bradshaw
>  wrote:
>> On Wed, Jul 20, 2011 at 9:40 PM, Vitja Makarov  
>> wrote:
>>> 2011/7/21 Robert Bradshaw :
 I'm not sure what we can do here--we make sure to emit the #include
 statements in the same order as they are encountered in the Cython
 sources because C is sensitive to this kind of thing, but we can't
 really "fix" C. I suppose we could cimport posix.fcntl from within
 posix.unistd to force an ordering.
>
> https://github.com/cython/cython/commit/55d5e576935d83c6bdadc593e36793aecffe0bae

I'm not familiar enough with the Cython internals to understand this
change; are #includes now generated in the C code in the order of the
corresponding cimports?

-- 
Lars Buitinck
Scientific programmer, ILPS
University of Amsterdam
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Strange cimport behaviour

2011-07-21 Thread Robert Bradshaw
On Thu, Jul 21, 2011 at 2:28 AM, Lars Buitinck  wrote:
> 2011/7/21 Robert Bradshaw :
>> On Wed, Jul 20, 2011 at 10:30 PM, Robert Bradshaw
>>  wrote:
>>> On Wed, Jul 20, 2011 at 9:40 PM, Vitja Makarov  
>>> wrote:
 2011/7/21 Robert Bradshaw :
> I'm not sure what we can do here--we make sure to emit the #include
> statements in the same order as they are encountered in the Cython
> sources because C is sensitive to this kind of thing, but we can't
> really "fix" C. I suppose we could cimport posix.fcntl from within
> posix.unistd to force an ordering.
>>
>> https://github.com/cython/cython/commit/55d5e576935d83c6bdadc593e36793aecffe0bae
>
> I'm not familiar enough with the Cython internals to understand this
> change; are #includes now generated in the C code in the order of the
> corresponding cimports?

Yes. Essentially, imagine every time you see a "cdef extern from
'header.h'" you add 'header.h' to your ordered list of #includes (if
it's not already there), and when you encounter a cimport, you "step
into" it and gather all the #includes it has before moving on.

- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Strange cimport behaviour

2011-07-21 Thread Stefan Behnel

Lars Buitinck, 21.07.2011 11:28:

2011/7/21 Robert Bradshaw:

https://github.com/cython/cython/commit/55d5e576935d83c6bdadc593e36793aecffe0bae


I'm not familiar enough with the Cython internals to understand this
change; are #includes now generated in the C code in the order of the
corresponding cimports?


That's been the case for several releases already. The change above only 
fixes a glitch where the #include was not generated at all.


Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] PEP 3135 -- New Super

2011-07-21 Thread Stefan Behnel

Robert Bradshaw, 21.07.2011 11:21:

On Thu, Jul 7, 2011 at 2:13 PM, Vitja Makarov wrote:

2011/7/8 mark florisson

In descr_get you create and return a new CyFunction with a __self__
set (note, not m_self) and in __call__ you put __self__ in the args
tuple and the function as m_self, and then call the CyFunction using
PyCFunction_Call. But copying and modifying PyCFunction_Call works
just as well I suppose :)


Yes, that would work) But I think CyFunction is too heavy for methods.
I've just realized that descr_get is called each time you access method,
each time you call it and so on.


True, but this is also the case for CPython methods and method
descriptors. There's also an inefficiency in calling bound methods in
that a new tuple needs to be created with the bound parameter as the
first component. Perhaps we could do away with that as well with
specialized unpacking code. It would simultaneously be a step forward
and backwards in terms of compatibility, but we could do away with
ever generating PyCFunctions, methods, and method descriptors.


That sounds like a really cool optimisation path. We could get rid of one 
indirection by replacing tp_call (originally PyCFunction_Call) directly by 
the real function entry point, i.e. the Python function wrapper (assuming 
the DefNode refactoring has taken place by then).


It would also (somewhat) simplify the argument unpacking code, as that 
would no longer have to care about the METH_O and METH_NOARGS special 
cases. They'd be handled naturally inside of the Python call wrapper.




Perhaps
a CEP is in order to nail down the behavior of the CyFunction object.


+1, totally helps in figuring out the details.

Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] PEP 3135 -- New Super

2011-07-21 Thread Vitja Makarov
2011/7/21 Stefan Behnel :
> Robert Bradshaw, 21.07.2011 11:21:
>>
>> On Thu, Jul 7, 2011 at 2:13 PM, Vitja Makarov wrote:
>>>
>>> 2011/7/8 mark florisson

 In descr_get you create and return a new CyFunction with a __self__
 set (note, not m_self) and in __call__ you put __self__ in the args
 tuple and the function as m_self, and then call the CyFunction using
 PyCFunction_Call. But copying and modifying PyCFunction_Call works
 just as well I suppose :)
>>>
>>> Yes, that would work) But I think CyFunction is too heavy for methods.
>>> I've just realized that descr_get is called each time you access method,
>>> each time you call it and so on.
>>
>> True, but this is also the case for CPython methods and method
>> descriptors. There's also an inefficiency in calling bound methods in
>> that a new tuple needs to be created with the bound parameter as the
>> first component. Perhaps we could do away with that as well with
>> specialized unpacking code. It would simultaneously be a step forward
>> and backwards in terms of compatibility, but we could do away with
>> ever generating PyCFunctions, methods, and method descriptors.
>

Yeah, I was thinking about that too.

> That sounds like a really cool optimisation path. We could get rid of one
> indirection by replacing tp_call (originally PyCFunction_Call) directly by
> the real function entry point, i.e. the Python function wrapper (assuming
> the DefNode refactoring has taken place by then).
>

+1 I think DefNode refactoring should be done first (Did we froget about that?)

> It would also (somewhat) simplify the argument unpacking code, as that would
> no longer have to care about the METH_O and METH_NOARGS special cases.
> They'd be handled naturally inside of the Python call wrapper.
>
>
>> Perhaps
>> a CEP is in order to nail down the behavior of the CyFunction object.
>
> +1, totally helps in figuring out the details.

CEP for refactored DefNode would be a good idea too.

As I rember the idea is to turn def node into cdef node with wrapper
call function just like cpdef buf without C-level access.
So here we have to decide how to how to match python function's
signature and cdef one.


-- 
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Cython 0.15 release

2011-07-21 Thread Robert Bradshaw
Stefan,

Are the lxml failures at
https://sage.math.washington.edu:8091/hudson/job/cython-devel-lxml-trunk-py27/
expected?

- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


[Cython] Cython 0.15 release candidate

2011-07-21 Thread Robert Bradshaw
Cython has seen an enormous amount of development since 0.14.1. If you
are not already using the latest version from the development
repository, we encourage you to try out the release candidate:
http://cython.org/release/Cython-0.15rc0.tar.gz

- Robert
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] [cython-users] Cython 0.15 release candidate

2011-07-21 Thread Robert Bradshaw
On Thu, Jul 21, 2011 at 4:24 PM, 최원준  wrote:
> what is the new feature of cython 0.15?

The release notes are still pretty bare-bones, but see:
http://wiki.cython.org/ReleaseNotes-0.15

> 2011/7/22 Robert Bradshaw 
>>
>> Cython has seen an enormous amount of development since 0.14.1. If you
>> are not already using the latest version from the development
>> repository, we encourage you to try out the release candidate:
>> http://cython.org/release/Cython-0.15rc0.tar.gz
>>
>> - Robert
>
>
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Cython 0.15 release candidate

2011-07-21 Thread Christoph Gohlke



On 7/21/2011 4:14 PM, Robert Bradshaw wrote:

Cython has seen an enormous amount of development since 0.14.1. If you
are not already using the latest version from the development
repository, we encourage you to try out the release candidate:
http://cython.org/release/Cython-0.15rc0.tar.gz

- Robert


Hi,

I get a test error on Windows with msvc9 due to the "with gil" statement 
producing invalid C code (at least for msvc9). Below is the offending 
code in with_gil.c. Moving the declaration of the __pyx_gilstate_save 
variable a couple of lines up fixes this. A patch is attached.


Christoph


/* "with_gil.pyx":347
 * raise Exception("This will be overridden")
 * finally:
 * with gil: # <<
 * raise Exception("Override exception!")
 *
 */
/*finally:*/ {
  int __pyx_why;
  PyObject *__pyx_exc_type, *__pyx_exc_value, *__pyx_exc_tb;
  int __pyx_exc_lineno;
  __pyx_exc_type = 0; __pyx_exc_value = 0; __pyx_exc_tb = 0; 
__pyx_exc_lineno = 0;

  #ifdef WITH_THREAD
  PyGILState_STATE __pyx_gilstate_save;
  #endif
  __pyx_why = 0; goto __pyx_L10;
diff --git a/Cython/Compiler/Nodes.py b/Cython/Compiler/Nodes.py
index 2d39240..2312e8d 100644
--- a/Cython/Compiler/Nodes.py
+++ b/Cython/Compiler/Nodes.py
@@ -5414,15 +5414,15 @@ class TryFinallyStatNode(StatNode):
 code.putln("int __pyx_why;")
 
 if error_label_used and self.preserve_exception:
+if self.is_try_finally_in_nogil:
+code.declare_gilstate()
+
 code.putln("PyObject *%s, *%s, *%s;" % Naming.exc_vars)
 code.putln("int %s;" % Naming.exc_lineno_name)
 exc_var_init_zero = ''.join(
 ["%s = 0; " % var for var in Naming.exc_vars])
 exc_var_init_zero += '%s = 0;' % Naming.exc_lineno_name
 code.putln(exc_var_init_zero)
-
-if self.is_try_finally_in_nogil:
-code.declare_gilstate()
 else:
 exc_var_init_zero = None
 
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Cython 0.15 release candidate

2011-07-21 Thread Lisandro Dalcin
On 21 July 2011 23:43, Christoph Gohlke  wrote:
>
>
> On 7/21/2011 4:14 PM, Robert Bradshaw wrote:
>>
>> Cython has seen an enormous amount of development since 0.14.1. If you
>> are not already using the latest version from the development
>> repository, we encourage you to try out the release candidate:
>> http://cython.org/release/Cython-0.15rc0.tar.gz
>>
>> - Robert
>
> Hi,
>
> I get a test error on Windows with msvc9 due to the "with gil" statement
> producing invalid C code (at least for msvc9). Below is the offending code
> in with_gil.c. Moving the declaration of the __pyx_gilstate_save variable a
> couple of lines up fixes this. A patch is attached.
>
> Christoph
>
>
>        /* "with_gil.pyx":347
>  *                 raise Exception("This will be overridden")
>  *         finally:
>  *             with gil:             # <<
>  *                 raise Exception("Override exception!")
>  *
>  */
>        /*finally:*/ {
>          int __pyx_why;
>          PyObject *__pyx_exc_type, *__pyx_exc_value, *__pyx_exc_tb;
>          int __pyx_exc_lineno;
>          __pyx_exc_type = 0; __pyx_exc_value = 0; __pyx_exc_tb = 0;
> __pyx_exc_lineno = 0;
>          #ifdef WITH_THREAD
>          PyGILState_STATE __pyx_gilstate_save;
>          #endif
>          __pyx_why = 0; goto __pyx_L10;
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
>

Your patch is OK. However now I'm wondering why not to initialize the
exc variables to 0 directly on declaration, like this:

PyObject *__pyx_exc_type = 0, *__pyx_exc_value =0, *__pyx_exc_tb = 0;

(anyway, I still prefer declaring the gilstate variable first)

-- 
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] Cython 0.15 release

2011-07-21 Thread Stefan Behnel

Robert Bradshaw, 22.07.2011 01:00:

Are the lxml failures at
https://sage.math.washington.edu:8091/hudson/job/cython-devel-lxml-trunk-py27/
expected?


Yes, the "test_xmlschema_import_file" test keeps crashing with a high 
probability. Some kind of threading related bug, no idea how to reproduce 
it in a debuggable way...


Just ignore those two jobs for the time being. I keep looking at the tests 
from time to time, because it's not *always* that bug that kills them.


Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] PEP 3135 -- New Super

2011-07-21 Thread Stefan Behnel

Vitja Makarov, 21.07.2011 20:57:

2011/7/21 Stefan Behnel:

Robert Bradshaw, 21.07.2011 11:21:


On Thu, Jul 7, 2011 at 2:13 PM, Vitja Makarov wrote:


2011/7/8 mark florisson


In descr_get you create and return a new CyFunction with a __self__
set (note, not m_self) and in __call__ you put __self__ in the args
tuple and the function as m_self, and then call the CyFunction using
PyCFunction_Call. But copying and modifying PyCFunction_Call works
just as well I suppose :)


Yes, that would work) But I think CyFunction is too heavy for methods.
I've just realized that descr_get is called each time you access method,
each time you call it and so on.


True, but this is also the case for CPython methods and method
descriptors. There's also an inefficiency in calling bound methods in
that a new tuple needs to be created with the bound parameter as the
first component. Perhaps we could do away with that as well with
specialized unpacking code. It would simultaneously be a step forward
and backwards in terms of compatibility, but we could do away with
ever generating PyCFunctions, methods, and method descriptors.




Yeah, I was thinking about that too.


That sounds like a really cool optimisation path. We could get rid of one
indirection by replacing tp_call (originally PyCFunction_Call) directly by
the real function entry point, i.e. the Python function wrapper (assuming
the DefNode refactoring has taken place by then).



+1 I think DefNode refactoring should be done first (Did we froget about that?)


It would also (somewhat) simplify the argument unpacking code, as that would
no longer have to care about the METH_O and METH_NOARGS special cases.
They'd be handled naturally inside of the Python call wrapper.



Perhaps
a CEP is in order to nail down the behavior of the CyFunction object.


+1, totally helps in figuring out the details.


CEP for refactored DefNode would be a good idea too.


Well, there's still the part I wrote for the generator CEP:

http://wiki.cython.org/enhancements/generators#Pythonfunctionrefactoring



As I rember the idea is to turn def node into cdef node with wrapper
call function just like cpdef buf without C-level access.
So here we have to decide how to how to match python function's
signature and cdef one.


I think that's mostly trivial - the argument unpacking code already does 
the mapping for us.


Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] hg-git shows 25 heads?

2011-07-21 Thread Stefan Behnel

Pauli Virtanen, 07.07.2011 21:06:

On Thu, 07 Jul 2011 20:10:30 +0200, Dag Sverre Seljebotn wrote:
[clip]

5 minutes of Googling didn't turn up anything... Do you think it would
be a bad idea to simply delete these branches, something like

git push origin :refs/*


That drops *all* branches -- not recommended :)

As far as I see,

git push origin :refs/pull/*

should not break anything. However, depending on how the stuff is set
up on the Github side, either the merge button stops working (would be
an improvement to the state of matters, IMHO), or alternatively,
the branches come back immediately whenever someone views pull requests.

So I don't think doing this would help much. Maybe best to ask the Github
guys what the heck is their creation doing here, and try to see if hg-git
can be made to work properly.


I asked and they already fixed it. There isn't a release yet, but the 
latest hg-git sources work nicely.


Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel


Re: [Cython] PEP 3135 -- New Super

2011-07-21 Thread Vitja Makarov
2011/7/22 Stefan Behnel :
> Vitja Makarov, 21.07.2011 20:57:
>>
>> 2011/7/21 Stefan Behnel:
>>>
>>> Robert Bradshaw, 21.07.2011 11:21:

 On Thu, Jul 7, 2011 at 2:13 PM, Vitja Makarov wrote:
>
> 2011/7/8 mark florisson
>>
>> In descr_get you create and return a new CyFunction with a __self__
>> set (note, not m_self) and in __call__ you put __self__ in the args
>> tuple and the function as m_self, and then call the CyFunction using
>> PyCFunction_Call. But copying and modifying PyCFunction_Call works
>> just as well I suppose :)
>
> Yes, that would work) But I think CyFunction is too heavy for methods.
> I've just realized that descr_get is called each time you access
> method,
> each time you call it and so on.

 True, but this is also the case for CPython methods and method
 descriptors. There's also an inefficiency in calling bound methods in
 that a new tuple needs to be created with the bound parameter as the
 first component. Perhaps we could do away with that as well with
 specialized unpacking code. It would simultaneously be a step forward
 and backwards in terms of compatibility, but we could do away with
 ever generating PyCFunctions, methods, and method descriptors.
>>>
>>
>> Yeah, I was thinking about that too.
>>
>>> That sounds like a really cool optimisation path. We could get rid of one
>>> indirection by replacing tp_call (originally PyCFunction_Call) directly
>>> by
>>> the real function entry point, i.e. the Python function wrapper (assuming
>>> the DefNode refactoring has taken place by then).
>>>
>>
>> +1 I think DefNode refactoring should be done first (Did we froget about
>> that?)
>>
>>> It would also (somewhat) simplify the argument unpacking code, as that
>>> would
>>> no longer have to care about the METH_O and METH_NOARGS special cases.
>>> They'd be handled naturally inside of the Python call wrapper.
>>>
>>>
 Perhaps
 a CEP is in order to nail down the behavior of the CyFunction object.
>>>
>>> +1, totally helps in figuring out the details.
>>
>> CEP for refactored DefNode would be a good idea too.
>
> Well, there's still the part I wrote for the generator CEP:
>
> http://wiki.cython.org/enhancements/generators#Pythonfunctionrefactoring
>
>
>> As I rember the idea is to turn def node into cdef node with wrapper
>> call function just like cpdef buf without C-level access.
>> So here we have to decide how to how to match python function's
>> signature and cdef one.
>
> I think that's mostly trivial - the argument unpacking code already does the
> mapping for us.
>

So the wrapper could pass all the args (including defaults) to
underlaying C-function.
That solves the problem.

-- 
vitja.
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel