Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
On 16 February 2011 03:11, Stefan Behnel wrote: > [forwarding to the list, from Yaroslav Halchenko] > > On Tue, 15 Feb 2011, Stefan Behnel wrote: >> >> Cool. That looks good. As you mentioned, it doesn't include the test >> runs yet, but it would be really great if you could get those >> running. > > ok - added testing during build, so now need to assure that everything > passes and all necessary dependencies are there. So, there is that > failing one I have mentioned before: > > ,--- > | == > | FAIL: test_embed (__main__.EmbedTest) > | -- > | Traceback (most recent call last): > | File "runtests.py", line 886, in test_embed > | "make PYTHON='%s' LIBDIR1='%s' test > make.output" % (sys.executable, > libdir)) == 0) > | AssertionError > `--- > > what could be done about it or should it be excluded? > I've pushed some fixes. Now this testcase should run from ancient Python 2.3 to head Python 3.2, both for static and sharedlib builds (but not in Windows). -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Fixing NumPy support for Python 3 (Stefan, please help!)
Stefan, what do you think about the patch below? This hunk is part of a series of fixes required to get numpy-dev working under Python 3.2. The root of the issue is that __cythonbufferdefaults__ keys&values end-up being "bytes" (this coercion is triggered in Interpreter.py). diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index 5b339da..b72deef 100755 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -12,6 +12,7 @@ cython.declare(error=object, warning=object, warn_once=object, Builtin=object, Symtab=object, Utils=object, find_coercion_error debug_disposal_code=object, debug_temp_alloc=object, debug_coerc +import sys import operator from Errors import error, warning, warn_once, InternalError, CompileError @@ -1136,6 +1137,8 @@ class StringNode(PyConstNode): return self.result_code def compile_time_value(self, env): +if sys.version_info[0] >= 3 and self.unicode_value: +return self.unicode_value return self.value -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On 17 February 2011 11:35, W. Trevor King wrote: > On Thu, Feb 17, 2011 at 10:53:15AM -0300, Lisandro Dalcin wrote: >> Cython could certainly support "cpdef struct", it is just a matter to >> define a proposal and find a contributor to implement it :-) > > Is there a CEP template (a la PEPs 9 and 12) that should be discussed > on the mailing list, or do I develop it free-form on the wiki [2]? > I would just develop it free-form on the wiki > p.s. should I be picking one of cython-dev@codespeak or > cython-devel@python? Is the shift not yet official? > Use cython-devel@python.org PS: Do we really need a full CEP for this? Do any of you object "cpdef struct" to automatically create a wrapper extension type for exposing to Python? I think all wee need to discuss is how to implement __cinit__(), and what to do with slots you want to use in C but cannot be mapped to a Python type (like pointers). -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] python 2.7/3.x and numpy-dev (Dag, I need a quick comment)
I'm working on a patch to get old, recent, and dev NumPy working in
2.7/3.x. So far, I had success, but I still have two failures like the
one pasted below.
Dag, could you elaborate a bit about the purpose of
__Pyx_BufFmt_CheckString() ? It is just a validity check for pep 3118
format strings? Do you expect the failure below to be hard to fix?
Just in case, the format string that triggers the failure is:
>>> memoryview(np.zeros((1,), dtype=np.dtype('b,i', align=False))).format
'T{b:f0:=i:f1:}'
==
FAIL: numpy_test ()
Doctest: numpy_test
--
Traceback (most recent call last):
File "/usr/local/python/3.2/lib/python3.2/doctest.py", line 2113, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for numpy_test
File "/u/dalcinl/Devel/Cython/cython/BUILD/run/c/numpy_test.cpython-32dm.so",
line 1, in numpy_test
--
File "/u/dalcinl/Devel/Cython/cython/BUILD/run/c/numpy_test.cpython-32dm.so",
line 155, in numpy_test
Failed example:
print(test_packed_align(np.zeros((1,), dtype=np.dtype('b,i', align=False
Exception raised:
Traceback (most recent call last):
File "/usr/local/python/3.2/lib/python3.2/doctest.py", line 1248, in __run
compileflags, 1), test.globs)
File "", line 1, in
print(test_packed_align(np.zeros((1,), dtype=np.dtype('b,i',
align=False
File "numpy_test.pyx", line 404, in numpy_test.test_packed_align
(numpy_test.c:6367)
ValueError: Buffer packing mode currently only allowed at
beginning of format string (this is a defect)
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
2011/2/18 Yaroslav Halchenko : > > awesome! I will test it asap -- just let me know if there would be > another commit for above or I should tune up $HOME ;) > https://github.com/cython/cython/commit/475e9a085654252d5a274dab2118b746e8bda4eb >> We are very near to be crystal clear... > > yeap > And now we should be crystal clear, finally! Could you confirm? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
2011/2/18 Yaroslav Halchenko : > > On Fri, 18 Feb 2011, Lisandro Dalcin wrote: >> > awesome! I will test it asap -- just let me know if there would be >> > another commit for above or I should tune up $HOME ;) >> https://github.com/cython/cython/commit/475e9a085654252d5a274dab2118b746e8bda4eb > > evil you -- and who would address: > Caller is responsible for deleting the directory when done with it. > ;-) > Well, I decided to not delete it just in case things go wrong and you need to check the output... I can easily fix that in a tearDown() method... Should I? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On 21 February 2011 15:26, Stefan Behnel wrote: > Robert Bradshaw, 21.02.2011 19:11: >> >> On Sun, Feb 20, 2011 at 1:26 AM, Stefan Behnel wrote: >>> >>> W. Trevor King, 20.02.2011 00:31: >>>> >>>> On Sat, Feb 19, 2011 at 02:04:16PM -0800, Robert Bradshaw wrote: >>>>> >>>>> On Sat, Feb 19, 2011 at 1:45 PM, W. Trevor King wrote: >>>>>> >>>>>> It is unclear to me what `cdef public struct` means. I think it >>>>>> should mean "Python bindings can alter this struct's definition", >>>>>> which doesn't make sense. >>>>> >>>>> I think it should mean "this struct is accessible from Python (as X)" >>>> >>>> Wouldn't that be "cdef readonly struct X"? >>> >>> The problem here is that "public" is used differently in different >>> contexts >>> - usually "exported at the C level" in this kind of context, with the >>> quirk >>> of meaning "modifiable at the Python level" for cdef class attributes. >>> >>> The potentially clearer "cpdef" means "overridable at the Python level" >>> as >>> well as "visible at the Python level", so it doesn't quite match the >>> second >>> meaning by itself. >>> >>> Structs won't be overridable at the Python level, I guess, so cpdef isn't >>> quite right. The intention here isn't to export them at the C level >>> either, >>> so "public" isn't right. >>> >>> We could tweak the "cpdef" meaning into "cdef thing mapped to the Python >>> level, and overridable if supported in the given context", which would >>> lead >>> to broader applicability. Then we could allow >> >> That's what I was thinking. >> >>> cpdef readonly struct ... >>> >>> I think that's a lot clearer than adding to the double meaning of >>> "public". >>> I would also prefer if Python accessible cdef class attributes were >>> defined >>> using "cpdef". We could potentially make that the preferred way in the >>> future? >> >> We could allow it, but -1 to disallowing "cdef class" > > With "preferred way", I was suggesting that we could *deprecate* > > cdef public int x > cdef readonly object y > > for cdef class properties in favour of > > cpdef int x > cpdef readonly object y > > and change the documentation accordingly etc., so that at least new users > get used to the new way. +1 > The old way would likely continue to be supported > until Cython 2.0 or so, for the holy cow's sake... > But with a deprecation warning, right? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] hasattr() swallows any exception (Py<3.2)
Take a look here: http://bugs.python.org/issue9666 'hasattr' default behaviour should be changed to suppress only AttributeError exceptions. Other should pass through. Should we do something about this in Cython? We currently use PyObject_HasAttr(), but even in Python 3.2 this is not the same as builtins.hasattr(), it swallows any exception (do you think this is a bug in core Python?). I'm inclined to fix the behavior for ALL Python versions to suppress only AttributeError. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] hasattr() swallows any exception (Py<3.2)
On 22 February 2011 18:21, Stefan Behnel wrote: > Lisandro Dalcin, 22.02.2011 21:41: >> >> I'm inclined to fix the behavior for ALL Python >> versions to suppress only AttributeError. > > How? Would you implement a hasattr() helper that uses PyObject_GetAttr() and > Does The Right Thing? > Yes, more or less the implementation of builtin_hasattr from bltinmodule.c from Py3.2 > In any case, I personally don't care so much about hasattr(), but I'm +1 for > Python compatibility, and +1 for getting the fixed Py3.2 behaviour in all > Python versions. There's a reason this was fixed in CPython, it's a pretty > clear bug. > OK, I'll go for it. Thanks. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] hasattr() swallows any exception (Py<3.2)
On 22 February 2011 19:09, Lisandro Dalcin wrote: > On 22 February 2011 18:21, Stefan Behnel wrote: >> Lisandro Dalcin, 22.02.2011 21:41: >>> >>> I'm inclined to fix the behavior for ALL Python >>> versions to suppress only AttributeError. >> >> How? Would you implement a hasattr() helper that uses PyObject_GetAttr() and >> Does The Right Thing? >> > > Yes, more or less the implementation of builtin_hasattr from > bltinmodule.c from Py3.2 > >> In any case, I personally don't care so much about hasattr(), but I'm +1 for >> Python compatibility, and +1 for getting the fixed Py3.2 behaviour in all >> Python versions. There's a reason this was fixed in CPython, it's a pretty >> clear bug. >> > > OK, I'll go for it. Thanks. > https://github.com/cython/cython/commit/90120314bcc8bd5a4f71f2629e1065f5c943071c -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Py_LIMITED_API
Should we try to support Py_LIMITED_API for Py>=3.2? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] adding support for __dict__ in extension types
Bringing up this old post...
On 21 June 2010 15:41, Robert Bradshaw wrote:
> On Jun 17, 2010, at 9:31 AM, Lisandro Dalcin wrote:
>
>> If we special case a __dict__ attribute in extension types, i.e:
>>
>> cdef class Foo:
>> cdef dict __dict__
>>
>> and fill type->tp_dictoffset, then we can support __dict__ in
>> extension types.
>>
>> What do you think?
>
> Sounds like a good idea to me. Note that we check tp_dictoffset for
> fast dispatching for cpdef methods (which would be correct as a dict
> lookup *would* be needed if __dict__ is available).
>
I still have this patch lying around in my disk. I remember Stefan had
some objections. For example, when the user ask for __dict__, a new
dict is unconditionally created (in CPython, type dict are allocated
on-demand). I propose to get this patch pushed now, and optimize
later (however, I really don't know how to safely implement this
optimization).
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
index a66d43b..204d474 100644
--- a/Cython/Compiler/ModuleNode.py
+++ b/Cython/Compiler/ModuleNode.py
@@ -24,6 +24,7 @@ import DebugFlags
from Errors import error, warning
from PyrexTypes import py_object_type
+from Builtin import dict_type as py_dict_type
from Cython.Utils import open_new_file, replace_suffix
from Code import UtilityCode
from StringEncoding import EncodedString
@@ -1088,10 +1089,18 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
code.putln("p->%s = %s%s;" % (
type.vtabslot_cname,
struct_type_cast, type.vtabptr_cname))
+entry = scope.lookup_here("__dict__")
+if entry and entry in py_attrs:
+code.putln("p->%s = PyDict_New();" % entry.cname)
+code.putln("if (p->%s == 0) { Py_DECREF(o); return 0;}" % entry.cname)
for entry in py_attrs:
-if scope.is_internal or entry.name == "__weakref__":
+if scope.is_internal:
# internal classes do not need None inits
code.putln("p->%s = 0;" % entry.cname)
+elif entry.name == "__dict__":
+pass
+elif entry.name == "__weakref__":
+code.putln("p->%s = 0;" % entry.cname)
else:
code.put_init_var_to_py_none(entry, "p->%s", nanny=False)
entry = scope.lookup_here("__new__")
@@ -2162,9 +2171,14 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
code.putln(
'if (__Pyx_SetAttrString(%s, "%s", (PyObject *)&%s) < 0) %s' % (
Naming.module_cname,
-scope.class_name,
-typeobj_cname,
-code.error_goto(entry.pos)))
+scope.class_name,
+typeobj_cname,
+code.error_goto(entry.pos)))
+dict_entry = scope.lookup_here("__dict__")
+if dict_entry:
+if not (dict_entry.type is py_object_type or
+dict_entry.type is py_dict_type):
+error(dict_entry.pos, "__dict__ slot must be of type 'object' or 'dict'")
weakref_entry = scope.lookup_here("__weakref__")
if weakref_entry:
if weakref_entry.type is py_object_type:
diff --git a/Cython/Compiler/Symtab.py b/Cython/Compiler/Symtab.py
index 0d0e576..afb54c2 100644
--- a/Cython/Compiler/Symtab.py
+++ b/Cython/Compiler/Symtab.py
@@ -37,7 +37,8 @@ def c_safe_identifier(cname):
# There are some C limitations on struct entry names.
if ((cname[:2] == '__'
and not (cname.startswith(Naming.pyrex_prefix)
- or cname == '__weakref__'))
+ or cname == '__weakref__'
+ or cname == '__dict__'))
or cname in iso_c99_keywords):
cname = Naming.pyrex_prefix + cname
return cname
diff --git a/Cython/Compiler/TypeSlots.py b/Cython/Compiler/TypeSlots.py
index 4565c7e..aaba929 100644
--- a/Cython/Compiler/TypeSlots.py
+++ b/Cython/Compiler/TypeSlots.py
@@ -427,7 +427,24 @@ class BaseClassSlot(SlotDescriptor):
self.slot_name,
base_type.typeptr_cname))
+class DictOffsetSlot(SlotDescriptor):
+# Slot descriptor for the dict offset slot.
+def __init__(self, name):
+SlotDescriptor.__init__(self, name,
Re: [Cython] adding support for __dict__ in extension types
On 28 February 2011 15:09, Robert Bradshaw wrote: > On Mon, Feb 28, 2011 at 8:33 AM, Lisandro Dalcin wrote: >> Bringing up this old post... >> >> On 21 June 2010 15:41, Robert Bradshaw wrote: >>> On Jun 17, 2010, at 9:31 AM, Lisandro Dalcin wrote: >>> >>>> If we special case a __dict__ attribute in extension types, i.e: >>>> >>>> cdef class Foo: >>>> cdef dict __dict__ >>>> >>>> and fill type->tp_dictoffset, then we can support __dict__ in >>>> extension types. >>>> >>>> What do you think? >>> >>> Sounds like a good idea to me. Note that we check tp_dictoffset for >>> fast dispatching for cpdef methods (which would be correct as a dict >>> lookup *would* be needed if __dict__ is available). >>> >> >> I still have this patch lying around in my disk. I remember Stefan had >> some objections. For example, when the user ask for __dict__, a new >> dict is unconditionally created (in CPython, type dict are allocated >> on-demand). I propose to get this patch pushed now, and optimize >> later (however, I really don't know how to safely implement this >> optimization). > > Note there's also the issue of cpdef methods--if the instance has a > __dict__ then a dict lookup must be performed for every method call > (to make sure it's not overridden). > But if the type do have a __dict__, then tp_dictoffset will be filled (this is in my patch), and cpdef methods should always go the dict lookup... Am I missing something? Now that you mention it, my patch should include tests for all this. I'll work on that. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Multiple modules in one compilation unit
On 2 March 2011 08:35, Stefan Behnel wrote: > Dag Sverre Seljebotn, 02.03.2011 11:54: >> >> Problem is that Fortran code often has...interesting...programming >> practices. Global variables abound, and are often initialised between >> modules. Imagine: >> >> settings_mod.set_alpha(0.34) >> print compute_mod.get_alpha_squared() >> >> This behaves quite differently with two static versions rather than one... > > Then I'd suggest always linking dynamically. > And where are you going to put your fortran shared libraries? Dynamic linking details varies wildly across platforms... I'm very much understand Dag's use case and concerns, and I do think that some research in all this is worth it. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Multiple modules in one compilation unit
On 2 March 2011 13:01, Stefan Behnel wrote: > Dag Sverre Seljebotn, 02.03.2011 16:37: >> >> On 03/02/2011 04:11 PM, Lisandro Dalcin wrote: >>> >>> On 2 March 2011 08:35, Stefan Behnel wrote: >>>> >>>> Dag Sverre Seljebotn, 02.03.2011 11:54: >>>>> >>>>> Problem is that Fortran code often has...interesting...programming >>>>> practices. Global variables abound, and are often initialised between >>>>> modules. Imagine: >>>>> >>>>> settings_mod.set_alpha(0.34) >>>>> print compute_mod.get_alpha_squared() >>>>> >>>>> This behaves quite differently with two static versions rather than >>>>> one... >>>> >>>> Then I'd suggest always linking dynamically. >>>> >>> And where are you going to put your fortran shared libraries? Dynamic >>> linking details varies wildly across platforms... I'm very much >>> understand Dag's use case and concerns, and I do think that some >>> research in all this is worth it. >> >> I'm not sure if there's much more to research at the moment -- Stefan says >> it is possible, and that's what I wanted to know at this stage. If I want >> it, I obviously need to implement it myself. (And if such a patch >> implements PEP 3121 and there's a demonstrated need for it with some >> users, >> I really can't see it getting rejected just out of it being in "poor >> taste"). >> >> I.e., I'm going to make Fwrap spit out multiple pyx files and worry about >> this later. If multiple .pyx in one .so was fundamentally impossible, I >> might have gone another route with Fwrap. That was all. > > The feature I could imagine becoming part of Cython is "compiling packages". > I.e. you'd call "cython" on a package and it would output a directory with a > single __init__.so that contains the modules compiled from all .pyx/.py > files in that package. Importing the package would then trigger an import of > that __init__.so, which in turn will execute code in its init__init__() > function to register the other modules. > $ cd /tmp/ $ mkdir pkg $ touch pkg/__init__.so $ python -c 'import pkg' Traceback (most recent call last): File "", line 1, in ImportError: No module named pkg So that's not going to work... The import machinery ignores __init__.so files. Instead, you need a "package.so" file, the initpackage() init function should in turn setup the package structure and initialize submodules. I've sent privately to Dag an example of how this should be done. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "__ipow__ has wrong number of arguments" is back here
On 2 March 2011 16:18, Vitja Makarov wrote: > Hi! > > I noticed that this error came back again. > > https://sage.math.washington.edu:8091/hudson/view/cython-devel/job/cython-devel-tests-py26-c/lastCompletedBuild/testReport/(root)/CythonRunTestCase/compiling__c__and_running_special_methods_T561/ > I've pushed a fix for __ipow__ to take just two arguments, like regular Python classes. https://github.com/cython/cython/commit/490fc80e6b494f8ed65f78fc2032d856ec75b8ef But I forgot to update test for ticket #561, now this should fixed: https://github.com/cython/cython/commit/829d6485b77b5688671959f840712c01e6886556 -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "__ipow__ has wrong number of arguments" is back here
On 2 March 2011 17:14, Lisandro Dalcin wrote: > On 2 March 2011 16:18, Vitja Makarov wrote: >> Hi! >> >> I noticed that this error came back again. >> >> https://sage.math.washington.edu:8091/hudson/view/cython-devel/job/cython-devel-tests-py26-c/lastCompletedBuild/testReport/(root)/CythonRunTestCase/compiling__c__and_running_special_methods_T561/ >> > > I've pushed a fix for __ipow__ to take just two arguments, like > regular Python classes. > https://github.com/cython/cython/commit/490fc80e6b494f8ed65f78fc2032d856ec75b8ef > > But I forgot to update test for ticket #561, now this should fixed: > https://github.com/cython/cython/commit/829d6485b77b5688671959f840712c01e6886556 > BTW, do you all agree with this change? I expect this change to break some code, particularly in Sage... -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Multiple modules in one compilation unit
On 2 March 2011 21:01, Greg Ewing wrote:
> Stefan Behnel wrote:
>>
>> you'd call "cython" on a package and it would output a directory with a
>> single __init__.so that contains the modules compiled from all .pyx/.py
>> files in that package. Importing the package would then trigger an import of
>> that __init__.so, which in turn will execute code in its init__init__()
>> function to register the other modules.
>
> I don't think it even has to be a directory with an __init__,
> it could just be an ordinary .so file with the name of the
> package.
>
> I just tried an experiment in Python:
>
> # onefilepackage.py
> import new, sys
> blarg = new.module("blarg")
> blarg.thing = "This is the thing"
> sys.modules["onefilepackage.blarg"] = blarg
>
> and two different ways of importing it:
>
>>>> from onefilepackage import blarg
>>>> blarg
>
>>>> blarg.thing
> 'This is the thing'
>
>>>> import onefilepackage.blarg
>>>> onefilepackage.blarg.thing
> 'This is the thing'
>
I'm hacking around these lines. However, I'm working to maintain
different modules in different C compilation units, in order to
workaround the obvious issue with duplicated global C symbols.
> So assuming the same thing works with a .so instead of a .py,
> all you need to do is emit a .so whose init function stuffs
> appropriate entries into sys.modules to make it look like
> a package.
>
However, the import machinery does not treat .so the same as *.pyx.
For example, I have a problem with Python 3. For .py modules, before
the module code starts to execute, the matching entries in sys.modules
is already there. The same happens in Python 2 for .so modules, as
Py_InitModule() add the entry in sys.modules early. However, in Python
3 that is not te case (and only for the .so, for .py is the same as in
Py2), the import machinery adds the entry later, after the
finalization of the module init function. I'm tempted to workaround
this by setting the entry in sys.modules right after the call to
PyModule_Create() ... What do you think about this? Any potential
issue?
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "__ipow__ has wrong number of arguments" is back here
On 3 March 2011 18:05, Robert Bradshaw wrote: > On Wed, Mar 2, 2011 at 5:47 PM, Lisandro Dalcin wrote: >> >> BTW, do you all agree with this change? > > To summarize, the C signature of ipow takes three arguments, but the > third is often/always garbage, so we shouldn't use it? > Yes. >> I expect this change to break some code, particularly in Sage... > > It'll be a compile time error fixing a potential runtime crash, so I > think that's a good move. OK, -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Out of order side effects of argument evaluation in function calls (ticket #654)
On 11 March 2011 09:45, Stefan Behnel wrote: > Hi, > > ticket 654 describes a code generation problem where the arguments to a cdef > function are not being evaluated in the order they are written down in the > code. > > http://trac.cython.org/cython_trac/ticket/654 > > This introduces problems when the arguments have side effects or are not > simple, e.g. they are function calls themselves or are taken from object > attributes or live in a closure. For example, > > f(g(a), a.x, h(a)) > > will produce different results depending on the evaluation order if g or h > change the value of a.x. > > However, apparently, the order of evaluation is only guaranteed by Python, > not by C. Now, the question is: what are the right semantics for Cython > here: follow Python or C? > > Personally, I think it would be nice to keep up Python's semantics, but when > I implemented this I broke quite some code in Sage (you may have noticed > that the sage-build project in Hudson has been red for a while). There are > things in C and especially in C++ that cannot be easily copied into a > temporary variable in order to make sure they are evaluated before the > following arguments. This is not a problem for Python function calls where > all arguments end up being copied (and often converted) anyway. It is a > problem for C function calls, though. > > What do you think about this? > Regarding our previous history, I would go for Python semantics... Perhaps you could add a compiler directive for these rare cases where you need/want C/C++ semantics? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bug in cython-mode.el under GNU Emacs 23.2.1
On 22 March 2011 02:21, Vitja Makarov wrote: > 2011/3/22 Rafe Kettler : >> I've just tried out cython-mode.el and I was a bit disappointed when it >> didn't work right off the bat. >> >> I added the following to my .emacs: >> >> (setq load-path (cons "~/lib/emacs-plugins" load-path)) >> (require 'cython-mode) >> >> this initially gave me the following error when I restarted emacs: >> >> Warning (initialization): An error occurred while loading >> `/home/rafe/.emacs': >> >> File error: Cannot open load file, python-mode >> >> To ensure normal operation, you should investigate and remove the >> cause of the error in your initialization file. Start Emacs with >> the `--debug-init' option to view a complete error backtrace. >> >> So, seeing that python-mode wasn't around, I changed line 3 of >> cython-mode.el from (require 'python-mode) to (require 'python) and >> everything works now. >> >> I got this mode from Github this morning and it is the latest version (you >> can see it at >> https://github.com/cython/cython/blob/master/Tools/cython-mode.el). >> >> I'm not an emacs lisp expert, so I'm not sure if there's something I'm >> missing here or if I somehow loaded the package correctly. Anyway, I thought >> you guys should know that I had a problem with the file but managed to get a >> fix. If my fix needs to be applied, I'll patch it on Github. >> >> This is GNU Emacs 23.2.1 x64 on Fedora 14, btw. >> >> Rafe >> > > Emacs23 have native support for python, you can install python-mode package: > > $ sudo apt-get install python-mode > > Instead of requiring python it's better to try python-mode first then python: > > (when (not (require 'python-mode nil t)) > (require 'python)) > Should we push this fix? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Cannot profile nogil function. Error or Warn?
Error compiling Cython file: ... cdef int PyMPE_Raise(int ierr) except -1 with gil: __Pyx_Raise(RuntimeError, "MPE logging error [code: %d]" % ierr, NULL) return 0 cdef inline int CHKERR(int ierr) nogil except -1: ^ /home/dalcinl/Devel/mpi4py-dev/src/MPE/helpers.pxi:22:5: Cannot profile nogil function. Do we REALLY want this to be an error? Why not just a warning? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] unspecified return type for extern C++ functions
In a previous thread, Robert asked about issuing a warning in the case of unspecified return type for extern C++ functions. I'm definitely +1 for it... Moreover, I would extend it for ANY extern C function. IMHO, this is a case for "explicit is better than implicit". -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cannot profile nogil function. Error or Warn?
On 29 March 2011 21:26, Lisandro Dalcin wrote: > Error compiling Cython file: > > ... > > cdef int PyMPE_Raise(int ierr) except -1 with gil: > __Pyx_Raise(RuntimeError, "MPE logging error [code: %d]" % ierr, NULL) > return 0 > > cdef inline int CHKERR(int ierr) nogil except -1: > ^ > > > /home/dalcinl/Devel/mpi4py-dev/src/MPE/helpers.pxi:22:5: Cannot > profile nogil function. > > > Do we REALLY want this to be an error? Why not just a warning? > OK, I pushed a fix. Without this, using -X profile=True cannot work with any pyx source that has nogil functions. Enabling profiling should not force users to change source code. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] implementation of cdef functions with default arguments
Perhaps I'm missing something, but why we need the intermediate struct? Why not generate a regular C function with all the args, and then generate the call providing arguments? We could even extend this to support kwargs for calling functions in Cython, 1 - The implementation would be cleaner, IMHO. 2 - These functions cannot be easily used in external C code (or course, C code should provide all the args) 3 - We could define default args for "cdef extern" C functions, Cython would provide the arg values on call. 4 - We could add support to pass values as kwargs (well, we could do that with the current implementation). 5 - Faster code? Comments? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] cython broken
Since the commit below, Cython fails to compile itself. That fix requires further work and definitely more tests. If that is impossible right now, I would ask the guilty parties to revert the change and continue working on this the bug tracker and repo clones. Please try to keep cython-dev repo clean. commit 3069c3e516fc7336b003861881623f30e168849e Author: Haoyu Bai Date: Thu Mar 31 04:19:14 2011 +0800 fix T477 by refactor FuncDefNode, so cython decorators can applied to cdef function See yourself: $ git checkout 3069c3e516fc7336b003861881623f30e168849e Note: checking out '3069c3e516fc7336b003861881623f30e168849e'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b new_branch_name HEAD is now at 3069c3e... fix T477 by refactor FuncDefNode, so cython decorators can applied to cdef function [dalcinl@trantor cython-dev]$ python setup.py --name Compiling module Cython.Compiler.Scanning ... Compiling module Cython.Compiler.Parsing ... Compiling module Cython.Compiler.Code ... Error compiling Cython file: ... self.cname = cname self.text = text self.escaped_value = StringEncoding.escape_byte_string(byte_string) self.py_strings = None def get_py_string_const(self, encoding, identifier=None, is_str=False): ^ Cython/Compiler/Code.py:316:4: Signature not compatible with previous declaration Error compiling Cython file: ... cdef public object text cdef public object escaped_value cdef public dict py_strings @cython.locals(intern=bint, is_str=bint, is_unicode=bint) cpdef get_py_string_const(self, encoding, identifier=*, is_str=*) ^ Cython/Compiler/Code.pxd:62:30: Previous declaration is here Compilation failed Cython -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Incompatibility with numpy-1.5.1 - fixed in master?!
On 15 April 2011 11:20, Yury V. Zaytsev wrote: > Hi folks! > > I have just ran into buffer protocol incompatibility problem with > numpy-1.5.1, which lead me to discover the following ticket (discussed > back in December 2010 on this list): > > http://trac.cython.org/cython_trac/ticket/630 > > In despair, I was about to try to see if there is anything I can do to > fix it myself. To this end I cloned your git repository and set up my > Python environment to use the latest bleeding edge version from there. > > To my surprise I discovered that my code started working and I don't > have the buffer interface problem that I was facing before anymore. > > So my suggestion would be to maybe test it more thoroughly and if it is > indeed the case, close the ticket. I tried to subscribe to it or leave a > comment, but I need an account which I can't register on my own. > I'm opposed to close the ticket. Cython cannot currently parse format strings according to the full spec. It worked for you just because of some quick fixes I've pushed for simple cases. > Still, I have the following question: which risks am I running by using > the bleeding edge version and is there any chance this could inflict > cancer on me? > Well, usually the bleeding edge is more featured and bug-free than latest release. However, right now, generator support was merged and you could have some performance regressions. But I'm using the bleeding edge for every day development work of mpi4py/petsc4py. > > Do you have plans to make a new point release which includes this very > important improvement anytime soon? > Not sure, we still have some (performance) regressions to fix, all of them related to the new generators support. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Incompatibility with numpy-1.5.1 - fixed in master?!
On 15 April 2011 11:48, Yury V. Zaytsev wrote: > On Fri, 2011-04-15 at 16:20 +0200, Yury V. Zaytsev wrote: > >> To my surprise I discovered that my code started working and I don't >> have the buffer interface problem that I was facing before anymore. > > I am under impression that the culprit was this commit: > > Commit SHA1: dd3da6c283a0750c9c6514991be719ac064e79b4 > Commit message: PEP 3118: fix for NULL Py_buffer arg > Committer: Lisandro Dalcin 2011-04-08 00:25:45 > This commit should be easy to cherry-pick and backport to latest release. However, there is a potential risk that user code implementing __getbuffer__ in extension classes do not make the check "if info == NULL: return", and then you get a SEGFAULT. AFAIK, all this business with Py_buffer pointer being NULL was added because of some obscure bug in Python==3.0.x, but I would expect people out there to be using Python >= 3.1.x, so not a big deal. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] libcpp.string operators
On 19 April 2011 21:22, Brent Pedersen wrote:
> On Tue, Apr 19, 2011 at 6:08 PM, Brent Pedersen wrote:
>> On Tue, Apr 19, 2011 at 2:45 PM, Brent Pedersen wrote:
>>> hi, i have been using a stub for the c++ in a lot of my work.
>>> i decided to have a go at doing a proper string.pxd, pasted here:
>>>
>>> https://gist.github.com/929604
>>>
>>> other than the operators, it's mostly implemented with tests. but when
>>> i try the operators
>>> with this test:
>>>
>>> def test_equal_operator(char *a, char *b):
>>> """
>>> >>> test_equal_operator("asdf", "asdf")
>>> True
>>> """
>>> cdef string s = string(a)
>>> cdef string t = string(b)
>>> cdef bint same = t == s
>>> return same
>>>
>>> and this declaration in the pxd:
>>>
>>> bint operator==(string&, string&)
>>
>>
>> it seems:
>>
>> bint operator==(string&)
>>
>> is the correct syntax. i had copied the stuff from vector.pxd
>>
>>>
>>> i get the error below. any ideas what i'm doing wrong?
>>> thanks,
>>> -brent
>>>
>>>
>>>
>>>
>>>
>>> === Got errors: ===
>>> 152:23: Invalid types for '==' (string, string)
>>>
>>>
>>> ==
>>> ERROR: runTest (__main__.CythonRunTestCase)
>>> compiling (cpp) and running cpp_stl_string
>>> --
>>> Traceback (most recent call last):
>>> File "runtests.py", line 569, in run
>>> self.runCompileTest()
>>> File "runtests.py", line 400, in runCompileTest
>>> self.test_directory, self.expect_errors, self.annotate)
>>> File "runtests.py", line 546, in compile
>>> self.assertEquals(None, unexpected_error)
>>> AssertionError: None != u"152:23: Invalid types for '==' (string, string)"
>>>
>>
>
> i updated the gist with the operators, didn't do iterators.
> https://gist.github.com/929604
>
Looks pretty good. Could you fork the devel repo, add the pxd and the
test at appropriate places and make a pull request? This is just in
order to give you credits for your work.
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Compiler error when @staticmethod is used in cdef class
On 27 April 2011 12:45, Bogdan Opanchuk wrote: > Hello, > > I am using Cython 0.14.1 with Python 2.7.1 on OSX 10.6.7 > > The following code: > --- > cdef class Test: > @staticmethod > def func(): > print "Static method" > --- > gives error when being cythoned: "Method func has wrong number of > arguments (0 declared, 1 or more expected)" > > And if one adds some parameters to this method (like "def func(a, > b):"), cython crashes with exception: > --- > ... long stack trace ... > File > "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Cython-0.14-py2.7-macosx-10.6-intel.egg/Cython/Compiler/TypeSlots.py", > line 100, in fixed_arg_type > return self.format_map[self.fixed_arg_format[i]] > KeyError: 'T' > --- > > So, is there a way to create static methods in cdef classes? This > workaround seems to work, but it is a bit ugly: > --- > def func(a, b): > print "Static method" > > cdef class Test: > func = func > --- > > Best regards, > Bogdan > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > This issue has bugged my from the fist day I've started to use Cython. However, never got time to look for a fix. Moreover, I think using @classmethod is better than @staticmethod. @classmethod gives you some extra context (the class) that could be useful in the case of inheritance, specially if your method is a factory function. So, as a workaround, try to use @classmethod, from an API point of view they are very similar, and you can take advantage of @classmethod extra context in the future. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] setup.py refusing to run cython
On 14 June 2011 16:20, Nathaniel Smith wrote:
> Hello Cython folks,
>
> This message (see below) is the second report I've gotten of a very
> strange build problem with a cython module. I'm just using the
> standard 'from Cython.Distutils import build_ext', 'cmdclass =
> {"build_ext": build_ext}' incantation, but for some reason setup.py is
> attempting to just run gcc directly without calling cython first to
> generate the .c file.
>
> You can see the setup.py here:
> https://code.google.com/p/scikits-sparse/source/browse/setup.py?name=scikits.sparse-0.1
>
> Anyone have any ideas?
>
Your setup.py is mixing setuptools and Cython.Distutils. Both are
monkeypatching distutils, and in general that's a very bad idea.
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to define C-consts in python module scope
On 18 July 2011 06:38, Vitja Makarov wrote: > cdef enum: > EV_READ = 1 > EV_WRITE = 2 > > Is there a way to put this constants into module dict? > I want to access this constants from pure python code, I tried this way: > > globals()['EV_READ'] = EV_READ > globals()['EV_WRITE'] = EV_WRITE > > But I don't like it, is there any other way? > cdef public enum: EV_READ = 1 EV_WRITE = 2 However, I do not like it, because I would like to use "public" for other meaning (API generation). Also note that using "public" will trigger the generation of a header file. Perhaps we should support "cpdef enum: ..." -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to define C-consts in python module scope
On 19 July 2011 02:24, Vitja Makarov wrote: > 2011/7/18 Robert Bradshaw : >> Trevor King and I discussed this quite a while back, but every time I >> got around to looking at his code (I don't think he ever created a >> formal pull request) something came up. The idea was that we could >> support cpdef structs and extern functions as well. >> > > That's interesting, I think I shouldn't hurry with my pull request. > > 2011/7/19 Robert Bradshaw : >> On Mon, Jul 18, 2011 at 4:34 PM, Greg Ewing >> wrote: >>> My suggestion is >>> >>> cdef exposed enum: >>> ... >> >> I agree, public is an overloaded word. This meaning is analogous to >> its use for cdef class members. Perhaps we should use "api" for api >> generation, and public used for Python-level access, with cpdef being >> the preferred form for declaring Python-accessible types. >> > > It seems to me that cpdef is more cythonic but exposed could be used > in this case: > > cdef extern from "ev.h": > exposed enum: > EV_READ > EV_WRITE > And what about this? cdef extern from "ev.h": cpdef enum: EV_READ EV_WRITE BTW, how is this supposed to work with *.pxd files? I think the values will be exposed just in the matching implementation .pyx file, and not with cimport, right? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to define C-consts in python module scope
On 19 July 2011 20:48, Robert Bradshaw wrote: > On Tue, Jul 19, 2011 at 3:02 PM, Lisandro Dalcin wrote: >> On 19 July 2011 02:24, Vitja Makarov wrote: >>> 2011/7/18 Robert Bradshaw : >>>> Trevor King and I discussed this quite a while back, but every time I >>>> got around to looking at his code (I don't think he ever created a >>>> formal pull request) something came up. The idea was that we could >>>> support cpdef structs and extern functions as well. >>>> >>> >>> That's interesting, I think I shouldn't hurry with my pull request. >>> >>> 2011/7/19 Robert Bradshaw : >>>> On Mon, Jul 18, 2011 at 4:34 PM, Greg Ewing >>>> wrote: >>>>> My suggestion is >>>>> >>>>> cdef exposed enum: >>>>> ... >>>> >>>> I agree, public is an overloaded word. This meaning is analogous to >>>> its use for cdef class members. Perhaps we should use "api" for api >>>> generation, and public used for Python-level access, with cpdef being >>>> the preferred form for declaring Python-accessible types. >>>> >>> >>> It seems to me that cpdef is more cythonic but exposed could be used >>> in this case: >>> >>> cdef extern from "ev.h": >>> exposed enum: >>> EV_READ >>> EV_WRITE >>> >> >> And what about this? >> >> cdef extern from "ev.h": >> cpdef enum: >> EV_READ >> EV_WRITE > > Yep, exactly. > >> BTW, how is this supposed to work with *.pxd files? I think the values >> will be exposed just in the matching implementation .pyx file, and not >> with cimport, right? > > It would be an error unless there's an corresponding .pyx file (which > could be empty). The idea is that one could also define extern cpdef > functions and structs and wrappers would be provided. > It would be an error? What do you mean? if you cpdef enum in foo.pxd, and have foo.pyx, then the enumerations should be exposed in the 'foo' module. But then if you "cimport foo" in bar.pyx, that should succeed but the enumerations should not be exposed in the "bar" module... Am I missing something? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to define C-consts in python module scope
On 20 July 2011 13:51, mark florisson wrote: > On 20 July 2011 18:06, Lisandro Dalcin wrote: >> On 19 July 2011 20:48, Robert Bradshaw wrote: >>> On Tue, Jul 19, 2011 at 3:02 PM, Lisandro Dalcin wrote: >>>> On 19 July 2011 02:24, Vitja Makarov wrote: >>>>> 2011/7/18 Robert Bradshaw : >>>>>> Trevor King and I discussed this quite a while back, but every time I >>>>>> got around to looking at his code (I don't think he ever created a >>>>>> formal pull request) something came up. The idea was that we could >>>>>> support cpdef structs and extern functions as well. >>>>>> >>>>> >>>>> That's interesting, I think I shouldn't hurry with my pull request. >>>>> >>>>> 2011/7/19 Robert Bradshaw : >>>>>> On Mon, Jul 18, 2011 at 4:34 PM, Greg Ewing >>>>>> wrote: >>>>>>> My suggestion is >>>>>>> >>>>>>> cdef exposed enum: >>>>>>> ... >>>>>> >>>>>> I agree, public is an overloaded word. This meaning is analogous to >>>>>> its use for cdef class members. Perhaps we should use "api" for api >>>>>> generation, and public used for Python-level access, with cpdef being >>>>>> the preferred form for declaring Python-accessible types. >>>>>> >>>>> >>>>> It seems to me that cpdef is more cythonic but exposed could be used >>>>> in this case: >>>>> >>>>> cdef extern from "ev.h": >>>>> exposed enum: >>>>> EV_READ >>>>> EV_WRITE >>>>> >>>> >>>> And what about this? >>>> >>>> cdef extern from "ev.h": >>>> cpdef enum: >>>> EV_READ >>>> EV_WRITE >>> >>> Yep, exactly. >>> >>>> BTW, how is this supposed to work with *.pxd files? I think the values >>>> will be exposed just in the matching implementation .pyx file, and not >>>> with cimport, right? >>> >>> It would be an error unless there's an corresponding .pyx file (which >>> could be empty). The idea is that one could also define extern cpdef >>> functions and structs and wrappers would be provided. >>> >> >> It would be an error? What do you mean? if you cpdef enum in foo.pxd, >> and have foo.pyx, then the enumerations should be exposed in the 'foo' >> module. But then if you "cimport foo" in bar.pyx, that should succeed >> but the enumerations should not be exposed in the "bar" module... Am I >> missing something? >> > > I believe Robert confirmed what you said and added to that that it > would be an error to have a cpdef extern enum in a .pxd without a > corresponding .pyx file. > But what an "error" means? Cython is going to fail compilation? Suppose I develop a package, and at setup.py install time my package installs somewhere a .pxd full of definitions in such a way that third-party code can cimport it. Then, while writing the third party code, you cimport the my package .pxd, and obviously there is no .pyx for my package, as you are writing YOUR code. What's going to happen in this case? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to define C-consts in python module scope
On 20 July 2011 15:32, mark florisson wrote:
> On 20 July 2011 20:04, Lisandro Dalcin wrote:
>> On 20 July 2011 13:51, mark florisson wrote:
>>> On 20 July 2011 18:06, Lisandro Dalcin wrote:
>>>> On 19 July 2011 20:48, Robert Bradshaw
>>>> wrote:
>>>>> On Tue, Jul 19, 2011 at 3:02 PM, Lisandro Dalcin
>>>>> wrote:
>>>>>> On 19 July 2011 02:24, Vitja Makarov wrote:
>>>>>>> 2011/7/18 Robert Bradshaw :
>>>>>>>> Trevor King and I discussed this quite a while back, but every time I
>>>>>>>> got around to looking at his code (I don't think he ever created a
>>>>>>>> formal pull request) something came up. The idea was that we could
>>>>>>>> support cpdef structs and extern functions as well.
>>>>>>>>
>>>>>>>
>>>>>>> That's interesting, I think I shouldn't hurry with my pull request.
>>>>>>>
>>>>>>> 2011/7/19 Robert Bradshaw :
>>>>>>>> On Mon, Jul 18, 2011 at 4:34 PM, Greg Ewing
>>>>>>>> wrote:
>>>>>>>>> My suggestion is
>>>>>>>>>
>>>>>>>>> cdef exposed enum:
>>>>>>>>> ...
>>>>>>>>
>>>>>>>> I agree, public is an overloaded word. This meaning is analogous to
>>>>>>>> its use for cdef class members. Perhaps we should use "api" for api
>>>>>>>> generation, and public used for Python-level access, with cpdef being
>>>>>>>> the preferred form for declaring Python-accessible types.
>>>>>>>>
>>>>>>>
>>>>>>> It seems to me that cpdef is more cythonic but exposed could be used
>>>>>>> in this case:
>>>>>>>
>>>>>>> cdef extern from "ev.h":
>>>>>>> exposed enum:
>>>>>>> EV_READ
>>>>>>> EV_WRITE
>>>>>>>
>>>>>>
>>>>>> And what about this?
>>>>>>
>>>>>> cdef extern from "ev.h":
>>>>>> cpdef enum:
>>>>>> EV_READ
>>>>>> EV_WRITE
>>>>>
>>>>> Yep, exactly.
>>>>>
>>>>>> BTW, how is this supposed to work with *.pxd files? I think the values
>>>>>> will be exposed just in the matching implementation .pyx file, and not
>>>>>> with cimport, right?
>>>>>
>>>>> It would be an error unless there's an corresponding .pyx file (which
>>>>> could be empty). The idea is that one could also define extern cpdef
>>>>> functions and structs and wrappers would be provided.
>>>>>
>>>>
>>>> It would be an error? What do you mean? if you cpdef enum in foo.pxd,
>>>> and have foo.pyx, then the enumerations should be exposed in the 'foo'
>>>> module. But then if you "cimport foo" in bar.pyx, that should succeed
>>>> but the enumerations should not be exposed in the "bar" module... Am I
>>>> missing something?
>>>>
>>>
>>> I believe Robert confirmed what you said and added to that that it
>>> would be an error to have a cpdef extern enum in a .pxd without a
>>> corresponding .pyx file.
>>>
>>
>> But what an "error" means? Cython is going to fail compilation?
>> Suppose I develop a package, and at setup.py install time my package
>> installs somewhere a .pxd full of definitions in such a way that
>> third-party code can cimport it. Then, while writing the third party
>> code, you cimport the my package .pxd, and obviously there is no .pyx
>> for my package, as you are writing YOUR code. What's going to happen
>> in this case?
>
> If you want to use cpdef in your .pxd, it means you want to expose
> those to Python. This means they must be importable from Python, which
> means you need an extension module that exposes the constants, which
> means you need a .pyx. So you ship a (possibly empty) .pyx file along
> with your .pxd. If you don't want to expose them to Python but only to
> Cython, don't use cpdef. This compile time error would be issued
> whenever a user does a cimport of a .pxd file that has a cpdef enum
> with no corresponding .pyx file, i.e. the pxd is unusable, you
> wouldn't ship it in the first place.
>
So supose I want to write a small wrapper for dlopen(), then I do:
# dl.pxd
cdef from extern from "dlfcn.h":
cpdef enum: RTLD_GLOBAL
void * dlopen()(const char *filename, int flag);
# dl.pyx
def open(filename, mode):
cdef void *handle = c_dlopen(fiename,handle)
return handle
Then the "dl" module namespace contains "dlopen" and "RTLD_GLOBAL"
Now, suppose I code my setup.py to build the ext module and install
dl.pxd as package data.
Now a user runs "pip install dl" and installs my package. She wants to
reuse my .pxd (installed somewhere) for calling dlopen() in its own
Cython code:
# user.pyx
from dl cimport RTLD_GLOBAL, c_dlopen
dlopen("somelibrary.so", RTLD_GLOBAL)
So the user code is going to fail? Then I cannot take advantage of
cpdef enum for developing my "dl" module, I have to go back to
manually exposing the enumerations
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to define C-consts in python module scope
On 20 July 2011 16:27, mark florisson wrote: > On 20 July 2011 21:13, Lisandro Dalcin wrote: >> On 20 July 2011 15:32, mark florisson wrote: >>> On 20 July 2011 20:04, Lisandro Dalcin wrote: >>>> On 20 July 2011 13:51, mark florisson wrote: >>>>> On 20 July 2011 18:06, Lisandro Dalcin wrote: >>>>>> On 19 July 2011 20:48, Robert Bradshaw >>>>>> wrote: >>>>>>> On Tue, Jul 19, 2011 at 3:02 PM, Lisandro Dalcin >>>>>>> wrote: >>>>>>>> On 19 July 2011 02:24, Vitja Makarov wrote: >>>>>>>>> 2011/7/18 Robert Bradshaw : >>>>>>>>>> Trevor King and I discussed this quite a while back, but every time I >>>>>>>>>> got around to looking at his code (I don't think he ever created a >>>>>>>>>> formal pull request) something came up. The idea was that we could >>>>>>>>>> support cpdef structs and extern functions as well. >>>>>>>>>> >>>>>>>>> >>>>>>>>> That's interesting, I think I shouldn't hurry with my pull request. >>>>>>>>> >>>>>>>>> 2011/7/19 Robert Bradshaw : >>>>>>>>>> On Mon, Jul 18, 2011 at 4:34 PM, Greg Ewing >>>>>>>>>> wrote: >>>>>>>>>>> My suggestion is >>>>>>>>>>> >>>>>>>>>>> cdef exposed enum: >>>>>>>>>>> ... >>>>>>>>>> >>>>>>>>>> I agree, public is an overloaded word. This meaning is analogous to >>>>>>>>>> its use for cdef class members. Perhaps we should use "api" for api >>>>>>>>>> generation, and public used for Python-level access, with cpdef being >>>>>>>>>> the preferred form for declaring Python-accessible types. >>>>>>>>>> >>>>>>>>> >>>>>>>>> It seems to me that cpdef is more cythonic but exposed could be used >>>>>>>>> in this case: >>>>>>>>> >>>>>>>>> cdef extern from "ev.h": >>>>>>>>> exposed enum: >>>>>>>>> EV_READ >>>>>>>>> EV_WRITE >>>>>>>>> >>>>>>>> >>>>>>>> And what about this? >>>>>>>> >>>>>>>> cdef extern from "ev.h": >>>>>>>> cpdef enum: >>>>>>>> EV_READ >>>>>>>> EV_WRITE >>>>>>> >>>>>>> Yep, exactly. >>>>>>> >>>>>>>> BTW, how is this supposed to work with *.pxd files? I think the values >>>>>>>> will be exposed just in the matching implementation .pyx file, and not >>>>>>>> with cimport, right? >>>>>>> >>>>>>> It would be an error unless there's an corresponding .pyx file (which >>>>>>> could be empty). The idea is that one could also define extern cpdef >>>>>>> functions and structs and wrappers would be provided. >>>>>>> >>>>>> >>>>>> It would be an error? What do you mean? if you cpdef enum in foo.pxd, >>>>>> and have foo.pyx, then the enumerations should be exposed in the 'foo' >>>>>> module. But then if you "cimport foo" in bar.pyx, that should succeed >>>>>> but the enumerations should not be exposed in the "bar" module... Am I >>>>>> missing something? >>>>>> >>>>> >>>>> I believe Robert confirmed what you said and added to that that it >>>>> would be an error to have a cpdef extern enum in a .pxd without a >>>>> corresponding .pyx file. >>>>> >>>> >>>> But what an "error" means? Cython is going to fail compilation? >>>> Suppose I develop a package, and at setup.py install time my package >>>> installs somewhere a .pxd full of definitions in such a way that >>>> third-party code can cimport it. Then, while writing the third party >>>> code, you cimpo
Re: [Cython] How to define C-consts in python module scope
On 20 July 2011 23:41, Stefan Behnel wrote: > Hi, > > I think we're getting way off course with this discussion. The following is > a good way to get back into a useful direction. > > Robert Bradshaw, 20.07.2011 23:24: >> >> it would be a runtime error when trying to import the >> objects. I suppose if cimport does not inject anything into the python >> namespace, then "cimport" could just work and "import" would work iff >> the module is available at runtime. > > +1 > +1, too... That was my whole point! -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython 0.15 release candidate
On 21 July 2011 23:43, Christoph Gohlke wrote:
>
>
> On 7/21/2011 4:14 PM, Robert Bradshaw wrote:
>>
>> Cython has seen an enormous amount of development since 0.14.1. If you
>> are not already using the latest version from the development
>> repository, we encourage you to try out the release candidate:
>> http://cython.org/release/Cython-0.15rc0.tar.gz
>>
>> - Robert
>
> Hi,
>
> I get a test error on Windows with msvc9 due to the "with gil" statement
> producing invalid C code (at least for msvc9). Below is the offending code
> in with_gil.c. Moving the declaration of the __pyx_gilstate_save variable a
> couple of lines up fixes this. A patch is attached.
>
> Christoph
>
>
> /* "with_gil.pyx":347
> * raise Exception("This will be overridden")
> * finally:
> * with gil: # <<<<<<<<<<<<<<
> * raise Exception("Override exception!")
> *
> */
> /*finally:*/ {
> int __pyx_why;
> PyObject *__pyx_exc_type, *__pyx_exc_value, *__pyx_exc_tb;
> int __pyx_exc_lineno;
> __pyx_exc_type = 0; __pyx_exc_value = 0; __pyx_exc_tb = 0;
> __pyx_exc_lineno = 0;
> #ifdef WITH_THREAD
> PyGILState_STATE __pyx_gilstate_save;
> #endif
> __pyx_why = 0; goto __pyx_L10;
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>
>
Your patch is OK. However now I'm wondering why not to initialize the
exc variables to 0 directly on declaration, like this:
PyObject *__pyx_exc_type = 0, *__pyx_exc_value =0, *__pyx_exc_tb = 0;
(anyway, I still prefer declaring the gilstate variable first)
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Should we drop support for CPython 2.3? (and maybe even 2.4?)
On 30 July 2011 06:21, David Cournapeau wrote: > On Sat, Jul 30, 2011 at 6:07 PM, Stefan Behnel wrote: >> David Cournapeau, 30.07.2011 10:52: >>> >>> Python 2.4 is still surprisingly >>> common. Working around it for C extensions can be pretty daunting. >> >> The same applies to Cython, obviously, although I do see the advantage of >> doing it in one place, and specifically in Cython, instead of dropping it on >> the users. > > I was not very clear: I meant that if you were to use python 2.5 or > above on a platform where only 2.4 is available, you would need to > build a lot of C extensions by yourself, which is out of reach to the > vast majority of our users. > > Typically, I had to use centos 5.x 2 years ago, and even for me, it > was quite difficult to build a python 2.6 environment I was > comfortable with (keeping in mind that a lot of "big-lab environments" > do not give you admin access). Centos 5.x end of life is in 2014. > I'm with David here. I do not object dropping support for 2.3, but would like 2.4 to be supported. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Willing to contribute
On 3 August 2011 17:53, Stefano wrote: > > Everything started with trying to speed up a Mandelbrot generator, which > heavily relies on complex numbers. Eventually I've found in "Mini Projects (or > something similar)" wiki that C99 `complex' is still to be implemented. So, I > decide to make a step further and try to hop into development. > That comment is outdated. Cython do support complex numbers in many different flavors (native C99 complex, C++ std::complex, raw C structs for pre-C99 compilers) -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] except clause on nogil functions
On 20 August 2011 06:55, mark florisson wrote: > > What about the syntax issue though? Should we fix that? > Long ago I also complained about that. Greg commented that we should really accept "except" and "gil/nogil" in any position, so that cdef int foo() except 0 with gil cdef int foo() with gil except 0 cdef int bar() nogil except 0 cdef int bar() except 0 nogil should be all valid. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Error when using Intel icl compiler on Windows
On 9 September 2011 05:26, Robert Bradshaw wrote: > Does it work with icc if you replace > > # define CYTHON_UNUSED __attribute__ ((__unused__)) > > with > > # define CYTHON_UNUSED __attribute__ ((unused)) > > ? > > > On Fri, Sep 9, 2011 at 12:28 AM, Christoph Gohlke wrote: >> Hello, >> >> compiling Cython 0.15 generated C code on Windows using the Intel Compiler >> 11.1 icl.exe results in an "expected a type specifier" error. >> >> For example: >> >> /* "numpy.pxd":190 >> * # experimental exception made for __getbuffer__ and >> __releasebuffer__ >> * # -- the details of this may change. >> * def __getbuffer__(ndarray self, Py_buffer* info, int flags): >> # <<<<<<<<<<<<<< >> * # This implementation of getbuffer is geared towards Cython >> * # requirements, and does not yet fullfill the PEP. >> */ >> >> static CYTHON_UNUSED int __pyx_pf_5numpy_7ndarray___getbuffer__(PyObject >> *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /*proto*/ >> ^ >> Error: expected a type specifier. >> >> The problem is the definition of CYTHON_UNUSED. >> >> Please consider the attached patch. It works for me but maybe the problem is >> icl version specific, not Windows specific. >> Robert, could you apply this patch? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Error when using Intel icl compiler on Windows
On 13 September 2011 01:09, Robert Bradshaw wrote: > On Mon, Sep 12, 2011 at 7:19 PM, Lisandro Dalcin wrote: >> On 9 September 2011 05:26, Robert Bradshaw >> wrote: >>> Does it work with icc if you replace >>> >>> # define CYTHON_UNUSED __attribute__ ((__unused__)) >>> >>> with >>> >>> # define CYTHON_UNUSED __attribute__ ((unused)) >>> >>> ? >>> >>> >>> On Fri, Sep 9, 2011 at 12:28 AM, Christoph Gohlke wrote: >>>> Hello, >>>> >>>> compiling Cython 0.15 generated C code on Windows using the Intel Compiler >>>> 11.1 icl.exe results in an "expected a type specifier" error. >>>> >>>> For example: >>>> >>>> /* "numpy.pxd":190 >>>> * # experimental exception made for __getbuffer__ and >>>> __releasebuffer__ >>>> * # -- the details of this may change. >>>> * def __getbuffer__(ndarray self, Py_buffer* info, int flags): >>>> # <<<<<<<<<<<<<< >>>> * # This implementation of getbuffer is geared towards Cython >>>> * # requirements, and does not yet fullfill the PEP. >>>> */ >>>> >>>> static CYTHON_UNUSED int __pyx_pf_5numpy_7ndarray___getbuffer__(PyObject >>>> *__pyx_v_self, Py_buffer *__pyx_v_info, int __pyx_v_flags); /*proto*/ >>>> ^ >>>> Error: expected a type specifier. >>>> >>>> The problem is the definition of CYTHON_UNUSED. >>>> >>>> Please consider the attached patch. It works for me but maybe the problem >>>> is >>>> icl version specific, not Windows specific. >>>> >> >> Robert, could you apply this patch? > > Done. I'm still wondering if it's broken for __INTEL_COMPILER > non-windows, but don't have any way to test it out. > In my Linux box, it seens to works just fine. However, now I think that the correct guard should be: # elif (defined(__ICC) || (defined(__INTEL_COMPILER)) && !defined(_MSC_VER) Anyway, the __ICC definition is obsolete. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Correct way of defining enums
2011/11/28 Stéfan van der Walt : > > I tried different ways of defining the enum, e.g. > > cdef extern from "ndarraytypes.h": > cpdef enum NPY_SEARCHSIDE: > NPY_SEARCHLEFT = 0 > NPY_SEARCHRIGHT = 1 > This is (near to, see below) the right way, as that NumPy enum do is declared in NumPy headers. > > But that doesn't work either. Any suggestions would be appreciated. > Try "ctypedef enum ..." -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Correct way of defining enums
On 30 November 2011 04:15, Robert Bradshaw wrote: > > There should probably be at least a mode to give warnings for untyped > arguments (and return types) of cdef functions. > Definitely +1. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fix integer width constant names in stdint.pxd
On 2 January 2012 22:37, Mansour Moufid wrote: > Now my issue is as follows. > > (I CCed the cython-users list if this question is more appropriate there.) > > I have a simple file, int.pyx: > > from libc.stdint cimport * > print long(UINT8_MAX) > print long(UINT16_MAX) > print long(UINT32_MAX) > print long(UINT64_MAX) > > with the usual setup.py stuff. Compiling and running: > > $ python setup.py build_ext --inplace > ... > int.c:566:3: warning: overflow in implicit constant conversion [-Woverflow] > ... > $ python -c 'import int' > 255 > 65535 > -1 > -1 > > So obviously there are overflows here. Checking int.c, I see: > > /* "int.pyx":2 > * from libc.stdint cimport * > * print long(UINT8_MAX) # <<<<<<<<<<<<<< > * print long(UINT16_MAX) > * print long(UINT32_MAX) > */ > __pyx_t_1 = PyInt_FromLong(UINT8_MAX); > > and so on... > > PyInt_FromLong is used for all these constants, regardless of > signedness or width, so any argument larger than LONG_MAX overflows, > *before* being converted to the arbitrary-size Python integer type. > > I don't know if this is a bug, or if I'm overlooking something. Is > there a way for me to use these constants with Python's arbitrary-size > integers? > All these constants are declared as "enum", so Cython promotes them to "int". Once again, Cython should have something like a "const" type qualifier to poperly declare these compile-time constants. As workaround, you could explicitly cast the constants like this "print long(UINT8_MAX)" -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fix integer width constant names in stdint.pxd
On 2 January 2012 23:00, Robert Bradshaw wrote: > On Mon, Jan 2, 2012 at 5:48 PM, Lisandro Dalcin wrote: >> On 2 January 2012 22:37, Mansour Moufid wrote: >>> Now my issue is as follows. >>> >>> (I CCed the cython-users list if this question is more appropriate there.) >>> >>> I have a simple file, int.pyx: >>> >>> from libc.stdint cimport * >>> print long(UINT8_MAX) >>> print long(UINT16_MAX) >>> print long(UINT32_MAX) >>> print long(UINT64_MAX) >>> >>> with the usual setup.py stuff. Compiling and running: >>> >>> $ python setup.py build_ext --inplace >>> ... >>> int.c:566:3: warning: overflow in implicit constant conversion [-Woverflow] >>> ... >>> $ python -c 'import int' >>> 255 >>> 65535 >>> -1 >>> -1 >>> >>> So obviously there are overflows here. Checking int.c, I see: >>> >>> /* "int.pyx":2 >>> * from libc.stdint cimport * >>> * print long(UINT8_MAX) # <<<<<<<<<<<<<< >>> * print long(UINT16_MAX) >>> * print long(UINT32_MAX) >>> */ >>> __pyx_t_1 = PyInt_FromLong(UINT8_MAX); >>> >>> and so on... >>> >>> PyInt_FromLong is used for all these constants, regardless of >>> signedness or width, so any argument larger than LONG_MAX overflows, >>> *before* being converted to the arbitrary-size Python integer type. >>> >>> I don't know if this is a bug, or if I'm overlooking something. Is >>> there a way for me to use these constants with Python's arbitrary-size >>> integers? >>> >> >> All these constants are declared as "enum", so Cython promotes them to >> "int". Once again, Cython should have something like a "const" type >> qualifier to poperly declare these compile-time constants. >> >> As workaround, you could explicitly cast the constants like this >> "print long(UINT8_MAX)" > > I'm leaning towards declaring them as being the proper type to begin > with; what's to be gained by declaring these extern values as enums > (=const)? At least with the larger types we should do this to avoid > patently incorrect behavior, and this way they would be consistant > with the actual C for arithmetic promotion, etc. > Not sure about recent Cython releases, but in older ones you cannot do: cdef char buf[UINT8_MAX] unless UINT8_MAX was declared in Cython as a compile time constant. However, I do agree that for the case of stdint.h, using matching types instead of "enum" is way better. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Upcoming issues with NumPy deprecated APIs and Cython's sizeof checks
I'm testing my code with numpy-dev. They are trying to discourage use
of deprecated APIs, this includes direct access to the ndarray struct.
In order to update your code, you have to pass -DNPY_NO_DEPRECATED_API
to the C compiler (or #define it before including NumPy headers).
However, they have implemented this feature by exposing the ndarray
type with just the Python object header:
https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/ndarraytypes.h#L695
Obviously, this interact bad with Cython's sizeof check, I'm getting
this runtime warning:
build/lib.linux-x86_64-2.7/petsc4py/lib/__init__.py:64:
RuntimeWarning: numpy.ndarray size changed, may indicate binary
incompatibility
I think there is nothing Cython can do about this (other than
special-casing NumPy to disable this VERY useful warning).
I've tried the patch below with success, but I'm not convinced...
Does any of you have a suggestion for NumPy folks about how to improve
this?
diff --git a/numpy/core/include/numpy/ndarraytypes.h
b/numpy/core/include/numpy/ndarraytypes.h
index 0288272..1fcbf52 100644
--- a/numpy/core/include/numpy/ndarraytypes.h
+++ b/numpy/core/include/numpy/ndarraytypes.h
@@ -695,6 +695,7 @@ typedef struct tagPyArrayObject_fields {
#ifdef NPY_NO_DEPRECATED_API
typedef struct tagPyArrayObject {
PyObject_HEAD
+char _npy_array_fields[sizeof(PyArrayObject_fields)-sizeof(PyObject)];
} PyArrayObject;
#else
/*
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Upcoming issues with NumPy deprecated APIs and Cython's sizeof checks
On 31 January 2012 11:29, mark florisson wrote: > On 30 January 2012 21:03, Lisandro Dalcin wrote: >> >> I think there is nothing Cython can do about this (other than >> special-casing NumPy to disable this VERY useful warning). > > Weird, shouldn't you be getting an error? Because the size of the > PyArrayObject should be less than what Cython expects. > Well, as long as your code does not access the structure fiels, everything is OK. > > I'm not sure this should be fixed in NumPy. Their entire point is that > people shouldn't use those attributes directly. > This makes me think about the all those arguments about the beauties of "private:" in C++ (and others OOP langs). IMHO, if they way to discourage access to these slots, they should declare them with some weird names. > I think numpy.pxd > should be fixed, but the problem is that some attributes might be used > in user code (especially shape), and we still want that to work in > nogil mode. You can still use PyArray_DATA(), PyArray_DIMS(), etc... > As such, I'm not sure what the best way of fixing it is, > without special casing these attributes in the compiler directly. > Maybe Dag will have some thoughts about this. > I'v just noticed now NumPy #defines NPY_SIZEOF_PYARRAYOBJECT, that could serve as a workaround for the sizeof check: https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/ndarraytypes.h#L707 -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Broken C-API generation for ext modules
The commit below from Stefan broke C-API generation for extension modules. The problem is that the code of __Pyx_ImportModule() and __Pyx_ImportType() depeds on the #definition of __Pyx_PyIdentifier_FromString , and such #define is not emitted in C-API headers. Stefan, given that __Pyx_PyIdentifier_FromString() is only used in __Pyx_ImportModule() and __Pyx_ImportType(), what's your opinion about just rolling back your commit? commit 500f9a40f5ad441c2c204d076cfc4f82a41d531b Author: Stefan Behnel Date: Sat Sep 10 00:14:05 2011 +0200 minor code simplification in utility functions by using macro for Py2/3 dependent PyString/PyUnicode_FromString() calls -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Broken C-API generation for ext modules
On 24 February 2012 08:00, Stefan Behnel wrote: > Lisandro Dalcin, 23.02.2012 22:00: >> The commit below from Stefan broke C-API generation for extension >> modules. The problem is that the code of __Pyx_ImportModule() and >> __Pyx_ImportType() depeds on the #definition of >> __Pyx_PyIdentifier_FromString , and such #define is not emitted in >> C-API headers. >> >> Stefan, given that __Pyx_PyIdentifier_FromString() is only used in >> __Pyx_ImportModule() and __Pyx_ImportType(), what's your opinion about >> just rolling back your commit? >> >> >> commit 500f9a40f5ad441c2c204d076cfc4f82a41d531b >> Author: Stefan Behnel >> Date: Sat Sep 10 00:14:05 2011 +0200 > > That's impressively old for a broken feature. > Indeed. I do not test for this feature on a regular basis in my projects, and I've been using latest Cython release in production runs. > >> minor code simplification in utility functions by using macro for >> Py2/3 dependent PyString/PyUnicode_FromString() calls > > https://github.com/cython/cython/commit/5a31a3d8d38d9d266886916432f1ebe621a2bc69 > > I pushed a fix here. Looks like the capi tests didn't detect this, though... > Thanks, I've improved the tests here: https://github.com/cython/cython/commit/6f343d9c82e611af5e437641583bdb88a926af93 BTW, if you run $ python runtests.py --no-cleanup module_api and next look at the generated api header: $ emacs BUILD/build/module_api/a_api.h you will notice that there are a lot of blank lines between groups of declarations and definitions. I guess this originates at UtilityCodeBase.load_utilities_from_file(), but could not figure out how to fix it... could you take a look? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] __cinit__ swallowing exceptions
This is something I really have no idea about how to fix, so I'll ask
any of you to do it.
How to reproduce. The quick example below should fail in the second to
last line in test_cinit.py, but it succeeds:
$ cat cinit.pyx
cdef class A:
def __cinit__(self, A a=None):
pass
$ cat test_cinit.py
import pyximport; pyximport.install()
from cinit import A
a = A(123)
print (a)
$ python test_cinit.py
/home/dalcinl/.pyxbld/temp.linux-x86_64-2.7/pyrex/cinit.c:1429:13:
warning: ‘__pyx_clear_code_object_cache’ defined but not used
[-Wunused-function]
I think the issue is bad code generation. Please try to follow the
flow below, when ArgTypeTest fails, then "goto __pyx_L1_error" but
__pyx_r is never initialized to -1, so the function (accidentally)
returns 0 indicating success. BTW, GCC warns about __pyx_r used before
initialization is some other code of mine (more complex, involving
inheritance, and the C compiler inlining code), but not for this
simple example.
static int __pyx_pw_5cinit_1A_1__cinit__(PyObject *__pyx_v_self,
PyObject *__pyx_args, PyObject *__pyx_kwds) {
...
int __pyx_r;
...
{
}
goto __pyx_L4_argument_unpacking_done;
...
__pyx_L4_argument_unpacking_done:;
if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_a),
__pyx_ptype_5cinit_A, 1, "a", 0))) {__pyx_filename = __pyx_f[0];
__pyx_lineno = 2; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
...
__pyx_L1_error:;
__pyx_L0:;
__Pyx_RefNannyFinishContext();
return __pyx_r;
}
BTW, valgrind is able to catch the issue:
$ touch cinit.pyx
$ CFLAGS=-O0 valgrind python test_cinit.py
==6735== Memcheck, a memory error detector
==6735== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==6735== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==6735== Command: python test_cinit.py
==6735==
/home/dalcinl/.pyxbld/temp.linux-x86_64-2.7/pyrex/cinit.c:1429:13:
warning: ‘__pyx_clear_code_object_cache’ defined but not used
[-Wunused-function]
==6735== Conditional jump or move depends on uninitialised value(s)
==6735==at 0x12DA4635: __pyx_tp_new_5cinit_A (cinit.c:548)
==6735==by 0x3A87C9DCF2: ??? (in /usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87C49192: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87CDE794: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87CE15A4: PyEval_EvalCodeEx (in
/usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87CE16D1: PyEval_EvalCode (in /usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87CFB9EB: ??? (in /usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87CFC7EF: PyRun_FileExFlags (in
/usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87CFD26E: PyRun_SimpleFileExFlags (in
/usr/lib64/libpython2.7.so.1.0)
==6735==by 0x3A87D0E744: Py_Main (in /usr/lib64/libpython2.7.so.1.0)
==6735==by 0x56F969C: (below main) (in /lib64/libc-2.14.90.so)
==6735==
==6735==
==6735== HEAP SUMMARY:
==6735== in use at exit: 8,870,441 bytes in 53,569 blocks
==6735== total heap usage: 391,334 allocs, 337,765 frees, 94,515,287
bytes allocated
==6735==
==6735== LEAK SUMMARY:
==6735==definitely lost: 0 bytes in 0 blocks
==6735==indirectly lost: 0 bytes in 0 blocks
==6735== possibly lost: 2,319,018 bytes in 15,424 blocks
==6735==still reachable: 6,551,423 bytes in 38,145 blocks
==6735== suppressed: 0 bytes in 0 blocks
==6735== Rerun with --leak-check=full to see details of leaked memory
==6735==
==6735== For counts of detected and suppressed errors, rerun with: -v
==6735== Use --track-origins=yes to see where uninitialised values come from
==6735== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
On 9 March 2012 16:22, Lisandro Dalcin wrote: > > PS: This is the only serious failure I get on OS X Lion. The embed > test is not working, but it is a Makefile issue. > UPDATE: The embed test do work with system Python 2.7. The failure is with EPD Python 7.2, it is caused by a bad definition of LINKFORSHARED in its installed config/Makefile. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
I'm basically experiencing the issues here: http://www.cocoabuilder.com/archive/xcode/310299-error-calling-builtin-expect-inside-omp-parallel-for.html Can you imagine any way to workaround it? PS: This is the only serious failure I get on OS X Lion. The embed test is not working, but it is a Makefile issue. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
On 10 March 2012 03:41, mark florisson wrote: > On 9 March 2012 14:22, Lisandro Dalcin wrote: >> I'm basically experiencing the issues here: >> http://www.cocoabuilder.com/archive/xcode/310299-error-calling-builtin-expect-inside-omp-parallel-for.html >> >> Can you imagine any way to workaround it? > > What a lovely C compiler bug... Did you file a bug report with gcc or > xcode? What version of gcc does Lion ship with? > I'm using a Mac just by accident, no idea (nor motivation) to report bugs to Apple. $ gcc --version i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00) Could that be the the bug is actually in the LLVM backend? > I suppose the macro would have to be disabled for OS X Lion inside > parallel sections (there seems to be __APPLE__ and __OSX__, I don't > know about detecting the OS X version), that's easy to do (undef and > redefine to no-ops before parallel section and redefine it again after > the section). I'll try fixing it during the sprints next week. > Perhaps switching to use a "omp_likely"/"omp_unlikely" macros inside OpenMP blocks would be nicer than defining/undefining? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
On 11 March 2012 09:46, mark florisson wrote: > On 10 March 2012 14:00, Stefan Behnel wrote: >> Lisandro Dalcin, 10.03.2012 10:51: >>> On 10 March 2012 03:41, mark florisson wrote: >>>> On 9 March 2012 14:22, Lisandro Dalcin wrote: >>>>> I'm basically experiencing the issues here: >>>>> http://www.cocoabuilder.com/archive/xcode/310299-error-calling-builtin-expect-inside-omp-parallel-for.html >>>>> >>>>> Can you imagine any way to workaround it? >>>> >>>> What a lovely C compiler bug... Did you file a bug report with gcc or >>>> xcode? What version of gcc does Lion ship with? >>>> >>> >>> I'm using a Mac just by accident, no idea (nor motivation) to report >>> bugs to Apple. >>> >>> $ gcc --version >>> i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. >>> build 5658) (LLVM build 2336.9.00) >>> >>> Could that be the the bug is actually in the LLVM backend? >>> >>>> I suppose the macro would have to be disabled for OS X Lion inside >>>> parallel sections (there seems to be __APPLE__ and __OSX__, I don't >>>> know about detecting the OS X version), that's easy to do (undef and >>>> redefine to no-ops before parallel section and redefine it again after >>>> the section). I'll try fixing it during the sprints next week. >>> >>> Perhaps switching to use a "omp_likely"/"omp_unlikely" macros inside >>> OpenMP blocks would be nicer than defining/undefining? >> >> Could that be coded into the macro or would it require to change the >> generated code? But at least it sounds like it would not impact code in >> functions that are being called from within the OpenMP blocks, would it? >> Just the code straight inside the block. A work-around could still have a >> substantial impact if it requires changes to the generated code. > > Yeah, that's why I suggested the undef/re-def approach around OpenMP > blocks. It's some code bloat, but only for the C preprocessor, so it > should be fine. > I still feel bad about this. What about just disabling branch prediction if OpenMP is ever used? Or perhaps just protect the definitions of likely/unlikely with some guard, such as users can disable them using a -D definition? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] OS X 10.7 Lion: GCC __builtin_expect unrecognized inside OpenMP blocks
On 14 March 2012 08:58, mark florisson wrote: > On 9 March 2012 14:22, Lisandro Dalcin wrote: > > Could you give this fix a try? > https://github.com/markflorisson88/cython/commit/2bffde15edc66c7416716051959e3b0cf1d6b41b > I'm still getting the linking error. Your fix does not resolve the issue :-( -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Exception instantiation
On 21 March 2012 11:03, Stefan Behnel wrote: > Hi, > I think this should change. We may consider leaving it as it is for known > exception types that are created without arguments, +1 > but otherwise, and > especially for user provided exceptions, we should take the safer route of > always instantiating them, like CPython does. +1 -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Question on "numpy_common.pxi" in NumPy tests
On 27 March 2012 17:20, Stefan Behnel wrote: > Hi, > > the NumPy related tests use a file "numpy_common.pxi" that contains this > useless code: > > """ > cdef extern from *: > bint FALSE "0" > void import_array() > void import_umath() > > if FALSE: > import_array() > import_umath() > """ > > Does this serve any purpose? It currently leads to build failures of the > tests in C++ mode because the "import_umath()" call seems to inline code > with a bare "return" statement, which is not allowed in the module init > function (which returns a reference to the module). > > Is there any reason why this can't just die? > Oh! long long ago Cython testsuite was able to run without a single C compiler warning... That numpy common file was a hack to silent "defined but not used" warnings about the import array/umath functions defined in NumPy headers. I think you can safely remove that file, it serves no useful purpose IMHO... -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] never used numpy.pxd, but now my code is failing
Is there any way to disable special-casing of numpy arrays? IMHO, if
I'm not using Cython's numpy.pxd file, Cython should let me decide how
to manage the beast.
Error compiling Cython file:
...
if ((nm != PyArray_DIM(aj, 0)) or
(nm != PyArray_DIM(av, 0)) or
(si*bs * sj*bs != sv)): raise ValueError(
("input arrays have incompatible shapes: "
"rows.shape=%s, cols.shape=%s, vals.shape=%s") %
(ai.shape, aj.shape, av.shape))
^
PETSc/petscmat.pxi:683:11: Cannot convert 'npy_intp *' to Python object
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] never used numpy.pxd, but now my code is failing
On 10 April 2012 22:53, Dag Sverre Seljebotn wrote:
> On 04/10/2012 09:52 PM, Dag Sverre Seljebotn wrote:
>>
>> On 04/10/2012 08:32 PM, Lisandro Dalcin wrote:
>>>
>>> Is there any way to disable special-casing of numpy arrays? IMHO, if
>>> I'm not using Cython's numpy.pxd file, Cython should let me decide how
>>> to manage the beast.
>>>
>>>
>>> Error compiling Cython file:
>>>
>>> ...
>>> if ((nm != PyArray_DIM(aj, 0)) or
>>> (nm != PyArray_DIM(av, 0)) or
>>> (si*bs * sj*bs != sv)): raise ValueError(
>>> ("input arrays have incompatible shapes: "
>>> "rows.shape=%s, cols.shape=%s, vals.shape=%s") %
>>> (ai.shape, aj.shape, av.shape))
>>> ^
>>>
>>>
>>> PETSc/petscmat.pxi:683:11: Cannot convert 'npy_intp *' to Python object
>>>
>>
>> Whoops, sorry about that. I patched on yet another hack here:
>>
>>
>> https://github.com/dagss/cython/commit/6f2271d2b3390d869a53d15b2b70769df029b218
>
>
> BTW, that's the _numpy branch.
>
The fix worked for me.
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Julialang
On 22 April 2012 08:10, Robert Bradshaw wrote: > Yes, Julia looks really cool. It's been on my radar for a while, but I > haven't had a chance to really try it out for anything yet. But I > hadn't thought about low-level Python/Cython <-> Julia integration. > That sounds very interesting. I wonder if Jython could give any > insight into to the tight interaction between two languages that are > usually used in isolation but have been made to call each other > (though there are a lot of differences too, e.g. we're not targeting > replacing the CPython interpreter (on first pass at least...)). > Are you all aware that "calling C" actually means a ctypes-like functionality based in dlopen()/dlsym() ? http://julialang.org/manual/calling-c-and-fortran-code/. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] help getting NumPy into a 32-bit Jenkins build environment
On 28 July 2012 06:37, Stefan Behnel wrote:
> Hi,
>
> I've been trying to set up a 32bit build environment on Jenkins. Most of it
> works nicely by just switching to "gcc -m32", but NumPy fails to build with
> that setup due to the lack of dependencies. It actually finds them when
> looking for their header files in the wrong (64bit) places, and then fails
> to build when it can't find the expected 32bit libraries.
>
> Is there a way to tell NumPy that I really don't care about BLAS, LAPACK
> and all of its friends and just want bare arrays? Or is there an easy way
> to install 32 bit versions of those dependencies on sage.math? I'd be fine
> with having them built and installed into a local directory (namely, in
> standard directory layout under /home/scoder/jenkins/Tools/libs32/).
>
>From doc/source/user/install.rst in numpy-dev:
Disabling ATLAS and other accelerated libraries
---
Usage of ATLAS and other accelerated libraries in Numpy can be disabled
via::
BLAS=None LAPACK=None ATLAS=None python setup.py build
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Bad interaction between cimported types and module cleanup
Basically, the module cleanup function nullifies the type ptr of
cimported parent types, but tp_dealloc slots of children types access
these null pointers, then you get a segfault. Looking at Py_Finalize()
, atexit functions are run BEFORE a gargabe collection step and the
destroy of all modules.
@Stefan, I remember you also had issues with module cleanup and object
deallocation. Did you find a solution?
Could you give a quick look at the following patch? Do you think the
extra pointer deref for getting Py_TYPE(o)->tp_base->tp_dealloc would
be a performance regression? BTW, The "if 0" branch is the original
code, the else branch is my proposed fix.
diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
index 2472de3..8758967 100644
--- a/Cython/Compiler/ModuleNode.py
+++ b/Cython/Compiler/ModuleNode.py
@@ -,7 +,12 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
if base_type:
tp_dealloc = TypeSlots.get_base_slot_function(scope, tp_slot)
if tp_dealloc is None:
-tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname
+if 0:
+# XXX bad interaction between
+# cimported types and module cleanup
+tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname
+else:
+tp_dealloc = "Py_TYPE(o)->tp_base->tp_dealloc"
code.putln(
"%s(o);" % tp_dealloc)
else:
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bad interaction between cimported types and module cleanup
On 3 August 2012 01:07, Stefan Behnel wrote:
> Stefan Behnel, 03.08.2012 06:04:
>> Lisandro, could you open a pull request with only the single line change
>> and a comment that properly explains why we are doing this?
>
> A code comment, just to be clear.
>
Is the comment blow clear enough for you?
Stefan, I've never used github seriously, and making a pull request
will require time I really do not have. The patch is really trivial,
any chance to push it directly? There is a policy is to use pull
requests for every change (sorry, I've not been following Cython
development closely)?
diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py
index 2472de3..255b047 100644
--- a/Cython/Compiler/ModuleNode.py
+++ b/Cython/Compiler/ModuleNode.py
@@ -,7 +,11 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode):
if base_type:
tp_dealloc = TypeSlots.get_base_slot_function(scope, tp_slot)
if tp_dealloc is None:
-tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname
+# Using the cimported base type pointer interacts
+# badly with module cleanup nullyfing these pointers.
+# Use instead the base type pointer from the
+# instance's type pointer.
+tp_dealloc = "Py_TYPE(o)->tp_base->tp_dealloc"
code.putln(
"%s(o);" % tp_dealloc)
else:
--
Lisandro Dalcin
---
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bad interaction between cimported types and module cleanup
On 4 August 2012 17:50, Stefan Behnel wrote: > Stefan Behnel, 04.08.2012 14:59: >> Lisandro Dalcin, 03.08.2012 18:51: >>> diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py >>> index 2472de3..255b047 100644 >>> --- a/Cython/Compiler/ModuleNode.py >>> +++ b/Cython/Compiler/ModuleNode.py >>> @@ -,7 +,11 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): >>> if base_type: >>> tp_dealloc = TypeSlots.get_base_slot_function(scope, tp_slot) >>> if tp_dealloc is None: >>> -tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname >>> +# Using the cimported base type pointer interacts >>> +# badly with module cleanup nullyfing these pointers. >>> +# Use instead the base type pointer from the >>> +# instance's type pointer. >>> +tp_dealloc = "Py_TYPE(o)->tp_base->tp_dealloc" >>> code.putln( >>> "%s(o);" % tp_dealloc) >>> else: >> >> Tried it, doesn't work. The problem is that this always goes through the >> actual type of the object, ignoring the type hierarchy completely, which >> kills the tp_dealloc() call chain and runs into an infinite loop starting >> from the second inheritance level (or a crash because of multiple DECREF >> calls on the same fields). > > Here is a fix that should handle this correctly. > > https://github.com/cython/cython/commit/a8393fa58741c9ae0647e8fdec5fee4ffd91ddf9 > > Basically, it checks the type pointer of cimported types on tp_traverse(), > tp_clear() and tp_dealloc() and in the unlikely case that they are NULL, it > walks the type hierarchy up to the point where it finds the current > function and then calls the one in the base type. > Nice! > It is somewhat involved, but I still doubt that it would lead to a real > slow down. Maybe tp_traverse() is the one that's a bit more time critical > than the others because it can be called more often, but the one additional > NULL check in the normal case still shouldn't hurt all that much. > OK. > It fixes the case in lxml for me, but I didn't shrink that down to a test > case yet. Lisandro, if you find the time, it would be nice if you could > write one up for the problem you ran into. > To properly test this we would need to namespace the __cleanup functions registered with atexit, I mean to name them 'cleanup', that way we could loop over registered functions to and call them "by hand". I you agree, I can take care of this and write some tests for module cleanup. -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Hashability of Python memoryviews
On 30 August 2012 14:54, Stefan Behnel wrote: > Hi, > > there's a ticket on the CPython bug tracker where correct hashability of > the "memoryview" type is being discussed. I don't have an opinion on it, > but some of you might. It could translate into a similar feature for > Cython's array and memory view types (no idea if hashability has been > considered there yet). > readonly != immutable -> unhashable -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] porting mpi4py to PyPy runtime
On 6 November 2012 16:22, Stefan Behnel wrote: > Lisandro Dalcin, 02.10.2012 17:06: >> After successfully porting mpi4py to PyPy 1.9, I want to share a few >> issues I've encountered. The most serious onesare on PyPy's side, but >> Cython do require a cuple of very minor fixes. >> [...] >> 4) mpi4py uses @classmethod decorator for many methods in cdef >> classes. I really do not know what's going on, but method lookups do >> not return the classmethod, but the original "unwrapped" method. I've >> found a way to trick it, but again, I have no idea why it is working. >> Basically, I have to execute hasattr() (in Python code, not Cython >> code) on every class method: The hack reduces to adding the following >> code at the very end of Cython code (note that I'm using "exec"): >> >> if PYPY: exec """ >> def _pypy_setup(): >> for klass in ( >> Status, >> Datatype, >> Request, >> Message, >> Op, >> Group, >> Info, >> Errhandler, >> Comm, >> Win, >> File, >> ): >> for name in klass.__dict__: >> meth = klass.__dict__[name] >> if (isinstance(meth, classmethod) or >> isinstance(meth, staticmethod)): >> hasattr(klass, name) >> _pypy_setup() >> del _pypy_setup >> """ >> >> I think (1) and (3) can be trivially handled in Cython, (3) is not an >> issue for Cython as PyBuffer_FillInfo() is not directly used. And >> about (4), we really need help from PyPy folks, they implement >> PyType_Modified() as a non-op >> (https://bitbucket.org/pypy/pypy/src/release-1.9/pypy/module/cpyext/typeobject.py#cl-726) >> but that is not playing well with Cython. > > https://bugs.pypy.org/issue1106 > > I had already opened a ticket for this one a while ago. > Can you test adding the following lines (add them after the Spam class) to see if them "fix" the issue? Such hint could be useful for PyPy folks: exec """ for name in Spam.__dict__: hasattr(Spam, name) """ -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Commit f2a4b09b broke petsc4py
https://github.com/cython/cython/commit/f2a4b09b94dc0783625dc869af0880742c29f58d I could not figure out how to fix it, but the following patch to the test case reproduces the problem: diff --git a/tests/run/tp_new_cimport.srctree b/tests/run/tp_new_cimport.srctree index d60d712..632172c 100644 --- a/tests/run/tp_new_cimport.srctree +++ b/tests/run/tp_new_cimport.srctree @@ -42,7 +42,7 @@ def test_sub(): a.pxd -cdef class ExtTypeA: +cdef api class ExtTypeA[type ExtTypeA_Type, object ExtTypeAObject]: cdef readonly attrA a.pyx -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Regresion in 0.19
See the example below, the last hunk fails with Cython 0.19. It seems Cython is confusing attribute access with constness. CYTHON=cython echo 'cdef import from *:' > defs.pxd echo 'enum:SIZE' >> defs.pxd echo 'from defs cimport *' > code1.pyx echo 'cdef char buf[SIZE]' >> code1.pyx $CYTHON code1.pyx echo 'cimport defs' > code2.pyx echo 'cdef char buf[defs.SIZE]' >> code2.pyx $CYTHON code2.pyx -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Git tag for 0.19.1?
I cannot find the tag 0.19.1 in the repository? Did you forget to push it? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] "Stack" checker for undefined behaviour in C code
On 31 October 2013 20:25, Stefan Behnel wrote: > Hi, > > I just came across this paper: > > http://pdos.csail.mit.edu/~xi/papers/stack-sosp13.pdf > > They describe an analysis tool that checks C code for bugs that exploit > undefined behaviour and that are thus up to the mercy of compiler > assumptions and "optimisations" to do the right thing or not. They made it > available on github: > > https://github.com/xiw/stack/ > > If anyone wants to take the time to set it up for checking some Cython > generated code, I'd be interested to see if it finds something. > I tested with mpi4py and got 0 warnings. I'll run it on Cython's testsuite. -- Lisandro Dalcin --- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Files with executable bit on
This is in branch 0.19.x. These files should not have the executable bit on. Can any of you please fix them? [dalcinl@macarena cython-dev]$ ls -al $(find Cython -name '*.py') | grep rwx -rwxr-xr-x 1 dalcinl staff4286 Feb 14 2013 Cython/Build/BuildExecutable.py -rwxr-xr-x 1 dalcinl staff 420289 Oct 15 23:39 Cython/Compiler/ExprNodes.py -rwxr-xr-x 1 dalcinl staff 137439 Oct 15 23:39 Cython/Compiler/PyrexTypes.py -- Lisandro Dalcin --- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] "Stack" checker for undefined behaviour in C code
On 1 November 2013 15:55, Lisandro Dalcin wrote: > On 31 October 2013 20:25, Stefan Behnel wrote: >> Hi, >> >> I just came across this paper: >> >> http://pdos.csail.mit.edu/~xi/papers/stack-sosp13.pdf >> >> They describe an analysis tool that checks C code for bugs that exploit >> undefined behaviour and that are thus up to the mercy of compiler >> assumptions and "optimisations" to do the right thing or not. They made it >> available on github: >> >> https://github.com/xiw/stack/ >> >> If anyone wants to take the time to set it up for checking some Cython >> generated code, I'd be interested to see if it finds something. >> > > I tested with mpi4py and got 0 warnings. I'll run it on Cython's testsuite. > > I got two warnings out of 'python setup.py build' Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.17387.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Plex/Actions.17412.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Runtime/refnanny.17706.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Compiler/Code.17659.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Compiler/FlowControl.17614.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Compiler/Lexicon.17437.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Compiler/Visitor.17577.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Compiler/Scanning.17462.ll Analyzing ./build/temp.linux-x86_64-2.7/home/dalcinl/Devel/cython-dev/Cython/Compiler/Parsing.17489.ll Generated 2 warnings, see pstack.txt for details. [dalcinl@kw2060 build.bak]$ cat pstack.txt --- bug: anti-dce model: | %523 = icmp ne i64 %522, 0, !dbg !1139 --> false : %518 = load %struct._object** @PyExc_UnboundLocalError, align 8, !dbg !1140 %519 = call %struct._object* (%struct._object*, i8*, ...)* @PyErr_Format(%struct._object* %518, i8* getelementptr inbounds ([49 x i8]* @.str100, i32 0, i32 0), i8* getelementptr inbounds ([6 x i8]* @.str52, i32 0, i32 0)), !dbg !1140 br label %605, !dbg !1143 stack: - /home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.c:6651:0 - /home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.c:1752:0 ncore: 1 core: - /home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.c:6511:0 - null pointer dereference --- bug: anti-simplify model: | %7699 = icmp ne %struct._object* %78, null, !dbg !4455 --> true stack: - /home/dalcinl/Devel/cython-dev/Cython/Compiler/FlowControl.c:16304:0 ncore: 1 core: - /home/dalcinl/Devel/cython-dev/Cython/Compiler/FlowControl.c:30198:0 - null pointer dereference -- Lisandro Dalcin --- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] "Stack" checker for undefined behaviour in C code
On 1 November 2013 21:53, Stefan Behnel wrote: > > Erm - interesting. I looked through the code lines above and couldn't find > anything that looked suspicious. I hope I used the same source version as > you did (latest master?). Maybe I was just blinded by macros, but what I > saw looked rather reasonable... > Oh! Sorry, I've used branch 0.19.x, I'll check master and report back my findings. -- Lisandro Dalcin --- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] "Stack" checker for undefined behaviour in C code
On 1 November 2013 23:47, Lisandro Dalcin wrote: > On 1 November 2013 21:53, Stefan Behnel wrote: >> >> Erm - interesting. I looked through the code lines above and couldn't find >> anything that looked suspicious. I hope I used the same source version as >> you did (latest master?). Maybe I was just blinded by macros, but what I >> saw looked rather reasonable... >> > > Oh! Sorry, I've used branch 0.19.x, I'll check master and report back > my findings. > OK, this is the commit in master I've used for test: $ git show commit 2cf64fd4e3e1f5d570d856c2e98fdc51930f1a50 Author: Robert Bradshaw Date: Fri Nov 1 09:05:24 2013 -0700 Fix executable bits. I got three warnings, see below. I understand the ones in _tempita.c and FlowControl.c, but not sure about the others. --- bug: anti-dce model: | %523 = icmp ne i64 %522, 0, !dbg !1146 --> false : %518 = load %struct._object** @PyExc_UnboundLocalError, align 8, !dbg !1147 %519 = call %struct._object* (%struct._object*, i8*, ...)* @PyErr_Format(%struct._object* %518, i8* getelementptr inbounds ([49 x i8]* @.str101, i32 0, i32 0), i8* getelementptr inbounds ([6 x i8]* @.str52, i32 0, i32 0)), !dbg !1147 br label %605, !dbg !1150 stack: - /home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.c:6632:0 - /home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.c:1734:0 ncore: 1 core: - /home/dalcinl/Devel/cython-dev/Cython/Plex/Scanners.c:6492:0 - null pointer dereference --- bug: anti-simplify model: | %2387 = icmp ne %struct._object* %__pyx_t_4.1, null, !dbg !3029 --> true stack: - /home/dalcinl/Devel/cython-dev/Cython/Tempita/_tempita.c:24184:0 ncore: 1 core: - /home/dalcinl/Devel/cython-dev/Cython/Tempita/_tempita.c:591:0 - null pointer dereference --- bug: anti-simplify model: | %7850 = icmp ne %struct._object* %78, null, !dbg !4512 --> true stack: - /home/dalcinl/Devel/cython-dev/Cython/Compiler/FlowControl.c:16478:0 ncore: 1 core: - /home/dalcinl/Devel/cython-dev/Cython/Compiler/FlowControl.c:30460:0 - null pointer dereference -- Lisandro Dalcin --- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] MPI_Type_create_hindexed_block() segfaults
I'm very sorry for this bogus email, please ignore.
On 21 April 2014 14:53, Lisandro Dalcin wrote:
> I believe the problem is in the following source code line
> (file:ompi_datatype_args.c, line:221):
>
> https://bitbucket.org/ompiteam/ompi-svn-mirror/src/v1.8/ompi/datatype/ompi_datatype_args.c?at=v1.8#cl-221
>
> I think you should just remove that bogus line, and that's all.
>
>
> [dalcinl@kw2060 openmpi]$ cat type_hindexed_block.c
> #include
> int main(int argc, char *argv[])
> {
> MPI_Aint disps[] = {0};
> MPI_Datatype datatype;
> MPI_Init(&argc, &argv);
> MPI_Type_create_hindexed_block(1, 1, disps, MPI_BYTE, &datatype);
> MPI_Finalize();
> return 0;
> }
> [dalcinl@kw2060 openmpi]$ mpicc type_hindexed_block.c
> [dalcinl@kw2060 openmpi]$ ./a.out
> [kw2060:20304] *** Process received signal ***
> [kw2060:20304] Signal: Segmentation fault (11)
> [kw2060:20304] Signal code: Address not mapped (1)
> [kw2060:20304] Failing at address: 0x6a
> [kw2060:20304] [ 0] /lib64/libpthread.so.0[0x327c40f750]
> [kw2060:20304] [ 1] /lib64/libc.so.6[0x327bc94126]
> [kw2060:20304] [ 2]
> /home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(ompi_datatype_set_args+0x7f1)[0x7f8f0158b62a]
> [kw2060:20304] [ 3]
> /home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(MPI_Type_create_hindexed_block+0x24d)[0x7f8f015cedc8]
> [kw2060:20304] [ 4] ./a.out[0x40080c]
> [kw2060:20304] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x327bc21d65]
> [kw2060:20304] [ 6] ./a.out[0x4006f9]
> [kw2060:20304] *** End of error message ***
> Segmentation fault (core dumped)
>
>
> --
> Lisandro Dalcin
> ---
> CIMEC (UNL/CONICET)
> Predio CONICET-Santa Fe
> Colectora RN 168 Km 472, Paraje El Pozo
> 3000 Santa Fe, Argentina
> Tel: +54-342-4511594 (ext 1016)
> Tel/Fax: +54-342-4511169
--
Lisandro Dalcin
---
CIMEC (UNL/CONICET)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1016)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
[Cython] MPI_Type_create_hindexed_block() segfaults
I believe the problem is in the following source code line
(file:ompi_datatype_args.c, line:221):
https://bitbucket.org/ompiteam/ompi-svn-mirror/src/v1.8/ompi/datatype/ompi_datatype_args.c?at=v1.8#cl-221
I think you should just remove that bogus line, and that's all.
[dalcinl@kw2060 openmpi]$ cat type_hindexed_block.c
#include
int main(int argc, char *argv[])
{
MPI_Aint disps[] = {0};
MPI_Datatype datatype;
MPI_Init(&argc, &argv);
MPI_Type_create_hindexed_block(1, 1, disps, MPI_BYTE, &datatype);
MPI_Finalize();
return 0;
}
[dalcinl@kw2060 openmpi]$ mpicc type_hindexed_block.c
[dalcinl@kw2060 openmpi]$ ./a.out
[kw2060:20304] *** Process received signal ***
[kw2060:20304] Signal: Segmentation fault (11)
[kw2060:20304] Signal code: Address not mapped (1)
[kw2060:20304] Failing at address: 0x6a
[kw2060:20304] [ 0] /lib64/libpthread.so.0[0x327c40f750]
[kw2060:20304] [ 1] /lib64/libc.so.6[0x327bc94126]
[kw2060:20304] [ 2]
/home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(ompi_datatype_set_args+0x7f1)[0x7f8f0158b62a]
[kw2060:20304] [ 3]
/home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(MPI_Type_create_hindexed_block+0x24d)[0x7f8f015cedc8]
[kw2060:20304] [ 4] ./a.out[0x40080c]
[kw2060:20304] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x327bc21d65]
[kw2060:20304] [ 6] ./a.out[0x4006f9]
[kw2060:20304] *** End of error message ***
Segmentation fault (core dumped)
--
Lisandro Dalcin
---
CIMEC (UNL/CONICET)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1016)
Tel/Fax: +54-342-4511169
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] error LNK2001: unresolved external symbol PyInit_init
On 20 August 2014 20:25, Sturla Molden wrote: > To build a Python extension for Win64 you must define the symbol MS_WIN64, > typically -DMS_WIN64. > Sturla, isn't this supposed to be handled in pyconfig.h (at least when using MSVC) ? I see these lines in PC/pyconfig.h (from Python sources): /* MSVC defines _WINxx to differentiate the windows platform types Note that for compatibility reasons _WIN32 is defined on Win32 *and* on Win64. For the same reasons, in Python, MS_WIN32 is defined on Win32 *and* Win64. Win32 only code must therefore be guarded as follows: #if defined(MS_WIN32) && !defined(MS_WIN64) Some modules are disabled on Itanium processors, therefore we have MS_WINI64 set for those targets, otherwise MS_WINX64 */ #ifdef _WIN64 #define MS_WIN64 #endif -- Lisandro Dalcin --- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] error LNK2001: unresolved external symbol PyInit_init
On 14 August 2014 08:20, Andriy Kornatskyy wrote: > When installing either from pip3 or downloading the source with python3 > setup.py install Could you please show us the full output of "python3 setup.py build" ? -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] PyPy3 fixes
1) Stefan, I had to push this fix, just to be sure, please double check. https://github.com/cython/cython/commit/6bd3f7b9e494d1259082aecfc0366da15fc105ec 2) PyPy3 defines some legacy (Py2) Py_TPFLAGS_XXX values. Do you like the following fix? IOW, I check if the value is #define'd, if not I #define it to 0 (zero): diff --git a/Cython/Utility/ModuleSetupCode.c b/Cython/Utility/ModuleSetupCode.c index 6477fb2..3ae8a0c 100644 --- a/Cython/Utility/ModuleSetupCode.c +++ b/Cython/Utility/ModuleSetupCode.c @@ -59,13 +59,16 @@ #define __Pyx_DefaultClassType PyType_Type #endif -#if PY_MAJOR_VERSION >= 3 +#if !defined(Py_TPFLAGS_CHECKTYPES) #define Py_TPFLAGS_CHECKTYPES 0 +#endif +#if !defined(Py_TPFLAGS_HAVE_INDEX) #define Py_TPFLAGS_HAVE_INDEX 0 +#endif +#if !defined(Py_TPFLAGS_HAVE_NEWBUFFER) #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif - -#if PY_VERSION_HEX < 0x030400a1 && !defined(Py_TPFLAGS_HAVE_FINALIZE) +#if !defined(Py_TPFLAGS_HAVE_FINALIZE) #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Py3, TypeConversion.c, PyLong internals
Robert, please look at this line: https://github.com/cython/cython/blob/0.22.x/Cython/Utility/TypeConversion.c#L559 You are still using a PyLong internal detail to test for signedness. I'm getting a strange error with PyPy3 that might be related to this, see this traceback: https://travis-ci.org/mpi4py/mpi4py/jobs/53793609#L5650 The code generating this traceback is linked below, the__setitem__method of the variable "m" is coded to only accept "usigned char" values. https://bitbucket.org/mpi4py/mpi4py/src/86e4235ee2ec31832031ce020421810c51d4910b/test/test_rma.py?at=master#cl-13 Any suggestions about how to check signedness in Python 3 (CPython and PyPy) without using Py_SIZE() ? -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Sorry: 0.22.x and master broken
Folks, I'm very sorry. In an attempt to fix issues with PyPy3, I pushed a fix to 0.22.x (https://github.com/cython/cython/commit/8573e7898a27ef607a3264909cc3bf24d4e13959) and merged to master, but now one test is failing with CPython 3 (https://travis-ci.org/cython/cython/jobs/54762494). I did not run the full test suite with python3, my fault. I'll look at this issue tomorrow and push a fix ASAP. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] PyPy3 fixes
On 18 March 2015 at 17:07, Stefan Behnel wrote: > LGTM. I tend to use the shorter "#ifndef" instead of "#if !defined()", though. Do you want me to change it in master? No problem, just ask for it. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] "long long" versus "PY_LONG_LONG"
At some point Cython lost the ability of using PY_LONG_LONG instead of "long long" in generated C code. The big offenders are the following two files: Cython/Utility/Overflow.c Cython/Utility/TypeConversion.c Do we want to continue using PY_LONG_LONG? Otherwise, what about Microsoft compilers? Going back to PY_LONG_LONG everywhere is quite easy right now, let me know and I'll contribute the patch for the upcoming 0.22.1 release (if that ever happens). -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Bringing Cython and PyPy closer together
On 18 February 2012 at 19:11, Stefan Behnel wrote: > Stefan Behnel, 18.02.2012 09:54: >> Stefan Behnel, 15.02.2012 12:32: >>> The current state of the discussion seems to be that PyPy provides ways to >>> talk to C code, but nothing as complete as CPython's C-API in the sense >>> that it allows efficient two-way communication between C code and Python >>> objects. Thus, we need to either improve this or look for alternatives. >>> >>> In order to get us more focussed on what can be done and what the >>> implications are, so that we may eventually be able to decide what should >>> be done, I started a Wiki page for a PyPy backend CEP (Cython Enhancement >>> Proposal). >>> >>> http://wiki.cython.org/enhancements/pypy >> >> The discussion so far makes me rather certain that the most promising >> short-term solution is to make Cython generate C code that PyPy's cpyext >> can handle. This should get us a rather broad set of running code somewhat >> quickly, while requiring the least design-from-scratch type of work in a >> direction that does not yet allow us to see if it will really make existing >> code work or not. > > Update: > > Amaury Forgeot d'Arc fiddled out a couple of fixes and hacks to make it run > (although with some clear bugs in the exception handling code). There is a > Jenkins job now to (try to) run the test suite of my own branch in the > latest PyPy nightly build: > > https://sage.math.washington.edu:8091/hudson/view/dev-scoder/job/cython-scoder-pypy-nightly/ > > It currently crashes rather badly at some point, but at least it looks like > it's actually getting somewhere. > One thing that Cython developers really need is PyPy defining a macro such as PYPY_VERSION_HEX in such a way us we can properly use conditional compilation. For example, a few days ago I was pushing PyPy fixes to Cython. I tried to use _PyLong_Sign in my patch, but the interpreter broke at runtime. This issue will be eventually fixed, I hope. Unce that happens, how can we know it is save to use the call for that pypy version and upwards? I mean, Cython should still support previous PyPy releases... -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "long long" versus "PY_LONG_LONG"
On 29 March 2015 at 13:33, Stefan Behnel wrote: > Lisandro Dalcin schrieb am 29.03.2015 um 12:17: >> At some point Cython lost the ability of using PY_LONG_LONG instead of >> "long long" in generated C code. The big offenders are the following >> two files: >> >> Cython/Utility/Overflow.c >> Cython/Utility/TypeConversion.c >> >> Do we want to continue using PY_LONG_LONG? Otherwise, what about >> Microsoft compilers? Going back to PY_LONG_LONG everywhere is quite >> easy right now, let me know and I'll contribute the patch for the >> upcoming 0.22.1 release (if that ever happens). > > If PyLL helps in any way, then, by all means, we should use it. > > And yes, there should be a 0.22.1 release. I selected changes for it > already, and saw that you did, too. We should give the current branch > another review to make sure we didn't forget anything else, and then > release it. > OK, please give me a day to push the PyLL patch. Thank you! -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Bringing Cython and PyPy closer together
On 29 March 2015 at 14:16, Stefan Behnel wrote:
> Lisandro Dalcin schrieb am 29.03.2015 um 12:23:
>> One thing that Cython developers really need is PyPy defining a macro
>> such as PYPY_VERSION_HEX in such a way us we can properly use
>> conditional compilation. For example, a few days ago I was pushing
>> PyPy fixes to Cython. I tried to use _PyLong_Sign in my patch, but the
>> interpreter broke at runtime. This issue will be eventually fixed, I
>> hope. Unce that happens, how can we know it is save to use the call
>> for that pypy version and upwards? I mean, Cython should still support
>> previous PyPy releases...
>
> Yes, it's unfortunate that cpyext isn't versioned. But PY_VERSION_HEX
> should still change sometimes over PyPy releases, so that might at least
> help a bit.
>
Well, you know that's not enough ;-)
> In general, however, you shouldn't expect any CPython internals to work in
> PyPy. Avoiding multiple C-API calls when a generic one exists is really the
> best of all strategies. For example, why read just the sign, when you can
> read the whole value? Everything else just risks aggregating slowness and
> running into buggy corners.
>
Well, I agree, however I do require _PyLong_Sign to work if I want to
report correct overflow error messages when converting PyLong values
to a C integral type. In CPython, we are using Py_SIZE for that, not
only for speed, but also to figure out the right overflow kind when
converting to unsigned integers.
> That being said, if you find anything that can be optimised, I'm happy to
> learn about your changes.
>
Again, this is not about optimization, but proper error reporting.
diff --git a/Cython/Utility/TypeConversion.c b/Cython/Utility/TypeConversion.c
index 4cd8229..fbac649 100644
--- a/Cython/Utility/TypeConversion.c
+++ b/Cython/Utility/TypeConversion.c
@@ -560,6 +560,10 @@ static CYTHON_INLINE {{TYPE}}
{{FROM_PY_FUNCTION}}(PyObject *x) {
if (unlikely(Py_SIZE(x) < 0)) {
goto raise_neg_overflow;
}
+#elif CYTHON_COMPILING_IN_PYPY
+if (unlikely(_PyLong_Sign(x) < 0)) {
+goto raise_neg_overflow;
+}
#endif
if (sizeof({{TYPE}}) <= sizeof(unsigned long)) {
__PYX_VERIFY_RETURN_INT({{TYPE}}, unsigned long,
PyLong_AsUnsignedLong(x))
--
Lisandro Dalcin
Research Scientist
Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
Numerical Porous Media Center (NumPor)
King Abdullah University of Science and Technology (KAUST)
http://numpor.kaust.edu.sa/
4700 King Abdullah University of Science and Technology
al-Khawarizmi Bldg (Bldg 1), Office # 4332
Thuwal 23955-6900, Kingdom of Saudi Arabia
http://www.kaust.edu.sa
Office Phone: +966 12 808-0459
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] "long long" versus "PY_LONG_LONG"
On 29 March 2015 at 14:19, Lisandro Dalcin wrote: > On 29 March 2015 at 13:33, Stefan Behnel wrote: >> Lisandro Dalcin schrieb am 29.03.2015 um 12:17: >>> At some point Cython lost the ability of using PY_LONG_LONG instead of >>> "long long" in generated C code. The big offenders are the following >>> two files: >>> >>> Cython/Utility/Overflow.c >>> Cython/Utility/TypeConversion.c >>> >>> Do we want to continue using PY_LONG_LONG? Otherwise, what about >>> Microsoft compilers? Going back to PY_LONG_LONG everywhere is quite >>> easy right now, let me know and I'll contribute the patch for the >>> upcoming 0.22.1 release (if that ever happens). >> >> If PyLL helps in any way, then, by all means, we should use it. >> >> And yes, there should be a 0.22.1 release. I selected changes for it >> already, and saw that you did, too. We should give the current branch >> another review to make sure we didn't forget anything else, and then >> release it. >> > > OK, please give me a day to push the PyLL patch. Thank you! > Done: [0.22.x] https://github.com/cython/cython/commit/60f0e2f99c3e587bc6d267608ad13e0f384b527c [master] https://github.com/cython/cython/commit/2e245fc1e3881ed66ce9e8a5e07c44071b1a499e -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Bringing Cython and PyPy closer together
On 29 March 2015 at 16:10, Stefan Behnel wrote: > Why not call PyObject_RichCompareBool() to make cpyext itself compare the > value to 0? That should definitely work. Let me try to put a patch together. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Bringing Cython and PyPy closer together
On 29 March 2015 at 20:30, Lisandro Dalcin wrote: > On 29 March 2015 at 16:10, Stefan Behnel wrote: >> Why not call PyObject_RichCompareBool() to make cpyext itself compare the >> value to 0? > > That should definitely work. Let me try to put a patch together. > Stefan, how to you feel about abusing of Py_False in the call to RichCompareBool ? -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Bringing Cython and PyPy closer together
On 29 March 2015 at 21:02, Stefan Behnel wrote: > Lisandro Dalcin schrieb am 29.03.2015 um 19:40: >> On 29 March 2015 at 20:30, Lisandro Dalcin wrote: >>> On 29 March 2015 at 16:10, Stefan Behnel wrote: >>>> Why not call PyObject_RichCompareBool() to make cpyext itself compare the >>>> value to 0? >>> >>> That should definitely work. Let me try to put a patch together. >> >> Stefan, how to you feel about abusing of Py_False in the call to >> RichCompareBool ? > > Wow - evil idea! :) > > But then, it's only for cpyext - as long as it works there and comes with a > comment, it's certainly as simple as it gets. bool is clearly defined as an > int subtype in Python. > > Just remember to check the error code. Py_SIZE() can't fail in CPython, but > comparisons can, sadly. > Please review: https://github.com/cython/cython/pull/377 Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Python 3 wheels including bytecode files.
After getting AppVeyor to work, I noticed that Python 3 wheels do include __pycache__ directory. This also happens with in OS X as well as Linux. I think the root of the issue resides in setup.py, in the build_ext command class workaround that seems to be a legacy thing from the time Cython required 2to3. BTW, in setup.py, compile_cython_modules() follows quite a different path in Python 2 versus 3. Unless some of you believe we should keep things that way, I'll try to fix setup.py to be as much independent as possible of the major Python version. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python 3 wheels including bytecode files.
On 2 October 2015 at 14:30, Stefan Behnel wrote: > Ah - note that Python 3.2 still uses the unicode fixer of 2to3, so the code > is not entirely legacy. Would be nice if it could be moved out of the way, > though. Especially since it will eventually disappear entirely. Yes, I'm aware of this. I'll push a fix in branch 0.23.x (I'm doing my tests in OS X), then see what happens with the AppVeyor builds. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python 3 wheels including bytecode files.
On 2 October 2015 at 15:07, Lisandro Dalcin wrote: > On 2 October 2015 at 14:30, Stefan Behnel wrote: >> Ah - note that Python 3.2 still uses the unicode fixer of 2to3, so the code >> is not entirely legacy. Would be nice if it could be moved out of the way, >> though. Especially since it will eventually disappear entirely. > > Yes, I'm aware of this. I'll push a fix in branch 0.23.x (I'm doing my > tests in OS X), then see what happens with the AppVeyor builds. > AppVeyor builds succeeded: https://ci.appveyor.com/project/cython/cython/build/0.23.x-4 The Python 3 wheels no longer have *.pyc files, so I'm confident this is fixed for other platforms as well. Stefan, could you manage to merge branch 0.23.x into master? PS: At some point, we should try to really run the testsuite in AppVeyor. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python 3 wheels including bytecode files.
On 2 October 2015 at 17:01, Stefan Behnel wrote: > Lisandro Dalcin schrieb am 02.10.2015 um 15:18: >> On 2 October 2015 at 15:07, Lisandro Dalcin wrote: >>> On 2 October 2015 at 14:30, Stefan Behnel wrote: >>>> Ah - note that Python 3.2 still uses the unicode fixer of 2to3, so the code >>>> is not entirely legacy. Would be nice if it could be moved out of the way, >>>> though. Especially since it will eventually disappear entirely. >>> >>> Yes, I'm aware of this. I'll push a fix in branch 0.23.x (I'm doing my >>> tests in OS X), then see what happens with the AppVeyor builds. >> >> AppVeyor builds succeeded: >> https://ci.appveyor.com/project/cython/cython/build/0.23.x-4 >> >> The Python 3 wheels no longer have *.pyc files, so I'm confident this >> is fixed for other platforms as well. >> >> Stefan, could you manage to merge branch 0.23.x into master? > > Done, thanks! github just told me that my merge landed as commit 1. :) > > I also changed a couple of minor things: > > https://github.com/cython/cython/commit/64da2480e88d8cd3eec2b9a5d2922307623c3cd5 > I don't like your change. Before, the self-cythonization was triggeres when you actually needed to compile the extensions, now a simple "setup.py --version" will do. Try yourself: $ git clean -dxf $ python setup.py --version > I guess the > ># XXX hack for '*.pyx ' sources > > referred to the setuptools quirk that stubbornly replaces .pyx by .c? > Yes, indeed. > >> PS: At some point, we should try to really run the testsuite in AppVeyor. > > What's not working there? > Right now, the "test" step in AppVeyor invokes "python setup.py test", which basically does nothing. I mean, you should run "python runtests.py" intead. https://github.com/cython/cython/blob/0.23.x/appveyor.yml#L73 -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Python 3 wheels including bytecode files.
On 3 October 2015 at 15:30, Stefan Behnel wrote: > Let's try it in master first. > > https://github.com/cython/cython/commit/bc88b4ae02510d68014839769a1ed47faa7af077 Stefan, I'm starting to fix issues here and there. I have no idea how to fix some tests, in such cases I'm just dissabling them at the command line. At some point, I'll need help to resolve them. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython 0.23.4 released
On 10 October 2015 at 15:46, Stefan Behnel wrote: > Hi everyone, > > here is a minor bug-fix release for the 0.23 release series that repairs > some problems and regressions. > > https://pypi.python.org/pypi/Cython/0.23.4 > > Complete changelog follows below. > > > You can get the signed release from here: > > http://cython.org/ > > http://cython.org/release/Cython-0.23.4.tar.gz > > http://cython.org/release/Cython-0.23.4.zip > > > SHA1 sums: > fc574c5050cd5a8e34435432e2a4a693353ed807 Cython-0.23.4.tar.gz > ad477040a22190b5506a3d8ddde646b7586d5703 Cython-0.23.4.zip > Stefan, should we upload wheels to PyPI? It is a bit of manual work, as we would need to download artifacts for every build here: https://ci.appveyor.com/project/cython/cython/build/0.23.x-31 then use twine to do the uploads. I do not have the PyPI password to do all by myself. -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Issue with function annotations
$ cython --version Cython version 0.27 $ cat tmp.pyx def f() -> (list, list): return [], [] $ cython tmp.pyx Error compiling Cython file: ... def f() -> (list, list): ^ tmp.pyx:1:12: C struct/union member cannot be a Python object Error compiling Cython file: ... def f() -> (list, list): ^ tmp.pyx:1:12: C struct/union member cannot be a Python object -- Lisandro Dalcin Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 0109 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459 ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
