[Cython] Add support for the offsetof() C macro
Hi, offsetof() is not supported by current Cython, and I have not found any workaround (except hardcoding offsets for a specific architecture and compiler, but this is obviously wrong). offsetof(struct_type, member) is a C macro that expands to the offset of the member in the struct, in bytes. Could you add support for it, perhaps like sizeof as a special case in the parser? And many thanks for the whole project! ☺ -- Emmanuel Gil Peyrot XMPP: OpenPGP: 24B1D609 pgpNAHf9fQpmU.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Compiler crash on unmatched kwarg
Hi, When the name of a kwarg in the definition and the declaration of a function doesn’t match, the compiler crash instead of returning a meaningful error. I have joined a testcase, it is reproductible at least on 0.19.1 and latest git. -- Emmanuel Gil Peyrot XMPP: OpenPGP: 24B1D609 testcase.tar.xz Description: Binary data pgpDMlz63gHBQ.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Assign from array to scalar
Hi,
A few weeks ago I started an implementation of that simple usecase:
cdef long a[2], x, y
# set a[0] and a[1] to some useful values
x, y = a
It does works, but I’m not used to the interactions between the
different parts of the code, I mainly hacked away where I could. I’m
especially dissatisfied with the code duplication I came with (to check
the correct types and size) and the location of the checks.
I added testcases for both the array assignation and the slices of that
same assignation I plan to implement, which seems to integrate
badly with my current implementation, making the code even more
unreadable.
I tried to find a commit implementing a similar feature so I could get
some inspiration but couldn’t find one. If someone has a hint about
that I’d be glad. :)
Btw, on a Cython.Compiler.ExprNodes.SliceIndexNode object, why does
stop_code() give an int while start_code() gives a string? It’d be
much better to return an int in both cases, so one could interpolate
without conversion from both python or the generated C.
Preliminary patch joined.
--
Emmanuel Gil Peyrot
XMPP:
OpenPGP: 24B1D609
diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py
index 2511eaf..bf04ff1 100755
--- a/Cython/Compiler/ExprNodes.py
+++ b/Cython/Compiler/ExprNodes.py
@@ -5615,6 +5615,11 @@ class SequenceNode(ExprNode):
def generate_assignment_code(self, rhs, code):
if self.starred_assignment:
self.generate_starred_assignment_code(rhs, code)
+elif (isinstance(self, TupleNode) and
+ isinstance(rhs, NameNode) and
+ isinstance(rhs.entry.type, PyrexTypes.CArrayType)):
+self.generate_parallel_assignment_code2(rhs, code)
+return
else:
self.generate_parallel_assignment_code(rhs, code)
@@ -5627,6 +5632,22 @@ class SequenceNode(ExprNode):
PyrexTypes.CFuncTypeArg("it", PyrexTypes.py_object_type, None),
]))
+def generate_parallel_assignment_code2(self, rhs, code):
+rhs_type = rhs.entry.type
+base_type = rhs_type.base_type
+size = rhs_type.size
+lhs_args = self.args
+assert len(lhs_args) == size
+for arg in lhs_args:
+if arg.entry.type is not base_type:
+raise ValueError("Wrong type for parallel assignment of '%s'
array: %s" %
+ (base_type, arg.entry.type))
+for i, arg in enumerate(lhs_args):
+code.putln("%s = %s[%s];" % (
+arg.result(),
+rhs.result(),
+i))
+
def generate_parallel_assignment_code(self, rhs, code):
# Need to work around the fact that generate_evaluation_code
# allocates the temps in a rather hacky way -- the assignment
diff --git a/Cython/Compiler/Nodes.py b/Cython/Compiler/Nodes.py
index fe7adb3..7987cc6 100644
--- a/Cython/Compiler/Nodes.py
+++ b/Cython/Compiler/Nodes.py
@@ -4509,6 +4509,20 @@ class SingleAssignmentNode(AssignmentNode):
self.lhs = self.lhs.analyse_target_types(env)
self.lhs.gil_assignment_check(env)
+if (isinstance(self.lhs, ExprNodes.TupleNode) and
+isinstance(self.rhs, ExprNodes.NameNode)):
+rhs_type = self.rhs.entry.type
+if isinstance(rhs_type, PyrexTypes.CArrayType):
+base_type = rhs_type.base_type
+size = rhs_type.size
+lhs_args = self.lhs.args
+assert len(lhs_args) == size
+for arg in lhs_args:
+if arg.entry.type is not base_type:
+break
+else:
+return self
+
if self.lhs.memslice_broadcast or self.rhs.memslice_broadcast:
self.lhs.memslice_broadcast = True
self.rhs.memslice_broadcast = True
diff --git a/tests/run/arrayassign.pyx b/tests/run/arrayassign.pyx
index 29c353e..f76ad2e 100644
--- a/tests/run/arrayassign.pyx
+++ b/tests/run/arrayassign.pyx
@@ -128,6 +128,59 @@ def test_ptr_literal_list_slice_end():
a[:5] = [1,2,3,4,5]
return (a[0], a[1], a[2], a[3], a[4])
+
+# The opposite, assignation from an array to multiple scalars.
+
+def test_array_to_scalars():
+"""
+>>> test_array_to_scalars()
+(1, 2, 3, 4, 5)
+"""
+cdef int l[5], a, b, c, d, e
+l[:] = [1,2,3,4,5]
+a, b, c, d, e = l
+return (a, b, c, d, e)
+
+def test_slice_all_to_scalars():
+"""
+>>> test_slice_all_to_scalars()
+(1, 2, 3, 4, 5)
+"""
+cdef int l[5], a, b, c, d, e
+l[:] = [1,2,3,4,5]
+a, b, c, d, e = l[:]
+return (a, b, c, d, e)
+
+def test_slice_start_to_scalars():
+"""
+>>> test_slice_start_to_scalars()
+(1, 2, 3, 4, 5)
+""
[Cython] Wrong order of __Pyx_DECREF when calling a function with an implicit str → char* conversion.
Hi, I was testing my cython codebase on top of pypy/cpyext, and I found a memory corruption. After investigating it with the pypy guys (thanks arigato!), we identified it as a cython bug: cdef void test(char *string): print(string) def run(array): test(array[0]) This code works fine under cpython, but looking at the generated code for the last line, the call to test is done with a pointer (__pyx_t_2) to some potentially deallocated string (__pyx_t_1), which isn’t freed just yet on cpython but is on pypy: __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_array, 0, …); __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_AsString(__pyx_t_1); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_f_1a_test(__pyx_t_2); The obvious solution is to swap the last two lines, it then works fine on pypy (although not necessarily if the test function stores the pointer somewhere, but it’s not cython’s fault then). This issue can also happen with an explicit cast: pointer = array[0] test(pointer) I’m not sure if it should be addressed the same way, because that would mean keeping a reference to array[0] for all the lifetime of the current scope, but it could still prevent obscure bugs like the memory corruption I had. -- Emmanuel Gil Peyrot pgpq8i4NyR8z2.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] [PATCH 1/2] Move ~/.pyxbld to $XDG_CACHE_HOME/pyxbld
---
pyximport/pyximport.py | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/pyximport/pyximport.py b/pyximport/pyximport.py
index 4fd7fe9..710c5eb 100644
--- a/pyximport/pyximport.py
+++ b/pyximport/pyximport.py
@@ -466,9 +466,10 @@ def install(pyximport=True, pyimport=False,
build_dir=None, build_in_temp=True,
will not work for most .py files, and will therefore only slow
down your imports. Use at your own risk.
-By default, compiled modules will end up in a ``.pyxbld``
-directory in the user's home directory. Passing a different path
-as ``build_dir`` will override this.
+By default, compiled modules will end up in a ``pyxbld`` directory
+in the directory pointed by the ``XDG_CACHE_HOME`` environment
+variable, or in ``~/.cache/pyxbld`` if ``XDG_CACHE_HOME`` isn’t
+set. Passing a different path as ``build_dir`` will override this.
``build_in_temp=False`` will produce the C files locally. Working
with complex dependencies and debugging becomes more easy. This
@@ -501,8 +502,9 @@ def install(pyximport=True, pyimport=False, build_dir=None,
build_in_temp=True,
runtime for .py files and Py2 for .pyx files.
"""
if not build_dir:
-build_dir = os.path.join(os.path.expanduser('~'), '.pyxbld')
-
+build_dir = os.path.join(os.environ.get('XDG_CACHE_HOME',
+os.path.join(os.path.expanduser('~'), '.cache')), 'pyxbld')
+
global pyxargs
pyxargs = PyxArgs() #$pycheck_no
pyxargs.build_dir = build_dir
--
2.3.5
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
[Cython] [PATCH 2/2] Make the Cython cache directory fallback to ~/.cache/cython if $XDG_CACHE_HOME isn’t set
---
Cython/Utils.py | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/Cython/Utils.py b/Cython/Utils.py
index 22f6db7..563989f 100644
--- a/Cython/Utils.py
+++ b/Cython/Utils.py
@@ -327,7 +327,8 @@ def get_cython_cache_dir():
1. CYTHON_CACHE_DIR
2. (OS X): ~/Library/Caches/Cython
(posix not OS X): XDG_CACHE_HOME/cython if XDG_CACHE_HOME defined
-3. ~/.cython
+3. ~/.cache/cython
+4. ~/.cython
"""
if 'CYTHON_CACHE_DIR' in os.environ:
@@ -338,8 +339,8 @@ def get_cython_cache_dir():
if sys.platform == 'darwin':
parent = os.path.expanduser('~/Library/Caches')
else:
-# this could fallback on ~/.cache
-parent = os.environ.get('XDG_CACHE_HOME')
+parent = os.environ.get('XDG_CACHE_HOME',
+os.path.join(os.path.expanduser('~'), '.cache'))
if parent and os.path.isdir(parent):
return os.path.join(parent, 'cython')
--
2.3.5
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [PATCH 1/2] Move ~/.pyxbld to $XDG_CACHE_HOME/pyxbld
Actually, I think it’d be better to move the pyxbld directory directly
under the cython cache directory, as it is part of cython itself.
Here is an attached patch updated with this behaviour.
--
Emmanuel Gil Peyrot
[PATCH 1/2] Move ~/.pyxbld to $XDG_CACHE_HOME/cython/pyxbld
---
pyximport/pyximport.py | 14 +-
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/pyximport/pyximport.py b/pyximport/pyximport.py
index 4fd7fe9..abed8e6 100644
--- a/pyximport/pyximport.py
+++ b/pyximport/pyximport.py
@@ -466,9 +466,11 @@ def install(pyximport=True, pyimport=False, build_dir=None, build_in_temp=True,
will not work for most .py files, and will therefore only slow
down your imports. Use at your own risk.
-By default, compiled modules will end up in a ``.pyxbld``
-directory in the user's home directory. Passing a different path
-as ``build_dir`` will override this.
+By default, compiled modules will end up in a ``cython/pyxbld``
+directory in the directory pointed by the ``XDG_CACHE_HOME``
+environment variable, or in ``~/.cache/cython/pyxbld`` if
+``XDG_CACHE_HOME`` isn’t set. Passing a different path as
+``build_dir`` will override this.
``build_in_temp=False`` will produce the C files locally. Working
with complex dependencies and debugging becomes more easy. This
@@ -501,8 +503,10 @@ def install(pyximport=True, pyimport=False, build_dir=None, build_in_temp=True,
runtime for .py files and Py2 for .pyx files.
"""
if not build_dir:
-build_dir = os.path.join(os.path.expanduser('~'), '.pyxbld')
-
+build_dir = os.path.join(os.environ.get('XDG_CACHE_HOME',
+os.path.join(os.path.expanduser('~'), '.cache')),
+'cython', 'pyxbld')
+
global pyxargs
pyxargs = PyxArgs() #$pycheck_no
pyxargs.build_dir = build_dir
--
2.3.5
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
[Cython] [PATCH] Add a pass transforming Python-style properties in cdef class into Cython-style properties.
This makes properties work at all, in cdef classes, and gives them
almost the same features as the “property something:” blocks. The only
missing feature being the ability to assignate a docstring to the
property itself, not only to the getter, setter and deleter.
Fixes T264.
---
Cython/Compiler/ParseTreeTransforms.py | 72
Cython/Compiler/Pipeline.py | 3 +-
tests/run/cdef_class_property_decorator_T264.pyx | 51 +
3 files changed, 125 insertions(+), 1 deletion(-)
create mode 100644 tests/run/cdef_class_property_decorator_T264.pyx
diff --git a/Cython/Compiler/ParseTreeTransforms.py
b/Cython/Compiler/ParseTreeTransforms.py
index b34f44b..67507ce 100644
--- a/Cython/Compiler/ParseTreeTransforms.py
+++ b/Cython/Compiler/ParseTreeTransforms.py
@@ -1270,6 +1270,78 @@ class WithTransform(CythonTransform, SkipDeclarations):
return node
+class PropertyTransform(ScopeTrackingTransform):
+"""Originally, this was the only place where decorators were
+transformed into the corresponding calling code. Now, this is
+done directly in DefNode and PyClassDefNode to avoid reassignments
+to the function/class name - except for cdef class methods. For
+those, the reassignment is required as methods are originally
+defined in the PyMethodDef struct.
+
+The IndirectionNode allows DefNode to override the decorator
+"""
+
+properties = {}
+
+def visit_DefNode(self, func_node):
+func_node = self.visit_FuncDefNode(func_node)
+if self.scope_type != 'cclass' or not func_node.decorators:
+return func_node
+return self.handle_decorators(func_node, func_node.decorators,
+ func_node.name)
+
+def handle_decorators(self, node, decorators, name):
+properties = self.properties.setdefault(self.scope_node, {})
+for decorator in decorators[::-1]:
+decorator_node = decorator.decorator
+if (isinstance(decorator_node, ExprNodes.NameNode) and
+decorator_node.name == 'property'):
+node.name = '__get__'
+node.decorators.remove(decorator)
+stat_list = [node]
+if name in properties:
+property = properties[name]
+property.body.stats = stat_list
+return []
+else:
+property = Nodes.PropertyNode(node.pos, name=name)
+property.doc = EncodedString(u'')
+property.body = Nodes.StatListNode(node.pos,
stats=stat_list)
+properties[name] = property
+return [property]
+elif (isinstance(decorator_node, ExprNodes.AttributeNode) and
+decorator_node.attribute == 'setter' and
+decorator_node.obj.name in properties):
+assert decorator_node.obj.name == name
+node.name = '__set__'
+node.decorators.remove(decorator)
+property = properties[name]
+stats = property.body.stats
+for i, stat in enumerate(stats):
+if stat.name == '__set__':
+stats[i] = node
+break
+else:
+stats.append(node)
+return []
+elif (isinstance(decorator_node, ExprNodes.AttributeNode) and
+decorator_node.attribute == 'deleter' and
+decorator_node.obj.name in properties):
+assert decorator_node.obj.name == name
+node.name = '__del__'
+node.decorators.remove(decorator)
+property = properties[name]
+stats = property.body.stats
+for i, stat in enumerate(stats):
+if stat.name == '__del__':
+stats[i] = node
+break
+else:
+stats.append(node)
+return []
+return node
+
+
class DecoratorTransform(ScopeTrackingTransform, SkipDeclarations):
"""Originally, this was the only place where decorators were
transformed into the corresponding calling code. Now, this is
diff --git a/Cython/Compiler/Pipeline.py b/Cython/Compiler/Pipeline.py
index 833e22c..0506aff 100644
--- a/Cython/Compiler/Pipeline.py
+++ b/Cython/Compiler/Pipeline.py
@@ -171,7 +171,7 @@ def create_pipeline(context, mode, exclude_classes=()):
from .ParseTreeTransforms import CreateClosureClasses, MarkClosureVisitor,
DecoratorTransform
from .ParseTreeTransforms import InterpretCompilerDirectives,
TransformBuiltinMethods
from .ParseTreeTransforms import ExpandInplaceOperators,
ParallelRangeTransform
-from .ParseTreeTransforms import CalculateQualifiedNamesTransform
+from .ParseTreeTransfor
[Cython] Implementing tp_richcmp using __eq__ etc.
Hello, I’m trying to implement the compilation of __eq__, __ne__, etc. in extension types, to have them available in pure mode (see ticket #130). I currently have an implementation which basically pastes the body.stats from those six functions in a __richcmp__ one using a template, but I find that very inelegant, as it removes the information about the original location of this code which is very bad for debugging, forces all six (or less) functions to use the same argument names (or rename them), and probably some other issues. So my idea was to convert each of those special function into a CFuncDefNode from a DefNode, in order to remove the Python wrapper, then to implement the new tp_richcmp slot as an util function directly in C. If this plan is sound, I’d like a few more informations, for now mostly about the DefNode to CFuncDefNode conversion, which I’ve been unable to do programmatically, with some later passes complaining about missing base_type or declarator properties. The next question is about determining in the C code whether a specific method exists, and otherwise returning NotImplemented instead. Thanks for listening, I’m not sure I’ve been very understandable, I’m still quite confused. ^^' -- Emmanuel Gil Peyrot signature.asc Description: PGP signature ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Manylinux wheels for Cython
On Mon, Mar 28, 2016 at 10:54:21AM -0700, Matthew Brett wrote: […] > I uploaded manylinux wheels for Cython 0.23.5. > > If you're on Linux, and you upgrade pip to 8.1.1 (current) you should > now get Cython via a manylinux wheel by default. > > Please do test and let me know of any problems. Thanks, this significantly improves the install-time required for poezio for people who don’t already have cython installed. Do you plan on publishing AArch64 and ARMv7h wheels as well? Those are the systems most impacted by the compile time of cython. > > Cheers, > > Matthew > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel -- Emmanuel Gil Peyrot ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
