Re: [Cython] Surprising behaviour wrt. generated tp_clear and tp_dealloc functions

[Torsten Landschoff]

On 04/23/2013 08:01 AM, Stefan Behnel wrote:
> Greg Ewing, 23.04.2013 01:16:
>> Only if subclassing of Slot and Context are forbidden.
> Right. Subtypes of a non-GC type can happily add attributes and start
> supporting cyclic garbage collection, including Python subclasses. So this
> only applies to "final" types and builtins. Not entirely uncommon,
> especially in the "keep this private thing alive until all referrers have
> died" use case, but I's say this restriction drops the priority a bit.
>
Does Cython have an equivalent of the "final class DoNoExtend { ... }"
of the Java world?
In any case, Greg has a good point. I seriously did not think about it
just because I did not plan to derive from those classes.

Greetings, Torsten

-- 
DYNAmore Gesellschaft fuer Ingenieurdienstleistungen mbH
Torsten Landschoff

Office Dresden
Tel: +49-(0)351-4519587
Fax: +49-(0)351-4519561

mailto:torsten.landsch...@dynamore.de
http://www.dynamore.de

DYNAmore Gesellschaft für FEM Ingenieurdienstleistungen mbH
Registration court: Stuttgart, HRB 733694
Managing director: Prof. Dr. Karl Schweizerhof, Dipl.-Math. Ulrich Franz

___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Surprising behaviour wrt. generated tp_clear and tp_dealloc functions

[Stefan Behnel]

Torsten Landschoff, 23.04.2013 10:06:
> On 04/23/2013 08:01 AM, Stefan Behnel wrote:
>> Greg Ewing, 23.04.2013 01:16:
>>> Only if subclassing of Slot and Context are forbidden.
>> Right. Subtypes of a non-GC type can happily add attributes and start
>> supporting cyclic garbage collection, including Python subclasses. So this
>> only applies to "final" types and builtins. Not entirely uncommon,
>> especially in the "keep this private thing alive until all referrers have
>> died" use case, but I's say this restriction drops the priority a bit.
>>
> Does Cython have an equivalent of the "final class DoNoExtend { ... }"
> of the Java world?

  @cython.final
  cdef class ExtType:
  ...

You can also declare a class @cython.internal to prevent it from showing up
in the module dict.

Stefan

___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel

[Cython] libc.string msvc incompatibility

[Dave Hirschfeld]

When trying to use strcasecmp from libc.string I get an error compiling with 
msvc:

error C3861: 'strcasecmp': identifier not found

It seems MS have decided to call it by another name - _stricmp

...amongst others:

http://botsikas.blogspot.co.uk/2011/12/strcasecmp-identifier-not-found-
when.html


I'm wondering if it's possible for cython to include the mentioned ifdef 
defines by default or otherwise solve the problem so that code is portable 
between compilers on Windows? I understand if it's not a high priority issue.

The workaround is for me to make sure all our code compiles with mingw which 
was on the radar anyway.

Thanks,
Dave

___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] libc.string msvc incompatibility

[Robert Bradshaw]

It would at the very least be worth putting a note on these functions,
but I'm not sure how far we should go with aliasing the
platform-specific alternatives. Certainly a .h file that you "cdef
extern from" include with these defines would be the shortest path to
getting things working on your side.

On Tue, Apr 23, 2013 at 7:56 AM, Dave Hirschfeld
 wrote:
>
> When trying to use strcasecmp from libc.string I get an error compiling with
> msvc:
>
> error C3861: 'strcasecmp': identifier not found
>
> It seems MS have decided to call it by another name - _stricmp
>
> ...amongst others:
>
> http://botsikas.blogspot.co.uk/2011/12/strcasecmp-identifier-not-found-
> when.html
>
>
> I'm wondering if it's possible for cython to include the mentioned ifdef
> defines by default or otherwise solve the problem so that code is portable
> between compilers on Windows? I understand if it's not a high priority issue.
>
> The workaround is for me to make sure all our code compiles with mingw which
> was on the radar anyway.
>
> Thanks,
> Dave
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> http://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel

[Cython] Unit testing Cython extensions

[Torsten Landschoff]

Hi again,

I am just wondering how to do unit testing on extensions built using
Cython with py.test.

My problem: The extension module I am working on is installed in our
global python environment (inside the build slaves as well as on local
machines) already. Before installing a new version of the extension
module, I'd like to run the unit tests. However, that way the original
module is tested.

I am trying to illustrate.

Doing TDD I create a new module which will fail the test:


torsten@sharokan:~/foo$ cat foo.pyx
def it_works():
return False

torsten@sharokan:~/foo$ cat tests/test_foo.py
import pprint, foo

def test_foo():
pprint.pprint(foo.__file__)
assert foo.it_works()

torsten@sharokan:~/foo$ py.test -v tests
 test
session starts

platform linux2 -- Python 2.7.3 -- pytest-2.3.4 --
/opt/dynasdk/loco2-precise/bin/python
plugins: cov, capturelog
collected 0 items / 1 errors

==
ERRORS
===
 ERROR
collecting tests/test_foo.py
_
tests/test_foo.py:1: in 
>   import sys, pprint, foo
E   ImportError: No module named foo
== 1 error
in 0.01 seconds
==

Sure, module foo does not exists. I could use pyximport, but I want to
check that the extension itself is correctly built.
Easily done:

torsten@sharokan:~/foo$ python setup.py build_ext -i
running build_ext
cythoning foo.pyx to foo.c
building 'foo' extension
creating build
creating build/temp.linux-x86_64-2.7
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes -fPIC
-I/opt/dynasdk/loco2-precise/include/python2.7 -c foo.c -o
build/temp.linux-x86_64-2.7/foo.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/foo.o
-L/opt/dynasdk/loco2-precise/lib -lpython2.7 -o /home/torsten/foo/foo.so

But alas, py.test still will not find foo.so, which is now installed
into the current directory (due to using the "-i" flag to build_ext).
Workaround: Overwrite PYTHONPATH:

torsten@sharokan:~/foo$ PYTHONPATH=`pwd` py.test
 test
session starts

platform linux2 -- Python 2.7.3 -- pytest-2.3.4
plugins: cov, capturelog
collected 1 items

tests/test_foo.py F

=
FAILURES
==
_
test_foo
__

def test_foo():
pprint.pprint(foo.__file__)
>   assert foo.it_works()
E   assert ()
E+  where  = foo.it_works

tests/test_foo.py:5: AssertionError
--
Captured stdout
--
'/home/torsten/foo/foo.so'
= 1 failed
in 0.02 seconds
==

Now it really loads our extension module and the failure is actually
genuine. Let's assume we have the old version installed in our current
Python environment:

torsten@sharokan:~/foo$ python setup.py install
[...]
creating

/usr/opt/dynasdk/loco2-precise/lib/python2.7/site-packages/foo-0.0-py2.7-linux-x86_64.egg
Extracting foo-0.0-py2.7-linux-x86_64.egg to
/usr/opt/dynasdk/loco2-precise/lib/python2.7/site-packages
Adding foo 0.0 to easy-install.pth file

Installed

/usr/opt/dynasdk/loco2-precise/lib/python2.7/site-packages/foo-0.0-py2.7-linux-x86_64.egg
[...]

Great. Now let's implement the feature that our test checks:

torsten@sharokan:~/foo$ vim foo.pyx
torsten@sharokan:~/foo$ cat foo.pyx
def it_works():
return True

The unit tests should pass now, after we rebuilt the extension:

torsten@sharokan:~/foo$ python setup.py build_ext -i
running build_ext
cythoning foo.pyx to foo.c
building 'foo' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes -fPIC
-I/opt/dynasdk/loco2-precise/include/python2.7 -c foo.c -o
build/temp.linux-x86_64-2.7/foo.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/foo.o
-L/opt/dynasdk/loco2-precise/lib -lpython2.

Re: [Cython] Unit testing Cython extensions

[Robert Bradshaw]

There is nothing special about Cython here, if you have "foo.py"
instead of "foo.so" you would be seeing exactly the same results. You
need to either set PYTHONPATH or have foo be in the same package
layout as your tests.

On Tue, Apr 23, 2013 at 1:51 PM, Torsten Landschoff
 wrote:
> Hi again,
>
> I am just wondering how to do unit testing on extensions built using Cython
> with py.test.
>
> My problem: The extension module I am working on is installed in our global
> python environment (inside the build slaves as well as on local machines)
> already. Before installing a new version of the extension module, I'd like
> to run the unit tests. However, that way the original module is tested.
>
> I am trying to illustrate.
>
> Doing TDD I create a new module which will fail the test:
>
>
> torsten@sharokan:~/foo$ cat foo.pyx
> def it_works():
> return False
>
> torsten@sharokan:~/foo$ cat tests/test_foo.py
> import pprint, foo
>
> def test_foo():
> pprint.pprint(foo.__file__)
> assert foo.it_works()
>
> torsten@sharokan:~/foo$ py.test -v tests
>  test session
> starts 
> platform linux2 -- Python 2.7.3 -- pytest-2.3.4 --
> /opt/dynasdk/loco2-precise/bin/python
> plugins: cov, capturelog
> collected 0 items / 1 errors
>
> == ERRORS
> ===
>  ERROR collecting
> tests/test_foo.py _
> tests/test_foo.py:1: in 
>>   import sys, pprint, foo
> E   ImportError: No module named foo
> == 1 error in 0.01
> seconds ==
>
> Sure, module foo does not exists. I could use pyximport, but I want to check
> that the extension itself is correctly built.
> Easily done:
>
> torsten@sharokan:~/foo$ python setup.py build_ext -i
> running build_ext
> cythoning foo.pyx to foo.c
> building 'foo' extension
> creating build
> creating build/temp.linux-x86_64-2.7
> gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall
> -Wstrict-prototypes -fPIC -I/opt/dynasdk/loco2-precise/include/python2.7 -c
> foo.c -o build/temp.linux-x86_64-2.7/foo.o
> gcc -pthread -shared build/temp.linux-x86_64-2.7/foo.o
> -L/opt/dynasdk/loco2-precise/lib -lpython2.7 -o /home/torsten/foo/foo.so
>
> But alas, py.test still will not find foo.so, which is now installed into
> the current directory (due to using the "-i" flag to build_ext). Workaround:
> Overwrite PYTHONPATH:
>
> torsten@sharokan:~/foo$ PYTHONPATH=`pwd` py.test
>  test session
> starts 
> platform linux2 -- Python 2.7.3 -- pytest-2.3.4
> plugins: cov, capturelog
> collected 1 items
>
> tests/test_foo.py F
>
> = FAILURES
> ==
> _ test_foo
> __
>
> def test_foo():
> pprint.pprint(foo.__file__)
>>   assert foo.it_works()
> E   assert ()
> E+  where  = foo.it_works
>
> tests/test_foo.py:5: AssertionError
> -- Captured
> stdout --
> '/home/torsten/foo/foo.so'
> = 1 failed in 0.02
> seconds ==
>
> Now it really loads our extension module and the failure is actually
> genuine. Let's assume we have the old version installed in our current
> Python environment:
>
> torsten@sharokan:~/foo$ python setup.py install
> [...]
> creating
> /usr/opt/dynasdk/loco2-precise/lib/python2.7/site-packages/foo-0.0-py2.7-linux-x86_64.egg
> Extracting foo-0.0-py2.7-linux-x86_64.egg to
> /usr/opt/dynasdk/loco2-precise/lib/python2.7/site-packages
> Adding foo 0.0 to easy-install.pth file
>
> Installed
> /usr/opt/dynasdk/loco2-precise/lib/python2.7/site-packages/foo-0.0-py2.7-linux-x86_64.egg
> [...]
>
> Great. Now let's implement the feature that our test checks:
>
> torsten@sharokan:~/foo$ vim foo.pyx
> torsten@sharokan:~/foo$ cat foo.pyx
> def it_works():
> return True
>
> The unit tests should pass now, after we rebuilt the extension:
>
> torsten@sharokan:~/foo$ python setup.py build_ext -i
> running build_ext
> cythoning foo.pyx to foo.c
> building 'foo' extension
> gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall
> -Wstrict-prototypes -fPIC -I/opt/dynasdk/loco2-precise/include/python2.7 -c
> foo.c

[Cython] Cython code producing different, incorrect results under Python 2.7 (not 3.x) under Cython 0.19

[Josh Warner]

Hi Cython devs,

Over in scikit-image we have traced an odd problem with a particular Cython
file to the 0.19 update of Cython. From at least Cython 0.15.1 (probably
earlier) through 0.18, `_mcp.pyx` in `skimage.graph` compiled and executed
correctly, passing all package tests on both Python 2.7 and Python 3.

After 0.19 was released and the Travis builds began using it, we began
getting 100% repeatable errors from the previously clean master branch
(example of an otherwise clean Python
2.7Travis
build; the
Python 3 build 
passedall
tests). All of these errors/failures trace back to this Cython file.
Oddly, the errors only happen on Python 2.7; our Python 3 Travis build
passes.

We are discussing this issue in scikit-image Github Issue
#534;
feel free to join the discussion there.

The .pyx Cython file is located
hereand
it has
an associated .pxd file
here.
It should be noted the file compiles and executes without errors, but its
output is now incorrect in Python 2.x.

In case the compiled results might be relevant, for your diffing pleasure here
is the compiled .c file from Cython
0.18which
passes all tests on both Python 2.7 and Python 3.x, while here
is the compiled .c file from Cython
0.19which
produces different, incorrect results in Python 2.7.

In the short term we are temporarily forcing Travis to use the 0.18 release
of Cython, but that isn't a viable long term solution.

It's possible the error is on our end, but seeing as it worked with prior
Cython releases we'd appreciate you taking a look.

Thanks!
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython code producing different, incorrect results under Python 2.7 (not 3.x) under Cython 0.19

[Stefan Behnel]

Hi,

Josh Warner, 24.04.2013 08:06:
> Over in scikit-image we have traced an odd problem with a particular Cython
> file to the 0.19 update of Cython. From at least Cython 0.15.1 (probably
> earlier) through 0.18, `_mcp.pyx` in `skimage.graph` compiled and executed
> correctly, passing all package tests on both Python 2.7 and Python 3.
> 
> After 0.19 was released and the Travis builds began using it, we began
> getting 100% repeatable errors from the previously clean master branch
> (example of an otherwise clean Python
> 2.7Travis
> build; the
> Python 3 build 
> passedall
> tests). All of these errors/failures trace back to this Cython file.
> Oddly, the errors only happen on Python 2.7; our Python 3 Travis build
> passes.
> 
> We are discussing this issue in scikit-image Github Issue
> #534;
> feel free to join the discussion there.
> 
> The .pyx Cython file is located
> hereand
> it has
> an associated .pxd file
> here.
> It should be noted the file compiles and executes without errors, but its
> output is now incorrect in Python 2.x.
> 
> In case the compiled results might be relevant, for your diffing pleasure here
> is the compiled .c file from Cython
> 0.18which
> passes all tests on both Python 2.7 and Python 3.x, while here
> is the compiled .c file from Cython
> 0.19which
> produces different, incorrect results in Python 2.7.
> 
> In the short term we are temporarily forcing Travis to use the 0.18 release
> of Cython, but that isn't a viable long term solution.
> 
> It's possible the error is on our end, but seeing as it worked with prior
> Cython releases we'd appreciate you taking a look.

Thanks for bringing this up. You could make it a little easier for us by
pointing us at the code that produces the incorrect results you are
experiencing. The set of failing tests seems to be quite small, but before
we start digging through your code, I'm sure you can provide pointers to
the relevant code snippets for a couple of these tests (i.e. the test code
itself and the major code parts that produce the results) much more quickly.

Stefan

___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython code producing different, incorrect results under Python 2.7 (not 3.x) under Cython 0.19

[Josh Warner]

Hi Stefan,

Every quantitative test is failing for _mcp.pyx (in
skimage.graph.tests.test_mcp.py), suggesting the problem is fairly global
to the MCP class. The three errors in skimage.graph.tests.test_spath.py are
due to incorrect MCP results which have no valid traceback path. No need to
investigate _spath.pyx.

The pure Python utility functions in _mcp.pyx appear to return correct
output, and I do not believe MCP.traceback is the culprit, rather, the
actual generated graph is incorrect. These errors appear in the MCP class,
so MCP_Geometric (subclass of MCP) is not the source.

Thus, I strongly suspect the problem lies within the MCP class, most likely
in MCP.find_costs(). Every failing test directly or indirectly uses this
method, and several of them directly test (and fail) on the returned cost
array.

Hopefully that helps,
Josh
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel