Re: [Cython] [cython-users] Re: Cython 0.17 beta 1 released
On Wed, 01 Aug 2012, mark florisson wrote: > Thanks for the fix. I also pushed a fix for one more test numpy_test > related to fused types dispatching. That passes all tests for me on 32 > bit linux. > > Yaroslav, could you give it a try on the Debian build servers? FWIW -- 0.16rc1-550-g8880c78 hasn't added nor fixed any of the failures on s390x in comparison to 0.16rc1-547-g68811fa ;-) -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.17 beta 1 released
Bradley M. Froehle, 01.08.2012 18:35: > Yes, this versioning has also impacted mpi4py which had to add some pretty > ugly code in setup.py to work around it: > > https://code.google.com/p/mpi4py/source/detail?r=841e9df > >> I am not sure what is the status on PEP 386 [1] (not yet adopted afaik) It's in state "Accepted". > but it might be worthwhile following it and/or existing >> disutils.version.StrictVersion since having >> >> In [2]: Cython.__version__ >> Out[2]: '0.17.beta1' >> >> is not digested well by some depending projects -- there were 2 failures to >> build against this beta release of Cython in Debian due to difficulty with >> parsing the version -- bzr and xpra I believe [2]. So may be it is worth >> making it '0.17b1' (and use corresponding tags accordingly)? I guess so. That would fit the StrictVersion scheme. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.17 beta 1 released
On Thu, 02 Aug 2012, Stefan Behnel wrote:
> Bradley M. Froehle, 01.08.2012 18:35:
> > Yes, this versioning has also impacted mpi4py which had to add some pretty
> > ugly code in setup.py to work around it:
> > https://code.google.com/p/mpi4py/source/detail?r=841e9df
> >> I am not sure what is the status on PEP 386 [1] (not yet adopted afaik)
> It's in state "Accepted".
ah right -- now it is in python3 (but not in python2):
(git)novo:~/proj/misc/cpython[2.7]git
$> git grep NormalizedVersion 2.7
$> git grep NormalizedVersion master
master:Doc/library/packaging.version.rst:.. class:: NormalizedVersion(self, s,
error_on_huge_major_num=True)
master:Doc/library/packaging.version.rst: >>> NormalizedVersion('1.2b1') <
NormalizedVersion('1.2')
master:Doc/library/packaging.version.rst: :class:`NormalizedVersion` is used
internally by :class:`VersionPredicate` to
master:Doc/library/packaging.version.rst: :class:`NormalizedVersion`.
master:Doc/library/packaging.version.rst: >>>
NormalizedVersion("irrational_version_number")
master:Lib/packaging/command/bdist_msi.py:from packaging.version import
NormalizedVersion
master:Lib/packaging/command/bdist_msi.py:class MSIVersion(NormalizedVersion):
master:Lib/packaging/errors.py:See `error_on_huge_major_num` option in
`NormalizedVersion` for details.
master:Lib/packaging/pypi/dist.py:from packaging.version import
(suggest_normalized_version, NormalizedVersion,
master:Lib/packaging/pypi/dist.py:self._version =
NormalizedVersion(version)
master:Lib/packaging/tests/test_version.py:from packaging.version import
NormalizedVersion as V
master:Lib/packaging/tests/test_version.py:
self.assertEqual(repr(V('1.0')), "NormalizedVersion('1.0')")
master:Lib/packaging/tests/test_version.py:TypeError: cannot compare
NormalizedVersion and str
master:Lib/packaging/tests/test_version.py:TypeError: cannot compare
NormalizedVersion and str
master:Lib/packaging/version.py:__all__ = ['NormalizedVersion',
'suggest_normalized_version',
master:Lib/packaging/version.py:class NormalizedVersion:
master:Lib/packaging/version.py:"""Create a NormalizedVersion instance
from a version string.
master:Lib/packaging/version.py:the introduction of
`NormalizedVersion` was version numbers like
master:Lib/packaging/version.py:if not isinstance(other,
NormalizedVersion):
master:Lib/packaging/version.py:if not isinstance(other,
NormalizedVersion):
master:Lib/packaging/version.py:If you have a version string that isn't
rational (i.e. NormalizedVersion
master:Lib/packaging/version.py:- 2312 (53.93%) match NormalizedVersion
without change
master:Lib/packaging/version.py:NormalizedVersion(s)
master:Lib/packaging/version.py:NormalizedVersion(rs)
master:Lib/packaging/version.py:return comp, NormalizedVersion(version)
master:Lib/packaging/version.py:version = NormalizedVersion(version)
> >> parsing the version -- bzr and xpra I believe [2]. So may be it is worth
> >> making it '0.17b1' (and use corresponding tags accordingly)?
> I guess so. That would fit the StrictVersion scheme.
just a side-note -- I didn't know that StrictVersion doesn't play nicely with
LooseVersion to any degree:
$> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
as SV; print SV("0.17") > LV("0.18")'
True
at least in python3 it pukes:
$> python3 -c 'from distutils.version import LooseVersion as LV, StrictVersion
as SV; print(SV("0.17") > LV("0.18"))'
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.2/distutils/version.py", line 70, in __gt__
c = self._cmp(other)
File "/usr/lib/python3.2/distutils/version.py", line 179, in _cmp
if self.version < other.version:
TypeError: unorderable types: tuple() < list()
--
Yaroslav O. Halchenko
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.17 beta 1 released
Yaroslav Halchenko, 02.08.2012 19:02:
> just a side-note -- I didn't know that StrictVersion doesn't play nicely with
> LooseVersion to any degree:
>
> $> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
> as SV; print SV("0.17") > LV("0.18")'
> True
>
> at least in python3 it pukes:
>
> $> python3 -c 'from distutils.version import LooseVersion as LV,
> StrictVersion as SV; print(SV("0.17") > LV("0.18"))'
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib/python3.2/distutils/version.py", line 70, in __gt__
> c = self._cmp(other)
> File "/usr/lib/python3.2/distutils/version.py", line 179, in _cmp
> if self.version < other.version:
> TypeError: unorderable types: tuple() < list()
That's surprising at first sight, but I don't think it hurts all that much.
People would normally use either of them, not both.
Anyway, the next release candidate of Cython will be called 0.17c1. Both
the LooseVersion and the NormalizedVersion should be able to handle that.
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.17 beta 1 released
On Thu, 02 Aug 2012, Stefan Behnel wrote:
> > just a side-note -- I didn't know that StrictVersion doesn't play nicely
> > with LooseVersion to any degree:
> > $> python -c 'from distutils.version import LooseVersion as LV,
> > StrictVersion as SV; print SV("0.17") > LV("0.18")'
> > True
> > at least in python3 it pukes:
> > $> python3 -c 'from distutils.version import LooseVersion as LV,
> > StrictVersion as SV; print(SV("0.17") > LV("0.18"))'
> > Traceback (most recent call last):
> > File "", line 1, in
> > File "/usr/lib/python3.2/distutils/version.py", line 70, in __gt__
> > c = self._cmp(other)
> > File "/usr/lib/python3.2/distutils/version.py", line 179, in _cmp
> > if self.version < other.version:
> > TypeError: unorderable types: tuple() < list()
> That's surprising at first sight, but I don't think it hurts all that much.
> People would normally use either of them, not both.
I understand that but absence of error in python2.x case (thus suggesting
supporting such a comparison) is misleading at least
> Anyway, the next release candidate of Cython will be called 0.17c1. Both
> the LooseVersion and the NormalizedVersion should be able to handle that.
not quite:
1.
$> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
as SV; print(SV("0.17") > SV("0.17c1"))'
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/distutils/version.py", line 40, in __init__
self.parse(vstring)
File "/usr/lib/python2.7/distutils/version.py", line 107, in parse
raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '0.17c1'
so it needs to be 0.17b1 I guess
$> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
as SV; print(SV("0.17") > SV("0.17b1"))'
True
2. LooseVersion is too loose to be used with standardized suffixes, so it
would not sort those release candidates appropriately
$> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
as SV; print(LV("0.17") > LV("0.17b1"))'
False
--
Yaroslav O. Halchenko
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Bad interaction between cimported types and module cleanup
Basically, the module cleanup function nullifies the type ptr of cimported parent types, but tp_dealloc slots of children types access these null pointers, then you get a segfault. Looking at Py_Finalize() , atexit functions are run BEFORE a gargabe collection step and the destroy of all modules. @Stefan, I remember you also had issues with module cleanup and object deallocation. Did you find a solution? Could you give a quick look at the following patch? Do you think the extra pointer deref for getting Py_TYPE(o)->tp_base->tp_dealloc would be a performance regression? BTW, The "if 0" branch is the original code, the else branch is my proposed fix. diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py index 2472de3..8758967 100644 --- a/Cython/Compiler/ModuleNode.py +++ b/Cython/Compiler/ModuleNode.py @@ -,7 +,12 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): if base_type: tp_dealloc = TypeSlots.get_base_slot_function(scope, tp_slot) if tp_dealloc is None: -tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname +if 0: +# XXX bad interaction between +# cimported types and module cleanup +tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname +else: +tp_dealloc = "Py_TYPE(o)->tp_base->tp_dealloc" code.putln( "%s(o);" % tp_dealloc) else: -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Fwd: [ANN] SIAM Conference on Computational Science & Engineering Submission Deadlines Approaching!
Dear Colleagues, the SIAM CSE13 conference will be held next year in Boston, and this is a conference that is well suited for much of the type of work that goes on in the open source scientific Python development community (and Julia). The conference is co-chaired by Hans-Petter Langtangen, well known around these parts for his several books on scientific computing with Python and for having led a campus-wide adoption of Python as the core computational foundation across the University of Oslo. I am also on the program committee, as well as Randy LeVeque and other Python-friendly folks. An excellent way to participate is to organize a one- or two-part minisymposium on a specific topic with a group of related speakers (instructions at http://www.siam.org/meetings/cse13/submissions.php). Please note that the MS deadline is fast approaching: August 13, 2012. If you have any further questions, don't hesitate to contact me or one of the other organizers if you feel they can address your concerns more directly: "Fernando Perez" "Randy LeVeque" (Reproducible research track) "Hans Petter Langtangen" (Conference co-chair) "Karen Willcox" (conference chair) -- Forwarded message -- From: Karen Willcox Date: Tue, Jul 24, 2012 at 6:08 AM Subject: [SIAM-CSE] SIAM Conference on Computational Science & Engineering Submission Deadlines Approaching! To: siam-...@siam.org *SIAM Conference on Computational Science & Engineering (CSE13)* February 25-March 1, 2013 The Westin Boston Waterfront, Boston, Massachusetts, USA ** ** SUBMISSION DEADLINES ARE APPROACHING! August 13, 2012: Minisymposium proposals September 10, 2012: Abstracts for contributed and minisymposium speakers Visit http://www.siam.org/meetings/cse13/submissions.php to submit. ** ** Twitter hashtag: #SIAMcse13 ** ** For more information about the conference, visit * http://www.siam.org/meetings/cse13/* or contact SIAM Conference Department at meeti...@siam.org. -- Karen Willcox Professor and Associate Department Head Department of Aeronautics and Astronautics, MIT http://acdl.mit.edu/willcox.html ___ SIAM-CSE mailing list To post messages to the list please send them to: siam-...@siam.org http://lists.siam.org/mailman/listinfo/siam-cse ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Add includes to generated header files
I'm working on some C++ projects and I'm using cython to embed python code. In an effort to make things easy for C++ devs not as familiar with cython/python I've made a small change to insert #includes in generated header files the same as is done in generated c code. This allows one who is using code processed with cython to just include the header and not worry about any other headers that may be required to match data types. Not having these headers is especially problematic because otherwise the error message presented at compilation can sometimes be quite cryptic. Patch is attached. Any reason this should not be done? Thanks, Jeff 0001-Updated-to-include-headers-in-.h-file-when-generated.patch Description: Binary data ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Add includes to generated header files
Jeff Copeland, 03.08.2012 02:18: > I'm working on some C++ projects and I'm using cython to embed python > code. In an effort to make things easy for C++ devs not as familiar > with cython/python I've made a small change to insert #includes in > generated header files the same as is done in generated c code. This > allows one who is using code processed with cython to just include the > header and not worry about any other headers that may be required to > match data types. > > Not having these headers is especially problematic because otherwise > the error message presented at compilation can sometimes be quite > cryptic. > > Patch is attached. Any reason this should not be done? Hmm, didn't try your patch, but I guess it inserts includes for *all* header files that the module that exports the C-API uses internally, right? If so, I'm not sure that's always wanted. I see the advantage, sure, but it may expose a lot of implementation details that external code may not normally have to care about. On the other hand, if extension types are exported, for example, it's somewhat unlikely that their struct would compile without at least some of those header files, because they almost always wrap some kind of externally defined struct, pointer or whatever kind of other data type. That sounds to me like we should make it a configurable option, something like "capi_reexport_cincludes". Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bad interaction between cimported types and module cleanup
Lisandro Dalcin, 03.08.2012 00:36: > Basically, the module cleanup function nullifies the type ptr of > cimported parent types, but tp_dealloc slots of children types access > these null pointers, then you get a segfault. Looking at Py_Finalize() > , atexit functions are run BEFORE a gargabe collection step and the > destroy of all modules. > > @Stefan, I remember you also had issues with module cleanup and object > deallocation. Did you find a solution? Nothing but setting the cleanup code level from 3 down to 2. > Could you give a quick look at the following patch? Looks like it should handle this case, yes. Maybe we could explicitly only apply it to cimported (i.e. non module local) types, but I guess that's what "tp_dealloc is None" already handles. > Do you think the > extra pointer deref for getting Py_TYPE(o)->tp_base->tp_dealloc would > be a performance regression? I have no idea, but I doubt that it would be any significant, given that the call is an indirect one already and that CPython is about to do something with its heap memory management shortly afterwards, with potentially a couple of additional DECREF calls in between. All of this should be costly enough to make the difference negligible. > diff --git a/Cython/Compiler/ModuleNode.py b/Cython/Compiler/ModuleNode.py > index 2472de3..8758967 100644 > --- a/Cython/Compiler/ModuleNode.py > +++ b/Cython/Compiler/ModuleNode.py > @@ -,7 +,12 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): > if base_type: > tp_dealloc = TypeSlots.get_base_slot_function(scope, tp_slot) > if tp_dealloc is None: > -tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname > +if 0: > +# XXX bad interaction between > +# cimported types and module cleanup > +tp_dealloc = "%s->tp_dealloc" % base_type.typeptr_cname > +else: > +tp_dealloc = "Py_TYPE(o)->tp_base->tp_dealloc" > code.putln( > "%s(o);" % tp_dealloc) > else: Any other opinions on this? Lisandro, could you open a pull request with only the single line change and a comment that properly explains why we are doing this? Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Bad interaction between cimported types and module cleanup
Stefan Behnel, 03.08.2012 06:04: > Lisandro, could you open a pull request with only the single line change > and a comment that properly explains why we are doing this? A code comment, just to be clear. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Add includes to generated header files
On Thu, Aug 2, 2012 at 8:43 PM, Stefan Behnel wrote: > Jeff Copeland, 03.08.2012 02:18: >> I'm working on some C++ projects and I'm using cython to embed python >> code. In an effort to make things easy for C++ devs not as familiar >> with cython/python I've made a small change to insert #includes in >> generated header files the same as is done in generated c code. This >> allows one who is using code processed with cython to just include the >> header and not worry about any other headers that may be required to >> match data types. >> >> Not having these headers is especially problematic because otherwise >> the error message presented at compilation can sometimes be quite >> cryptic. >> >> Patch is attached. Any reason this should not be done? > > Hmm, didn't try your patch, but I guess it inserts includes for *all* > header files that the module that exports the C-API uses internally, right? Yes, you've got it. Certainly quick and dirty to satisfy my immediate need. > If so, I'm not sure that's always wanted. I see the advantage, sure, but it > may expose a lot of implementation details that external code may not > normally have to care about. > > On the other hand, if extension types are exported, for example, it's > somewhat unlikely that their struct would compile without at least some of > those header files, because they almost always wrap some kind of externally > defined struct, pointer or whatever kind of other data type. Exactly. > That sounds to me like we should make it a configurable option, something > like "capi_reexport_cincludes". I like that thought. The other thing I'd though of is perhaps some syntax in the pyx itself to designate an include file as one that should show in a generated header. When I was looking for a solution to the issue that's what I started looking for in the documents but of course came up empty. I figured it may be something like the case for inclusion in the c file but with a public/api keyword thrown in: cdef extern public from "spam.h": pass Syntax change would probably be more involved. I'll have a go at doing a patch with the option added. Do you mean as a command-line option to cython? Jeff ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Re: Cython 0.17 beta 1 released
Yaroslav Halchenko, 02.08.2012 21:54:
> On Thu, 02 Aug 2012, Stefan Behnel wrote:
>>> just a side-note -- I didn't know that StrictVersion doesn't play nicely
>>> with LooseVersion to any degree:
>
>>> $> python -c 'from distutils.version import LooseVersion as LV,
>>> StrictVersion as SV; print SV("0.17") > LV("0.18")'
>>> True
>
>>> at least in python3 it pukes:
>
>>> $> python3 -c 'from distutils.version import LooseVersion as LV,
>>> StrictVersion as SV; print(SV("0.17") > LV("0.18"))'
>>> Traceback (most recent call last):
>>> File "", line 1, in
>>> File "/usr/lib/python3.2/distutils/version.py", line 70, in __gt__
>>> c = self._cmp(other)
>>> File "/usr/lib/python3.2/distutils/version.py", line 179, in _cmp
>>> if self.version < other.version:
>>> TypeError: unorderable types: tuple() < list()
>
>> That's surprising at first sight, but I don't think it hurts all that much.
>> People would normally use either of them, not both.
>
> I understand that but absence of error in python2.x case (thus suggesting
> supporting such a comparison) is misleading at least
>
>> Anyway, the next release candidate of Cython will be called 0.17c1. Both
>> the LooseVersion and the NormalizedVersion should be able to handle that.
>
> not quite:
>
> 1.
>
> $> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
> as SV; print(SV("0.17") > SV("0.17c1"))'
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib/python2.7/distutils/version.py", line 40, in __init__
> self.parse(vstring)
> File "/usr/lib/python2.7/distutils/version.py", line 107, in parse
> raise ValueError, "invalid version number '%s'" % vstring
> ValueError: invalid version number '0.17c1'
>
>
> so it needs to be 0.17b1 I guess
>
> $> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
> as SV; print(SV("0.17") > SV("0.17b1"))'
> True
>
>
> 2. LooseVersion is too loose to be used with standardized suffixes, so it
> would not sort those release candidates appropriately
>
> $> python -c 'from distutils.version import LooseVersion as LV, StrictVersion
> as SV; print(LV("0.17") > LV("0.17b1"))'
> False
Great. Then we'll have a "release candidate" called "beta" for the sake of
backwards compatibility. I'm not entirely happy about that.
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] help getting NumPy into a 32-bit Jenkins build environment
Lisandro Dalcin, 31.07.2012 18:59: > On 28 July 2012 06:37, Stefan Behnel wrote: >> I've been trying to set up a 32bit build environment on Jenkins. Most of it >> works nicely by just switching to "gcc -m32", but NumPy fails to build with >> that setup due to the lack of dependencies. It actually finds them when >> looking for their header files in the wrong (64bit) places, and then fails >> to build when it can't find the expected 32bit libraries. >> >> Is there a way to tell NumPy that I really don't care about BLAS, LAPACK >> and all of its friends and just want bare arrays? Or is there an easy way >> to install 32 bit versions of those dependencies on sage.math? I'd be fine >> with having them built and installed into a local directory (namely, in >> standard directory layout under /home/scoder/jenkins/Tools/libs32/). >> > >>From doc/source/user/install.rst in numpy-dev: > > > Disabling ATLAS and other accelerated libraries > --- > > Usage of ATLAS and other accelerated libraries in Numpy can be disabled > via:: > > BLAS=None LAPACK=None ATLAS=None python setup.py build Cool, looks like that did the trick. https://sage.math.washington.edu:8091/hudson/job/py27m-ext-hg/ARCH=m32/12/ Thanks! Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
