Re: [Cython] python 2.7/3.x and numpy-dev (Dag, I need a quick comment)
On 02/17/2011 07:11 PM, Lisandro Dalcin wrote:
I'm working on a patch to get old, recent, and dev NumPy working in
2.7/3.x. So far, I had success, but I still have two failures like the
one pasted below.
Dag, could you elaborate a bit about the purpose of
__Pyx_BufFmt_CheckString() ? It is just a validity check for pep 3118
format strings? Do you expect the failure below to be hard to fix?s.
Yes, it compares the string with the RTTI that is saved for the type
that is expected by the code.
I remember there was something about the '=' specifier that meant it was
not completely trivial to fix, but still, it's just about doing it. The
code in question is a bit convoluted; the reason I did that is because I
wanted to allow things to match if the binary layouts matched even if
the 'struct structure' wasn't the same...
As for user code, a quick hack around this is 'cast=True' in the buffer
spec.
Please file a bug...
Dag Sverre
Just in case, the format string that triggers the failure is:
memoryview(np.zeros((1,), dtype=np.dtype('b,i', align=False))).format
'T{b:f0:=i:f1:}'
==
FAIL: numpy_test ()
Doctest: numpy_test
--
Traceback (most recent call last):
File "/usr/local/python/3.2/lib/python3.2/doctest.py", line 2113, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for numpy_test
File "/u/dalcinl/Devel/Cython/cython/BUILD/run/c/numpy_test.cpython-32dm.so",
line 1, in numpy_test
--
File "/u/dalcinl/Devel/Cython/cython/BUILD/run/c/numpy_test.cpython-32dm.so",
line 155, in numpy_test
Failed example:
print(test_packed_align(np.zeros((1,), dtype=np.dtype('b,i',
align=False
Exception raised:
Traceback (most recent call last):
File "/usr/local/python/3.2/lib/python3.2/doctest.py", line 1248, in
__run
compileflags, 1), test.globs)
File "", line 1, in
print(test_packed_align(np.zeros((1,), dtype=np.dtype('b,i',
align=False
File "numpy_test.pyx", line 404, in numpy_test.test_packed_align
(numpy_test.c:6367)
ValueError: Buffer packing mode currently only allowed at
beginning of format string (this is a defect)
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] How to update python grammar in Windows?
I tried to add a statement to Python, but I don't know how to update the grammar header files such as graminit.c&graminit.h after I updated the grammar file (Garmmar/Garmmar). It seems pgenmain.c could do this, but these is no pgen.exe in the build output of Visual Studio. How can I update the grammar files in Windows? Thanks, Thomson ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] How to update python grammar in Windows?
Thomson, 18.02.2011 10:10: I tried to add a statement to Python, but I don't know how to update the grammar header files such as graminit.c&graminit.h after I updated the grammar file (Garmmar/Garmmar). It seems pgenmain.c could do this, but these is no pgen.exe in the build output of Visual Studio. How can I update the grammar files in Windows? Wrong list, actually doubly so. This is the core developers mailing list of the Cython project. Welcome! :) You may want to ask your question on comp.lang.python (or "python-l...@python.org"). Or maybe the python-dev list on python.org, in case it's really CPython core development related (in which case you should be subscribed to python-dev anyway). But I'd try c.l.py first. Stefan ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fixing NumPy support for Python 3 (Stefan, please help!)
Lisandro Dalcin, 18.02.2011 15:38:
On 18 February 2011 02:19, Stefan Behnel wrote:
Lisandro Dalcin, 17.02.2011 17:24:
OK, I've found an alternative workaround. What do you think?
diff --git a/Cython/Compiler/Interpreter.py
b/Cython/Compiler/Interpreter.py
index 83cb184..9fb5fe5 100644
--- a/Cython/Compiler/Interpreter.py
+++ b/Cython/Compiler/Interpreter.py
@@ -6,6 +6,7 @@ For now this only covers parse tree to value conversion of
compile-time values.
"""
+import sys
from Nodes import *
from ExprNodes import *
from Errors import CompileError
@@ -44,6 +45,10 @@ def interpret_compiletime_options(optlist, optdict,
type_env=None, type_args=())
else:
raise CompileError(node.pos, "Type not allowed here.")
else:
+if (sys.version_info[0]>=3 and
+isinstance(node, StringNode) and
+node.unicode_value is not None):
+return (node.unicode_value, node.pos)
return (node.compile_time_value(empty_scope), node.pos)
if optlist:
@@ -52,6 +57,7 @@ def interpret_compiletime_options(optlist, optdict,
type_env=None, type_args=())
assert isinstance(optdict, DictNode)
new_optdict = {}
for item in optdict.key_value_pairs:
-new_optdict[item.key.value] = interpret(item.value,
item.key.value)
+new_key, dummy = interpret(item.key, None)
+new_optdict[new_key] = interpret(item.value, item.key.value)
optdict = new_optdict
return (optlist, new_optdict)
This still isn't something that looks right. It just does the same thing at
a different place. Actually, I'm not sure there is a way to "get it right".
IIUC, the purpose of that code is to bring stuff from .pyx code to
Python world at Cython runtime, that code is trying to "coerce" a
DictNode object to a regular Python dict to be used later for
implementing compiler directives. So, if your pyx/pxd says
{"mode":"strided"} , and that is going to be compiler directive
options, you need to map StringNode -> __builtin__.str (and then use
node.value in Py3 or node.unicode_value in Py3).
Stefan, look at numpy.pxd, you will see $cdef __cythonbufferdefaults__
= {"mode": "strided"}$. W need to "convert" that declaration in a
regular Python dict to be used in Buffer.py, in such a way that
$"mode" in option_dict$ or $option_dict["mode"]$ work. Currently, in
Python 3, the key,value pair endup being bytes, then things fails...
I still think that what I'm doing is not so bad. After all, I'm
hacking a method called interpret_compiletime_options(), the purpose
of its output is not code generation.
Hmm, right. I guess that'll do then.
Please go ahead and push this in. If it helps NumPy users in Py3, it's
worth getting out rather sooner than later.
Stefan
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
And here are the first fruits: unittests fail on some platforms (also I have forgotten that $HOME is not guarantee to be writable, but that is a pure packaging-related side-point): All logs are available from https://buildd.debian.org/build.cgi?pkg=cython&dist=sid armel, powerpc, s390 (hppa is not yet done but probably the same) -- big endians: e.g. https://buildd.debian.org/fetch.cgi?pkg=cython;ver=0.14.1-3;arch=armel;stamp=1298052709 I could be wrong but it seems they all boil down to FAIL: Doctest: dictintindex.__test__.test_get_char_neg (line 1) so the others fail as well, e.g. FAIL: Doctest: c_int_types_T255 FAIL: Doctest: c_int_types_T255.__test__.test_char (line 90) FAIL: Doctest: dictintindex.__test__.test_get_char_neg (line 1) otherwise -- amazingly nice ;-) On Thu, 17 Feb 2011, Yaroslav Halchenko wrote: > > > | AssertionError > > > `--- > > > what could be done about it or should it be excluded? > > I've pushed some fixes. Now this testcase should run from ancient > > Python 2.3 to head Python 3.2, both for static and sharedlib builds > > (but not in Windows). > first of all THANKS for the patch -- I picked it up into 0.14.1-2 debian > package. Now tests are enabled and I just uploaded 0.14.1-2 into Debian > -- lets see how it would go across architectures ;-) > 2nd -- THANKS for ...: at first I got confused why I saw no commits > since Dec in HG clone of Cython I had, and then mentioned that you > moved to using GIT and github. Awesome and thank you for taking care > about my sanity (although unintentionally I guess ;) ) -- =--= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
2011/2/18 Yaroslav Halchenko : > > awesome! I will test it asap -- just let me know if there would be > another commit for above or I should tune up $HOME ;) > https://github.com/cython/cython/commit/475e9a085654252d5a274dab2118b746e8bda4eb >> We are very near to be crystal clear... > > yeap > And now we should be crystal clear, finally! Could you confirm? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Thu, Feb 17, 2011 at 8:38 PM, W. Trevor King wrote: > On Thu, Feb 17, 2011 at 3:53 PM, Robert Bradshaw wrote: >> On Thu, Feb 17, 2011 at 3:12 PM, W. Trevor King wrote: >>> On Thu, Feb 17, 2011 at 01:25:10PM -0800, Robert Bradshaw wrote: On Thu, Feb 17, 2011 at 5:29 AM, W. Trevor King wrote: > On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote: >> On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King wrote: >> > What I'm missing is a way to bind the ModuleScope namespace to a name >> > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib, >> > name)` will work in expose.pyx. >> >> You have also hit into the thorny issue that .pxd files are used for >> many things. They may be pure C library declarations with no Python >> module backing, they may be declarations of (externally implemented) >> Python modules (such as numpy.pxd), or they may be declarations for >> Cython-implemented modules. >> >> Here's another idea, what if extern blocks could contain cpdef >> declarations, which would automatically generate a Python-level >> wrappers for the declared members (if possible, otherwise an error)? > > Ah, this sounds good! Of the three .pxd roles you list above, > external Python modules (e.g. numpy) and Cython-implemented modules > (e.g. matched .pxd/.pyx) both already have a presence in Python-space. > What's missing is a way to give (where possible) declarations of > external C libraries a Python presence. cpdef fills this hole nicely, > since its whole purpose is to expose Python interfaces to > C-based elements. In the case of external Python modules, I'm not so sure we want to monkey-patch our stuff in >>> >>> I don't think any of the changes we are suggesting would require >>> changes to existing code, so .pxd-s with external implementations >>> wouldn't be affected unless they brough the changes upon themselves. >> >> Say, in numpy.pxd, I have >> >> cdef extern from "...": >> cpdef struct obscure_internal_struct: >> ... >> >> Do we add an "obscure_internal_struct" onto the (global) numpy module? >> What if it conflicts with a (runtime) name? This is the issue I'm >> bringing up. > > Defining a cpdef *and* a non-matching external implementation should > raise a compile-time error. I agree that there is a useful > distinction between external-C-library and external-Python-module .pxd > wrappers. Perhaps your matching blank .py or .pyx file could serve as > a marker that the .pxd file should be inflated into its own full > fledged python module. I'm not even sure how you would go about > adding attributes to the numpy module. When/how would the > Cython-created attributes get added? Yes, this is exactly the issue. > In the external-C-library case, if you define something incompatible > in the matching .pyx or .py file, Cython will be able to see it and > die with an appropriate error, so you can resolve your programming > mistake. That is only if you have a matching .pyx of .py file. Of course, there could be a module by that name in the user's runtime environment that we don't know about, but that currently is not in conflict with a .pxd only file. > If you try to override anything in a .so compiled module at runtime, > you'd get the same kind of error you currently do trying to rebind a > compiled class' method. That's the desired behavior for statically-bound globals, but implementing it is not so trivial. (and where would we do it--on the first import of a cimporting module?) >>> >>> Compilation is an issue. I think that .pxd files should be able to be >>> cythoned directly, since then they Cython can build any wrappers they >>> request. If the file has a matching .pyx file, cythoning either one >>> should compile both together, since they'll produce a single Python >>> .so module. >> >> In this case, I think it may make more sense to consider cimporting >> stuff from .pyx files if no .pxd file is present. > > Can you cimport .pyx files that lack matching .pxd files? > >> In any case, this is a big change. I don't like the idea of always >> creating such a module (it may be empty, or have name conflicts) > > It shouldn't take to long to compile an empty module ;). And odds are > noone will waste time importing it either. > >> nor the idea of conditionally compiling it (does one have to look at >> the contents and really understand Cython to see if a Python shadow >> will be created?) > > I agree here. > > Under the mantra "explicit is better than implicit", we could have > users add something like > > cdef module "modname" > > to any .pxd files that should be inflated into Python modules. .pxd > files without such a tag would receive the current treatment, error on > any cpdef, etc. The drawback of this approach is that it makes Cython > more complicated, but if both behaviors are reasonable, there's > pro
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
On Fri, 18 Feb 2011, Lisandro Dalcin wrote: > > awesome! I will test it asap -- just let me know if there would be > > another commit for above or I should tune up $HOME ;) > https://github.com/cython/cython/commit/475e9a085654252d5a274dab2118b746e8bda4eb evil you -- and who would address: Caller is responsible for deleting the directory when done with it. ;-) > >> We are very near to be crystal clear... > > yeap > And now we should be crystal clear, finally! Could you confirm? I will... manually first on one of those platforms... I will let you know of cause ;) -- =--= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
2011/2/18 Yaroslav Halchenko : > > On Fri, 18 Feb 2011, Lisandro Dalcin wrote: >> > awesome! I will test it asap -- just let me know if there would be >> > another commit for above or I should tune up $HOME ;) >> https://github.com/cython/cython/commit/475e9a085654252d5a274dab2118b746e8bda4eb > > evil you -- and who would address: > Caller is responsible for deleting the directory when done with it. > ;-) > Well, I decided to not delete it just in case things go wrong and you need to check the output... I can easily fix that in a tearDown() method... Should I? -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
> >> > awesome! I will test it asap -- just let me know if there would be > >> > another commit for above or I should tune up $HOME ;) > >> https://github.com/cython/cython/commit/475e9a085654252d5a274dab2118b746e8bda4eb > > evil you -- and who would address: > > Caller is responsible for deleting the directory when done with it. > > ;-) > Well, I decided to not delete it just in case things go wrong and you > need to check the output... I can easily fix that in a tearDown() > method... Should I? Well, that is up to you guys -- I do not think there is anything in Debian policy/guidelines about cleaning up TMPDIR. I, personally, would have preferred them being cleaned up. -- =--= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fwd: Re: Cython builds on various Debian platforms
confirming now -- built/tested fine manually on s390, uploaded, and now built across nearly all (few still building/testing) archs: https://buildd.debian.org/pkg.cgi?pkg=cython On Fri, 18 Feb 2011, Yaroslav Halchenko wrote: > > >> We are very near to be crystal clear... > > > yeap > > And now we should be crystal clear, finally! Could you confirm? > I will... manually first on one of those platforms... I will let you > know of cause ;) -- =--= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
