[Cython] Outdated `hg export` on cython-devel homepage
The `hg export` on the cython-devel page [1] should probably be changed to (or additionally list) `git format-patch`, now that Cython's versioned in Git. Keeping the `hg export` reference might be useful for Mercurial lovers using hg-git. [1]: http://mail.python.org/mailman/listinfo/cython-devel -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpx5jnKkfqoR.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
This thread is coming over to cython-dev (and the new cython-devel) from cython-users because it turns out it will probably require chaning the Cython code. To get everyone who hasn't been following on cython-users up to speed, here's a summary of what I'm trying to do: That's what I was trying to give with this: On Wed, Feb 09, 2011 at 12:23:25PM -0500, W. Trevor King wrote: > I'm wrapping an external C library with Cython, so I have `mylib.pxd`: > > cdef extern from 'mylib.h' > enum: CONST_A > enum: CONST_B > ... > > where I declare each constant macro from the library's header `mylib.h`: > > #define CONST_A 1 > #define CONST_B 2 > ... > > Now I want to expose those constants in Python, so I have `expose.pyx`: > > cimport mylib > > CONST_A = mylib.CONST_A > CONST_B = mylib.CONST_B > ... > > But the last part seems pretty silly. I'd like to do something like > > cimport mylib > import sys > > for name in dir(mylib): > setattr(sys.modules[__name__], name, getattr(mylib, name)) > > which compiles fine, but fails to import with... Looking into the Cython internals, everything defined in mylib.pxd is stored as `Entry`s in a `ModuleScope`, and... On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote: > On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King wrote: > > What I'm missing is a way to bind the ModuleScope namespace to a name > > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib, > > name)` will work in expose.pyx. > > You have also hit into the thorny issue that .pxd files are used for > many things. They may be pure C library declarations with no Python > module backing, they may be declarations of (externally implemented) > Python modules (such as numpy.pxd), or they may be declarations for > Cython-implemented modules. > > It seems like it would be easier to generate some kind of wrapper > > class (PxdModule?) for mylib when it is cimported (at compile time), > > and then further interactions would take care of themselves (at run > > time). > > Would such an object be created anew for every module that cimports > the declaration file? Hmm, That doesn't sound very nice, does it. However, .pxd files declaring C libraries have no Python-space presence, so that was my initial idea. > I have toyed with the idea of subclassing the module object itself for > better support of C-level attributes from the Python (and Cython) > namespaces. Sorry, I don't understand "better support of C-level attributes". Can you give an example? > Here's another idea, what if extern blocks could contain cpdef > declarations, which would automatically generate a Python-level > wrappers for the declared members (if possible, otherwise an error)? Ah, this sounds good! Of the three .pxd roles you list above, external Python modules (e.g. numpy) and Cython-implemented modules (e.g. matched .pxd/.pyx) both already have a presence in Python-space. What's missing is a way to give (where possible) declarations of external C libraries a Python presence. cpdef fills this hole nicely, since its whole purpose is to expose Python interfaces to C-based elements. A side effect of this cpdef change would be that now even bare .pxd files (no matching .pyx) would have a Python presence, so You could do something like cimport mylib as mylib_c import mylib as mylib_py import sys # Access through Python for name in dir(mylib_py): setattr(sys.modules[__name__], name, getattr(mylib_py, name)) # Direct C access cdef get_a(): return mylib_c.CONST_A Where the Python access would be the new feature, and list all cpdef-ed stuff. However, from Parsing.py:2369: error(pos, "C struct/union/enum cannot be declared cpdef") From pyrex_differences.rst: If a function is declared :keyword:`cpdef` it can be called from and overridden by both extension and normal python subclasses. I believe the reason that cpdef-ed enums and similar are currently illegal is confusion between "can be called from Python" and "can be overridden from Python". I think these should be just like methods already are, in that you can "override" a method by subclassing it, but not by rebinding the name in the base class: >>> import pyximport; pyximport.install() >>> import rectangle as R >>> r = R.Rectangle(1, 2, 3, 4) >>> r.area = lambda(self): r.x1' Traceback (most recent call last): File "", line 1, in AttributeError: 'rectangle.Rectangle' object attribute 'area' is read-only Where rectangle.pyx is a minorly patched version of the last example fr
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Thu, Feb 17, 2011 at 08:29:41AM -0500, W. Trevor King wrote: > cpdef struct Foo: > cpdef public int intA > cpdef readonly int intB > cdef void *ptr Oops, for consistency with classes, the variables declarations should read `cdef public` and and `cdef readonly`. Perhaps `cdef struct` too, to match `cdef class`? I get a bit confused, because for some things (functions, methods) `cpdef` adds a Python interface. For others (attributes) it's `cdef public/readonly`. There are even some things (classes), where a plain `cdef` is enough to provide a Python interface. Perhaps I am just missing some subtle distinction between the effects of the various incantations? -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpNmFAslJJpG.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] [PATCH] Add .gitignores to cython and cython-docs
Here's a pair of patches doing just that. I also ignore *.c in cython, because all .c files are currently auto-generated. Perhaps that will not always be the case? If it seems to risky, feel free to leave that part out. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt From a3916ac0ac058e43c9aa75e0a66312618be73edf Mon Sep 17 00:00:00 2001 From: W. Trevor King Date: Thu, 17 Feb 2011 09:16:21 -0500 Subject: [PATCH] Add *.c to .hgignore and create analogous .gitignore. --- .gitignore | 15 +++ .hgignore |1 + 2 files changed, 16 insertions(+), 0 deletions(-) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 000..2187b82 --- /dev/null +++ b/.gitignore @@ -0,0 +1,15 @@ +*.pyc +*.c +*.swp + +Cython/Compiler/Lexicon.pickle +BUILD/ +build/ +dist/ +.coverage +*~ +*.orig +*.rej +*.dep + +tags diff --git a/.hgignore b/.hgignore index 6fa7131..e005c72 100644 --- a/.hgignore +++ b/.hgignore @@ -1,6 +1,7 @@ syntax: glob *.pyc +*.c *.swp Cython/Compiler/Lexicon.pickle -- 1.7.3.4 From bf5bf5d3874cdcf34940b634c56dbeb35faa4137 Mon Sep 17 00:00:00 2001 From: W. Trevor King Date: Thu, 17 Feb 2011 09:20:30 -0500 Subject: [PATCH 2/2] Create .gitignore analogous to current .hgignore. --- .gitignore |6 ++ 1 files changed, 6 insertions(+), 0 deletions(-) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 000..d431956 --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +*.pyc +*~ +.*.swp + +build/ +_build/ -- 1.7.3.4 pgpAONMwGmDpc.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Thu, Feb 17, 2011 at 10:53:15AM -0300, Lisandro Dalcin wrote: > Cython could certainly support "cpdef struct", it is just a matter to > define a proposal and find a contributor to implement it :-) Is there a CEP template (a la PEPs 9 and 12) that should be discussed on the mailing list, or do I develop it free-form on the wiki [2]? p.s. should I be picking one of cython-dev@codespeak or cython-devel@python? Is the shift not yet official? [2]: http://wiki.cython.org/enhancements/ -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgptjVXzT6lHZ.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Thu, Feb 17, 2011 at 01:25:10PM -0800, Robert Bradshaw wrote: > On Thu, Feb 17, 2011 at 5:29 AM, W. Trevor King wrote: > > On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote: > >> On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King wrote: > >> > What I'm missing is a way to bind the ModuleScope namespace to a name > >> > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib, > >> > name)` will work in expose.pyx. > >> > >> You have also hit into the thorny issue that .pxd files are used for > >> many things. They may be pure C library declarations with no Python > >> module backing, they may be declarations of (externally implemented) > >> Python modules (such as numpy.pxd), or they may be declarations for > >> Cython-implemented modules. > >> > >> Here's another idea, what if extern blocks could contain cpdef > >> declarations, which would automatically generate a Python-level > >> wrappers for the declared members (if possible, otherwise an error)? > > > > Ah, this sounds good! Of the three .pxd roles you list above, > > external Python modules (e.g. numpy) and Cython-implemented modules > > (e.g. matched .pxd/.pyx) both already have a presence in Python-space. > > What's missing is a way to give (where possible) declarations of > > external C libraries a Python presence. cpdef fills this hole nicely, > > since its whole purpose is to expose Python interfaces to > > C-based elements. > > In the case of external Python modules, I'm not so sure we want to > monkey-patch our stuff in I don't think any of the changes we are suggesting would require changes to existing code, so .pxd-s with external implementations wouldn't be affected unless they brough the changes upon themselves. > (and where would we do it--on the first import of a cimporting > module?) Compilation is an issue. I think that .pxd files should be able to be cythoned directly, since then they Cython can build any wrappers they request. If the file has a matching .pyx file, cythoning either one should compile both together, since they'll produce a single Python .so module. > > A side effect of this cpdef change would be that now even bare .pxd > > files (no matching .pyx) would have a Python presence, > > Where would it live? Would we just create this module (in essence, > acting as if there was an empty .pyx file sitting there as well)? On > this note, it may be worth pursuing the idea of a "cython helper" > module where common code and objects could live. I'm not sure exactly what you mean by "cython helper", but this sounds like my 'bare .pyx can create a Python .so module idea above. > > so You could do > > something like > > > > cimport mylib as mylib_c > > import mylib as mylib_py > > import sys > > > > # Access through Python > > for name in dir(mylib_py): > > setattr(sys.modules[__name__], name, getattr(mylib_py, name)) > > I think this smells worse than "import *" Aha, thanks ;). I was stuck in my old .pxd-files-don't-create-modules-by-themselves mindset. Obviously, once they do, any Python code can access the contents directly and I can throw out all this indirection. > > However, from Parsing.py:2369: > > > >error(pos, "C struct/union/enum cannot be declared cpdef") > > > > From pyrex_differences.rst: > > > >If a function is declared :keyword:`cpdef` it can be called from > >and overridden by both extension and normal python subclasses. > > > > I believe the reason that cpdef-ed enums and similar are currently > > illegal is confusion between "can be called from Python" and "can be > > overridden from Python". > > The reason that error statement is there is because it had no meaning, > so an error was better than just ignoring it. Why does it have no meaning? I understand that it's not implemented yet, but a cpdef-ed enum or struct seems just as valid an idea as a cpdef-ed method. > > Unions don't really have a Python parallel, > > They can be a cdef class wrapping the union type. But I would think coercion would be difficult. Unions are usually (in my limited experience) for "don't worry about the type, just make sure it fits in X bytes". How would union->Python conversion work? > >cpdef struct Foo: > >cpdef public int intA > >cpdef readonly int intB > >cdef void *ptr > > > > We would both declare the important members of the C struct (as we can > > already
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Thu, Feb 17, 2011 at 3:53 PM, Robert Bradshaw wrote: > On Thu, Feb 17, 2011 at 3:12 PM, W. Trevor King wrote: >> On Thu, Feb 17, 2011 at 01:25:10PM -0800, Robert Bradshaw wrote: >>> On Thu, Feb 17, 2011 at 5:29 AM, W. Trevor King wrote: >>> > On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote: >>> >> On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King wrote: >>> >> > What I'm missing is a way to bind the ModuleScope namespace to a name >>> >> > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib, >>> >> > name)` will work in expose.pyx. >>> >> >>> >> You have also hit into the thorny issue that .pxd files are used for >>> >> many things. They may be pure C library declarations with no Python >>> >> module backing, they may be declarations of (externally implemented) >>> >> Python modules (such as numpy.pxd), or they may be declarations for >>> >> Cython-implemented modules. >>> >> >>> >> Here's another idea, what if extern blocks could contain cpdef >>> >> declarations, which would automatically generate a Python-level >>> >> wrappers for the declared members (if possible, otherwise an error)? >>> > >>> > Ah, this sounds good! Of the three .pxd roles you list above, >>> > external Python modules (e.g. numpy) and Cython-implemented modules >>> > (e.g. matched .pxd/.pyx) both already have a presence in Python-space. >>> > What's missing is a way to give (where possible) declarations of >>> > external C libraries a Python presence. cpdef fills this hole nicely, >>> > since its whole purpose is to expose Python interfaces to >>> > C-based elements. >>> >>> In the case of external Python modules, I'm not so sure we want to >>> monkey-patch our stuff in >> >> I don't think any of the changes we are suggesting would require >> changes to existing code, so .pxd-s with external implementations >> wouldn't be affected unless they brough the changes upon themselves. > > Say, in numpy.pxd, I have > > cdef extern from "...": >cpdef struct obscure_internal_struct: >... > > Do we add an "obscure_internal_struct" onto the (global) numpy module? > What if it conflicts with a (runtime) name? This is the issue I'm > bringing up. Defining a cpdef *and* a non-matching external implementation should raise a compile-time error. I agree that there is a useful distinction between external-C-library and external-Python-module .pxd wrappers. Perhaps your matching blank .py or .pyx file could serve as a marker that the .pxd file should be inflated into its own full fledged python module. I'm not even sure how you would go about adding attributes to the numpy module. When/how would the Cython-created attributes get added? In the external-C-library case, if you define something incompatible in the matching .pyx or .py file, Cython will be able to see it and die with an appropriate error, so you can resolve your programming mistake. If you try to override anything in a .so compiled module at runtime, you'd get the same kind of error you currently do trying to rebind a compiled class' method. >>> (and where would we do it--on the first import of a cimporting >>> module?) >> >> Compilation is an issue. I think that .pxd files should be able to be >> cythoned directly, since then they Cython can build any wrappers they >> request. If the file has a matching .pyx file, cythoning either one >> should compile both together, since they'll produce a single Python >> .so module. > > In this case, I think it may make more sense to consider cimporting > stuff from .pyx files if no .pxd file is present. Can you cimport .pyx files that lack matching .pxd files? > In any case, this is a big change. I don't like the idea of always > creating such a module (it may be empty, or have name conflicts) It shouldn't take to long to compile an empty module ;). And odds are noone will waste time importing it either. > nor the idea of conditionally compiling it (does one have to look at > the contents and really understand Cython to see if a Python shadow > will be created?) I agree here. Under the mantra "explicit is better than implicit", we could have users add something like cdef module "modname" to any .pxd files that should be inflated into Python modules. .pxd files without such a tag would receive the current treatment, error on any cpdef, etc. The drawback of this approach is that it m
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Fri, Feb 18, 2011 at 02:08:04PM -0800, Robert Bradshaw wrote: > On Thu, Feb 17, 2011 at 8:38 PM, W. Trevor King wrote: > > On Thu, Feb 17, 2011 at 3:53 PM, Robert Bradshaw wrote: > >> On Thu, Feb 17, 2011 at 3:12 PM, W. Trevor King wrote: > >>> On Thu, Feb 17, 2011 at 01:25:10PM -0800, Robert Bradshaw wrote: > >>>> On Thu, Feb 17, 2011 at 5:29 AM, W. Trevor King wrote: > >>>> > On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote: > >>>> >> On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King wrote: > >>>> >> > What I'm missing is a way to bind the ModuleScope namespace to a > >>>> >> > name > >>>> >> > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib, > >>>> >> > name)` will work in expose.pyx. > >>>> >> > >>>> >> You have also hit into the thorny issue that .pxd files are used for > >>>> >> many things. They may be pure C library declarations with no Python > >>>> >> module backing, they may be declarations of (externally implemented) > >>>> >> Python modules (such as numpy.pxd), or they may be declarations for > >>>> >> Cython-implemented modules. > >>>> >> > >>>> >> Here's another idea, what if extern blocks could contain cpdef > >>>> >> declarations, which would automatically generate a Python-level > >>>> >> wrappers for the declared members (if possible, otherwise an error)? > >>>> > > >>>> > Ah, this sounds good! Of the three .pxd roles you list above, > >>>> > external Python modules (e.g. numpy) and Cython-implemented modules > >>>> > (e.g. matched .pxd/.pyx) both already have a presence in Python-space. > >>>> > What's missing is a way to give (where possible) declarations of > >>>> > external C libraries a Python presence. cpdef fills this hole nicely, > >>>> > since its whole purpose is to expose Python interfaces to > >>>> > C-based elements. > >>>> > >>>> In the case of external Python modules, I'm not so sure we want to > >>>> monkey-patch our stuff in > >>> > >>> I don't think any of the changes we are suggesting would require > >>> changes to existing code, so .pxd-s with external implementations > >>> wouldn't be affected unless they brough the changes upon themselves. > >> > >> Say, in numpy.pxd, I have > >> > >> cdef extern from "...": > >>cpdef struct obscure_internal_struct: > >>... > >> > >> Do we add an "obscure_internal_struct" onto the (global) numpy module? > >> What if it conflicts with a (runtime) name? This is the issue I'm > >> bringing up. > > > > Defining a cpdef *and* a non-matching external implementation should > > raise a compile-time error. I agree that there is a useful > > distinction between external-C-library and external-Python-module .pxd > > wrappers. Perhaps your matching blank .py or .pyx file could serve as > > a marker that the .pxd file should be inflated into its own full > > fledged python module. I'm not even sure how you would go about > > adding attributes to the numpy module. When/how would the > > Cython-created attributes get added? > > Yes, this is exactly the issue. Ah, I'm retracting my agreement on the external-C-library and external-Python-module .pxd wrappers. There is no difference in how their .pxd files should be treated, and I now agree that .pxd files should not generate .so modules unless they have a paried .py/.pyx file. > > If you try to override anything in a .so compiled module at runtime, > > you'd get the same kind of error you currently do trying to rebind a > > compiled class' method. > > That's the desired behavior for statically-bound globals, but > implementing it is not so trivial. It's currently implemented for classes. Are modules that different from classes? >>> import types >>> types.ModuleType.__class__.__mro__ (, ) So you've got your standard __setattr__ to override with an error-message generator. What is the implementation difficulty? I am also unclear about the distinction between cdef and cpdef (perhaps this should be a new thread?). cpdef means "I'm declaring something with C and Python interfaces. The Python interfa
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Thu, Feb 17, 2011 at 03:53:13PM -0800, Robert Bradshaw wrote: > On Thu, Feb 17, 2011 at 3:12 PM, W. Trevor King wrote: > > * Extending class cdef/cdpef/public/readonly handling to cover enums, > > stucts, and possibly unions. > > This seems like the best first step. > > > Problems with me getting started now: > > > > * I don't know where I should start mucking about in the source. > > ...The other files to look at would be Parsing.py,.. I've got the the Parsing pretty much working, but portions of the test suite are now failing with things like === Expected errors: === 1:5: function definition in pxd file must be declared 'cdef inline' 4:5: inline function definition in pxd file cannot be 'public' 7:5: inline function definition in pxd file cannot be 'api' === Got errors: === 1:5: function definition in pxd file must be declared 'cdef inline' 4:12: inline function definition in pxd file cannot be 'public' 7:5: inline function definition in pxd file cannot be 'api' while cythoning tests/errors/e_func_in_pxd_support.pxd: cdef foo(): return 1 cdef public inline foo2(): return 1 cdef api inline foo3(): return 1 The current Cython implementation does not consider 'cdef' to be part of the node code, and my current implementation does not consider any qualifiers to be part of the node code. It's unclear to me what the "right" position is for the start of a function (or variable, class, ...) should be, or how errors should be formatted. My initial impression is that Node.pos should point to the beginning of the def (here 1:1, 4:1, and 7:1), but that positions of the various qualifiers (cdef, public, inline, api, ...) should be cached so that the error points to the offending qualifier. Thoughs? -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpWK2nBwsk8N.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 19, 2011 at 12:52:38PM -0800, Robert Bradshaw wrote: > On Sat, Feb 19, 2011 at 11:35 AM, W. Trevor King wrote: > > On Thu, Feb 17, 2011 at 03:53:13PM -0800, Robert Bradshaw wrote: > >> On Thu, Feb 17, 2011 at 3:12 PM, W. Trevor King wrote: > >> > * Extending class cdef/cdpef/public/readonly handling to cover enums, > >> > stucts, and possibly unions. > >> > >> This seems like the best first step. > >> > >> > Problems with me getting started now: > >> > > >> > * I don't know where I should start mucking about in the source. > >> > >> ...The other files to look at would be Parsing.py,.. > > > > I've got the the Parsing pretty much working, but portions of the test > > suite are now failing with things like > > > >=== Expected errors: === > >1:5: function definition in pxd file must be declared 'cdef inline' > >4:5: inline function definition in pxd file cannot be 'public' > >7:5: inline function definition in pxd file cannot be 'api' > > > > > >=== Got errors: === > >1:5: function definition in pxd file must be declared 'cdef inline' > >4:12: inline function definition in pxd file cannot be 'public' > >7:5: inline function definition in pxd file cannot be 'api' > > > > while cythoning tests/errors/e_func_in_pxd_support.pxd: > > > >cdef foo(): > >return 1 > > > >cdef public inline foo2(): > >return 1 > > > >cdef api inline foo3(): > >return 1 > > > > The current Cython implementation does not consider 'cdef' to be part > > of the node code, and my current implementation does not consider any > > qualifiers to be part of the node code. > > > > It's unclear to me what the "right" position is for the start of a > > function (or variable, class, ...) should be, or how errors should be > > formatted. My initial impression is that Node.pos should point to the > > beginning of the def (here 1:1, 4:1, and 7:1), but that positions of > > the various qualifiers (cdef, public, inline, api, ...) should be > > cached so that the error points to the offending qualifier. > > For better or for worse, Cython follows the same inane declarator > rules as C, so > > cdef int a, *b, c[5], (*d)(int) > > is valid. This colors how c function declarations are expressed in the > tree. Qualifiers don't really live in the tree and I'm not sure > plumbing the "qualifier node" positions around would be worth the > added complexity. So should I go ahead and update the expected error messages in my branch? -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpb5IXeiawOW.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 19, 2011 at 12:47:41PM -0800, Robert Bradshaw wrote: > On Sat, Feb 19, 2011 at 11:22 AM, W. Trevor King wrote: > > >> > If you try to override anything in a .so compiled module at runtime, > >> > you'd get the same kind of error you currently do trying to rebind a > >> > compiled class' method. > >> > >> That's the desired behavior for statically-bound globals, but > >> implementing it is not so trivial. > > > > It's currently implemented for classes. Are modules that different > > from classes? > > > >>>> import types > >>>> types.ModuleType.__class__.__mro__ > >(, ) > > > > So you've got your standard __setattr__ to override with an > > error-message generator. What is the implementation difficulty? > > You can't just add a __setattr__ to a module and have it work, it's a > class-level slot. And you don't want to modify all module classes. To > do this you have to subclass module itself and insert that in place > during import. So annoying but possible? Not particularly critical either way, though, since you could always say "don't rebind module-level stuff" in a project's docs, even if you don't enforce that at the Cython level. > > I am also unclear about the distinction between cdef and cpdef > > (perhaps this should be a new thread?). > > > > cpdef means "I'm declaring something with C and Python interfaces. > > The Python interface is a thin wrapper which can be rebound to an > > object of any type, leaving the static C interface inaccessible from > > Python." > > No, you can't re-bind cdef class methods. Despite Cython's attempt to > homogenize things for the user, extension classes are quite different > than "normal" Python classes. This is a Python C/API issue. Do you mean `cpdef class methods`? If so, you're right: $ cat square.pyx cdef class A (object): cdef public int value cpdef square(self): return self.value**2 $ python -c 'import pyximport; pyximport.install(); import square; square.A.square = lambda self: self.value' Traceback (most recent call last): File "", line 1, in TypeError: can't set attributes of built-in/extension type 'square.A' You can't override them in instances either: $ python -c 'import pyximport; pyximport.install(); import square; a = square.A(); a.square = lambda self: self.value' Traceback (most recent call last): File "", line 1, in AttributeError: 'square.A' object attribute 'square' is read-only But you can override them in subclasses, which is, I suppose, the point of cpdef for methods. > > cdef [private] means "I'm declaring something that only has a C > > interface." > > cdef public means "I'm declaring something with C and Python > > interfaces backed by C data. Python code can alter the C data." > > cdef readonly means "I'm declaring something with C and Python > > interfaces backed by C data. Python code cannot alter the C data." > > cdef means "back this by a C variable" Ah, ok. That makes more sense. > > However, the filename <-> module mapping is troublesome for backing > > externally-implemented Python modules (e.g. numpy). If you wanted to > > write a .pxd file backing numpy.random, how would you go about getting > > your module installed in Cython/Includes/numpy/random.pxd or another > > path that cython would successfully match with `cimport numpy.random`? > > Note that extern blocks (by definition) declare where things come from. They declare where the .pxd file looks for .h files, but not where .pyx files look for the .pxd file. > >> > cdef public struct X: > >> > int x > >> > readonly int z > >> > private int z > >> > > >> > I would perhaps say that non-Pythonable non-private members in public > >> > structs would be a compile error. > >> > >> +1, keep it safe at the beginning. > > > > -1, keep the code clean and the interface consistent ;). I think the > > struct syntax should be identical to the class syntax, with the > > exception that you can't bind methods to structs. That's the only > > real difference between structs and classes, isn't it? > > In C++, the only difference between structs and classes is that struct > members are public by default. (Not saying
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 19, 2011 at 02:04:16PM -0800, Robert Bradshaw wrote: > On Sat, Feb 19, 2011 at 1:45 PM, W. Trevor King wrote: > > On Sat, Feb 19, 2011 at 12:47:41PM -0800, Robert Bradshaw wrote: > >> On Sat, Feb 19, 2011 at 11:22 AM, W. Trevor King wrote: > >> > However, the filename <-> module mapping is troublesome for backing > >> > externally-implemented Python modules (e.g. numpy). If you wanted to > >> > write a .pxd file backing numpy.random, how would you go about getting > >> > your module installed in Cython/Includes/numpy/random.pxd or another > >> > path that cython would successfully match with `cimport numpy.random`? > >> > >> Note that extern blocks (by definition) declare where things come from. > > > > They declare where the .pxd file looks for .h files, but not where > > .pyx files look for the .pxd file. > > Sorry, I should have said extern blocks that make cdef class > declarations (such as our numpy.pxd). It doesn't look like there are cdef class declarations in numpy.pxd: cython $ grep class Cython/Includes/numpy.pxd ctypedef class numpy.dtype [object PyArray_Descr]: ctypedef extern class numpy.flatiter [object PyArrayIterObject]: ctypedef extern class numpy.broadcast [object PyArrayMultiIterObject]: ctypedef class numpy.ndarray [object PyArrayObject]: ctypedef extern class numpy.ufunc [object PyUFuncObject]: This still doesn't explain how .pxd files specify which external implemented Python modules they correspond to. > >> >> > cdef public struct X: > >> >> > int x > >> >> > readonly int z > >> >> > private int z > >> >> > > >> >> > I would perhaps say that non-Pythonable non-private members in public > >> >> > structs would be a compile error. > >> >> > >> >> +1, keep it safe at the beginning. > >> > > >> > -1, keep the code clean and the interface consistent ;). I think the > >> > struct syntax should be identical to the class syntax, with the > >> > exception that you can't bind methods to structs. That's the only > >> > real difference between structs and classes, isn't it? > >> > >> In C++, the only difference between structs and classes is that struct > >> members are public by default. (Not saying that C++ is always the > >> model to follow, but it gives precedent). And structs can have > >> function members, that's how to do OOP in C. > > > > Oh. Even more reason to have identical struct and class handling in > > Cython ;). > > > > It is unclear to me what `cdef public struct` means. I think it > > should mean "Python bindings can alter this struct's definition", > > which doesn't make sense. > > I think it should mean "this struct is accessible from Python (as X)" Wouldn't that be "cdef readonly struct X"? > > Shouldn't the syntax for public members be > > > >cdef struct X: > >cdef public: > >int x > >readonly int y > >private int z > > -1 on nesting things like this. Rather than make a struct visible from > Python iff any of its members are, A struct is visible from python iff it is declared public or readonly: cdef public struct X: ... or cdef readonly struct X: ... I don't think the visibility of the struct as a whole should have any effect over the visibility of its members, so you should be able to specify member visibility explicitly with per-member granularity (as you currently can for classes). I was assuming that structs would be public by default (like classes), but that is obviously configurable. > I think it makes more sense to put > the declaration on the struct itself. We could support > > cdef public struct X: > int x # public > > cdef readonly struct Y: > int y # readonly > > cdef [private] struct Z: > int z # private, as we don't even have Z in the Python namespace, > and no wrapper is created. The problems with this are: * It's differnent from how we handle the almost identical class case. * It makes it impossible to define, for example a public struct with C-only attributes: cdef public struct X: cdef public int a cdef private void* ptr Obviously, public attributes of private structs should raise compile-time Cython errors. > >> > If safety with a new feature is a concern, a warning like > >> > "EXPERIMEN
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 19, 2011 at 04:41:27PM -0800, Robert Bradshaw wrote: > On Sat, Feb 19, 2011 at 3:31 PM, W. Trevor King wrote: > > On Sat, Feb 19, 2011 at 02:04:16PM -0800, Robert Bradshaw wrote: > >> On Sat, Feb 19, 2011 at 1:45 PM, W. Trevor King wrote: > >> > On Sat, Feb 19, 2011 at 12:47:41PM -0800, Robert Bradshaw wrote: > >> >> On Sat, Feb 19, 2011 at 11:22 AM, W. Trevor King > >> >> wrote: > >> >> > However, the filename <-> module mapping is troublesome for backing > >> >> > externally-implemented Python modules (e.g. numpy). If you wanted to > >> >> > write a .pxd file backing numpy.random, how would you go about getting > >> >> > your module installed in Cython/Includes/numpy/random.pxd or another > >> >> > path that cython would successfully match with `cimport numpy.random`? > >> >> > >> >> Note that extern blocks (by definition) declare where things come from. > >> > > >> > They declare where the .pxd file looks for .h files, but not where > >> > .pyx files look for the .pxd file. > >> > >> Sorry, I should have said extern blocks that make cdef class > >> declarations (such as our numpy.pxd). > > > > It doesn't look like there are cdef class declarations in numpy.pxd: > > > >cython $ grep class Cython/Includes/numpy.pxd > >ctypedef class numpy.dtype [object PyArray_Descr]: > >ctypedef extern class numpy.flatiter [object PyArrayIterObject]: > >ctypedef extern class numpy.broadcast [object > > PyArrayMultiIterObject]: > >ctypedef class numpy.ndarray [object PyArrayObject]: > >ctypedef extern class numpy.ufunc [object PyUFuncObject]: > > > > This still doesn't explain how .pxd files specify which external > > implemented Python modules they correspond to. > > "numpy.dtype" is the fully qualified name of the class it's > declaring--the module is "numpy." Hmm, that means that cimport numpy a = numpy.numpy.ndarray also compiles. Wouldn't it make more sense to mirror numpy's module structure when backing it? You do have nested .pxd modules for cpython, libc, etc. > >> >> >> > cdef public struct X: > >> >> >> > int x > >> >> >> > readonly int z > >> >> >> > private int z > >> >> >> > > >> >> >> > I would perhaps say that non-Pythonable non-private members in > >> >> >> > public > >> >> >> > structs would be a compile error. > >> >> >> > >> >> >> +1, keep it safe at the beginning. > >> >> > > >> >> > -1, keep the code clean and the interface consistent ;). I think the > >> >> > struct syntax should be identical to the class syntax, with the > >> >> > exception that you can't bind methods to structs. That's the only > >> >> > real difference between structs and classes, isn't it? > >> >> > >> >> In C++, the only difference between structs and classes is that struct > >> >> members are public by default. (Not saying that C++ is always the > >> >> model to follow, but it gives precedent). And structs can have > >> >> function members, that's how to do OOP in C. > >> > > >> > Oh. Even more reason to have identical struct and class handling in > >> > Cython ;). > >> > > >> > It is unclear to me what `cdef public struct` means. I think it > >> > should mean "Python bindings can alter this struct's definition", > >> > which doesn't make sense. > >> > >> I think it should mean "this struct is accessible from Python (as X)" > > > > Wouldn't that be "cdef readonly struct X"? > > > >> > Shouldn't the syntax for public members be > >> > > >> >cdef struct X: > >> >cdef public: > >> >int x > >> >readonly int y > >> >private int z > >> > >> -1 on nesting things like this. Rather than make a struct visible from > >> Python iff any of its members are, > > > > A struct is visible from python iff it is declared public or readonly: > > > >cdef public struct X: > >... > &
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Mon, Feb 21, 2011 at 10:09:42AM +1300, Greg Ewing wrote: > W. Trevor King wrote: > > Hmm, that means that > > > > cimport numpy > > a = numpy.numpy.ndarray > > > > also compiles. > > Compiles and runs, or just compiles? You're right. Cython compilation worked, but it didn't run: $ export CFLAGS="-I/usr/lib/python2.6/site-packages/numpy/core/include/" $ python -c 'import pyximport; pyximport.install(); import np;' Traceback (most recent call last): ... File "np.pyx", line 3, in init np (/.../np.c:2846) a = numpy.numpy.ndarray ImportError: Building module failed: ['NameError: numpy\n'] Sorry for muddying the waters with that claim. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpl7psMqItPR.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 19, 2011 at 06:31:26PM -0500, W. Trevor King wrote: > On Sat, Feb 19, 2011 at 02:04:16PM -0800, Robert Bradshaw wrote: > > On Sat, Feb 19, 2011 at 1:45 PM, W. Trevor King wrote: > > > Ah. Sorry for all the c(p)def/qualifier confusion, but I'm trying to > > > consolidate the way these are handled in Parsing/Nodes/Symtab and I > > > want to make sure I don't implement the wrong interpretation. Can you > > > clarify how one knows if "public" means "expose a read/write Python > > > interface to this object" or "expose this symbol to external C code"? > > > > Public has had several different meanings. I wish there were a spec > > and full grammer for Cython, but there's not (yet?). The meaning is > > implicit in the code, and there's enough users out there that we > > should stay backwards compatible. It may be worth doing some > > backwards-incompatible normalization before we hit 1.0, but compiling > > the entire Python grammar is higher priority than that. > > Since I'm going to have lots of similar stuff (classes, enums, > structs, unions) all with the same (hopefully) cdef/cpdef/visibility > stuff for members, I'd like to consolidate now. I will of course, add > special-case code as necessary to support the current syntax, which > can then be removed whenever you think it is appropriate, but writing > separate, near-identical handlers for each type seems like a recipe > for disaster ;). > > I'll look to the code for guidance on public, and try to work out the > appropriate meaning during the parse phase. I've been working on a more explicit parser that removes the ambiguity behind the various visibilities. This will help me ensure proper impolementation of my cdef-ed enums/structs/..., and make it easier to update visibility syntax in the future. Take a look and let me know if you think this is a useful direction to take: git: http://www.physics.drexel.edu/~wking/code/git/cython.git branch: cdef-enums-stucts-and-unions gitweb: http://www.physics.drexel.edu/~wking/code/git/gitweb.cgi?p=cython.git;a=log;h=refs/heads/cdef-enums-stucts-and-unions -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpExMhfTUJMY.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Tue, Feb 22, 2011 at 08:18:21PM +0100, Stefan Behnel wrote:
> W. Trevor King, 22.02.2011 18:55:
> > I've been working on a more explicit parser that removes the ambiguity
> > behind the various visibilities. This will help me ensure proper
> > impolementation of my cdef-ed enums/structs/..., and make it easier to
> > update visibility syntax in the future. Take a look and let me know
> > if you think this is a useful direction to take:
>
> First thing that caught my eyes: I hope you do not intend to leave the
> logging usage in the parser. This is seriously performance critical code
> that Cython compiles down to pretty fast C code. Note that you can use a
> debugger and Python's profiling/tracing interface to find out what's
> happening, without code impact.
I can strip them out afterwards, but it helps me figure out what I've
broken if I shift too much around at the same time.
I don't know enough about Python's trace module to know if I can turn
on tracing only for functions defined in a single module or not, since
otherwise its hard for me to separate signal from noise.
> Some of the log statements span more than one line, which makes it trickier
> to strip them out with sed&friends (but backing out the initial changeset
> would likely make it simple enough to remove the rest manually).
Hmm, perhaps I'll condense the logging statements down onto one (long)
line a piece, that will make it easy to comment/uncomment them with
sed/emacs/etc. I suppose once Cython can compile the logging module
we could leave them in with reduced overhead ;).
> Also note that it's best to write runnable tests ("tests/run/"). The
> tests in "tests/compile/" are only compiled and imported. See the
> hacking guide in the wiki. I know you're not there yet with your
> implementation, I'm just mentioning it.
Thanks for the tip.
> CtxAttribute is the wrong name, though. And the class copy
> implementation gets even more generic than it already was in the
> Ctx. I'm not a big fan of that, especially not in the parser. For
> one, it prevents changing the classes into cdef classes, which had
> long been on my list for Ctx.
An easy, if uglier, workaround would be to prepend attributes with the
class name, e.g. CBinding.visibility -> CBinding.c_binding_visiblity.
Then the Ctx class could subclass the current CtxAttribute classes
instead of binding instances of each of them. That way Ctx would keep
its traditional flat attribute namespace and easy deepcopy, but
eveyone in Nodes, etc. that will use the attributes would become
class-name dependent.
The CtxAttribute class is, as its docstring says, just a hook for its
deepcopy method. With an alternative deepcopy implementation,
CtxAttribute could be replaced with the standard `object`, so don't
worry too much about its name at this point ;).
> CSource: doesn't sound like quite the right name - it does not describe a C
> source file but information that Cython has about non-Cython things.
It's a container for attributes that describe the presence and
location of backing C definitions.
* cdef: "Will there be a backing C defintion?
* extern: "Has someone else already written it?"
* name/namespace: "What did they call it?"
If you'd rather I called the class something else, I'm certainly
willing to change it.
> I also doubt that Cython allows you to call an attribute "cdef", you'll
> need to change that.
It seems to work for me:
>>> import Cython.Compiler.Parsing as P
>>> P.__file__
'Cython/Compiler/Parsing.so'
>>> c = P.CSource()
>>> dir(c)
[..., 'cdef', 'deepcopy', 'extern', 'name', 'namespace']
>>> c.cdef
0
>>> c.cdef = 1
>>> c.cdef
1
However, I agree that it's generally a bad idea to play around with
keywords. I'll revert it to the wordier-but-less-confusing
`cdef_flag`.
--
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
pgptJivrf8KRn.pgp
Description: PGP signature
___
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
[Cython] (was: Cython .pxd introspection: listing defined constants)
W. Trevor King, 22.02.2011 18:55: > I've been working on a more explicit parser that removes the > ambiguity behind the various visibilities. This will help me ensure > proper impolementation of my cdef-ed enums/structs/..., and make it > easier to update visibility syntax in the future. Take a look and > let me know if you think this is a useful direction to take: The refactoring continues as I'm moving my new binding classes into Symtab. This is, of course, leading to lots of changes, but I've been running the test suite before each commit to make sure I don't go to far astray. Anything I miss will eventually lead to a better test suite ;). I'm currently writing up new versions of most Scope methods that use my classes, and I'll replace the original methods once I've updated all the code that calls them. Next on the list will be the Nodes themselves, at which point I think I'll be positioned to put in the `cdef struct` and whatnot that got this whole thing started. Since there has been a fair amount of churn, I though I'd ask for some more feedback on the general direction I'm headed: http://www.physics.drexel.edu/~wking/code/git/gitweb.cgi?p=cython.git;a=log;h=refs/heads/cdef-enums-stucts-and-unions As I said in my previous response to Stefan, I don't really care what the attributes in my classes are called, or how the attributes are partitioned between them. With the current Scope work, I'm leaning towards a single Bindings class that subclasses all of my current bindings, since I have found very few cases where an entire class of nodes only needs one of the bindings, and if we have to pass them all around together, they might as well be a single class. The current names only appear in my new code, so sed & friends will make it easy to tweak things later. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpm55zSfxLt6.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] (was: Cython .pxd introspection: listing defined constants)
On Thu, Feb 24, 2011 at 05:57:21PM -0800, Robert Bradshaw wrote: > On of my primary motivations to moving to github (or similar) was > nicely annotated diffs between branches and the ability to do > line-by-line comments. If you could also push to your fork there, > that'd be great (otherwise I will ;). Ah, Stephan's comment about needing to comment via the mailing list makes more sense now ;). Pushed: https://github.com/wking/cython/tree/cdef-enums-stucts-and-unions -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgplQI3b6iBJX.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Fri, Feb 25, 2011 at 11:11:03PM -0800, Robert Bradshaw wrote: > On Tue, Feb 22, 2011 at 12:02 PM, W. Trevor King wrote: > > An easy, if uglier, workaround would be to prepend attributes with the > > class name, e.g. CBinding.visibility -> CBinding.c_binding_visiblity. > > Then the Ctx class could subclass the current CtxAttribute classes > > instead of binding instances of each of them. That way Ctx would keep > > its traditional flat attribute namespace and easy deepcopy, but > > eveyone in Nodes, etc. that will use the attributes would become > > class-name dependent. > > I'd be up for flattening this. In particular, changing every > "entry.name" to "entry.python_binding.name" seems to be a lot of churn > and extra verbiage for not much benefit. The only overlap I see is > name and visibility, and keeping name/cname and adding cvisibility > would be preferable to me. That works for me, but I see possible ambiguity in cname. From the "external C code" docs: The other way is to use a C name specification to give different Cython and C names to the C function. Suppose, for example, that you want to wrap an external function called eject_tomato(). If you declare it as: cdef extern void c_eject_tomato "eject_tomato" (float speed) then its name inside the Cython module will be c_eject_tomato, whereas its name in C will be eject_tomato. In this case I was eventually going to use c_source.name = eject_tomato c_source.namespace = ... c_binding.name = c_eject_tomato c_binding.namespace = ... python_binding.name = eject_tomato I'm not sure how Cython handles name/cname with this case at the moment, but it seems like there are two cnames to keep track of. > > The CtxAttribute class is, as its docstring says, just a hook for its > > deepcopy method. With an alternative deepcopy implementation, > > CtxAttribute could be replaced with the standard `object`, so don't > > worry too much about its name at this point ;). > > You mean shallow copy? I meant deepcopy, since that seems to be the point of Ctx cloning. At the moment, however, Ctx deep copies only require Binding shallow copies. If someone crazy wanted to add mutable attrubites to binding classes, the Binding deepcopy code would have had to be adjusted. None of this matters though, if we move back to a flat Ctx attribute space. > >> CSource: doesn't sound like quite the right name - it does not describe a C > >> source file but information that Cython has about non-Cython things. > > > > It's a container for attributes that describe the presence and > > location of backing C definitions. > > > > * cdef: "Will there be a backing C defintion? > > * extern: "Has someone else already written it?" > > * name/namespace: "What did they call it?" > > > > If you'd rather I called the class something else, I'm certainly > > willing to change it. > > It seems a bit odd to me, but if need be we can rename it later. > However, csource and c_binding seem rather redundant to me, but as > mentioned above I think it's better just to flatten it all. > > The changes to parsing look decent to me, but admittedly there's a lot > of renaming churn, so I could have missed something. I'll go back and revert out the name changes if you'll confirm that there is only one meaning for cname. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpkwFDxnnHeJ.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Expected errors of tests/errors/cdef_members_T517.pxd
I'm splitting Symtab visibilities into explicit C and Python visibilities, but am having trouble reproducing the expected error messages for cdef_members_T517: $ python runtests.py cdef_members_T517 Python 2.6.6 (r266:84292, Dec 8 2010, 09:53:33) [GCC 4.4.4] Running tests against Cython 0.14.1+ === Expected errors: === 5:24: C attribute of type 'VoidP' cannot be accessed from Python 5:24: Cannot convert 'VoidP' to Python object 6:24: C attribute of type 'VoidP' cannot be accessed from Python 6:24: Cannot convert 'VoidP' to Python object 6:24: Cannot convert Python object to 'VoidP' 14:22: C attribute of type 'Foo' cannot be accessed from Python 14:22: Cannot convert Python object to 'Foo' === Got errors: === 5:24: C attribute of type 'VoidP' cannot be accessed from Python 5:24: Cannot convert 'VoidP' to Python object 6:24: C attribute of type 'VoidP' cannot be accessed from Python 6:24: Cannot convert 'VoidP' to Python object 6:24: Cannot convert Python object to 'VoidP' 14:22: Cannot convert Python object to 'Foo' So my code is not raising: 14:22: C attribute of type 'Foo' cannot be accessed from Python The relevant lines from tests/errors/cdef_members_T517.pxd are: $ grep -n ^ tests/errors/cdef_members_T517.pyx ... 8:ctypedef struct Foo: 9:int i 10: 11:cdef class Bar: 12:cdef Foo foo0 13:cdef readonly Foo foo2 14:cdef public Foo foo1 15:pass ... I see two options: 1) Foo attributes are readable from Python, in which case the expected errors should be changed to match mine. 2) Foo attributes are not readable from Python, in which case the expected errors should be extended with 13:22: C attribute of type 'Foo' cannot be accessed from Python and probably also also 14:22: Cannot convert 'Foo' to Python object I think (2) is more reasonable, until I get my public Python enums, stucts, and unions branch working, after which it would switch to (1) if the Foo struct was declared with readonly (vs. private) Python visibility. You can't alter the struct definition from Python, so a public Python visibility doesn't really make sense for the struct itself. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpnXJYnxARaN.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 26, 2011 at 10:01:43AM -0800, Robert Bradshaw wrote: > On Sat, Feb 26, 2011 at 3:48 AM, W. Trevor King wrote: > > On Fri, Feb 25, 2011 at 11:11:03PM -0800, Robert Bradshaw wrote: > >> On Tue, Feb 22, 2011 at 12:02 PM, W. Trevor King wrote: > >> > An easy, if uglier, workaround would be to prepend attributes with the > >> > class name, e.g. CBinding.visibility -> CBinding.c_binding_visiblity. > >> > Then the Ctx class could subclass the current CtxAttribute classes > >> > instead of binding instances of each of them. That way Ctx would keep > >> > its traditional flat attribute namespace and easy deepcopy, but > >> > eveyone in Nodes, etc. that will use the attributes would become > >> > class-name dependent. > >> > >> I'd be up for flattening this. In particular, changing every > >> "entry.name" to "entry.python_binding.name" seems to be a lot of churn > >> and extra verbiage for not much benefit. The only overlap I see is > >> name and visibility, and keeping name/cname and adding cvisibility > >> would be preferable to me. > > > > That works for me, but I see possible ambiguity in cname. From the > > "external C code" docs: > > > >The other way is to use a C name specification to give different > >Cython and C names to the C function. Suppose, for example, that > >you want to wrap an external function called eject_tomato(). If > >you declare it as: > > > >cdef extern void c_eject_tomato "eject_tomato" (float speed) > > > >then its name inside the Cython module will be c_eject_tomato, > >whereas its name in C will be eject_tomato. > > > > In this case I was eventually going to use > > > >c_source.name = eject_tomato > >c_source.namespace = ... > >c_binding.name = c_eject_tomato > >c_binding.namespace = ... > >python_binding.name = eject_tomato > > > > I'm not sure how Cython handles name/cname with this case at the > > moment, but it seems like there are two cnames to keep track of. > > In this case, cname is "eject_tomato" (which actually gets emitted in > the C source file to use it) and name (in both Python and Cython > namespace) is "c_eject_tomato." What is the "Cython namespace"? Is that what you get when you cimport something (vs. the Python namespace being what you get when you import something). If so, then that sounds like just two names, so name/cname it is. > >> > The CtxAttribute class is, as its docstring says, just a hook for its > >> > deepcopy method. With an alternative deepcopy implementation, > >> > CtxAttribute could be replaced with the standard `object`, so don't > >> > worry too much about its name at this point ;). > >> > >> You mean shallow copy? > > > > I meant deepcopy, since that seems to be the point of Ctx cloning. At > > the moment, however, Ctx deep copies only require Binding shallow > > copies. If someone crazy wanted to add mutable attrubites to binding > > classes, the Binding deepcopy code would have had to be adjusted. > > None of this matters though, if we move back to a flat Ctx attribute > > space. > > OK. As an aside, do you know about the copy.deepcopy() method? Yup, but I was trying to avoid pulling in non-Cython dependencies since you guys seem to have gone through a lot of trouble to make the Parsing module fast in C. > >> >> CSource: doesn't sound like quite the right name - it does not describe > >> >> a C > >> >> source file but information that Cython has about non-Cython things. > >> > > >> > It's a container for attributes that describe the presence and > >> > location of backing C definitions. > >> > > >> > * cdef: "Will there be a backing C defintion? > >> > * extern: "Has someone else already written it?" > >> > * name/namespace: "What did they call it?" > >> > > >> > If you'd rather I called the class something else, I'm certainly > >> > willing to change it. > >> > >> It seems a bit odd to me, but if need be we can rename it later. > >> However, csource and c_binding seem rather redundant to me, but as > >> mentioned above I think it's better just to flatten it all. > >> > >> The changes to parsing look decent to me, but admittedly there's a lot > >> of renaming
Re: [Cython] Expected errors of tests/errors/cdef_members_T517.pxd
On Sat, Feb 26, 2011 at 10:15:38AM -0800, Robert Bradshaw wrote: > On Sat, Feb 26, 2011 at 5:08 AM, W. Trevor King wrote: > > The relevant lines from tests/errors/cdef_members_T517.pxd are: > > > >$ grep -n ^ tests/errors/cdef_members_T517.pyx > >... > >8:ctypedef struct Foo: > >9:int i > >10: > >11:cdef class Bar: > >12:cdef Foo foo0 > >13:cdef readonly Foo foo2 > >14:cdef public Foo foo1 > >15:pass > >... > > ... > > What's less clear is the behavior of > >b = Bar() >b.foo1 = Foo(i=1) >b.foo1.i = 100 >print b.foo1 >saved = b.foo1 >del b >print saved Is the problem choosing between: 1) Copy struct data into its own PyObject and let Python manage the memory. 2) Pass around a pointer to Cython-managed struct data. I suppose I don't really know enough about how Cython handles this yet to really see the problem or be able to help with a solution ;). -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpPEITgq5Y4p.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Sat, Feb 26, 2011 at 07:11:48PM -0800, Robert Bradshaw wrote: > On Sat, Feb 26, 2011 at 6:14 PM, W. Trevor King wrote: > > On Sat, Feb 26, 2011 at 10:01:43AM -0800, Robert Bradshaw wrote: > >> On Sat, Feb 26, 2011 at 3:48 AM, W. Trevor King wrote: > >> > On Fri, Feb 25, 2011 at 11:11:03PM -0800, Robert Bradshaw wrote: > >> >> On Tue, Feb 22, 2011 at 12:02 PM, W. Trevor King > >> >> wrote: > >> >> > An easy, if uglier, workaround would be to prepend attributes with the > >> >> > class name, e.g. CBinding.visibility -> CBinding.c_binding_visiblity. > >> >> > Then the Ctx class could subclass the current CtxAttribute classes > >> >> > instead of binding instances of each of them. That way Ctx would keep > >> >> > its traditional flat attribute namespace and easy deepcopy, but > >> >> > eveyone in Nodes, etc. that will use the attributes would become > >> >> > class-name dependent. > >> >> > >> >> I'd be up for flattening this. In particular, changing every > >> >> "entry.name" to "entry.python_binding.name" seems to be a lot of churn > >> >> and extra verbiage for not much benefit. The only overlap I see is > >> >> name and visibility, and keeping name/cname and adding cvisibility > >> >> would be preferable to me. > >> > > >> > That works for me, but I see possible ambiguity in cname. From the > >> > "external C code" docs: > >> > > >> >The other way is to use a C name specification to give different > >> >Cython and C names to the C function. Suppose, for example, that > >> >you want to wrap an external function called eject_tomato(). If > >> >you declare it as: > >> > > >> >cdef extern void c_eject_tomato "eject_tomato" (float speed) > >> > > >> >then its name inside the Cython module will be c_eject_tomato, > >> >whereas its name in C will be eject_tomato. > >> > > >> > In this case I was eventually going to use > >> > > >> >c_source.name = eject_tomato > >> >c_source.namespace = ... > >> >c_binding.name = c_eject_tomato > >> >c_binding.namespace = ... > >> >python_binding.name = eject_tomato > >> > > >> > I'm not sure how Cython handles name/cname with this case at the > >> > moment, but it seems like there are two cnames to keep track of. > >> > >> In this case, cname is "eject_tomato" (which actually gets emitted in > >> the C source file to use it) and name (in both Python and Cython > >> namespace) is "c_eject_tomato." > > > > What is the "Cython namespace"? Is that what you get when you cimport > > something (vs. the Python namespace being what you get when you import > > something). > > Yes, that would be a good way to understand it. Okay, I'm back to a flat attribute namespace in my branch, and I'm moving through Nodes removing the places where I currently collapse back to an overloaded visibility. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpsvqE3jNR30.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
Previous discussions in this thread [1,2] have discussed the issues associated with overloading the 'public' keyword. For an example of the difficulties this causes, see the ugly workaround [3] in my recent commit [4]. Definately worth fixing this syntax. How do syntax changes with deprecations work? The Parsing module doesn't seem to be setup to handle things like that. [1] http://mail.python.org/pipermail/cython-devel/2011-February/81.html [2] http://mail.python.org/pipermail/cython-devel/2011-February/86.html [3] https://github.com/wking/cython/commit/a918d1466928ea712817df18c3bc66d9c4c5c53f#L1R2552 [4] https://github.com/wking/cython/commit/a918d1466928ea712817df18c3bc66d9c4c5c53f -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgp1dbiqfsl8X.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants
On Wed, Mar 02, 2011 at 06:08:12PM -0800, Robert Bradshaw wrote: > On Wed, Mar 2, 2011 at 5:54 PM, W. Trevor King wrote: > > Previous discussions in this thread [1,2] have discussed the issues > > associated with overloading the 'public' keyword. For an example of > > the difficulties this causes, see the ugly workaround [3] in my recent > > commit [4]. Definately worth fixing this syntax. > > > > How do syntax changes with deprecations work? The Parsing module > > doesn't seem to be setup to handle things like that. > > It's not. > > We rarely deprecate syntax, and we should at least give people a > chance to move away from the old syntax first to the new first, so > we'll have to support both for a bit at least. When we do deprecate > the old way of doing things, I think we'll just use > Errors.warning(...). But how would you know which syntax version the source file was aiming for? There should probably be a way to explicitly specify a Cython syntax version, probably from the command line and/or setup.py file, e.g. pyrex_syntax_version = (, , ) as an argument to Cython.Distutils.extension.Extension, which would select the appropriate parser. The default (if the version was not explicitly set), would be to use the current parser. With this setup, projects would only be forced to upgrade if Cython removed support for their syntax entirely, and they could choose to upgrade whenever was most convienient for them. I think the syntax version's major/minor number should probably match the Cython version in which they were introduced. Parser fixes and compatible extensions would bump the number, and you'd probably want to deprecate buggy versions more quickly. -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgp6tWU52M0Ba.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] cdef-enums-stucts-and-unions progress update
Done: * Cleaned all overloaded visibilities out of Parsing and Symtab. * Merged current trunk. Todo: * Remove older Symtab interface so I can remove the WTK_* methods. * Activate `shadow` functionality (via AnalyseDeclarationsTransform.visit_CStructOrUnionDefNode()). Feel free to comment via github: https://github.com/wking/cython/commits/cdef-enums-stucts-and-unions I'm not quite sure how this shadow thing is supposed to work, but I'll take a stab at getting it going if a more seasoned Cythonista is not available for comment ;). -- This email may be signed or encrypted with GPG (http://www.gnupg.org). The GPG signature (if present) will be attached as 'signature.asc'. For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt pgpO0mjdm54pj.pgp Description: PGP signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Time to close relative import ticket on Trac?
Browsing around today I noticed that the relative import ticket is still open [1], even though support for relative imports landed in 0.15 [2]. There's also CEP 307 (Generators and PEP 342 Coroutines) [1], which is still listed as “planning, undecided” but also seems to be fixed in 0.15. The RTD page is also frozen at 0.15pre [3]. Cheers, Trevor [1]: cython-devel@python.org [2]: http://wiki.cython.org/ReleaseNotes-0.15 [3]: https://cython.readthedocs.org/en/latest/ -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
