Re: [Cython] Fused Types
On 28 April 2011 23:30, Robert Bradshaw wrote: > On Thu, Apr 28, 2011 at 1:31 PM, mark florisson > wrote: >> On 28 April 2011 22:12, Robert Bradshaw wrote: >>> On Thu, Apr 28, 2011 at 12:48 PM, mark florisson >>> wrote: >>> So I fixed all that, but I'm currently wondering about the proposed cython.typeof(). I believe it currently returns a string with the type name, and not the type itself. >>> >>> Yes. This is just because there's not really anything better to return >>> at this point. We should "fix" this at some point in the future. >>> So I think it would be inconsistent to suddenly start allowing comparison with 'is' and 'isinstance' and such. >>> >>> I'm open to other suggestions, but would like an expression that >>> resolves at compile time to true/false (and we need to do branch >>> pruning on it). Note that type() is not good enough, because it has >>> different semantics, i.e. >>> >>> cdef object o = [] >>> typeof(o), type(o) >>> >>> so lets not touch that one. >> >> Right, so for fused types I don't mind string comparison with >> cython.typeof(), but retrieval of the actual type for casts and >> declaration remains open. I'd be fine with something like >> cython.gettype(). > > I'd rather re-use typeof here than introduce yet another special > method. On the other hand, allowing parentheses in a type declaration > requires complicating the syntax even more. > > I'm not sure this is needed, couldn't one do > > cdef foo(fused_type a): > cdef fused_type b = a + func() > > and all instances of fused_type get specialized in the body. Right, that is supported already. >> On the other hand, the user could already do this thing by manually >> declaring another fused type that would perhaps be resolved based on >> its usage. e.g. for my original example: >> >> ctypedef char *string_t >> ctypedef cython.fused_type(int, string_t) attrib_t >> >> cdef func(struct_t mystruct, int i): >> cdef attrib_t var = mystruct.attrib + i >> >> Is this something that should be supported anyway? > > OK, I take back what I said, I was looking at the RHS, not the LHS. If > one needs to specialize in this manner, explicitly creating two > branches should typically be enough. The same for casting. The one > exception (perhaps) is "my_fused_type complex." Otherwise it's > starting to feel too much like C++ template magic and complexity for > little additional benefit. Ok, branching on the type sounds fine to me. It brings one problem though: because you cannot declare the variables of your variable type (the type of say, mystruct.attrib), you will need to do multiple declarations outside of your branches. So in my example: cdef func(struct_t mystruct, int i): cdef string_t string_var cdef int int_var if typeof(mystruct) is typeof(int): int_var = mystruct.attrib + i ... else: string_var = mystruct.attrib + i ... But this is probably not a common case, so likely not an issue. >> Currently fused >> types can only be used when appearing as argument types (in the >> function body and as return value). > I'm also wondering if it would be useful to allow actual type retrieval, which could be used in declarations and casts. For instance consider fusing two structs with the same attribute name but different attribute types. Perhaps in your code you want to introduce a variable compatible with such a type, e.g. consider this: ctypdef struct A: int attrib ctypedef struct B: char *attrib ctypedef cython.fused_type(A, B) struct_t cdef func(struct_t mystruct, int i): cdef cython.gettype(mystruct.attrib) var = mystruct.attrib + i ... What do you think? >>> >>> Yes, I was thinking that would be supported as well. >>> >>> - Robert >>> ___ >>> cython-devel mailing list >>> cython-devel@python.org >>> http://mail.python.org/mailman/listinfo/cython-devel >>> >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel >> > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 28 April 2011 23:59, Robert Bradshaw wrote: > On Thu, Apr 28, 2011 at 2:29 PM, mark florisson > wrote: >> On 28 April 2011 22:31, mark florisson wrote: >>> On 28 April 2011 22:12, Robert Bradshaw >>> wrote: On Thu, Apr 28, 2011 at 12:48 PM, mark florisson wrote: > So I fixed all that, but I'm currently wondering about the proposed > cython.typeof(). I believe it currently returns a string with the type > name, and not the type itself. Yes. This is just because there's not really anything better to return at this point. We should "fix" this at some point in the future. > So I think it would be inconsistent to > suddenly start allowing comparison with 'is' and 'isinstance' and > such. I'm open to other suggestions, but would like an expression that resolves at compile time to true/false (and we need to do branch pruning on it). Note that type() is not good enough, because it has different semantics, i.e. cdef object o = [] typeof(o), type(o) so lets not touch that one. >>> >>> Right, so for fused types I don't mind string comparison with >>> cython.typeof(), but retrieval of the actual type for casts and >>> declaration remains open. I'd be fine with something like >>> cython.gettype(). >> >> It seems that this isn't optimized yet, but it looks to me like it >> wouldn't be very hard to do so. At least == and != could be resolved >> at compile time if the other operand is a string literal. > > Yes. We could consider supporting "typeof(x) is typeof(double)" or > even "typeof(int*)" etc. as well to not tie ourselves to strings. Or > perhaps some other syntax we haven't thought of. Alternatively, it > would be nice if it involved the literal fused_type as that's what > we're really branching on, e.g. > > ctypedef cython.fused_type(A, B) AorB > > cdef foo(AorB x): > if AorB[A]: # or .A or something > ... > > it's hard to come up with something that plays nicely with pointer types. I like the typeof(my_type_here). So if we support listing types in typeof(), then you could also call typeof() on AorB to specialize. > - Robert > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 29 April 2011 06:32, Robert Bradshaw wrote: > On Thu, Apr 28, 2011 at 7:09 PM, Stefan Behnel wrote: >> mark florisson, 28.04.2011 23:29: >>> >>> On 28 April 2011 22:31, mark florisson wrote: On 28 April 2011 22:12, Robert Bradshaw wrote: > > On Thu, Apr 28, 2011 at 12:48 PM, mark florisson wrote: > >> So I fixed all that, but I'm currently wondering about the proposed >> cython.typeof(). I believe it currently returns a string with the type >> name, and not the type itself. > > Yes. This is just because there's not really anything better to return > at this point. We should "fix" this at some point in the future. > >> So I think it would be inconsistent to >> suddenly start allowing comparison with 'is' and 'isinstance' and >> such. > > I'm open to other suggestions, but would like an expression that > resolves at compile time to true/false (and we need to do branch > pruning on it). Note that type() is not good enough, because it has > different semantics, i.e. > > cdef object o = [] > typeof(o), type(o) > > so lets not touch that one. Right, so for fused types I don't mind string comparison with cython.typeof(), but retrieval of the actual type for casts and declaration remains open. I'd be fine with something like cython.gettype(). >>> >>> It seems that this isn't optimized yet, but it looks to me like it >>> wouldn't be very hard to do so. At least == and != could be resolved >>> at compile time if the other operand is a string literal. >> >> Well, the obvious place where this would happen for free would be constant >> folding. But that runs *way* before type analysis, so all it sees is the >> typeof() call, not the string. > > Actually, to support code like > > ctypedef cython.fused_type(object, double) my_fused_type > > cdef foo(my_fused_type x): > if my_fused_type is object: > print x.attr > else: > cdef my_fused_type *ptr = &x > > we need to resolve this branch and prune "dead code" before type > analysis, so I'm thinking it may need to be a special phase, perhaps > part of the fused-type-specialization phase. Here's another idea: > > cdef foo(numeric x): > if numeric in floating: > ... > elif numeric is long: > ... > else: > ... > print numeric is double # after the specialization pass, this > would be a literal True/False node. Hmm, that one looks pretty ok. We could also do typeof(numeric) is typeof(floating), and if any of the fused types are unresolved it works like 'in' instead of 'is'. Although in that case people could also write typeof(floating) is typeof(numeric) which would behave like 'in' with reversed operands :) > Again, this would all be resolved before type analysis. In terms of > syntactically supporting pointers, we could either support "fused_type > is typeof(void*)" or require typedefs for types not representable by > an ExprNode. (This would make declaring fused types syntactically > simpler as well...) I prefer the latter. Me too, cython.fused_type() currently just parses identifiers only. > - Robert > > >> Optimising this would thus basically require a second (simpler?) constant >> folding step, or at least an additional hook during (or after) type analysis >> that does the comparison. I wouldn't mind too much, given that type analysis >> can end up injecting some rather handy information into the tree. So a >> second (even complete) constant folding step *after* type analysis may still >> introduce some nice optimisations. >> >> We may actually consider splitting constant folding into two steps - one >> that calculates the results as early as possible (and maybe does only >> safe&obvious tree replacements, e.g. boolean values, strings, etc.) and a >> second step that replaces subtrees by constant nodes once it knows the final >> type of the result. >> >> Stefan >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel >> > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
Fri, 29 Apr 2011 10:23:55 +0200, mark florisson wrote: [clip] > Ok, branching on the type sounds fine to me. It brings one problem > though: because you cannot declare the variables of your variable type > (the type of say, mystruct.attrib), you will need to do multiple > declarations outside of your branches. So in my example: > > cdef func(struct_t mystruct, int i): > cdef string_t string_var > cdef int int_var > > if typeof(mystruct) is typeof(int): > int_var = mystruct.attrib + i > ... > else: >string_var = mystruct.attrib + i >... > > But this is probably not a common case, so likely not an issue. Are you planning to special-case the "real_t complex" syntax? Shooting from the sidelines, one more generic solution might be, e.g., ctypedef cython.fused_type(A, B) struct_t ctypedef cython.fused_type(float, double, paired=struct_t) real_t ctypedef cython.fused_type(int_t, string_t, paired=struct_t) var_t and just restrict the specialization to cases that make sense. -- Pauli Virtanen ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 29 April 2011 11:03, Pauli Virtanen wrote: > Fri, 29 Apr 2011 10:23:55 +0200, mark florisson wrote: > [clip] >> Ok, branching on the type sounds fine to me. It brings one problem >> though: because you cannot declare the variables of your variable type >> (the type of say, mystruct.attrib), you will need to do multiple >> declarations outside of your branches. So in my example: >> >> cdef func(struct_t mystruct, int i): >> cdef string_t string_var >> cdef int int_var >> >> if typeof(mystruct) is typeof(int): >> int_var = mystruct.attrib + i >> ... >> else: >> string_var = mystruct.attrib + i >> ... >> >> But this is probably not a common case, so likely not an issue. > > Are you planning to special-case the "real_t complex" syntax? Shooting > from the sidelines, one more generic solution might be, e.g., I'm sorry, I'm not sure what syntax you are referring to. Are you talking about actual complex numbers? > ctypedef cython.fused_type(A, B) struct_t > ctypedef cython.fused_type(float, double, paired=struct_t) real_t > ctypedef cython.fused_type(int_t, string_t, paired=struct_t) var_t > > and just restrict the specialization to cases that make sense. The paired means you're declaring types of attributes? > -- > Pauli Virtanen > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
Fri, 29 Apr 2011 11:30:19 +0200, mark florisson wrote: > On 29 April 2011 11:03, Pauli Virtanen wrote: [clip] >> Are you planning to special-case the "real_t complex" syntax? Shooting >> from the sidelines, one more generic solution might be, e.g., > > I'm sorry, I'm not sure what syntax you are referring to. Are you > talking about actual complex numbers? This: On 28 April 2011 23:30, Robert Bradshaw wrote: > OK, I take back what I said, I was looking at the RHS, not the LHS. If > one needs to specialize in this manner, explicitly creating two > branches should typically be enough. The same for casting. The one > exception (perhaps) is "my_fused_type complex." Otherwise it's > starting to feel too much like C++ template magic and complexity for > little additional benefit. That is, declaring a complex type matching a real one. >> ctypedef cython.fused_type(A, B) struct_t >>ctypedef cython.fused_type(float, double, paired=struct_t) real_t >> ctypedef cython.fused_type(int_t, string_t, paired=struct_t) var_t >> >> and just restrict the specialization to cases that make sense. > > The paired means you're declaring types of attributes? No, just that real_t is specialized to float whenever struct_t is specialized to A and to double when B. Or a more realistic example, ctypedef cython.fused_type(float, double) real_t ctypedef cython.fused_type(float complex, double complex) complex_t cdef real_plus_one(complex_t a): real_t b = a.real return b + 1 which I suppose would not be a very unusual thing in numerical codes. This would also allow writing the case you had earlier as cdef cython.fused_type(string_t, int, paired=struct_t) attr_t cdef func(struct_t mystruct, int i): cdef attr_t var if typeof(mystruct) is typeof(int): var = mystruct.attrib + i ... else: var = mystruct.attrib + i ... Things would need to be done explicitly instead of implicitly, though, but it would remove the need for any special handling of the "complex" keyword. -- Pauli Virtanen ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 29 April 2011 12:28, Pauli Virtanen wrote: > Fri, 29 Apr 2011 11:30:19 +0200, mark florisson wrote: >> On 29 April 2011 11:03, Pauli Virtanen wrote: > [clip] >>> Are you planning to special-case the "real_t complex" syntax? Shooting >>> from the sidelines, one more generic solution might be, e.g., >> >> I'm sorry, I'm not sure what syntax you are referring to. Are you >> talking about actual complex numbers? > > This: > > On 28 April 2011 23:30, Robert Bradshaw > wrote: >> OK, I take back what I said, I was looking at the RHS, not the LHS. If >> one needs to specialize in this manner, explicitly creating two >> branches should typically be enough. The same for casting. The one >> exception (perhaps) is "my_fused_type complex." Otherwise it's >> starting to feel too much like C++ template magic and complexity for >> little additional benefit. > > That is, declaring a complex type matching a real one. Ah, I see what you mean now. >>> ctypedef cython.fused_type(A, B) struct_t >>> ctypedef cython.fused_type(float, double, paired=struct_t) real_t >>> ctypedef cython.fused_type(int_t, string_t, paired=struct_t) var_t >>> >>> and just restrict the specialization to cases that make sense. >> >> The paired means you're declaring types of attributes? > > No, just that real_t is specialized to float whenever struct_t is specialized > to A and to double when B. Or a more realistic example, > > ctypedef cython.fused_type(float, double) real_t > ctypedef cython.fused_type(float complex, double complex) complex_t > > cdef real_plus_one(complex_t a): > real_t b = a.real > return b + 1 > > which I suppose would not be a very unusual thing in numerical codes. > This would also allow writing the case you had earlier as > > cdef cython.fused_type(string_t, int, paired=struct_t) attr_t > > cdef func(struct_t mystruct, int i): > cdef attr_t var > > if typeof(mystruct) is typeof(int): > var = mystruct.attrib + i > ... > else: > var = mystruct.attrib + i > ... > > Things would need to be done explicitly instead of implicitly, though, > but it would remove the need for any special handling of > the "complex" keyword. I see, so it's like a mapping. So, I didn't realize that you can't do this: def func(arbitrary_type complex x): ... But if we just allow that for fused types, then couldn't we simply do ctypedef cython.fused_type(float, double) real_t cdef real_plus_one(real_t complex a): real_t b = a.real return b + 1 ? Then you don't need to pair anything. Perhaps we could introduce real_t as a type, just like numeric and floating. So I guess special-casing complex sounds fine with me. Perhaps real_t should be builtin (not as an attribute of the Cython module), so the parser can just recognize it immediately? > -- > Pauli Virtanen > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 04/29/2011 12:53 PM, mark florisson wrote: On 29 April 2011 12:28, Pauli Virtanen wrote: Fri, 29 Apr 2011 11:30:19 +0200, mark florisson wrote: On 29 April 2011 11:03, Pauli Virtanen wrote: [clip] Are you planning to special-case the "real_t complex" syntax? Shooting from the sidelines, one more generic solution might be, e.g., I'm sorry, I'm not sure what syntax you are referring to. Are you talking about actual complex numbers? This: On 28 April 2011 23:30, Robert Bradshaw wrote: OK, I take back what I said, I was looking at the RHS, not the LHS. If one needs to specialize in this manner, explicitly creating two branches should typically be enough. The same for casting. The one exception (perhaps) is "my_fused_type complex." Otherwise it's starting to feel too much like C++ template magic and complexity for little additional benefit. That is, declaring a complex type matching a real one. Ah, I see what you mean now. ctypedef cython.fused_type(A, B) struct_t ctypedef cython.fused_type(float, double, paired=struct_t) real_t ctypedef cython.fused_type(int_t, string_t, paired=struct_t) var_t and just restrict the specialization to cases that make sense. The paired means you're declaring types of attributes? No, just that real_t is specialized to float whenever struct_t is specialized to A and to double when B. Or a more realistic example, ctypedef cython.fused_type(float, double) real_t ctypedef cython.fused_type(float complex, double complex) complex_t cdef real_plus_one(complex_t a): real_t b = a.real return b + 1 which I suppose would not be a very unusual thing in numerical codes. Did you mean ctypedef cython.fused_type(float complex, double complex, paired=real_t) complex_t ? This would also allow writing the case you had earlier as cdef cython.fused_type(string_t, int, paired=struct_t) attr_t cdef func(struct_t mystruct, int i): cdef attr_t var if typeof(mystruct) is typeof(int): var = mystruct.attrib + i ... else: var = mystruct.attrib + i ... Things would need to be done explicitly instead of implicitly, though, but it would remove the need for any special handling of the "complex" keyword. I see, so it's like a mapping. So, I didn't realize that you can't do this: def func(arbitrary_type complex x): ... We could support this, but I don't think it is powerful enough. I see some code that require pairings like this: ctypedef cython.fused_type(float, double, float complex, \ double complex) complex_or_float_t ctypedef cython.fused_type(float, double, float, \ double) only_float_t cdef func(complex_or_float_t x): cdef only_float_t y ... So IIUC, one could here add "paired=complex_or_float_t" to say that only_float_t links positionally to corresponding types in complex_or_float_t. Perhaps "pair_up_with="? "given_by="? Anyway, I'm wondering if the special case of complex could be handled by having magical built-in fused types for the floating point for these purposes, and that these would suffice *shrug*. Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
Fri, 29 Apr 2011 12:53:06 +0200, mark florisson wrote: [clip] > But if we just allow that for fused types, then couldn't we simply do > > ctypedef cython.fused_type(float, double) real_t > > cdef real_plus_one(real_t complex a): > real_t b = a.real > return b + 1 > > ? Then you don't need to pair anything. Perhaps we could introduce > real_t as a type, just like numeric and floating. So I guess > special-casing complex sounds fine with me. Special-casing for complex sounds ugly to me; "complex float" is a type name in the same way as "long int" is, and writing "real_t complex" feels a bit like writing "int_t long". But of course, if this feature is really only needed for declaring complex variants of real types, then how it is done doesn't matter so much. But, IMHO, such a special casing would be a somewhat weird language feature. -- Pauli Virtanen ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 29 April 2011 13:37, Pauli Virtanen wrote: > Fri, 29 Apr 2011 12:53:06 +0200, mark florisson wrote: > [clip] >> But if we just allow that for fused types, then couldn't we simply do >> >> ctypedef cython.fused_type(float, double) real_t >> >> cdef real_plus_one(real_t complex a): >> real_t b = a.real >> return b + 1 >> >> ? Then you don't need to pair anything. Perhaps we could introduce >> real_t as a type, just like numeric and floating. So I guess >> special-casing complex sounds fine with me. > > Special-casing for complex sounds ugly to me; "complex float" > is a type name in the same way as "long int" is, and writing > "real_t complex" feels a bit like writing "int_t long". > > But of course, if this feature is really only needed for declaring > complex variants of real types, then how it is done doesn't matter > so much. But, IMHO, such a special casing would be a somewhat weird > language feature. Hmm, indeed, it's pretty weird. I'm fine with the pairing also, although I'm still not sure how common this case is, and if we really want to support it. Wouldn't good old C promotion work for this? e.g. if the type is either float or double, just declare your variable double? > -- > Pauli Virtanen > > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On 29 April 2011 06:32, Robert Bradshaw wrote: > On Thu, Apr 28, 2011 at 7:09 PM, Stefan Behnel wrote: >> mark florisson, 28.04.2011 23:29: >>> >>> On 28 April 2011 22:31, mark florisson wrote: On 28 April 2011 22:12, Robert Bradshaw wrote: > > On Thu, Apr 28, 2011 at 12:48 PM, mark florisson wrote: > >> So I fixed all that, but I'm currently wondering about the proposed >> cython.typeof(). I believe it currently returns a string with the type >> name, and not the type itself. > > Yes. This is just because there's not really anything better to return > at this point. We should "fix" this at some point in the future. > >> So I think it would be inconsistent to >> suddenly start allowing comparison with 'is' and 'isinstance' and >> such. > > I'm open to other suggestions, but would like an expression that > resolves at compile time to true/false (and we need to do branch > pruning on it). Note that type() is not good enough, because it has > different semantics, i.e. > > cdef object o = [] > typeof(o), type(o) > > so lets not touch that one. Right, so for fused types I don't mind string comparison with cython.typeof(), but retrieval of the actual type for casts and declaration remains open. I'd be fine with something like cython.gettype(). >>> >>> It seems that this isn't optimized yet, but it looks to me like it >>> wouldn't be very hard to do so. At least == and != could be resolved >>> at compile time if the other operand is a string literal. >> >> Well, the obvious place where this would happen for free would be constant >> folding. But that runs *way* before type analysis, so all it sees is the >> typeof() call, not the string. > > Actually, to support code like > > ctypedef cython.fused_type(object, double) my_fused_type > > cdef foo(my_fused_type x): > if my_fused_type is object: > print x.attr > else: > cdef my_fused_type *ptr = &x > > we need to resolve this branch and prune "dead code" before type > analysis, so I'm thinking it may need to be a special phase, perhaps > part of the fused-type-specialization phase. Here's another idea: > > cdef foo(numeric x): > if numeric in floating: > ... > elif numeric is long: > ... > else: > ... > print numeric is double # after the specialization pass, this > would be a literal True/False node. > > Again, this would all be resolved before type analysis. In terms of > syntactically supporting pointers, we could either support "fused_type > is typeof(void*)" or require typedefs for types not representable by > an ExprNode. (This would make declaring fused types syntactically > simpler as well...) I prefer the latter. > > - Robert > I made both things work, you can use both typeof() on expressions and you can compare types using 'is' and 'in'. However, with typeof() it doesn't remove branches, so if you're doing incompatible stuff you need to do the type matching. It could be supported, but currently it isn't because TransformBuiltinMethods runs after AnalyseDeclarationsTransform (for which the reason is not immediately apparent). With the type matching it matches on exactly 'if src_type is dst_type:' so you can't use 'and' and such... perhaps I should turn these expression into a node with the constant value first and then see if the result of the entire expression is known at compile time? ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
Fri, 29 Apr 2011 16:59:22 +0200, mark florisson wrote: [clip] > Hmm, indeed, it's pretty weird. I'm fine with the pairing also, although > I'm still not sure how common this case is, and if we really want to > support it. Wouldn't good old C promotion work for this? e.g. if the > type is either float or double, just declare your variable double? In practice mostly yes I guess; IIRC single-precision arithmetic is not any faster than double precision nowadays anyway. However, (playing the devil's advocate :) (i) Theoretically, someone might also want to write code that works both for "double" and for "long double complex". You probably wouldn't want to use "long double" for the double-precision specialization. (ii) One reason to use floats could be the 2x size advantage. In principle someone might want to deal with huge float and float complex arrays. (Sounds like a somewhat rare use case though.) (iii) If you want to wrap a library that already provides complex float functions in both precisions, having a specialized real type could come handy sometimes. But in practice, I guess the "fused_type(double, double complex)" would be the most common use case. Maybe it's best to wait until enough concrete examples accumulate before implementing anything more --- I guess e.g. the pairing feature wouldn't be too difficult to add if it turns out something like that is really needed. Pauli ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] What *is* a fused type?
I seem to have missed the beginning of the discussion about this fused type business. Is there a document somewhere describing what a "fused type" is and what it's meant to be used for? -- Greg ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] What *is* a fused type?
On Fri, Apr 29, 2011 at 4:59 PM, Greg Ewing wrote: > I seem to have missed the beginning of the discussion about this > fused type business. Is there a document somewhere describing > what a "fused type" is and what it's meant to be used for? http://wiki.cython.org/enhancements/fusedtypes In short, it's a very basic type parameterization. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] speed.pypy.org
2011/4/27 Stefan Behnel : > Robert Bradshaw, 27.04.2011 19:08: >> >> On Wed, Apr 27, 2011 at 12:45 AM, Stefan Behnel wrote: >>> >>> Actually, if we want a proper history, I'd suggest a separate codespeed >>> installation somewhere. >> >> Makes sense. How many CPU-hours does it take? > > Including the Cython build, it's more like 25 minutes currently, given that > we only support a smaller part of the benchmark suite. It will obviously > take longer when we start supporting the larger benchmarks, such as Django > templates or Twisted. > > >> If it's not to >> intensive, we could probably run it, say, daily as a normal-priority >> job. > > We could certainly do that for now, and check again when we see that it > starts running substantially longer. > Nice thing! Can I run this tests at home? -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On Fri, Apr 29, 2011 at 8:04 AM, mark florisson wrote: > On 29 April 2011 06:32, Robert Bradshaw wrote: >> On Thu, Apr 28, 2011 at 7:09 PM, Stefan Behnel wrote: >>> mark florisson, 28.04.2011 23:29: On 28 April 2011 22:31, mark florisson wrote: > > On 28 April 2011 22:12, Robert Bradshaw wrote: >> >> On Thu, Apr 28, 2011 at 12:48 PM, mark florisson wrote: >> >>> So I fixed all that, but I'm currently wondering about the proposed >>> cython.typeof(). I believe it currently returns a string with the type >>> name, and not the type itself. >> >> Yes. This is just because there's not really anything better to return >> at this point. We should "fix" this at some point in the future. >> >>> So I think it would be inconsistent to >>> suddenly start allowing comparison with 'is' and 'isinstance' and >>> such. >> >> I'm open to other suggestions, but would like an expression that >> resolves at compile time to true/false (and we need to do branch >> pruning on it). Note that type() is not good enough, because it has >> different semantics, i.e. >> >> cdef object o = [] >> typeof(o), type(o) >> >> so lets not touch that one. > > Right, so for fused types I don't mind string comparison with > cython.typeof(), but retrieval of the actual type for casts and > declaration remains open. I'd be fine with something like > cython.gettype(). It seems that this isn't optimized yet, but it looks to me like it wouldn't be very hard to do so. At least == and != could be resolved at compile time if the other operand is a string literal. >>> >>> Well, the obvious place where this would happen for free would be constant >>> folding. But that runs *way* before type analysis, so all it sees is the >>> typeof() call, not the string. >> >> Actually, to support code like >> >> ctypedef cython.fused_type(object, double) my_fused_type >> >> cdef foo(my_fused_type x): >> if my_fused_type is object: >> print x.attr >> else: >> cdef my_fused_type *ptr = &x >> >> we need to resolve this branch and prune "dead code" before type >> analysis, so I'm thinking it may need to be a special phase, perhaps >> part of the fused-type-specialization phase. Here's another idea: >> >> cdef foo(numeric x): >> if numeric in floating: >> ... >> elif numeric is long: >> ... >> else: >> ... >> print numeric is double # after the specialization pass, this >> would be a literal True/False node. >> >> Again, this would all be resolved before type analysis. In terms of >> syntactically supporting pointers, we could either support "fused_type >> is typeof(void*)" or require typedefs for types not representable by >> an ExprNode. (This would make declaring fused types syntactically >> simpler as well...) I prefer the latter. >> >> - Robert >> > > I made both things work, you can use both typeof() on expressions and > you can compare types using 'is' and 'in'. Cool. > However, with typeof() it > doesn't remove branches, so if you're doing incompatible stuff you > need to do the type matching. It could be supported, but currently it > isn't because TransformBuiltinMethods runs after > AnalyseDeclarationsTransform (for which the reason is not immediately > apparent). For one thing, it's the declaration analysis that allows you to figure out what name nodes are built-ins. > With the type matching it matches on exactly 'if src_type is > dst_type:' so you can't use 'and' and such... perhaps I should turn > these expression into a node with the constant value first and then > see if the result of the entire expression is known at compile time? Yes, that's exactly what I was thinking you should do. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Fused Types
On Fri, Apr 29, 2011 at 3:53 AM, mark florisson wrote: > On 29 April 2011 12:28, Pauli Virtanen wrote: >> Fri, 29 Apr 2011 11:30:19 +0200, mark florisson wrote: >>> On 29 April 2011 11:03, Pauli Virtanen wrote: >> [clip] Are you planning to special-case the "real_t complex" syntax? Shooting from the sidelines, one more generic solution might be, e.g., >>> >>> I'm sorry, I'm not sure what syntax you are referring to. Are you >>> talking about actual complex numbers? >> >> This: >> >> On 28 April 2011 23:30, Robert Bradshaw >> wrote: >>> OK, I take back what I said, I was looking at the RHS, not the LHS. If >>> one needs to specialize in this manner, explicitly creating two >>> branches should typically be enough. The same for casting. The one >>> exception (perhaps) is "my_fused_type complex." Otherwise it's >>> starting to feel too much like C++ template magic and complexity for >>> little additional benefit. >> >> That is, declaring a complex type matching a real one. > > Ah, I see what you mean now. > ctypedef cython.fused_type(A, B) struct_t ctypedef cython.fused_type(float, double, paired=struct_t) real_t ctypedef cython.fused_type(int_t, string_t, paired=struct_t) var_t and just restrict the specialization to cases that make sense. >>> >>> The paired means you're declaring types of attributes? >> >> No, just that real_t is specialized to float whenever struct_t is specialized >> to A and to double when B. Or a more realistic example, >> >> ctypedef cython.fused_type(float, double) real_t >> ctypedef cython.fused_type(float complex, double complex) complex_t >> >> cdef real_plus_one(complex_t a): >> real_t b = a.real >> return b + 1 >> >> which I suppose would not be a very unusual thing in numerical codes. >> This would also allow writing the case you had earlier as >> >> cdef cython.fused_type(string_t, int, paired=struct_t) attr_t >> >> cdef func(struct_t mystruct, int i): >> cdef attr_t var >> >> if typeof(mystruct) is typeof(int): >> var = mystruct.attrib + i >> ... >> else: >> var = mystruct.attrib + i >> ... >> >> Things would need to be done explicitly instead of implicitly, though, >> but it would remove the need for any special handling of >> the "complex" keyword. If we're going to introduce pairing, another option would be ctypedef fused_type((double complex, double), (float complex, float)) (complex_t, real_t) though I'm not sure I like that either. We're not trying to create the all-powerful templating system here, and anything that can be done with pairing can be done (though less elegantly) via branching on the types, or, as Pauli mentions, using a wider type is often (but not always) a viable option. We talked of supporting fused types in classes, one could do the same for structs, thus cdef fused_type(int, double) int_or_double cdef my_struct: int_or_double attr cdef func1(int_or_double x): cdef my_struct my_struct.attr = x and cdef func2(my_struct s) cdef int_or_double x = s.attr could work. Maybe we'd require an explicit "my_struct[int_or_double]". This would solve a large number of the remaining cases, and complex is like a struct + native arithmetic/promotion. > I see, so it's like a mapping. So, I didn't realize that you can't do this: > > def func(arbitrary_type complex x): > ... > > But if we just allow that for fused types, then couldn't we simply do > > ctypedef cython.fused_type(float, double) real_t > > cdef real_plus_one(real_t complex a): > real_t b = a.real > return b + 1 > > ? Then you don't need to pair anything. That's what I was thinking. It's a bit special, but on the other hand a very natural pairing that one would want to be able to do with a very natural syntax (no one reading that code would wonder what it means). > Perhaps we could introduce > real_t as a type, just like numeric and floating. So I guess > special-casing complex sounds fine with me. Perhaps real_t should be > builtin (not as an attribute of the Cython module), so the parser can > just recognize it immediately? I'd rather not add new keywords to the language, and I don't see what advantage letting real_t be known to the parser woud be, we can ship a pxd file with the common fused types. - Robert ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
[Cython] Fwd: Proposal for a common benchmark suite
[Seems like this was meant to go to this list but didn't make it there. Forwarding from python-dev] Original-Message Subject: Proposal for a common benchmark suite Date: Thu, 28 Apr 2011 20:55:19 +0200 From: DasIch Hello, As announced in my GSoC proposal I'd like to announce which benchmarks I'll use for the benchmark suite I will work on this summer. As of now there are two benchmark suites (that I know of) which receive some sort of attention, those are the ones developed as part of the PyPy project[1] which is used for http://speed.pypy.org and the one initially developed for Unladen Swallow which has been continued by CPython[2]. The PyPy benchmarks contain a lot of interesting benchmarks some explicitly developed for that suite, the CPython benchmarks have an extensive set of microbenchmarks in the pybench package as well as the previously mentioned modifications made to the Unladen Swallow benchmarks. I'd like to "simply" merge both suites so that no changes are lost. However I'd like to leave out the waf benchmark which is part of the PyPy suite, the removal was proposed on pypy-dev for obvious deficits[3]. It will be easier to add a better benchmark later than replacing it at a later point. Unless there is a major issue with this plan I'd like to go forward with this. .. [1]: https://bitbucket.org/pypy/benchmarks .. [2]: http://hg.python.org/benchmarks .. [3]: http://mailrepository.com/pypy-dev.codespeak.net/msg/3627509/ ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
