Re: [Cython] memoryview slices can't be None?
On 02/03/2012 12:09 AM, mark florisson wrote: On 2 February 2012 21:38, Dag Sverre Seljebotn wrote: On 02/02/2012 10:16 PM, mark florisson wrote: On 2 February 2012 12:19, Dag Sverre Seljebotn wrote: I just realized that cdef int[:] a = None raises an exception; even though I'd argue that 'a' is of the "reference" kind of type where Cython usually allow None (i.e., "cdef MyClass b = None" is allowed even if type(None) is NoneType). Is this a bug or not, and is it possible to do something about it? Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel Yeah I disabled that quite early. It was supposed to be working but gave a lot of trouble in cases (segfaults, mainly). At the time I was trying to get rid of all the segfaults and get the basic functionality working, so I disabled it. Personally, I have never liked how things Well, you can segfault quite easily with cdef MyClass a = None print a.field so it doesn't make sense to slices different from cdef classes IMO. can be None unchecked. I personally prefer to write cdef foo(obj=None): cdef int[:] a if obj is None: obj = ... a = obj Often you forget to write 'not None' when declaring the parameter (and apparently that it only allowed for 'def' functions). As such, I never bothered to re-enable it. However, it does support control flow with uninitialized slices, and will raise an error if it is uninitialized. Do we want this behaviour (e.g. for consistency)? When in doubt, go for consistency. So +1 for that reason. I do believe that setting stuff to None is rather vital in Python. What I typically do is more like this: def f(double[:] input, double[:] out=None): if out is None: out = np.empty_like(input) ... Having to use another variable name is a bit of a pain. (Come on -- do you use "a" in real code? What do you actually call "the other obj"? I sometimes end up with "out_" and so on, but it creates smelly code quite quickly.) No, it was just a contrived example. It's easy to segfault with cdef classes anyway, so decent nonechecking should be implemented at some point, and then memoryviews would use the same mechanisms. Java has decent null-checking... The problem with none checking is that it has to occur at every point. Well, using control flow analysis etc. it doesn't really. E.g., for i in range(a.shape[0]): print i a[i] *= 3 can be unrolled and none-checks inserted as print 0 if a is None: raise a[0] *= 3 for i in range(1, a.shape[0]): print i a[i] *= 3 # no need for none-check It's very similar to what you'd want to do to pull boundschecking out of the loop... With initialized slices the control flow knows when the slices are initialized, or when they might not be (and it can raise a compile-time or runtime error, instead of a segfault if you're lucky). I'm fine with implementing the behaviour, I just always left it at the bottom of my todo list. Wasn't saying you should do it, just checking. I'm still not sure about this. I think what I'd really like is a) Stop cdef classes from being None as well b) Sort-of deprecate cdef in favor of cast/assertion type statements that help the type inferences: def f(arr): if arr is None: arr = ... arr = int[:](arr) # equivalent to "cdef int[:] arr = arr", but # acts as statement, with a specific point # for the none-check ... or even: def f(arr): if arr is None: return 'foo' else: arr = int[:](arr) # takes effect *here*, does none-check ... # arr still typed as int[:] here If we can make this work well enough with control flow analysis I'd never cdef declare local vars again :-) Dag ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] memoryview slices can't be None?
On 02/03/2012 06:53 PM, Dag Sverre Seljebotn wrote: On 02/03/2012 12:09 AM, mark florisson wrote: On 2 February 2012 21:38, Dag Sverre Seljebotn wrote: On 02/02/2012 10:16 PM, mark florisson wrote: On 2 February 2012 12:19, Dag Sverre Seljebotn wrote: I just realized that cdef int[:] a = None raises an exception; even though I'd argue that 'a' is of the "reference" kind of type where Cython usually allow None (i.e., "cdef MyClass b = None" is allowed even if type(None) is NoneType). Is this a bug or not, and is it possible to do something about it? Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel Yeah I disabled that quite early. It was supposed to be working but gave a lot of trouble in cases (segfaults, mainly). At the time I was trying to get rid of all the segfaults and get the basic functionality working, so I disabled it. Personally, I have never liked how things Well, you can segfault quite easily with cdef MyClass a = None print a.field so it doesn't make sense to slices different from cdef classes IMO. can be None unchecked. I personally prefer to write cdef foo(obj=None): cdef int[:] a if obj is None: obj = ... a = obj Often you forget to write 'not None' when declaring the parameter (and apparently that it only allowed for 'def' functions). As such, I never bothered to re-enable it. However, it does support control flow with uninitialized slices, and will raise an error if it is uninitialized. Do we want this behaviour (e.g. for consistency)? When in doubt, go for consistency. So +1 for that reason. I do believe that setting stuff to None is rather vital in Python. What I typically do is more like this: def f(double[:] input, double[:] out=None): if out is None: out = np.empty_like(input) ... Having to use another variable name is a bit of a pain. (Come on -- do you use "a" in real code? What do you actually call "the other obj"? I sometimes end up with "out_" and so on, but it creates smelly code quite quickly.) No, it was just a contrived example. It's easy to segfault with cdef classes anyway, so decent nonechecking should be implemented at some point, and then memoryviews would use the same mechanisms. Java has decent null-checking... The problem with none checking is that it has to occur at every point. Well, using control flow analysis etc. it doesn't really. E.g., for i in range(a.shape[0]): print i a[i] *= 3 can be unrolled and none-checks inserted as print 0 if a is None: raise a[0] *= 3 for i in range(1, a.shape[0]): print i a[i] *= 3 # no need for none-check It's very similar to what you'd want to do to pull boundschecking out of the loop... With initialized slices the control flow knows when the slices are initialized, or when they might not be (and it can raise a compile-time or runtime error, instead of a segfault if you're lucky). I'm fine with implementing the behaviour, I just always left it at the bottom of my todo list. Wasn't saying you should do it, just checking. I'm still not sure about this. I think what I'd really like is a) Stop cdef classes from being None as well I'm sorry, this was out of place and hardly constructive. I beg you all to ignore the above sentence. The below proposal still works for slices only though, which we can still define the semantics for. I rather like it. Is type-inference + control flow analysis powerful enough to pull this off with some minor changes? b) Sort-of deprecate cdef in favor of cast/assertion type statements that help the type inferences: def f(arr): if arr is None: arr = ... arr = int[:](arr) # equivalent to "cdef int[:] arr = arr", but # acts as statement, with a specific point # for the none-check ... or even: def f(arr): if arr is None: return 'foo' else: arr = int[:](arr) # takes effect *here*, does none-check ... # arr still typed as int[:] here If we can make this work well enough with control flow analysis I'd never cdef declare local vars again :-) Dag ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] memoryview slices can't be None?
On 3 February 2012 17:53, Dag Sverre Seljebotn wrote: > On 02/03/2012 12:09 AM, mark florisson wrote: >> >> On 2 February 2012 21:38, Dag Sverre Seljebotn >> wrote: >>> >>> On 02/02/2012 10:16 PM, mark florisson wrote: On 2 February 2012 12:19, Dag Sverre Seljebotn wrote: > > > I just realized that > > cdef int[:] a = None > > raises an exception; even though I'd argue that 'a' is of the > "reference" > kind of type where Cython usually allow None (i.e., "cdef MyClass b = > None" > is allowed even if type(None) is NoneType). Is this a bug or not, and > is > it > possible to do something about it? > > Dag Sverre > ___ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel Yeah I disabled that quite early. It was supposed to be working but gave a lot of trouble in cases (segfaults, mainly). At the time I was trying to get rid of all the segfaults and get the basic functionality working, so I disabled it. Personally, I have never liked how things >>> >>> >>> >>> Well, you can segfault quite easily with >>> >>> cdef MyClass a = None >>> print a.field >>> >>> so it doesn't make sense to slices different from cdef classes IMO. >>> >>> can be None unchecked. I personally prefer to write cdef foo(obj=None): cdef int[:] a if obj is None: obj = ... a = obj Often you forget to write 'not None' when declaring the parameter (and apparently that it only allowed for 'def' functions). As such, I never bothered to re-enable it. However, it does support control flow with uninitialized slices, and will raise an error if it is uninitialized. Do we want this behaviour (e.g. for consistency)? >>> >>> >>> >>> When in doubt, go for consistency. So +1 for that reason. I do believe >>> that >>> setting stuff to None is rather vital in Python. >>> >>> What I typically do is more like this: >>> >>> def f(double[:] input, double[:] out=None): >>> if out is None: >>> out = np.empty_like(input) >>> ... >>> >>> Having to use another variable name is a bit of a pain. (Come on -- do >>> you >>> use "a" in real code? What do you actually call "the other obj"? I >>> sometimes >>> end up with "out_" and so on, but it creates smelly code quite quickly.) >> >> >> No, it was just a contrived example. >> >>> It's easy to segfault with cdef classes anyway, so decent nonechecking >>> should be implemented at some point, and then memoryviews would use the >>> same >>> mechanisms. Java has decent null-checking... >>> >> >> The problem with none checking is that it has to occur at every point. > > > Well, using control flow analysis etc. it doesn't really. E.g., > > for i in range(a.shape[0]): > print i > a[i] *= 3 > > can be unrolled and none-checks inserted as > > print 0 > if a is None: raise > a[0] *= 3 > for i in range(1, a.shape[0]): > print i > a[i] *= 3 # no need for none-check > > It's very similar to what you'd want to do to pull boundschecking out of the > loop... > Oh, definitely. Both optimizations may not always be possible to do, though. The optimization (for boundschecking) is easier for prange() than range(), as you can immediately raise an exception as the exceptional condition may be issued at any iteration. What do you do with bounds checking when some accesses are in-bound, and some are out-of-bound? Do you immediately raise the exception? Are we fine with aborting (like Fortran compilers do when you ask them for bounds checking)? And how do you detect that the code doesn't already raise an exception or break out of the loop itself to prevent the out-of-bound access? (Unless no exceptions are propagating and no break/return is used, but exceptions are so very common). >> With initialized slices the control flow knows when the slices are >> initialized, or when they might not be (and it can raise a >> compile-time or runtime error, instead of a segfault if you're lucky). >> I'm fine with implementing the behaviour, I just always left it at the >> bottom of my todo list. > > > Wasn't saying you should do it, just checking. > > I'm still not sure about this. I think what I'd really like is > > a) Stop cdef classes from being None as well > > b) Sort-of deprecate cdef in favor of cast/assertion type statements that > help the type inferences: > > def f(arr): > if arr is None: > arr = ... > arr = int[:](arr) # equivalent to "cdef int[:] arr = arr", but > # acts as statement, with a specific point > # for the none-check > ... > > or even: > > def f(arr): > if arr is None: > return 'foo' > else: > arr = int[:](arr) # takes effect *here*, does none-check > ... > # arr still typed as int[:] here
Re: [Cython] memoryview slices can't be None?
On 3 February 2012 18:06, mark florisson wrote: > On 3 February 2012 17:53, Dag Sverre Seljebotn > wrote: >> On 02/03/2012 12:09 AM, mark florisson wrote: >>> >>> On 2 February 2012 21:38, Dag Sverre Seljebotn >>> wrote: On 02/02/2012 10:16 PM, mark florisson wrote: > > > On 2 February 2012 12:19, Dag Sverre Seljebotn > wrote: >> >> >> I just realized that >> >> cdef int[:] a = None >> >> raises an exception; even though I'd argue that 'a' is of the >> "reference" >> kind of type where Cython usually allow None (i.e., "cdef MyClass b = >> None" >> is allowed even if type(None) is NoneType). Is this a bug or not, and >> is >> it >> possible to do something about it? >> >> Dag Sverre >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > > > > Yeah I disabled that quite early. It was supposed to be working but > gave a lot of trouble in cases (segfaults, mainly). At the time I was > trying to get rid of all the segfaults and get the basic functionality > working, so I disabled it. Personally, I have never liked how things Well, you can segfault quite easily with cdef MyClass a = None print a.field so it doesn't make sense to slices different from cdef classes IMO. > can be None unchecked. I personally prefer to write > > cdef foo(obj=None): > cdef int[:] a > if obj is None: > obj = ... > a = obj > > Often you forget to write 'not None' when declaring the parameter (and > apparently that it only allowed for 'def' functions). > > As such, I never bothered to re-enable it. However, it does support > control flow with uninitialized slices, and will raise an error if it > is uninitialized. Do we want this behaviour (e.g. for consistency)? When in doubt, go for consistency. So +1 for that reason. I do believe that setting stuff to None is rather vital in Python. What I typically do is more like this: def f(double[:] input, double[:] out=None): if out is None: out = np.empty_like(input) ... Having to use another variable name is a bit of a pain. (Come on -- do you use "a" in real code? What do you actually call "the other obj"? I sometimes end up with "out_" and so on, but it creates smelly code quite quickly.) >>> >>> >>> No, it was just a contrived example. >>> It's easy to segfault with cdef classes anyway, so decent nonechecking should be implemented at some point, and then memoryviews would use the same mechanisms. Java has decent null-checking... >>> >>> The problem with none checking is that it has to occur at every point. >> >> >> Well, using control flow analysis etc. it doesn't really. E.g., >> >> for i in range(a.shape[0]): >> print i >> a[i] *= 3 >> >> can be unrolled and none-checks inserted as >> >> print 0 >> if a is None: raise >> a[0] *= 3 >> for i in range(1, a.shape[0]): >> print i >> a[i] *= 3 # no need for none-check >> >> It's very similar to what you'd want to do to pull boundschecking out of the >> loop... >> > > Oh, definitely. Both optimizations may not always be possible to do, > though. The optimization (for boundschecking) is easier for prange() > than range(), as you can immediately raise an exception as the > exceptional condition may be issued at any iteration. What do you do > with bounds checking when some accesses are in-bound, and some are > out-of-bound? Do you immediately raise the exception? Are we fine with > aborting (like Fortran compilers do when you ask them for bounds > checking)? And how do you detect that the code doesn't already raise > an exception or break out of the loop itself to prevent the > out-of-bound access? (Unless no exceptions are propagating and no > break/return is used, but exceptions are so very common). > >>> With initialized slices the control flow knows when the slices are >>> initialized, or when they might not be (and it can raise a >>> compile-time or runtime error, instead of a segfault if you're lucky). >>> I'm fine with implementing the behaviour, I just always left it at the >>> bottom of my todo list. >> >> >> Wasn't saying you should do it, just checking. >> >> I'm still not sure about this. I think what I'd really like is >> >> a) Stop cdef classes from being None as well >> >> b) Sort-of deprecate cdef in favor of cast/assertion type statements that >> help the type inferences: >> >> def f(arr): >> if arr is None: >> arr = ... >> arr = int[:](arr) # equivalent to "cdef int[:] arr = arr", but >> # acts as statement, with a specific point >> # for the none-che
Re: [Cython] memoryview slices can't be None?
On 3 February 2012 18:07, mark florisson wrote: > On 3 February 2012 18:06, mark florisson wrote: >> On 3 February 2012 17:53, Dag Sverre Seljebotn >> wrote: >>> On 02/03/2012 12:09 AM, mark florisson wrote: On 2 February 2012 21:38, Dag Sverre Seljebotn wrote: > > On 02/02/2012 10:16 PM, mark florisson wrote: >> >> >> On 2 February 2012 12:19, Dag Sverre Seljebotn >> wrote: >>> >>> >>> I just realized that >>> >>> cdef int[:] a = None >>> >>> raises an exception; even though I'd argue that 'a' is of the >>> "reference" >>> kind of type where Cython usually allow None (i.e., "cdef MyClass b = >>> None" >>> is allowed even if type(None) is NoneType). Is this a bug or not, and >>> is >>> it >>> possible to do something about it? >>> >>> Dag Sverre >>> ___ >>> cython-devel mailing list >>> cython-devel@python.org >>> http://mail.python.org/mailman/listinfo/cython-devel >> >> >> >> Yeah I disabled that quite early. It was supposed to be working but >> gave a lot of trouble in cases (segfaults, mainly). At the time I was >> trying to get rid of all the segfaults and get the basic functionality >> working, so I disabled it. Personally, I have never liked how things > > > > Well, you can segfault quite easily with > > cdef MyClass a = None > print a.field > > so it doesn't make sense to slices different from cdef classes IMO. > > >> can be None unchecked. I personally prefer to write >> >> cdef foo(obj=None): >> cdef int[:] a >> if obj is None: >> obj = ... >> a = obj >> >> Often you forget to write 'not None' when declaring the parameter (and >> apparently that it only allowed for 'def' functions). >> >> As such, I never bothered to re-enable it. However, it does support >> control flow with uninitialized slices, and will raise an error if it >> is uninitialized. Do we want this behaviour (e.g. for consistency)? > > > > When in doubt, go for consistency. So +1 for that reason. I do believe > that > setting stuff to None is rather vital in Python. > > What I typically do is more like this: > > def f(double[:] input, double[:] out=None): > if out is None: > out = np.empty_like(input) > ... > > Having to use another variable name is a bit of a pain. (Come on -- do > you > use "a" in real code? What do you actually call "the other obj"? I > sometimes > end up with "out_" and so on, but it creates smelly code quite quickly.) No, it was just a contrived example. > It's easy to segfault with cdef classes anyway, so decent nonechecking > should be implemented at some point, and then memoryviews would use the > same > mechanisms. Java has decent null-checking... > The problem with none checking is that it has to occur at every point. >>> >>> >>> Well, using control flow analysis etc. it doesn't really. E.g., >>> >>> for i in range(a.shape[0]): >>> print i >>> a[i] *= 3 >>> >>> can be unrolled and none-checks inserted as >>> >>> print 0 >>> if a is None: raise >>> a[0] *= 3 >>> for i in range(1, a.shape[0]): >>> print i >>> a[i] *= 3 # no need for none-check >>> >>> It's very similar to what you'd want to do to pull boundschecking out of the >>> loop... >>> >> >> Oh, definitely. Both optimizations may not always be possible to do, >> though. The optimization (for boundschecking) is easier for prange() >> than range(), as you can immediately raise an exception as the >> exceptional condition may be issued at any iteration. What do you do >> with bounds checking when some accesses are in-bound, and some are >> out-of-bound? Do you immediately raise the exception? Are we fine with >> aborting (like Fortran compilers do when you ask them for bounds >> checking)? And how do you detect that the code doesn't already raise >> an exception or break out of the loop itself to prevent the >> out-of-bound access? (Unless no exceptions are propagating and no >> break/return is used, but exceptions are so very common). >> With initialized slices the control flow knows when the slices are initialized, or when they might not be (and it can raise a compile-time or runtime error, instead of a segfault if you're lucky). I'm fine with implementing the behaviour, I just always left it at the bottom of my todo list. >>> >>> >>> Wasn't saying you should do it, just checking. >>> >>> I'm still not sure about this. I think what I'd really like is >>> >>> a) Stop cdef classes from being None as well >>> >>> b) Sort-of deprecate cdef in favor of cast/assertion type statements that >>> help the type inferences: >>> >>> def f(arr): >>> if arr is None: >>
Re: [Cython] memoryview slices can't be None?
On 02/03/2012 07:07 PM, mark florisson wrote: On 3 February 2012 18:06, mark florisson wrote: On 3 February 2012 17:53, Dag Sverre Seljebotn wrote: On 02/03/2012 12:09 AM, mark florisson wrote: On 2 February 2012 21:38, Dag Sverre Seljebotn wrote: On 02/02/2012 10:16 PM, mark florisson wrote: On 2 February 2012 12:19, Dag Sverre Seljebotn wrote: I just realized that cdef int[:] a = None raises an exception; even though I'd argue that 'a' is of the "reference" kind of type where Cython usually allow None (i.e., "cdef MyClass b = None" is allowed even if type(None) is NoneType). Is this a bug or not, and is it possible to do something about it? Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel Yeah I disabled that quite early. It was supposed to be working but gave a lot of trouble in cases (segfaults, mainly). At the time I was trying to get rid of all the segfaults and get the basic functionality working, so I disabled it. Personally, I have never liked how things Well, you can segfault quite easily with cdef MyClass a = None print a.field so it doesn't make sense to slices different from cdef classes IMO. can be None unchecked. I personally prefer to write cdef foo(obj=None): cdef int[:] a if obj is None: obj = ... a = obj Often you forget to write 'not None' when declaring the parameter (and apparently that it only allowed for 'def' functions). As such, I never bothered to re-enable it. However, it does support control flow with uninitialized slices, and will raise an error if it is uninitialized. Do we want this behaviour (e.g. for consistency)? When in doubt, go for consistency. So +1 for that reason. I do believe that setting stuff to None is rather vital in Python. What I typically do is more like this: def f(double[:] input, double[:] out=None): if out is None: out = np.empty_like(input) ... Having to use another variable name is a bit of a pain. (Come on -- do you use "a" in real code? What do you actually call "the other obj"? I sometimes end up with "out_" and so on, but it creates smelly code quite quickly.) No, it was just a contrived example. It's easy to segfault with cdef classes anyway, so decent nonechecking should be implemented at some point, and then memoryviews would use the same mechanisms. Java has decent null-checking... The problem with none checking is that it has to occur at every point. Well, using control flow analysis etc. it doesn't really. E.g., for i in range(a.shape[0]): print i a[i] *= 3 can be unrolled and none-checks inserted as print 0 if a is None: raise a[0] *= 3 for i in range(1, a.shape[0]): print i a[i] *= 3 # no need for none-check It's very similar to what you'd want to do to pull boundschecking out of the loop... Oh, definitely. Both optimizations may not always be possible to do, though. The optimization (for boundschecking) is easier for prange() than range(), as you can immediately raise an exception as the exceptional condition may be issued at any iteration. What do you do with bounds checking when some accesses are in-bound, and some are out-of-bound? Do you immediately raise the exception? Are we fine with aborting (like Fortran compilers do when you ask them for bounds checking)? And how do you detect that the code doesn't already raise an exception or break out of the loop itself to prevent the out-of-bound access? (Unless no exceptions are propagating and no break/return is used, but exceptions are so very common). With initialized slices the control flow knows when the slices are initialized, or when they might not be (and it can raise a compile-time or runtime error, instead of a segfault if you're lucky). I'm fine with implementing the behaviour, I just always left it at the bottom of my todo list. Wasn't saying you should do it, just checking. I'm still not sure about this. I think what I'd really like is a) Stop cdef classes from being None as well b) Sort-of deprecate cdef in favor of cast/assertion type statements that help the type inferences: def f(arr): if arr is None: arr = ... arr = int[:](arr) # equivalent to "cdef int[:] arr = arr", but # acts as statement, with a specific point # for the none-check ... or even: def f(arr): if arr is None: return 'foo' else: arr = int[:](arr) # takes effect *here*, does none-check ... # arr still typed as int[:] here If we can make this work well enough with control flow analysis I'd never cdef declare local vars again :-) Hm, what about the following? def f(arr): if arr is None: return 'foo' cdef int[:] arr # arr may not be None The above would work in general, until the declaration is lexically encountered, the object is typed
Re: [Cython] memoryview slices can't be None?
On 3 February 2012 18:15, Dag Sverre Seljebotn wrote: > On 02/03/2012 07:07 PM, mark florisson wrote: >> >> On 3 February 2012 18:06, mark florisson >> wrote: >>> >>> On 3 February 2012 17:53, Dag Sverre Seljebotn >>> wrote: On 02/03/2012 12:09 AM, mark florisson wrote: > > > On 2 February 2012 21:38, Dag Sverre Seljebotn > wrote: >> >> >> On 02/02/2012 10:16 PM, mark florisson wrote: >>> >>> >>> >>> On 2 February 2012 12:19, Dag Sverre Seljebotn >>> wrote: I just realized that cdef int[:] a = None raises an exception; even though I'd argue that 'a' is of the "reference" kind of type where Cython usually allow None (i.e., "cdef MyClass b = None" is allowed even if type(None) is NoneType). Is this a bug or not, and is it possible to do something about it? Dag Sverre ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel >>> >>> >>> >>> >>> Yeah I disabled that quite early. It was supposed to be working but >>> gave a lot of trouble in cases (segfaults, mainly). At the time I was >>> trying to get rid of all the segfaults and get the basic >>> functionality >>> working, so I disabled it. Personally, I have never liked how things >> >> >> >> >> Well, you can segfault quite easily with >> >> cdef MyClass a = None >> print a.field >> >> so it doesn't make sense to slices different from cdef classes IMO. >> >> >>> can be None unchecked. I personally prefer to write >>> >>> cdef foo(obj=None): >>> cdef int[:] a >>> if obj is None: >>> obj = ... >>> a = obj >>> >>> Often you forget to write 'not None' when declaring the parameter >>> (and >>> apparently that it only allowed for 'def' functions). >>> >>> As such, I never bothered to re-enable it. However, it does support >>> control flow with uninitialized slices, and will raise an error if it >>> is uninitialized. Do we want this behaviour (e.g. for consistency)? >> >> >> >> >> When in doubt, go for consistency. So +1 for that reason. I do believe >> that >> setting stuff to None is rather vital in Python. >> >> What I typically do is more like this: >> >> def f(double[:] input, double[:] out=None): >> if out is None: >> out = np.empty_like(input) >> ... >> >> Having to use another variable name is a bit of a pain. (Come on -- do >> you >> use "a" in real code? What do you actually call "the other obj"? I >> sometimes >> end up with "out_" and so on, but it creates smelly code quite >> quickly.) > > > > No, it was just a contrived example. > >> It's easy to segfault with cdef classes anyway, so decent nonechecking >> should be implemented at some point, and then memoryviews would use >> the >> same >> mechanisms. Java has decent null-checking... >> > > The problem with none checking is that it has to occur at every point. Well, using control flow analysis etc. it doesn't really. E.g., for i in range(a.shape[0]): print i a[i] *= 3 can be unrolled and none-checks inserted as print 0 if a is None: raise a[0] *= 3 for i in range(1, a.shape[0]): print i a[i] *= 3 # no need for none-check It's very similar to what you'd want to do to pull boundschecking out of the loop... >>> >>> Oh, definitely. Both optimizations may not always be possible to do, >>> though. The optimization (for boundschecking) is easier for prange() >>> than range(), as you can immediately raise an exception as the >>> exceptional condition may be issued at any iteration. What do you do >>> with bounds checking when some accesses are in-bound, and some are >>> out-of-bound? Do you immediately raise the exception? Are we fine with >>> aborting (like Fortran compilers do when you ask them for bounds >>> checking)? And how do you detect that the code doesn't already raise >>> an exception or break out of the loop itself to prevent the >>> out-of-bound access? (Unless no exceptions are propagating and no >>> break/return is used, but exceptions are so very common). >>> > With initialized slices the control flow knows when the slices are > initialized, or when they might not be (and it can raise a > compile-time or runtime error, instead of a segfault if you're lucky). > I'm fine with implementing the behaviour, I just always left it at the > bottom of my todo list.
Re: [Cython] Bug in Cython producing incorrect C code
2012/1/26 mark florisson : > On 26 January 2012 19:27, Stefan Behnel wrote: >> mark florisson, 26.01.2012 20:15: >>> On 26 January 2012 18:53, Stefan Behnel wrote: mark florisson, 26.01.2012 16:20: > I think this problem can trivially be solved by creating a ProxyNode > that should never be replaced by any transform, but it's argument may > be replaced. So you wrap self.rhs in a ProxyNode and use that to > create your CloneNodes. I can't see what a ProxyNode would do that a CloneNode shouldn't do anyway. >>> >>> It wouldn't be a replacement, merely an addition (an extra indirection). >> >> What I was trying to say was that a ProxyNode would always be required by a >> CloneNode, but I don't see where a ProxyNode would be needed outside of a >> CloneNode. So it seems rather redundant and I don't know if we need a >> separate node for it. > > Yes it would be needed only for that, but I think the only real > alternative is to not use CloneNode at all, i.e. make the > transformation Dag mentioned, where you create new rhs (NameNode?) > references to the temporary result. > Now it seems to be the only case when we got problem like this. It means that clones may be safely created at very late stage. So transforming CascadeAssignment into SingleAssignments doesn't solve generic problem. I tried to implement conditional inlining the same problem may happen there (ConditionalCallNode owns arguments and replaces SimpleCallNode's args with clones). Splitting analyse_expressions() would help. On the other hand moving this optimization after OptimizeBuiltinCalls() would help too. -- vitja. ___ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] memoryview slices can't be None?
On 3 February 2012 18:06, mark florisson wrote: > On 3 February 2012 17:53, Dag Sverre Seljebotn > wrote: >> On 02/03/2012 12:09 AM, mark florisson wrote: >>> >>> On 2 February 2012 21:38, Dag Sverre Seljebotn >>> wrote: On 02/02/2012 10:16 PM, mark florisson wrote: > > > On 2 February 2012 12:19, Dag Sverre Seljebotn > wrote: >> >> >> I just realized that >> >> cdef int[:] a = None >> >> raises an exception; even though I'd argue that 'a' is of the >> "reference" >> kind of type where Cython usually allow None (i.e., "cdef MyClass b = >> None" >> is allowed even if type(None) is NoneType). Is this a bug or not, and >> is >> it >> possible to do something about it? >> >> Dag Sverre >> ___ >> cython-devel mailing list >> cython-devel@python.org >> http://mail.python.org/mailman/listinfo/cython-devel > > > > Yeah I disabled that quite early. It was supposed to be working but > gave a lot of trouble in cases (segfaults, mainly). At the time I was > trying to get rid of all the segfaults and get the basic functionality > working, so I disabled it. Personally, I have never liked how things Well, you can segfault quite easily with cdef MyClass a = None print a.field so it doesn't make sense to slices different from cdef classes IMO. > can be None unchecked. I personally prefer to write > > cdef foo(obj=None): > cdef int[:] a > if obj is None: > obj = ... > a = obj > > Often you forget to write 'not None' when declaring the parameter (and > apparently that it only allowed for 'def' functions). > > As such, I never bothered to re-enable it. However, it does support > control flow with uninitialized slices, and will raise an error if it > is uninitialized. Do we want this behaviour (e.g. for consistency)? When in doubt, go for consistency. So +1 for that reason. I do believe that setting stuff to None is rather vital in Python. What I typically do is more like this: def f(double[:] input, double[:] out=None): if out is None: out = np.empty_like(input) ... Having to use another variable name is a bit of a pain. (Come on -- do you use "a" in real code? What do you actually call "the other obj"? I sometimes end up with "out_" and so on, but it creates smelly code quite quickly.) >>> >>> >>> No, it was just a contrived example. >>> It's easy to segfault with cdef classes anyway, so decent nonechecking should be implemented at some point, and then memoryviews would use the same mechanisms. Java has decent null-checking... >>> >>> The problem with none checking is that it has to occur at every point. >> >> >> Well, using control flow analysis etc. it doesn't really. E.g., >> >> for i in range(a.shape[0]): >> print i >> a[i] *= 3 >> >> can be unrolled and none-checks inserted as >> >> print 0 >> if a is None: raise >> a[0] *= 3 >> for i in range(1, a.shape[0]): >> print i >> a[i] *= 3 # no need for none-check >> >> It's very similar to what you'd want to do to pull boundschecking out of the >> loop... >> > > Oh, definitely. Both optimizations may not always be possible to do, > though. The optimization (for boundschecking) is easier for prange() > than range(), as you can immediately raise an exception as the > exceptional condition may be issued at any iteration. What do you do > with bounds checking when some accesses are in-bound, and some are > out-of-bound? Do you immediately raise the exception? Are we fine with > aborting (like Fortran compilers do when you ask them for bounds > checking)? And how do you detect that the code doesn't already raise > an exception or break out of the loop itself to prevent the > out-of-bound access? (Unless no exceptions are propagating and no > break/return is used, but exceptions are so very common). I enabled bound checking in nogil contexts: https://github.com/markflorisson88/cython/commit/73c6b0ea8e7e1c243e87b3966ade834b02664a4f . It's not optimized yet, but at least it doesn't force users to use boundscheck(False), it just hints that it would be faster to disable the bounds checking. When we actually start optimizing these things (e.g. moving it outside loops etc), it might also be useful to consider inlining functions at the Cython level (otherwise optimizations cannot escape the function). >>> With initialized slices the control flow knows when the slices are >>> initialized, or when they might not be (and it can raise a >>> compile-time or runtime error, instead of a segfault if you're lucky). >>> I'm fine with implementing the behaviour, I just always left it at the >>> bottom of my todo list. >> >> >> Wasn't say
