[Numpy-discussion] Scipy 2017 NumPy sprint
Hi All, I will be running the NumPy sprint at Scipy 2017 and I'm trying to put together a suitable list of things to sprint on. In my experience, sprinting on NumPy is hard, enhancements generally need lengthy review and even finding and doing simple bug fixes can take time. What I have in mind at this point, apart from what might be a getting started tutorial, could mostly be classified as janitorial work. 1. Triage issues and close those that no longer apply. This is mind numbing work, but it has been almost three years since the last pass. 2. Move the contents of `numpy/doc` into `doc/source` and make them normal *.rst files. 3. Convert the doctest in `numpy/lib/tests/test_polynomial.py` to regular tests. Might be tricky as it mostly checks print formatting. ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Scipy 2017 NumPy sprint
On Thu, Jun 29, 2017 at 12:07 PM, Charles R Harris < charlesr.har...@gmail.com> wrote: > Hi All, > > I will be running the NumPy sprint at Scipy 2017 and I'm trying to put > together a suitable list of things to sprint on. In my experience, > sprinting on NumPy is hard, enhancements generally need lengthy review and > even finding and doing simple bug fixes can take time. What I have in mind > at this point, apart from what might be a getting started tutorial, could > mostly be classified as janitorial work. > > >1. Triage issues and close those that no longer apply. This is mind >numbing work, but it has been almost three years since the last pass. >2. Move the contents of `numpy/doc` into `doc/source` and make them >normal *.rst files. >3. Convert the doctest in `numpy/lib/tests/test_polynomial.py` to >regular tests. Might be tricky as it mostly checks print formatting. > > > Oops, sent prematurely by accident. Anyway, I'm looking for suggestions and comments as to things to do that would be useful. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Scipy 2017 NumPy sprint
On Thu, Jun 29, 2017, at 11:09, Charles R Harris wrote:
>
> On Thu, Jun 29, 2017 at 12:07 PM, Charles R Harris
> wrote:>> I will be running the NumPy sprint at
> Scipy 2017 and I'm trying to
>> put together a suitable list of things to sprint on. In my
>> experience, sprinting on NumPy is hard, enhancements generally
>> need lengthy review and even finding and doing simple bug fixes
>> can take time. What I have in mind at this point, apart from what
>> might be a getting started tutorial, could mostly be classified as
>> janitorial work.
Here's a random idea: how about building a NumPy gallery? scikit-
{image,learn} has it, and while those projects may have more visual
datasets, I can imagine something along the lines of Nicolas Rougier's
beautiful book:
http://www.labri.fr/perso/nrougier/from-python-to-numpy/
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Scipy 2017 NumPy sprint
On Thu, Jun 29, 2017 at 12:15 PM, Stefan van der Walt
wrote:
> On Thu, Jun 29, 2017, at 11:09, Charles R Harris wrote:
>
>
> On Thu, Jun 29, 2017 at 12:07 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
>
> I will be running the NumPy sprint at Scipy 2017 and I'm trying to put
> together a suitable list of things to sprint on. In my experience,
> sprinting on NumPy is hard, enhancements generally need lengthy review and
> even finding and doing simple bug fixes can take time. What I have in mind
> at this point, apart from what might be a getting started tutorial, could
> mostly be classified as janitorial work.
>
>
> Here's a random idea: how about building a NumPy gallery?
> scikit-{image,learn} has it, and while those projects may have more visual
> datasets, I can imagine something along the lines of Nicolas Rougier's
> beautiful book:
>
> http://www.labri.fr/perso/nrougier/from-python-to-numpy/
>
So that would be added in the numpy /numpy.org
repo?
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Scipy 2017 NumPy sprint
Charles R Harris kirjoitti 29.06.2017 klo 20:45:
> Here's a random idea: how about building a NumPy gallery?
> scikit-{image,learn} has it, and while those projects may have more
> visual datasets, I can imagine something along the lines of Nicolas
> Rougier's beautiful book:
>
> http://www.labri.fr/perso/nrougier/from-python-to-numpy/
> /numpy.org
> repo?
Or https://scipy-cookbook.readthedocs.io/ ?
(maybe minus bitrot and images added :)
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Scipy 2017 NumPy sprint
> Convert the doctest in `numpy/lib/tests/test_polynomial.py` to regular tests. Might be tricky as it mostly checks print formatting. Port scipy's refguide-check and enhance/fix/improve code examples in docstrings? Also somewhat janitorial though. ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] proposed changes to array printing in 1.14
Hello all,
There are various updates to array printing in preparation for numpy
1.14. See https://github.com/numpy/numpy/pull/9139/
Some are quite likely to break other projects' doc-tests which expect a
particular str or repr of arrays, so I'd like to warn the list in case
anyone has opinions.
The current proposed changes, from most to least painful by my
reckoning, are:
1.
For float arrays, an extra space previously used for the sign position
will now be omitted in many cases. Eg, `repr(arange(4.))` will now
return 'array([0., 1., 2., 3.])' instead of 'array([ 0., 1., 2., 3.])'.
2.
The printing of 0d arrays is overhauled. This is a bit finicky to
describe, please see the release note in the PR. As an example of the
effect of this, the `repr(np.array(0.))` now prints as 'array(0.)`
instead of 'array(0.0)'. Also the repr of 0d datetime arrays is now like
"array('2005-04-04', dtype='datetime64[D]')" instead of
"array(datetime.date(2005, 4, 4), dtype='datetime64[D]')".
3.
User-defined dtypes which did not properly implement their `repr` (and
`str`) should do so now. Otherwise it now falls back to
`object.__repr__`, which will return something ugly like
``. (Previously you could depend on
only implementing the `item` method and the repr of that would be
printed. But no longer, because this risks infinite recursions.).
4.
Bool arrays of size 1 with a 'True' value will now omit a space, so that
`repr(array([True]))` is now 'array([True])' instead of
'array([ True])'.
Allan
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] proposed changes to array printing in 1.14
To add to Allan's message: point (2), the printing of 0-d arrays, is
the one that is the most important in the sense that it rectifies a
really strange situation, where the printing cannot be logically
controlled by the same mechanism that controls >=1-d arrays (see PR).
While point 3 can also be considered a bug fix, 1 & 4 are at some
level matters of taste; my own reason for supporting their
implementation now is that the 0-d arrays already forces me (or,
specifically, astropy) to rewrite quite a few doctests, and I'd rather
have everything in one go -- in this respect, it is a pity that this
is separate from the earlier change in printing for structured arrays
(which was also much for the better, but broke a lot of doctests).
-- Marten
On Thu, Jun 29, 2017 at 3:38 PM, Allan Haldane wrote:
> Hello all,
>
> There are various updates to array printing in preparation for numpy
> 1.14. See https://github.com/numpy/numpy/pull/9139/
>
> Some are quite likely to break other projects' doc-tests which expect a
> particular str or repr of arrays, so I'd like to warn the list in case
> anyone has opinions.
>
> The current proposed changes, from most to least painful by my
> reckoning, are:
>
> 1.
> For float arrays, an extra space previously used for the sign position
> will now be omitted in many cases. Eg, `repr(arange(4.))` will now
> return 'array([0., 1., 2., 3.])' instead of 'array([ 0., 1., 2., 3.])'.
>
> 2.
> The printing of 0d arrays is overhauled. This is a bit finicky to
> describe, please see the release note in the PR. As an example of the
> effect of this, the `repr(np.array(0.))` now prints as 'array(0.)`
> instead of 'array(0.0)'. Also the repr of 0d datetime arrays is now like
> "array('2005-04-04', dtype='datetime64[D]')" instead of
> "array(datetime.date(2005, 4, 4), dtype='datetime64[D]')".
>
> 3.
> User-defined dtypes which did not properly implement their `repr` (and
> `str`) should do so now. Otherwise it now falls back to
> `object.__repr__`, which will return something ugly like
> ``. (Previously you could depend on
> only implementing the `item` method and the repr of that would be
> printed. But no longer, because this risks infinite recursions.).
>
> 4.
> Bool arrays of size 1 with a 'True' value will now omit a space, so that
> `repr(array([True]))` is now 'array([True])' instead of
> 'array([ True])'.
>
> Allan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Array blitting (pasting one array into another)
Hello all I often need to copy one array into another array, given an offset. This is how the "blit" function can be understood, i.e. in every graphical lib there is such a function. The common definition is like: blit ( dest, src, offset ): where dest is destination array, src is source array and offset is coordinates in destination where the src should pe blitted. Main feature of such function is that it never gives an error, so if the source does not fit into the destination array, it is simply trimmed. And respectively if there is no intersection area then nothing happens. Hope this is clear. So to make it work with Numpy arrays one need to calculate the slices before copying the data. I cannot find any Numpy or Python method to help with that so probably it does not exist yet. If so, my proposal is to add a Numpy method which helps with that. Namely the proposal will be to add a method which returns the slices for the intersection areas of two arbitrary arrays, given an offset, so then one can "blit" the array into another with simple assignment =. Here is a Python function I use for 2d arrays now: def interslice ( dest, src, offset ): y,x = offset H,W = dest.shape h,w = src.shape dest_starty = max (y,0) dest_endy = min (y+h,H) dest_startx = max (x,0) dest_endx = min (x+w,W) src_starty = 0 src_endy = h if y<0: src_starty = -y by = y+h - H # Y bleed if by>0: src_endy = h - by src_startx = 0 src_endx = w if x<0: src_startx = -x bx = x+w - W # X bleed if bx>0: src_endx = w - bx dest_sliceY = slice(dest_starty,dest_endy) dest_sliceX = slice(dest_startx,dest_endx) src_sliceY = slice(src_starty, src_endy) src_sliceX = slice(src_startx, src_endx) if dest_endy <= dest_starty: print "No Y intersection !" dest_sliceY = ( slice(0, 0) ) src_sliceY = ( slice(0, 0) ) if dest_endx <= dest_startx: print "No X intersection !" dest_sliceX = ( slice(0, 0) ) src_sliceX = ( slice(0, 0) ) dest_slice = ( dest_sliceY, dest_sliceX ) src_slice = ( src_sliceY, src_sliceX ) return ( dest_slice, src_slice ) -- I have intentionally made it expanded and without contractions so that it is better understandable. It returns the intersection area of two arrays given an offset. First returned tuple element is the slice for DEST array and the second element is the slice for SRC array. If there is no intersection along one of the axis at all it returns the corresponding slice as (0,0) With this helper function one can blit arrays easily e.g. example code: W = 8; H = 8 DEST = numpy.ones([H,W], dtype = "uint8") w = 4; h = 1 SRC = numpy.zeros([h,w], dtype = "uint8") SRC[:]=8 offset = (0,9) ds, ss = interslice (DEST, SRC, offset ) # blit SRC into DEST DEST[ds] = SRC[ss] So changing the offset one can observe how the SRC array is trimmed if it is crosses the DEST boundaries. I think it is very useful function in general and it has well defined behaviour. It has usage not only for graphics, but actually any data copying-pasting between arrays. So I am looking forward to comments on this proposal. Mikhail ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array blitting (pasting one array into another)
This is a useful idea certainly. I would recommended extending it to an arbitrary number of axes. You could either raise an error if the ndim of the two arrays are unequal, or allow a broadcast of a lesser ndimmed src array. - Joe On Jun 29, 2017 20:17, "Mikhail V" wrote: > Hello all > > I often need to copy one array into another array, given an offset. > This is how the "blit" function can be understood, i.e. in > every graphical lib there is such a function. > The common definition is like: > blit ( dest, src, offset ): > where dest is destination array, src is source array and offset is > coordinates in destination where the src should pe blitted. > Main feature of such function is that it never gives an error, > so if the source does not fit into the destination array, it is simply > trimmed. > And respectively if there is no intersection area then nothing happens. > > Hope this is clear. > So to make it work with Numpy arrays one need to calculate the > slices before copying the data. > I cannot find any Numpy or Python method to help with that so probably > it does not exist yet. > If so, my proposal is to add a Numpy method which helps with that. > Namely the proposal will be to add a method which returns > the slices for the intersection areas of two arbitrary arrays, given an > offset, > so then one can "blit" the array into another with simple assignment =. > > Here is a Python function I use for 2d arrays now: > > def interslice ( dest, src, offset ): > y,x = offset > H,W = dest.shape > h,w = src.shape > > dest_starty = max (y,0) > dest_endy = min (y+h,H) > dest_startx = max (x,0) > dest_endx = min (x+w,W) > > src_starty = 0 > src_endy = h > if y<0: src_starty = -y > by = y+h - H # Y bleed > if by>0: src_endy = h - by > > src_startx = 0 > src_endx = w > if x<0: src_startx = -x > bx = x+w - W # X bleed > if bx>0: src_endx = w - bx > > dest_sliceY = slice(dest_starty,dest_endy) > dest_sliceX = slice(dest_startx,dest_endx) > src_sliceY = slice(src_starty, src_endy) > src_sliceX = slice(src_startx, src_endx) > if dest_endy <= dest_starty: > print "No Y intersection !" > dest_sliceY = ( slice(0, 0) ) > src_sliceY = ( slice(0, 0) ) > if dest_endx <= dest_startx: > print "No X intersection !" > dest_sliceX = ( slice(0, 0) ) > src_sliceX = ( slice(0, 0) ) > dest_slice = ( dest_sliceY, dest_sliceX ) > src_slice = ( src_sliceY, src_sliceX ) > return ( dest_slice, src_slice ) > > > -- > > I have intentionally made it expanded and without contractions > so that it is better understandable. > It returns the intersection area of two arrays given an offset. > First returned tuple element is the slice for DEST array and the > second element is the slice for SRC array. > If there is no intersection along one of the axis at all > it returns the corresponding slice as (0,0) > > With this helper function one can blit arrays easily e.g. example code: > > W = 8; H = 8 > DEST = numpy.ones([H,W], dtype = "uint8") > w = 4; h = 1 > SRC = numpy.zeros([h,w], dtype = "uint8") > SRC[:]=8 > offset = (0,9) > ds, ss = interslice (DEST, SRC, offset ) > > # blit SRC into DEST > DEST[ds] = SRC[ss] > > So changing the offset one can observe how the > SRC array is trimmed if it is crosses the DEST boundaries. > I think it is very useful function in general and it has > well defined behaviour. It has usage not only for graphics, > but actually any data copying-pasting between arrays. > > So I am looking forward to comments on this proposal. > > > Mikhail > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array blitting (pasting one array into another)
On Fri, 2017-06-30 at 02:16 +0200, Mikhail V wrote: > Hello all > > I often need to copy one array into another array, given an offset. > This is how the "blit" function can be understood, i.e. in > every graphical lib there is such a function. > The common definition is like: > blit ( dest, src, offset ): > where dest is destination array, src is source array and offset is > coordinates in destination where the src should pe blitted. > Main feature of such function is that it never gives an error, > so if the source does not fit into the destination array, it is > simply trimmed. > And respectively if there is no intersection area then nothing > happens. > > Hope this is clear. > So to make it work with Numpy arrays one need to calculate the > slices before copying the data. > I cannot find any Numpy or Python method to help with that so > probably > it does not exist yet. > If so, my proposal is to add a Numpy method which helps with that. > Namely the proposal will be to add a method which returns > the slices for the intersection areas of two arbitrary arrays, given > an offset, > so then one can "blit" the array into another with simple assignment > =. > > Here is a Python function I use for 2d arrays now: > > def interslice ( dest, src, offset ): > y,x = offset > H,W = dest.shape > h,w = src.shape > > dest_starty = max (y,0) > dest_endy = min (y+h,H) > dest_startx = max (x,0) > dest_endx = min (x+w,W) > > src_starty = 0 > src_endy = h > if y<0: src_starty = -y > by = y+h - H # Y bleed > if by>0: src_endy = h - by > > src_startx = 0 > src_endx = w > if x<0: src_startx = -x > bx = x+w - W # X bleed > if bx>0: src_endx = w - bx > > dest_sliceY = slice(dest_starty,dest_endy) > dest_sliceX = slice(dest_startx,dest_endx) > src_sliceY = slice(src_starty, src_endy) > src_sliceX = slice(src_startx, src_endx) > if dest_endy <= dest_starty: > print "No Y intersection !" > dest_sliceY = ( slice(0, 0) ) > src_sliceY = ( slice(0, 0) ) > if dest_endx <= dest_startx: > print "No X intersection !" > dest_sliceX = ( slice(0, 0) ) > src_sliceX = ( slice(0, 0) ) > dest_slice = ( dest_sliceY, dest_sliceX ) > src_slice = ( src_sliceY, src_sliceX ) > return ( dest_slice, src_slice ) > > > -- > > I have intentionally made it expanded and without contractions > so that it is better understandable. > It returns the intersection area of two arrays given an offset. > First returned tuple element is the slice for DEST array and the > second element is the slice for SRC array. > If there is no intersection along one of the axis at all > it returns the corresponding slice as (0,0) > > With this helper function one can blit arrays easily e.g. example > code: > > W = 8; H = 8 > DEST = numpy.ones([H,W], dtype = "uint8") > w = 4; h = 1 > SRC = numpy.zeros([h,w], dtype = "uint8") > SRC[:]=8 > offset = (0,9) > ds, ss = interslice (DEST, SRC, offset ) > > # blit SRC into DEST > DEST[ds] = SRC[ss] > > So changing the offset one can observe how the > SRC array is trimmed if it is crosses the DEST boundaries. > I think it is very useful function in general and it has > well defined behaviour. It has usage not only for graphics, > but actually any data copying-pasting between arrays. > > So I am looking forward to comments on this proposal. > First, the slice object provides some help: ``` In [8]: s = slice(1, 40, 2) In [9]: s.indices(20) # length of dimension Out[9]: (1, 20, 2) # and the 40 becomes 20 ``` Second, there is almost no overhead of creating a view, so just create the views first (it may well be faster). Then use the result to see how large they actually are and index those (a second time) instead of creating new slice objects. - Sebastian > > Mikhail > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
