[Numpy-discussion] what would you expect A[none] to do?
In my case, what it does is: A.shape = (5760,) A[none] -> (1, 5760) In my case, use of none here is just a mistake. But why would you want this to be accepted at all, and how should it be interpreted? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] what would you expect A[none] to do?
Neal Becker wrote: > In my case, what it does is: > > A.shape = (5760,) > A[none] -> (1, 5760) > > In my case, use of none here is just a mistake. But why would you want > this to be accepted at all, and how should it be interpreted? Actually, in my particular case, if it just acted as a noop, returning the original array, that would have been perfect. No idea if that's a good result in general. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] what would you expect A[none] to do?
On Do, 2015-12-31 at 11:36 -0500, Neal Becker wrote: > Neal Becker wrote: > > > In my case, what it does is: > > > > A.shape = (5760,) > > A[none] -> (1, 5760) > > > > In my case, use of none here is just a mistake. But why would you > > want > > this to be accepted at all, and how should it be interpreted? > > Actually, in my particular case, if it just acted as a noop, > returning the > original array, that would have been perfect. No idea if that's a > good > result in general. > We have `np.newaxis` with `np.newaxis is None` for the same thing. `None` inserts a new axes, it is documented to do so in the indexing documentation, so I will ask you to check it if you have more questions. If you want a noop, you should probably use `...` or `Ellipsis`. - Sebastian > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] what would you expect A[none] to do?
Slicing with None adds a new dimension. It's a common paradigm, though usually you'd use A[np.newaxis] or A[np.newaxis, ...] instead for readibility. (np.newaxis is None, but it's a lot more readable) There's a good argument to be made that slicing with a single None shouldn't add a new axis, and only the more readable forms like A[None, :], A[..., None], etc should. However, that would rather seriously break backwards compatibility. There's a fair amount of existing code that assumes "A[None]" prepends a new axis. On Thu, Dec 31, 2015 at 10:36 AM, Neal Becker wrote: > Neal Becker wrote: > > > In my case, what it does is: > > > > A.shape = (5760,) > > A[none] -> (1, 5760) > > > > In my case, use of none here is just a mistake. But why would you want > > this to be accepted at all, and how should it be interpreted? > > Actually, in my particular case, if it just acted as a noop, returning the > original array, that would have been perfect. No idea if that's a good > result in general. > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Dynamic array list implementation
On Wed, Dec 30, 2015 at 6:34 AM, Nicolas P. Rougier < nicolas.roug...@inria.fr> wrote: > > > On 28 Dec 2015, at 19:58, Chris Barker wrote: > > > > >>> python benchmark.py > > Python list, append 10 items: 0.01161 > > Array list, append 10 items: 0.46854 > > > > are you pre-allocating any extra space? if not -- it's going to be > really, really pokey when adding a little bit at a time. > > > Yes, I’m preallocating but it might not be optimal at all given your > implementation is much faster. > I’ll try to adapt your code. Thanks. sounds good -- I'll try to take a look at yours soon - maybe we can merge the projects. MIne is only operational in one small place, I think. -CHB > > > > > With my Accumulator class: > > > > > https://github.com/PythonCHB/NumpyExtras/blob/master/numpy_extras/accumulator.py > > > > I pre-allocate a larger numpy array to start, and it gets re-allocated, > with some extra, when filled, using ndarray.resize() > > > > this is quite fast. > > > > These are settable parameters in the class: > > > > DEFAULT_BUFFER_SIZE = 128 # original buffer created. > > BUFFER_EXTEND_SIZE = 1.25 # array.array uses 1+1/16 -- that seems small > to me. > > > > > > I looked at the code in array.array (and list, I think), and it does > stuff to optimize very small arrays, which I figured wasn't the use-case > here :-) > > > > But I did a bunch of experimentation, and as long as you pre-allocate > _some_ it doesn't make much difference how much :-) > > > > BTW, > > > > I just went in an updated and tested the Accumulator class code -- it > needed some tweaks, but it's working now. > > > > The cython version is in an unknown state... > > > > some profiling: > > > > In [11]: run profile_accumulator.py > > > > > > In [12]: timeit accum1(1) > > > > 100 loops, best of 3: 3.91 ms per loop > > > > In [13]: timeit list1(1) > > > > 1000 loops, best of 3: 1.15 ms per loop > > > > These are simply appending 10,000 integers in a loop -- with teh list, > the list is turned into a numpy array at the end. So it's still faster to > accumulate in a list, then make an array, but only a about a factor of 3 -- > I think this is because you are staring with a python integer -- with the > accumulator function, you need to be checking type and pulling a native > integer out with each append. but a list can append a python object with no > type checking or anything. > > > > Then the conversion from list to array is all in C. > > > > Note that the accumulator version is still more memory efficient... > > > > In [14]: timeit accum2(1) > > > > 100 loops, best of 3: 3.84 ms per loop > > > > this version pre-allocated the whole internal buffer -- not much faster > the buffer re-allocation isn't a big deal (thanks to ndarray.resize using > realloc(), and not creating a new numpy array) > > > > In [24]: timeit list_extend1(10) > > > > 100 loops, best of 3: 4.15 ms per loop > > > > In [25]: timeit accum_extend1(10) > > > > 1000 loops, best of 3: 1.37 ms per loop > > > > This time, the stuff is added in chunks 100 elements at a time -- the > chunks being created ahead of time -- a list with range() the first time, > and an array with arange() the second. much faster to extend with arrays... > > > > -CHB > > > > > > > > -- > > > > Christopher Barker, Ph.D. > > Oceanographer > > > > Emergency Response Division > > NOAA/NOS/OR&R(206) 526-6959 voice > > 7600 Sand Point Way NE (206) 526-6329 fax > > Seattle, WA 98115 (206) 526-6317 main reception > > > > chris.bar...@noaa.gov > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@scipy.org > > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] what would you expect A[none] to do?
TBH, I wouldn't have expected it to work, but now that I see it, it does make some sense. I would have thought that it would error out as being ambiguous (prepend? append?). I have always used ellipses to make it explicit where the new axis should go. But, thinking in terms of how regular indexing works, I guess it isn't all that ambiguous. Ben Root On Thu, Dec 31, 2015 at 11:56 AM, Joe Kington wrote: > Slicing with None adds a new dimension. It's a common paradigm, though > usually you'd use A[np.newaxis] or A[np.newaxis, ...] instead for > readibility. (np.newaxis is None, but it's a lot more readable) > > There's a good argument to be made that slicing with a single None > shouldn't add a new axis, and only the more readable forms like A[None, :], > A[..., None], etc should. > > However, that would rather seriously break backwards compatibility. > There's a fair amount of existing code that assumes "A[None]" prepends a > new axis. > > On Thu, Dec 31, 2015 at 10:36 AM, Neal Becker wrote: > >> Neal Becker wrote: >> >> > In my case, what it does is: >> > >> > A.shape = (5760,) >> > A[none] -> (1, 5760) >> > >> > In my case, use of none here is just a mistake. But why would you want >> > this to be accepted at all, and how should it be interpreted? >> >> Actually, in my particular case, if it just acted as a noop, returning the >> original array, that would have been perfect. No idea if that's a good >> result in general. >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> https://mail.scipy.org/mailman/listinfo/numpy-discussion >> > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy funding update
On Wed, Dec 30, 2015 at 10:54 AM, Ralf Gommers wrote: > > Hi all, > > A quick good news message: OSDC has made a $5k contribution to NumFOCUS, which is split between support for a women in technology workshop and support for Numpy: http://www.numfocus.org/blog/osdc-donates-5k-to-support-numpy-women-in-tech > This was a very nice surprise to me, and a first sign that the FSA (fiscal sponsorship agreement) we recently signed with NumFOCUS is going to yield significant benefits for Numpy. > > NumFOCUS is also doing a special end-of-year fundraiser. Funds donated (up to $5k) will be tripled by anonymous sponsors: http://www.numfocus.org/blog/numfocus-end-of-year-fundraising-drive-5000-matching-gift-challenge > So think of Numpy (or your other favorite NumFOCUS-sponsored project of course) if you're considering a holiday season charitable gift! That sounds great! Do we have any concrete plans for spending that money, yet? -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] what would you expect A[none] to do?
On Thu, Dec 31, 2015 at 12:10 PM, Benjamin Root wrote: > TBH, I wouldn't have expected it to work, but now that I see it, it does > make some sense. I would have thought that it would error out as being > ambiguous (prepend? append?). I have always used ellipses to make it > explicit where the new axis should go. But, thinking in terms of how regular > indexing works, I guess it isn't all that ambiguous. Yeah, I'm not really a fan of the rule that indexing with too-few axes implicitly adds a "..." on the right A[0] -> A[0, ...] but given that we do have that rule, then A[None] -> A[None, ...] does make sense. -n -- Nathaniel J. Smith -- http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
