Re: [Numpy-discussion] converting a C bytes array to two dimensional numpy array

2019-07-19 Thread Chris Barker - NOAA Federal
You can also directly build a numpy array from a pointer with the numpy API. And I recommend Cython as an interface to make these things easy. This does mean you’d need to have the numpy lib at build time, .which may be a downside. -CHB Christopher Barker, Ph.D. Oceanographer Emergency Respon

Re: [Numpy-discussion] defining a NumPy API standard?

2019-06-02 Thread Chris Barker - NOAA Federal
> Exactly. This is great, thanks Marten. I agree with pretty much everything in > this list. For my part, a few things immediately popped out at my that I disagree with. ;-) Which does not mean it isn’t a useful exercise, but it does mean we should expect a fair bit of debate. But I do think we

Re: [Numpy-discussion] GSoD - Technical Writter

2019-05-19 Thread Chris Barker - NOAA Federal
> a responsive web page may not be an absolute requirement, but still it may be nice to be able to read the docs from a tablet or smartphone. Unfortunately I am not familiar yet with Sphinx, but I hope that it can be integrated with Jekyll or Hugo, and then one of their templates can be used. Sp

Re: [Numpy-discussion] Style guide for numpy code?

2019-05-14 Thread Chris Barker - NOAA Federal
;s best if variables that will appear in formulae are short, so more > terms can fit in one 80 character line. > > Overall, having and following a style makes code easier to read. And, as > an added bonus, if you take care to be consistent, you will write slower, > view your code

Re: [Numpy-discussion] Style guide for numpy code?

2019-05-09 Thread Chris Barker - NOAA Federal
Oops, Somehow that got sent before I was done. (Like my use of the passive voice there?) Here is a complete message: Do any of you know of a style guide for computational / numpy code? I don't mean code that will go into numpy itself, but rather, users code that uses numpy (and scipy, and...)

Re: [Numpy-discussion] grant proposal for core scientific Python projects (rejected)

2019-05-04 Thread Chris Barker - NOAA Federal
now say you know >>> yet another person at NASA who has no idea this even exists ... :) >>> Not only do I not know of that, but I know of NASA policies that make >>> it very difficult for NASA civil servants to contribute to open source >>> projects -- quite hy

Re: [Numpy-discussion] grant proposal for core scientific Python projects (rejected)

2019-05-04 Thread Chris Barker - NOAA Federal
On May 4, 2019, at 9:00 AM, Ralf Gommers Okay never mind, this is apparently happening already: https://hackmd.io/YbxTpC1ZT_aEapTqydmHCA. Please jump in there instead:) Slightly different focus than I had in mind, but yes, it makes sense to join that effort. -CHB Ralf > Cheers, > Ralf > >

Re: [Numpy-discussion] grant proposal for core scientific Python projects (rejected)

2019-05-02 Thread Chris Barker - NOAA Federal
Sounds like this is a NASA specific thing, in which case, I guess someone at NASA would need to step up. I’m afraid I know no pythonistas at NASA. But I’ll poke around NOAA to see if there’s anything similar. -CHB On Apr 25, 2019, at 1:04 PM, Ralf Gommers wrote: On Sat, Apr 20, 2019 at 12:4

Re: [Numpy-discussion] adding Quansight Labs as institutional partner

2019-04-16 Thread Chris Barker - NOAA Federal
Another consideration is that some organizations, the US federal government, for instance, cannot make donations. I wouldn’t be surprised if some Universities might have similar restrictions on using grant money to make a donation to a non-profit. Is there any way for NumFocus to accept funds in a

Re: [Numpy-discussion] Numpy-discussion

2019-01-29 Thread Chris Barker - NOAA Federal
> On Jan 29, 2019, at 12:15 AM, Matti Picus wrote: >> Perhaps you could suggest they add this explanation (or if it is in our >> documentation point out where you think would be appropriate) since it seems >> many people have problems with dtypes and overflow. arange() is particularly problem

Re: [Numpy-discussion] pytest, fixture and parametrize

2018-08-09 Thread Chris Barker - NOAA Federal
> On Aug 8, 2018, at 5:40 PM, Matti Picus wrote: > We have these guidelines http://www.numpy.org/devdocs/reference/testing.html, Thanks Matti — that’s clearly the place to document best practices. > It was updated for pytest in the 1.15 release, but could use some more > editing and refinement

Re: [Numpy-discussion] pytest, fixture and parametrize

2018-08-08 Thread Chris Barker - NOAA Federal
BTW: with pytest.raises(AnException): Is another nice feature. -CHB ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] pytest, fixture and parametrize

2018-08-08 Thread Chris Barker - NOAA Federal
> as to whether we are should use pytest fixtures and parametrized classes and > functions. Absolutely! > the disadvantage is being dependent on pytest as unittest does not support that functionality. Which is the whole point of using pytest, yes? I’m very opinionated about this, but I really

Re: [Numpy-discussion] Roadmap proposal, v3

2018-07-25 Thread Chris Barker - NOAA Federal
> Obviously the string dtype proposal in the roadmap is only a sketch at this > point :). > > I do think that options listed currently (encoded strings with fixed-width > storage and variable length strings) cover the breadth of proposals from last > time. We may not want to implement all of the

Re: [Numpy-discussion] Roadmap proposal, v3

2018-07-25 Thread Chris Barker - NOAA Federal
Great work, thanks! I see this: “- Fixed width encoded strings (utf8, latin1, ...)” And a bit of discussion in the PR. But I think there are key questions to be addressed in handling strings in numpy. I know we’ve had a lot of discussion about it on this list over the years — but is there a p

Re: [Numpy-discussion] Setting custom dtypes and 1.14

2018-01-25 Thread Chris Barker - NOAA Federal
> On Jan 25, 2018, at 4:06 PM, Allan Haldane wrote: >> 1) This is a known change with good reason? > . The > change occurred because the old assignment behavior was dangerous, and > was not doing what you thought. OK, that’s a good reason! >> A) improve the error message. > > Good idea. I'll s

Re: [Numpy-discussion] Using np.frombuffer and cffi.buffer on array of C structs (problem with struct member padding)

2018-01-25 Thread Chris Barker - NOAA Federal
The numpy dtype constructor takes an “align” keyword that will pad it for you. https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.dtype.html -CHB ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailm

Re: [Numpy-discussion] Using np.frombuffer and cffi.buffer on array of C structs (problem with struct member padding)

2018-01-25 Thread Chris Barker - NOAA Federal
Sent from my iPhone > On Jan 25, 2018, at 6:51 AM, Joe wrote: > > Hello, > > how I could dynamically handle the dtype of a structured array when reading > an array of C structs with np.frombuffer (regarding the member padding in the > struct). > > So far I manually adjusted the dtype of the stru

Re: [Numpy-discussion] NumPy 1.14.0rc1 release

2017-12-14 Thread Chris Barker - NOAA Federal
Thanks Chuck! And a huge thanks to that awesome list of contributors!!! -Chris Sent from my iPhone On Dec 13, 2017, at 2:55 PM, Charles R Harris wrote: Hi All, On behalf of the NumPy team, I am pleased to announce NumPy 1.14.0rc1. Numpy 1.14.0rc1 is the result of seven months of work and con

Re: [Numpy-discussion] Type annotations for NumPy

2017-11-28 Thread Chris Barker - NOAA Federal
(a) it would be good if NumPy type annotations could include an “array_like” type that allows lists, tuples, etc. I think that would be a sequence — already supported by the Typing system. (b) I’ve always thought (since PEP561) that it would be cool for type annotations to replace compiler type

Re: [Numpy-discussion] Type annotations for NumPy

2017-11-28 Thread Chris Barker - NOAA Federal
On Nov 25, 2017, at 3:35 PM, Matthew Rocklin wrote: Thoughts on basing this on a more generic Array type rather than the np.ndarray? This would actually be more consistent with the current python typing approach. I can imagine other nd-array libraries (XArray, Tensorflow, Dask.array) wanting t

Re: [Numpy-discussion] List comprehension and loops performances with NumPy arrays

2017-10-10 Thread Chris Barker - NOAA Federal
Andrea, One note: transposing is almost free — it just rearranges the strides — I.e. changed how the array is interpreted. It doesn’t actually move the data around. -CHB Sent from my iPhone On Oct 7, 2017, at 2:58 AM, Andrea Gavana wrote: Apologies, correct timeit code this time (I had gotten

Re: [Numpy-discussion] Only integer scalar arrays can be converted to a scalar index

2017-09-15 Thread Chris Barker - NOAA Federal
No thoughts on optimizing memory, but that indexing error probably comes from np.mean producing float results. An astype call shoulder that work. -CHB Sent from my iPhone On Sep 15, 2017, at 5:51 PM, Robert McLeod wrote: On Fri, Sep 15, 2017 at 2:37 PM, Elliot Hallmark wrote: > Nope. Numpy

Re: [Numpy-discussion] UC Berkeley hiring developers to work on NumPy

2017-05-14 Thread Chris Barker - NOAA Federal
Awesome! This is really great news. Does this mean is several person-years of funding secured? -CHB > On May 13, 2017, at 10:47 PM, Nathaniel Smith wrote: > > Hi all, > > As some of you know, I've been working for... quite some time now to > try to secure funding for NumPy. So I'm excited that

Re: [Numpy-discussion] proposal: smaller representation of string arrays

2017-04-26 Thread Chris Barker - NOAA Federal
> > I DO recommend Latin-1 As a default encoding ONLY for "mostly ascii, with > > a few extra characters" data. With all the sloppiness over the years, there > > are way to many files like that. > > That sloppiness that you mention is precisely the "unknown encoding" problem. Exactly -- but fro

Re: [Numpy-discussion] proposal: smaller representation of string arrays

2017-04-25 Thread Chris Barker - NOAA Federal
> On Apr 25, 2017, at 12:38 PM, Nathaniel Smith wrote: > Eh... First, on Windows and MacOS, filenames are natively Unicode. Yeah, though once they are stored I. A text file -- who the heck knows? That may be simply unsolvable. > s. And then from in Python, if you want to actually work with those

Re: [Numpy-discussion] proposal: smaller representation of string arrays

2017-04-25 Thread Chris Barker - NOAA Federal
Actually, for what it's worth, the FITS spec says that in such values trailing spaces are not significant, see page 7: https://fits.gsfc.nasa.gov/standard40/fits_standard40draft1.pdf But they're not really relevant to numpy's situation, because as here you need to do elaborate de-quoting before the

Re: [Numpy-discussion] proposal: smaller representation of string arrays

2017-04-25 Thread Chris Barker - NOAA Federal
A compact dtype for mostly-ascii text: > I'm a little confused about exactly what you're trying to do. Actually, *I* am not trying to do anything here -- I'm the one that said computers are so big and fast now that we shouldn't whine about 4 bytes for a characterbut this whole conversation s