[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

On Wed, Aug 30, 2023 at 4:25 AM Ralf Gommers wrote: > > > On Tue, Aug 29, 2023 at 4:08 PM Nathan wrote: > >> The NEP was merged in draft form, see below. >> >> https://numpy.org/neps/nep-0055-string_dtype.html >> > > This is a really nice NEP, thanks Nathan! I see that questions and > constructi

[Numpy-discussion] Re: Adding NumpyUnpickler to Numpy 1.26 and future Numpy 2.0

On Mon, Oct 9, 2023 at 2:29 PM Nathan wrote: > However, one thing we can do now is, for that one particular symbol that > we know is going to be in every pickle file and probably never elsewhere, > is intercept that one import and instead of generating a generic warning > about np.core being depr

[Numpy-discussion] Re: Fixing definition of reduceat for Numpy 2.0?

On Fri, Dec 22, 2023 at 12:34 PM Martin Ling wrote: > Hi folks, > > I don't follow numpy development in much detail these days but I see > that there is a 2.0 release planned soon. > > Would this be an opportunity to change the behaviour of 'reduceat'? > > This issue has been open in some form si

[Numpy-discussion] Re: NEP 56: array API standard support in the main numpy namespace

On Sun, Jan 7, 2024 at 8:08 AM Ralf Gommers wrote: > This NEP will supersede the following NEPs: > > - :ref:`NEP30` (never implemented) > - :ref:`NEP31` (never implemented) > - :ref:`NEP37` (never implemented; the ``__array_module__`` idea is > basically > the same as ``__array_namespace__``) >

Re: [Numpy-discussion] __array_ufunc__ counting down to launch, T-24 hrs.

I agree with Nathaniel -- let's finish the design doc first. On Thu, Mar 30, 2017 at 10:08 PM, Nathaniel Smith wrote: > On Thu, Mar 30, 2017 at 7:40 PM, Charles R Harris > wrote: > > Hi All, > > > > Just a note that the __array_ufunc__ PR is ready to merge. If you are > > interested, you can re

Re: [Numpy-discussion] proposal: smaller representation of string arrays

Julian -- thanks for taking this on. NumPy's handling of strings on Python 3 certainly needs fixing. On Thu, Apr 20, 2017 at 9:47 AM, Anne Archibald wrote: > Variable-length encodings, of which UTF-8 is obviously the one that makes > good handling essential, are indeed more complicated. But is i

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Thu, Apr 20, 2017 at 10:43 AM, Chris Barker wrote: > On Thu, Apr 20, 2017 at 10:26 AM, Stephan Hoyer wrote: > >> I agree with Anne here. Variable-length encoding would be great to have, >> but even fixed length UTF-8 (in terms of memory usage, not characters) >> woul

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Thu, Apr 20, 2017 at 11:53 AM, Robert Kern wrote: > I don't know of a format off-hand that works with numpy uniform-length > strings and Unicode as well. HDF5 (to my recollection) supports arrays of > NULL-terminated, uniform-length ASCII like FITS, but only variable-length > UTF8 strings. >

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Thu, Apr 20, 2017 at 12:17 PM, Robert Kern wrote: > On Thu, Apr 20, 2017 at 12:05 PM, Stephan Hoyer wrote: > > > > On Thu, Apr 20, 2017 at 11:53 AM, Robert Kern > wrote: > >> > >> I don't know of a format off-hand that works with numpy uniform-lengt

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Fri, Apr 21, 2017 at 11:34 AM, Chris Barker wrote: > 1) Use with/from Python -- both creating and working with numpy arrays. > > In this case, we want something compatible with Python's string (i.e. full > Unicode supporting) and I think should be as transparent as possible. > Python's strin

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Mon, Apr 24, 2017 at 10:04 AM, Chris Barker wrote: > latin-1 or latin-9 buys you (over ASCII): > > ... > > - round-tripping of binary data (at least with Python's encoding/decoding) > -- ANY string of bytes can be decodes as latin-1 and re-encoded to get the > same bytes back. You may get garb

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Mon, Apr 24, 2017 at 11:13 AM, Chris Barker wrote: > On the other hand, if this is the use-case, perhaps we really want an >> encoding closer to "Python 2" string, i.e, "unknown", to let this be >> signaled more explicitly. I would suggest that "text[unknown]" should >> support operations like

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Mon, Apr 24, 2017 at 4:08 PM, Robert Kern wrote: > Let me make a counter-proposal for your latin-1 dtype (your #2) that might > address your, Thomas's, and Julian's use cases: > > 2) We want a single-byte-per-character, NULL-terminated string dtype that > can be used to represent mostly-ASCII

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Mon, Apr 24, 2017 at 7:41 PM, Nathaniel Smith wrote: > But also, is it important whether strings we're loading/saving to an > HDF5 file have the same in-memory representation in numpy as they > would in the file? I *know* [1] no-one is reading HDF5 files using > np.memmap :-). Of course they

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Tue, Apr 25, 2017 at 9:21 PM Robert Kern wrote: > On Tue, Apr 25, 2017 at 6:27 PM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > > > The maximum length of an UTF-8 character is 4 bytes, so we could use > that to size arrays by character length. The advantage over UTF-32 is that > it

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Wed, Apr 26, 2017 at 3:27 PM, Chris Barker wrote: > When a numpy user wants to put a string into a numpy array, they should > know how long a string they can fit -- with "length" defined how python > strings define it. > Sorry, I remain unconvinced (for the reasons that Robert, Nathaniel and

Re: [Numpy-discussion] proposal: smaller representation of string arrays

On Wed, Apr 26, 2017 at 4:49 PM, Nathaniel Smith wrote: > It's worthwhile enough that both major HDF5 bindings don't support Unicode > arrays, despite user requests for years. The sticking point seems to be the > difference between HDF5's view of a Unicode string array (defined in size > by the b

Re: [Numpy-discussion] Proposal: np.search() to complement np.searchsorted()

On Tue, May 9, 2017 at 9:46 AM, Martin Spacek wrote: > Looking at my own habits and uses, it seems to me that finding the indices > of matching values of one array in another is a more common use case than > finding insertion indices of one array into another sorted array. So, I > propose that np

Re: [Numpy-discussion] NumPy v1.13.0rc1 released.

>- Likhith Chitneni + >>- Loïc Estève >>- Mads Ohm Larsen >>- Marein Könings + >>- Marten van Kerkwijk >>- Martin Thoma >>- Martino Sorbaro + >>- Marvin Schmidt + >>- Matthew Brett >>- Matthias Busson

Re: [Numpy-discussion] Proposal: np.search() to complement np.searchsorted()

I like the idea of a strategy keyword argument. strategy='auto' leaves the door open for future improvements, e.g., if we ever add hash tables to numpy. For the algorithm, I think we actually want to sort the needles array as well in most (all?) cases. If haystack is also sorted, advancing thorou

Re: [Numpy-discussion] UC Berkeley hiring developers to work on NumPy

On Mon, May 22, 2017 at 11:52 AM, Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > My sentence "adapt the typical academic rule for conflicts of > interests to PRs, that non-trivial ones cannot be merged by someone > who has a conflict of interest with the author, i.e., it cannot be a > s

Re: [Numpy-discussion] Controlling NumPy __mul__ method or forcing it to use __rmul__ of the "other"

I answered your question on StackOverflow: https://stackoverflow.com/questions/40694380/forcing-multiplication-to-use-rmul-instead-of-numpy-array-mul-or-byp/44634634#44634634 In brief, you need to set __array_priority__ or __array_ufunc__ on your object. On Mon, Jun 19, 2017 at 5:27 AM, Ilhan Pol

Re: [Numpy-discussion] Controlling NumPy __mul__ method or forcing it to use __rmul__ of the "other"

n Mon, Jun 19, 2017 at 11:07 AM, Ilhan Polat > wrote: > >> Thank you. I didn't know that it existed. Is there any place where I can >> get a feeling for a sane priority number compared to what's being done in >> production? Just to make sure I'm not stepping on a

Re: [Numpy-discussion] Vector stacks

I would also prefer separate functions. These are much easier to understand that custom operator overloads. Side note: implementing this class with __array_ufunc__ for ndarray @ cvec actually isn't possible to do currently, until we fix this bug: https://github.com/numpy/numpy/issues/9028 On Sat,

Re: [Numpy-discussion] Scipy 2017 NumPy sprint

On Sun, Jul 2, 2017 at 8:33 AM Sebastian Berg wrote: > If someone who does subclasses/array-likes or so (e.g. like Stefan > Hoyer ;)) and is interested, and also we do some > teleconferencing/chatting (and I have time) I might be interested > in discussing and possibly trying to develop the n

Re: [Numpy-discussion] Scipy 2017 NumPy sprint

On Wed, Jul 5, 2017 at 10:40 AM, Chris Barker wrote: > Along those lines, there was some discussion of having a set of utilities > (or maybe eve3n an ABC?) that would make it easier to create a ndarray-like > object. > > That is, the boilerplate needed for multi-dimensional indexing and > slicing

Re: [Numpy-discussion] Scipy 2017 NumPy sprint

On Thu, Jul 6, 2017 at 4:42 AM, Ben Rowland wrote: > Slightly off topic, but as someone who has just spent a fair amount of > time implementing various > subclasses of nd-array, I am interested (and a little concerned), that the > consensus is not to use > them. Is there anything available which

Re: [Numpy-discussion] Scipy 2017 NumPy sprint

On Thu, Jul 6, 2017 at 9:42 AM, Chris Barker wrote: > In NumPy, this could go along with NDArrayOperatorsMixins in >> numpy/lib/mixins.py >> >> > > Yes! I had no idea that existed. > It's brand new

Re: [Numpy-discussion] ENH: ratio function to mimic diff

This is an interesting idea, but I don't understand the use cases for this function. In particular, what would you use n-th order ratios for? One use case I can think of is estimating the slope of a log-scaled plot. But here exp(diff(log(x))) is an easy substitute. I guess ratio() would work in c

Re: [Numpy-discussion] Tensor Contraction (HPTT) and Tensor Transposition (TCL)

On Wed, Aug 16, 2017 at 2:39 AM, Paul Springer wrote: > > What version of Numpy are you comparing to? Note that in 1.13 you can > enable some optimization in einsum, and the coming 1.14 makes that the > default and uses CBLAS when possible. > > I was using 1.10.4; however, I am currently running

Re: [Numpy-discussion] Why are empty arrays False?

I agree, this behavior seems actively harmful. Let's fix it. On Fri, Aug 18, 2017 at 2:45 PM, Michael Lamparski wrote: > Greetings, all. I am troubled. > > The TL;DR is that `bool(array([])) is False` is misleading, dangerous, and > unnecessary. Let's begin with some examples: > > >>> bool(np.a

Re: [Numpy-discussion] Interface numpy arrays to Matlab?

If you can use Octave instead of Matlab, I've had a very good experience with Oct2Py: https://github.com/blink1073/oct2py On Mon, Aug 28, 2017 at 12:20 PM, Neal Becker wrote: > I've searched but haven't found any decent answer. I need to call Matlab > from python. Matlab has a python module fo

Re: [Numpy-discussion] Is there a better way to write a stacked matrix multiplication

I would certainly use einsum. It is almost perfect for these use cases, e.g., np.einsum('ki,kij,kj->k', A, inv(B), A) On Thu, Oct 26, 2017 at 12:38 PM Charles R Harris wrote: > On Thu, Oct 26, 2017 at 12:11 PM, Daniele Nicolodi > wrote: > >> Hello, >> >> is there a better way to write the dot p

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

Hi Will, We spent a *long time* sorting out the messy details of __array_ufunc__ [1], especially for handling interactions between different types, e.g., between numpy arrays, non-numpy array-like objects, builtin Python objects, objects that override arithmetic to act in non-numpy-like ways, and

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

On Thu, Nov 2, 2017 at 9:45 AM wrote: > similar, scipy.special has ufuncs > what units are those? > > Most code that I know (i.e. scipy.stats and statsmodels) does not use only > "normal mathematical operations with ufuncs" > I guess there are a lot of "abnormal" mathematical operations > where j

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

On Thu, Nov 2, 2017 at 12:42 PM Nathan Goldbaum wrote: > Would this issue be ameliorated given Nathaniel's proposal to try to move > away from subclasses and towards storing data in dtypes? Or would that just > mean that xarray would need to ban dtypes it doesn't know about? > Yes, I think custo

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

Maybe the best of both worlds would require explicit opt-in for classes that shouldn't be coerced, e.g., xarray.register_data_type(MyArray) or maybe better yet ;) xarray.void_my_nonexistent_warranty_its_my_fault_if_my_buggy_duck_array_breaks_everything(MyArray) On Thu, Nov 2, 2017 at 3:39 PM Mart

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

On Thu, Nov 2, 2017 at 3:35 PM Nathan Goldbaum wrote: > Ah, but what if the dtype modifies the interface? That might sound evil, > but it's something that's been proposed. For example, if I wanted to > replace yt's YTArray in a backward compatibile way with a dtype and just > use plain ndarrays e

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

On Mon, Nov 6, 2017 at 2:29 PM Ryan May wrote: > On Mon, Nov 6, 2017 at 3:18 PM, Chris Barker > wrote: > >> Klunky, and maybe we could come up with a standard way to do it and >> include that in numpy, but I'm not sure that ABCs are the way to do it. >> > > ABCs are *absolutely* the way to go ab

Re: [Numpy-discussion] is __array_ufunc__ ready for prime-time?

On Tue, Nov 7, 2017 at 12:23 PM Chris Barker wrote: > > And then a third abc for indexing support, although, I am not sure how >> that could get implemented... > > > This is the really tricky one -- all ABCs really check is the existence of > methods -- making sure they behave the same way is up

Re: [Numpy-discussion] np.vstack vs. np.stack

I'm pretty sure I wrote the offending line in the vstack() docs. The original motivation for stack() was that stacking behavior of hstack(), vstack() and dstack() was somewhat inconsistent, especially with regard to lower dimensional input. stack() is conceptually much simpler and more general. T

Re: [Numpy-discussion] np.vstack vs. np.stack

On Thu, Nov 9, 2017 at 2:49 PM Allan Haldane wrote: > Maybe we should reword the vstack docstring so that it doesn't imply > that vstack is going away. It should say something weaker > like "the functions np.stack, np.concatenate, and np.block are often > more general/useful/less confusing altern

[Numpy-discussion] Type annotations for NumPy

There's been growing interest in supporting PEP-484 style type annotations in NumPy: https://github.com/numpy/numpy/issues/7370 This would allow NumPy users to add type-annotations to their code that uses NumPy, which they could check with mypy, pycharm or pytype. For example: def f(x: np.ndarray

Re: [Numpy-discussion] Type annotations for NumPy

On Sat, Nov 25, 2017 at 7:21 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > A question of perhaps broader scope than what you were asking for, and > more out of curiosity than anything else, but can one mix type > annotations with others? E.g., in astropy, we have a decorator that >

Re: [Numpy-discussion] Type annotations for NumPy

On Sat, Nov 25, 2017 at 3:34 PM Matthew Rocklin wrote: > Thoughts on basing this on a more generic Array type rather than the > np.ndarray? I can imagine other nd-array libraries (XArray, Tensorflow, > Dask.array) wanting to reuse this work. For dask.array in particular we > would want to copy

Re: [Numpy-discussion] Type annotations for NumPy

On Tue, Nov 28, 2017 at 5:11 PM Robert T. McGibbon wrote: > I'm strongly in support of this proposal. Type annotations have really > helped me write more correct code. > > I started working on numpy type stubs a few months ago. I needed a mypy > plugin to support shape-aware functions. Those who

Re: [Numpy-discussion] Type annotations for NumPy

This discussion has died down, but I don't want to lose momentum . It sounds like there is at least strong interest from a subset of our community in type annotations. Are there any objections to the first part of my plan, to start developing type stubs for NumPy in separate repository? We'll com

Re: [Numpy-discussion] Type annotations for NumPy

OK, in that case let's get to work over in https://github.com/numpy/numpy_stubs! On Tue, Dec 5, 2017 at 2:43 PM Fernando Perez wrote: > On Tue, Dec 5, 2017 at 2:19 PM, Nathaniel Smith wrote: > >> On Tue, Dec 5, 2017 at 10:04 AM, Stephan Hoyer wrote: >> > This discu

Re: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars?

On Tue, Dec 12, 2017 at 5:07 PM Mark Campanelli wrote: > I think I saw some other discussion recently about numpy joining forces > with Python 3's gradual type system. Is there any draft formal proposal for > this yet? If numpy+scipy wants to scale to "bigger" projects, I think it > behooves the

Re: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars?

On Tue, Dec 12, 2017 at 6:20 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > The real magic happens when you ducktype, and ensure your function > works both for arrays and scalars on its own. This is more often > possible than you might think! Sadly, this still doesn't work in a typ

Re: [Numpy-discussion] Moving NumPy's PRNG Forward

On Fri, Jan 19, 2018 at 6:57 AM Robert Kern wrote: > As an alternative, we may also want to leave `np.random.RandomState` > entirely fixed in place as deprecated legacy code that is never updated. > This would allow current unit tests that depend on the stream-compatibility > that we previously p

[Numpy-discussion] NumPy should not silently promote numbers to strings

This is one of my oldest NumPy pain-points: >>> np.array([1, 2, 'three']) array(['1', '2', 'three'], dtype='https://github.com/pydata/xarray/pull/1847), but mostly just hides bugs until later. It's certainly very un-Pythonic. The sane promotion rule would be `np.promote_types(str, float) ->

Re: [Numpy-discussion] NumPy should not silently promote numbers to strings

On Thu, Feb 8, 2018 at 11:00 PM Eric Wieser wrote: > Presumably you would extend that to all (str, np.number), or even (str, > np.generic_)? > Yes, I'm currently doing (np.character, np.number) and (np.character, np.bool_). But only in direct consultation with the diagram of NumPy's type hierarch

Re: [Numpy-discussion] Permissable NumPy logo usage

I don't know the history of the NumPy logo, or who officially owns the rights to NumPy's branding at this point. In principle, that might be NumFOCUS, but the logo far predates NumFOCUS and NumFOCUS's fiscal sponsorship of NumPy. Looking at the Git history, it looks like David Cournapeau added it t

Re: [Numpy-discussion] new NEP: np.AbstractArray and np.asabstractarray

Hi Nathaniel, Thanks for starting the discussion! Like Marten says, I think it would be useful to more clearly define what it means to be an abstract array. ndarray has lots of methods/properties that expose internal implementation (e.g., view, strides) that presumably we don't want to require as

Re: [Numpy-discussion] new NEP: np.AbstractArray and np.asabstractarray

On Thu, Mar 8, 2018 at 5:54 PM Juan Nunez-Iglesias wrote: > On Fri, Mar 9, 2018, at 5:56 AM, Stephan Hoyer wrote: > > Marten's case 1: works exactly like ndarray, but stores data differently: > parallel arrays (e.g., dask.array), sparse arrays (e.g., > https://gith

Re: [Numpy-discussion] Where to discuss NEPs (was: Re: new NEP: np.AbstractArray and np.asabstractarray)

I also have a slight preference for managing the discussion on GitHub, which is a bit more fully featured than email for long discussion (e.g., it supports code formatting and editing comments). But I'm really OK either way as long as discussion is kept in one place. We could still stipulate that

Re: [Numpy-discussion] Where to discuss NEPs (was: Re: new NEP: np.AbstractArray and np.asabstractarray)

I'll note that we basically used GitHub for revising __array_ufunc__ NEP, and I think that worked out better for everyone involved. The discussion was a little too specialized and high volume to be well handled on the mailing list. On Fri, Mar 9, 2018 at 8:58 AM Stephan Hoyer wrote: >

Re: [Numpy-discussion] PR to add an initializer kwarg to ufunc.reduce (and similar functions)

This looks like a very logical addition to the reduce interface. It has my support! I would have preferred the more descriptive name "initial_value", but consistency with functools.reduce makes a compelling case for "initializer". On Sun, Mar 25, 2018 at 1:15 PM Eric Wieser wrote: > To reiterat

Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions

On Sun, Apr 29, 2018 at 2:48 AM Matti Picus wrote: > The proposed solution to issue #9029 is to extend the meaning of a > signature so "syntax like (n?,k),(k,m?)->(n?,m?) could mean that n and m > are optional dimensions; if missing in the input, they're treated as 1, and > then dropped from the

Re: [Numpy-discussion] summary of "office Hours" open discusison April 25

I'm happy to chat about how pandas has done things. It's worth noting that although it may *look* like Jeff Reback is a full-time maintainer (he does a lot of work!), he has actually been maintaining pandas as a side-project. Mostly the project bumbles along without a clear direction, somewhat simi

Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions

On Wed, May 2, 2018 at 8:39 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > I think we should not decide too readily on what is "reasonable" to > expect for a ufunc input. > I agree strongly with this. I can think of a couple of other use-cases off hand: - xarray.Dataset is a dict-l

Re: [Numpy-discussion] NumPy sprint May 24-25 at BIDS

I will also be attending, on at least Thursday (and hopefully Friday, too). Best, Stephan On Thu, May 17, 2018 at 1:40 PM Jaime Fernández del Río < jaime.f...@gmail.com> wrote: > $#!#, was looking at the wrong calendar month: Thursday half day, Friday > all day. > > Jaime > > On Thu, May 17, 201

Re: [Numpy-discussion] matmul as a ufunc

On Mon, May 21, 2018 at 5:42 PM Matti Picus wrote: > - create a wrapper that can convince the ufunc mechanism to call > __array_ufunc__ even on functions that are not true ufuncs > I am somewhat opposed to this approach, because __array_ufunc__ is about overloading ufuncs, and as soon as we rela

Re: [Numpy-discussion] matmul as a ufunc

On Mon, May 28, 2018 at 7:36 PM Eric Wieser wrote: > which ensure that it is still well defined (as the identity) on 1d arrays. > > This strikes me as a bad idea. There’s already enough confusion from > beginners that array_1d.T is a no-op. If we introduce a matrix-transpose, > it should either e

Re: [Numpy-discussion] Adding take_along_axis and put_along_axis functions

As I'm sure I stated in the GItHub discussion, I strongly support adding these functions to NumPy. This logic is non-trivial to get right and is quite broadly useful. These names also seem natural to me. On Mon, May 28, 2018 at 8:07 PM Eric Wieser wrote: > These functions provide a vectorized w

Re: [Numpy-discussion] Where to discuss NEPs (was: Re: new NEP: np.AbstractArray and np.asabstractarray)

Reviving this discussion -- I don't really care what our policy is, but can we make a decision one way or the other about where we discuss NEPs? We've had a revival of NEP writing recently, so this is very timely. Previously, I was in slight favor of doing discussion on GitHub. Now that I've start

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On Wed, May 30, 2018 at 11:15 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > My PR provides the ability to indicate in the signature that a core > dimension can be broadcast, by using a suffix of "|1". Thus, the > signature of `all_equal` would become: > > ``` > (n|1),(n|1)->() > ```

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On Thu, May 31, 2018 at 4:21 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > I think the case for frozen dimensions is much more solid that just > `cross1d` - there are many operations that work on size-3 vectors. > Indeed, as I noted in the PR, I have just been wrapping a > Standards

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On Wed, May 30, 2018 at 5:01 PM Matthew Harrigan wrote: > "short-cut to automatically return False if m != n", that seems like a > silent bug > I guess it depends on the use-cases. This is how np.array_equal() works: https://docs.scipy.org/doc/numpy/reference/generated/numpy.array_equal.html We

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On Fri, Jun 1, 2018 at 2:42 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Having done that, I felt the examples actually justified the frozen > dimensions quite well. Given that you're the who expressed most doubts > about them, could you have a look? Ideally, I'd avoid having to wr

[Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

== :Author: Stephan Hoyer :Author: Matthew Rocklin :Status: Draft :Type: Standards Track :Created: 2018-05-29 Abstact --- We propose a protocol to allow arguments of numpy functions to define how that function operates on them. This allows other libraries that implement NumPy's high leve

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 8:19 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > My more general comment is one of speed: for *normal* operation > performance should be impacted as minimally as possible. I think this is a > serious issue and feel strongly it *has* to be possible to avoid a

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 11:12 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > On Sun, Jun 3, 2018 at 2:00 PM, Hameer Abbasi > wrote: > >> >>- Objects that don’t implement ``__array_function__`` should be >>treated as having returned ``np.NotImplementedButCoercible``. >>

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 4:25 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > I think one might still want to know *where* the type occurs (e.g., as an > output or index would have different implications). > This in certainly true in general, but given the complete flexibility of __arr

Re: [Numpy-discussion] NEP: Random Number Generator Policy

On Sat, Jun 2, 2018 at 12:06 PM Robert Kern wrote: > We propose first freezing ``RandomState`` as it is and developing a new RNG > subsystem alongside it. This allows anyone who has been relying on our old > stream-compatibility guarantee to have plenty of time to migrate. > ``RandomState`` will

Re: [Numpy-discussion] NEP: Random Number Generator Policy

On Sun, Jun 3, 2018 at 5:39 PM Robert Kern wrote: > You and I both agree that this is an anti-pattern for numpy.random, but >> certainly there is plenty of code that relies on the stability of random >> numbers when seeds are set by np.random.seed(). Similar to the case for >> RandomState, we wou

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 5:44 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Although I'm still not 100% convinced by NotImplementedButCoercible, I do > like the idea that this is the default for items that do not implement > `__array_function__`. And it might help avoid trying to find

Re: [Numpy-discussion] NEP: Random Number Generator Policy

On Sun, Jun 3, 2018 at 8:22 PM Ralf Gommers wrote: > It may be worth having a look at test suites for scipy, statsmodels, > scikit-learn, etc. and estimate how much work this NEP causes those > projects. If the devs of those packages are forced to do large scale > migrations from RandomState to S

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Sun, Jun 3, 2018 at 9:54 PM Hameer Abbasi wrote: > Mixed return values of NotImplementedButCoercible and NotImplemented would > still result in TypeError, and there would be no second chances for > overloads. > > > I would like to differ with you here: It can be quite useful to have > second c

Re: [Numpy-discussion] A roadmap for NumPy - longer term planning

PEP-574 isn't on the roadmap (yet!), but I think we would clearly welcome it. Like all NumPy improvements, it would need to implemented by an interested party. On Mon, Jun 4, 2018 at 1:52 AM Antoine Pitrou wrote: > > Hi, > > Do you plan to consider trying to add PEP 574 / pickle5 support? There's

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Mon, Jun 4, 2018 at 5:39 AM Matthew Harrigan wrote: > Should there be discussion of typing (pep-484) or abstract base classes in > this nep? Are there any requirements on the result returned by > __array_function__? > This is a good question that should be addressed in the NEP. Currently, we

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Mon, Jun 4, 2018 at 7:35 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Hi Stephan, > > Another potential consideration in favor of NotImplementedButCoercible is >> for subclassing: we could use it to write the default implementations of >> ndarray.__array_ufunc__ and ndarray.__ar

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Tue, Jun 5, 2018 at 12:35 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Things would, I think, make much more sense if `ndarray.__array_ufunc__` > (or `*_function__`) actually *were* the implementation for array-only. But > while that is something I'd like to eventually get to, i

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Tue, Jun 5, 2018 at 2:47 PM Matti Picus wrote: > What is the difference between the `func` provided as the first argument > to `__array_function__` and `__array_ufunc__` and the "non-overloaded > version of the provided function"? > The ""non-overloaded version of the provided function" is en

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

On Fri, Jun 8, 2018 at 8:58 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > I think we're getting to the stage where an updated text would be useful. > Yes, I plan to work on this over the weekend. Stay tuned! > For that, you may want to consider an actual implementation of, e.g.,

Re: [Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

(offlist) To clarify, by "where_i_am" you mean something like the name of the argument where it was found? On Fri, Jun 8, 2018 at 4:49 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > and in particular how the implementation finds out where its own instances >>> are located. >>> >> >

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

In Sun, Jun 10, 2018 at 4:31 PM Eric Wieser wrote: > Thanks for the writeup Marten, > Indeed, thank you Marten! > This hits on an interesting alternative to frozen dimensions - np.cross > could just become a regular ufunc with signature np.dtype((float64, 3)), > np.dtype((float64, 3)) → np.dtype

Re: [Numpy-discussion] NEP: Random Number Generator Policy

On Sun, Jun 10, 2018 at 8:10 PM Ralf Gommers wrote: > On Sun, Jun 10, 2018 at 5:57 PM, Robert Kern > wrote: > >> > Conclusion: the current proposal will cause work for the vast majority >> of libraries that depends on numpy. The total amount of that work will >> certainly not be counted in perso

Re: [Numpy-discussion] Dropping Python 3.4 support for NumPy 1.16

This sounds good to me. Most of the downstream projects I work with have already dropped Python 3.4 support. On Wed, Jun 13, 2018 at 2:30 PM Charles R Harris wrote: > Hi All, > > I think NumPy 1.16 would be a good time to drop Python 3.4 support. We > will want to do that anyway once we drop 2.7

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On Mon, Jun 11, 2018 at 11:59 PM Eric Wieser wrote: > I don’t understand your alternative here. If we overload np.matmul using > *array_function*, then it would not use *ether* of these options for > writing the operation in terms of other gufuncs. It would simply look for > an *array_function* a

Re: [Numpy-discussion] NEP: Random Number Generator Policy

> > This is a little weird; "mtrand" is an implementation detail already. >> There's exactly 3 instances of that in scikit-learn, so replacing those >> with a sane name (with a long timeline, say 4 numpy versions at least plus >> a major version number bump) doesn't seem unreasonable. >> > > Everyt

Re: [Numpy-discussion] question about array slicing and element assignment

You will need to convert "a[(2,3,5),][mask]" into a single indexing expression, e.g, by using utility functions like np.nonzero() on mask. NumPy can't support assignment in chained indexing. On Tue, Jun 19, 2018 at 1:25 PM Emil Sidky wrote: > Hello, > The following is an example where an array e

[Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

Cheers, Stephan = Simplified and explicit advanced indexing = :Author: Sebastian Berg :Author: Stephan Hoyer :Status: Draft :Type: Standards Track :Created: 2015-08-27 Abstract NumPy's "advanced" indexing support for indexi

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

On Tue, Jun 26, 2018 at 4:34 PM Robert Kern wrote: > I maintain that considering deprecation is premature at this time. Please > take it out of this NEP. Let us get a feel for how people actually use > .oindex/.vindex. Then we can talk about deprecation. This NEP gets my > enthusiastic approval,

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

On Tue, Jun 26, 2018 at 9:38 AM Eric Wieser wrote: > We can expose some of the internals > > These could be expressed as methods on the internal indexing objects I > proposed in the first reply to this thread, which has seen no responses. > > I think Hameer Abbasi is looking for something like >

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

On Tue, Jun 26, 2018 at 6:39 PM Robert Kern wrote: > I'd still prefer not talking deprecation, per se, in this NEP (but my > objection is weaker). I would definitely start adding in informative, noisy > warnings in these cases, though. Along the lines of, "Hey, this is a dodgy > construction that

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

On Tue, Jun 26, 2018 at 12:46 AM Robert Kern wrote: > I think having more self-contained descriptions of the semantics of each > of these would be a good idea. The current description of `.vindex` spends > more time talking about what it doesn't do, compared to the other methods, > than what it d

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

On Tue, Jun 26, 2018 at 10:22 PM Robert Kern wrote: > We certainly could make the conservative choice of only adopting 4 for now >> and leaving further cleanup for later. I guess this uncertainty about >> whether direct indexing should be more like vindex[] or oindex[] in the >> long term is a go

[Numpy-discussion] Revised NEP-18, __array_function__ protocol

also included below. Best, Stephan === A dispatch mechanism for NumPy's high level array functions === :Author: Stephan Hoyer :Author: Matthew Rocklin :Author: Marten van Kerkwij

  1   2   3   >