[Numpy-discussion] Re: Fortran-style support in ctypeslib.as_array()?

2023-03-23 Thread Eric Wieser
Hi Monte, This strikes me as a slightly strange request; ctypes is intended to interface with the C memory model, which has no native representation of fortran arrays. The underlying workings of `as_array` is to cast your ctype pointer into a ctypes array object, and then pass that into numpy. Tha

Re: [Numpy-discussion] Drop LGTM testing.

2021-08-14 Thread Eric Wieser
On Sat, Aug 14, 2021 at 2:35 PM Eric Wieser > wrote: > >> This might be worth creating a github issue for simply so we can tag >> someone working at LGTM; they've been helpful in the past, and it's >> possible we just need to fiddle with some configuration to make i

Re: [Numpy-discussion] Drop LGTM testing.

2021-08-14 Thread Eric Wieser
This might be worth creating a github issue for simply so we can tag someone working at LGTM; they've been helpful in the past, and it's possible we just need to fiddle with some configuration to make it work. It's also worth noting that LGTM runs against C code too; so even if we disable it for p

Re: [Numpy-discussion] copy="never" discussion and no deprecation cycle?

2021-06-21 Thread Eric Wieser
Stefan, that sketch is more complicated than it needs to be - `np.copy` is a python function, so you can just attach the attributes directly! (although maybe there are implications for static typing) ``` class CopyFlag(enum.Enum): IF_NEEDED = 0 ALWAYS = 1 NEVER = 2 np.copy.IF_NEEDED =

Re: [Numpy-discussion] copy="never" discussion and no deprecation cycle?

2021-06-16 Thread Eric Wieser
I agree with Stephan, but even 3 seems dangerous to me. Any code that wraps a numpy function and accepts a `copy` parameter (especially `__array_function__`) is likely to contain `if copy` somewhere, which would result in entirely (but likely silently) the wrong behavior for `copy="never"`. An impo

Re: [Numpy-discussion] linalg.det for fractions

2021-05-16 Thread Eric Wieser
Numpy implements linalg.det by going through LAPACK, which only knows about f4, f8, c8, and c16 data types. Your request amounts to wanting an `O` dtype implementation. I think this is a totally reasonable request as we already have such an implementation for `np.matmul`; but it won't be particula

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

2021-05-15 Thread Eric Wieser
on in another message in the next few days. On Thu, 13 May 2021 at 17:06, eliaskoromilas wrote: > Eric Wieser wrote > >> Yes, sorry, had been a while since I had looked it up: > >> > >> https://docs.python.org/3/c-api/memory.html#c.PyMemAllocatorEx > > > &g

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

2021-05-11 Thread Eric Wieser
rity isn't the only motivation - I was considering compatibility. Consider a user who's already written a shiny stateful C++ allocator, and wants to use it with numpy. I've made a gist at https://gist.github.com/eric-wieser/6d0fde53fc1ba7a2fa4ac208467f2ae5 which demonstrates how to hook a

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

2021-05-10 Thread Eric Wieser
> The Python version of this does have a `void *ctx`, but I am not sure if the use for this is actually valuable for the NumPy use-cases. Do you mean "the CPython version"? If so, can you link a reference? > While I like the `PyObject *` idea, I am also not sure that it helps much. If we want a

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

2021-05-06 Thread Eric Wieser
store pointers to allocators this doesn't matter. Eric On Thu, 6 May 2021 at 12:43, Matti Picus wrote: > > On 6/5/21 2:07 pm, Eric Wieser wrote: > > The NEP looks good, but I worry the API isn't flexible enough. My two > > main concerns are: > > > > ###

Re: [Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

2021-05-06 Thread Eric Wieser
The NEP looks good, but I worry the API isn't flexible enough. My two main concerns are: ### Stateful allocators Consider an allocator that aligns to `N` bytes, where `N` is configurable from a python call in someone else's extension module. Where do they store `N`? They can hide it in `PyDataMe

Re: [Numpy-discussion] Is there a defined way to "unpad" an array, and if not, should there be?

2021-04-13 Thread Eric Wieser
Some other options here that avoid the need for a new function: * Add a `return_view` argument to `pad`, such that for `padded, orig = np.pad(arr, ..., return_view=True)`, `orig == arr` and `orig.base is padded`. This is useful if `padded` is modified in place, but less useful otherwise. It has th

Re: [Numpy-discussion] MAINT: Use of except-pass blocks

2021-04-06 Thread Eric Wieser
I think the add_docstring one is best left alone, since if it fails once it will probably fail for every docstring in the system, and the logging would be pure noise. The ma/core one is suspicious - I can't think of any examples where an error would occur, but if you're interested, I'd encourage y

Re: [Numpy-discussion] Programmatically contracting multiple tensors

2021-03-12 Thread Eric Wieser
Einsum has a secret integer argument format that appears in the Examples section of the `np.einsum` docs, but appears not to be mentioned at all in the parameter listing. Eric On Sat, 13 Mar 2021 at 00:25, Michael Lamparski wrote: > Greetings, > > I have something in my code where I can receive

Re: [Numpy-discussion] ENH: Proposal to add atleast_nd function

2021-02-12 Thread Eric Wieser
t; in a positive direction and I’d prefer it to what’s there now: > > In [1]: import numpy as np > In [2]: np.atleast_3d(np.ones(4)).shape > Out[2]: (1, 4, 1) > > There might be some linear algebraic reason why those axis positions make > sense, but I’m not aware of it... > &

Re: [Numpy-discussion] ENH: Proposal to add atleast_nd function

2021-02-11 Thread Eric Wieser
I did a quick search of matplotlib, and found a few uses of all three functions: * https://github.com/matplotlib/matplotlib/blob/fed55c63a314351cd39a12783f385009782c06e1/lib/matplotlib/_layoutgrid.py#L441-L446 This one isn't really numpy at all, and is really just a shorthand for normalizing an

Re: [Numpy-discussion] ENH: Proposal to add atleast_nd function

2021-02-11 Thread Eric Wieser
> I find that the at_least{1,2,3}d functions are useful for sanitizing inputs IMO, this type of "sanitization" goes against "In the face of ambiguity, refuse the temptation to guess". Instead of using `at_least{n}d`, it could be argued that `if np.ndim(x) != n: raise ValueError` is a safer bet, wh

Re: [Numpy-discussion] np.{bool,float,int} deprecation

2020-12-11 Thread Eric Wieser
> you might want to discuss this with us at the array API standard > https://github.com/data-apis/array-api (which is currently in RFC > stage). The spec uses bool as the name for the boolean dtype. I don't fully understand this argument - `np.bool` is already not the boolean dtype. Either: * Th

Re: [Numpy-discussion] np.{bool,float,int} deprecation

2020-12-07 Thread Eric Wieser
If the CI noise in downstream libraries is particularly painful, we could switch to `PendingDeprecationWarning` instead of `DeprecationWarning` to make it easier to add the warnings to an ignore list. I think this might make the warning less visible to end users though, who are the users that this

Re: [Numpy-discussion] datetime64: Remove deprecation warning when constructing with timezone

2020-11-05 Thread Eric Wieser
Without weighing in yet on how I feel about the deprecation, you can see some discussion about why this was originally deprecated in the PR that introduced the warning: https://github.com/numpy/numpy/pull/6453 Eric On Thu, Nov 5, 2020, 20:13 Noam Yorav-Raphael wrote: > Hi, > > I suggest removi

Re: [Numpy-discussion] Request for comments on PEP 637 - Support for indexing with keyword arguments

2020-09-25 Thread Eric Wieser
I agree with Stephan's suggestion of having no default value for positional indices, and letting the user supply it. It seems I replied badly to the mail on the python-ideas list, and my response ended up as a separate thread, at https://mail.python.org/archives/list/python-id...@python.org/thread

Re: [Numpy-discussion] number datetime64 dtypes

2020-09-10 Thread Eric Wieser
It's interesting to confirm that people are aware of this syntax! This is intended but perhaps not useful behavior. `datetime64[15D]` is a type that stores dates by the nearest date that is a multiple of 15 days from the unix epoch. Arguably there isn't a situation where using `15D` makes a whole

Re: [Numpy-discussion] [Feature Request] Add alias of np.concatenate as np.concat

2020-06-06 Thread Eric Wieser
I agree with all of Ralf's points, except for perhaps this one: > I also don't see a reason to conform to TensorFlow (or PyTorch, or Matlab, or whichever other library) Python itself has a name for this function, `operator.concat` - so _maybe_ this sets a strong enough precedent for us to add an

Re: [Numpy-discussion] log of negative real numbers -> RuntimeWarning: invalid value encountered in log

2020-05-25 Thread Eric Wieser
One explanation for this behavior is that doing otherwise would be slow. Consider an array like arr = np.array([1]*10**6 + [-1]) ret = np.log(arr) Today, what happens is: - The output array is allocated as np.double - The input array is iterated over, and log evaluated on each element in

Re: [Numpy-discussion] Using nditer + external_loop to Always Iterate by Column

2020-05-19 Thread Eric Wieser
Hi Will, To force an iteration to run along certain axes, I believe you should be using `op_axes`. Your diagnosis is correct that `external_loop` is trying to help you be more optimal, since it's purpose is exactly that: optimization. Unfortunately, if you use `op_axes` you'll run into https://gi

Re: [Numpy-discussion] Deprecate Promotion of numbers to strings?

2020-04-30 Thread Eric Wieser
> Another larger visible change will be code such as: > > np.concatenate([np.array(["string"]), np.array([2])]) > > will result in an error instead of returning a string array. (Users > will have to cast manually here.) I wonder if we can lessen the blow by allowing `np.concatenate([np.array([

Re: [Numpy-discussion] Proposal: add `force=` or `copy=` kwarg to `__array__` interface

2020-04-24 Thread Eric Wieser
Perhaps worth mentioning that we've discussed this sort of API before, in https://github.com/numpy/numpy/pull/11897. Under that proposal, the api would be something like: * `copy=True` - always copy, like it is today * `copy=False` - copy if needed, like it is today * `copy=np.never_copy` - never

Re: [Numpy-discussion] Is `numpy.lib.shape_base.normalize_axis_index` considered part of the public API?

2020-04-06 Thread Eric Wieser
he > > pull > > request in which the function was created is > > https://github.com/numpy/numpy/pull/8584. Whether or not the function > > was to be public is discussed starting here: > > https://github.com/numpy/numpy/pull/8584#issuecomment-28117939

[Numpy-discussion] Deprecating ndarray.tostring()

2020-03-30 Thread Eric Wieser
Hi all, Just a heads up that in https://github.com/numpy/numpy/pull/15867 I plan to deprecate ndarray.tostring(), which is just a confusing way to spell ndarray.tobytes(). This function has been documented as a compatibility alias since NumPy 1.9, but never emitted a warning upon use. Given array

Re: [Numpy-discussion] Put type annotations in NumPy proper?

2020-03-24 Thread Eric Wieser
> Putting > aside ndarray, as more challenging, even annotations for numpy functions > and method parameters with built-in types would help, as a start. This is a good idea in principle, but one thing concerns me. If we add type annotations to numpy, does it become an error to have numpy-stubs i

Re: [Numpy-discussion] NEP 37: A dispatch protocol for NumPy-like modules

2020-02-05 Thread Eric Wieser
> scipy.linalg is a superset of numpy.linalg This isn't completely accurate - numpy.linalg supports almost all operations* over stacks of matrices via gufuncs, but scipy.linalg does not appear to. Eric *: not lstsq due to an ungeneralizable public API On Wed, 5 Feb 2020 at 17:38, Ralf Gommers

Re: [Numpy-discussion] Adding an nd generalization of np.ma.mask_rowscols

2020-01-17 Thread Eric Wieser
refer a more functional approach: Where we take in an input 1-D or N-D > boolean array in addition to a masked array with multiple axes over which > to extend the mask. > > > > *From: *NumPy-Discussion gmail@python.org> on behalf of Eric Wieser < > wieser.eric+nu...@gmail.com

[Numpy-discussion] Adding an nd generalization of np.ma.mask_rowscols

2020-01-17 Thread Eric Wieser
Today, numpy has a np.ma.mask_rowcols function, which stretches masks along the full length of an axis. For example, given the matrix:: >>> a2d = np.zeros((3, 3), dtype=int) >>> a2d[1, 1] = 1 >>> a2d = np.ma.masked_equal(a2d, 1) >>> print(a2d) [[0 0 0] [0 -- 0] [0 0 0]] The API allows:: >>> pr

Re: [Numpy-discussion] argmax() indexes to value

2019-11-01 Thread Eric Wieser
> On my system plain fancy indexing is fastest Hardly surprising, since take_along_axis is doing that under the hood, after constructing the index for you :) https://github.com/numpy/numpy/blob/v1.17.0/numpy/lib/shape_base.py#L58-L172 I deliberately didn't expose the internal function that const

Re: [Numpy-discussion] Problem with np.savetxt

2019-10-10 Thread Eric Wieser
You're trying to read a file with a name of literally `${d}.log`, which is unlikely to be the name of your file. `${}` is bash syntax, not python syntax. This has drifted out of numpy territory and into "how to coordinate between bash and python" territory - I'd perhaps recommend you ask this to a

Re: [Numpy-discussion] Forcing gufunc to error with size zero input

2019-09-28 Thread Eric Wieser
Can you just raise an exception in the gufuncs inner loop? Or is there no mechanism to do that today? I don't think you were proposing that core dimensions should _never_ be allowed to be 0, but if you were I disagree. I spent a fair amount of work enabling that for linalg because it provided some

Re: [Numpy-discussion] NEP 31 — Context-local and global overrides of the NumPy API

2019-09-09 Thread Eric Wieser
> In other words `np.arange(100)` (but with a completely different syntax, probably hidden away only for libraries to use). It sounds an bit like you're describing factory classmethods there. Is the solution to this problem to move (leaving behind aliases) `np.arange` to `ndarray.arange`, `np.zero

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

2019-06-25 Thread Eric Wieser
One other approach here that perhaps treads a little too close to np.matrix: class MatrixOpWrapper: def __init__(self, arr): # todo: accept axis arguments here? self._array = arr # todo: assert that arr.ndim >= 2 / call atleast1d @property def T(self): return linalg.t

Re: [Numpy-discussion] new MaskedArray class

2019-06-23 Thread Eric Wieser
I think we’d need to consider separately the operation on the mask and on the data. In my proposal, the data would always do np.sum(array, where=~mask), while how the mask would propagate might depend on the mask itself, I quite like this idea, and I think Stephan’s strawman design is actually pla

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

2019-06-23 Thread Eric Wieser
gt; > > > However, we will need some way to filter for intent, as the > > > people > > > > who write this code are the ones who didn’t read docs on it at > > > the > > > > time, and so there might be a fair amount of noise even if it > > > fi

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

2019-06-23 Thread Eric Wieser
This might be contentious, but I wonder if, with a long enough deprecation cycle, we can change the meaning of .T. That would look like: * Emit a future warning on `more_than_2d.T` with a message like "in future .T will transpose just the last two dimensions, not all dimensions. Use are.transpose(

Re: [Numpy-discussion] New release note strategy after branching 1.17.

2019-06-12 Thread Eric Wieser
It's worth linking to the issue where this discussion started, so we avoid repeating ourselves - https://github.com/numpy/numpy/issues/13707. Eric On Wed, Jun 12, 2019, 11:51 Nathaniel Smith wrote: > It might be worth considering a tool like 'towncrier'. It's automation to > support the workfl

Re: [Numpy-discussion] defining a NumPy API standard?

2019-06-02 Thread Eric Wieser
Some of your categories here sound like they might be suitable for ABCs that provide mixin methods, which is something I think Hameer suggested in the past. Perhaps it's worth re-exploring that avenue. Eric On Sat, Jun 1, 2019, 18:18 Marten van Kerkwijk wrote: > > Our API is huge. A simple coun

Re: [Numpy-discussion] Style guide for numpy code?

2019-05-09 Thread Eric Wieser
Joe, While most of your style suggestions are reasonable, I would actually recommend the opposite of the first point you make in (a)., especially if you're trying to write generic reusable code. > For example, an item count is always an integer, but a distance is always a > float. This is close

Re: [Numpy-discussion] Boolean arrays with nulls?

2019-04-18 Thread Eric Wieser
One option here would be to use masked arrays: arr = np.ma.zeros(3, dtype=bool) arr[0] = True arr[1] = False arr[2] = np.ma.masked giving masked_array(data=[True, False, --], mask=[False, False, True], fill_value=True) On Thu, 18 Apr 2019 at 10:51, Stuart Reynolds wrote: >

Re: [Numpy-discussion] Porting code for Numpy 1.13+ to get rid of "boolean index did not match indexed array along dimension 1" error

2019-02-12 Thread Eric Wieser
It looks like your code is wrong, and numpy 1.12 happened to let you get away with it This line: evals = evals[evals > tolerance] Reduces the eigenvalues to only those which are greater than the tolerance When you do U[:, evals > tolerance], evals > tolerance is just going to be [True, True, ..

Re: [Numpy-discussion] py3k.os_fspath enforces inconsist rules with python3 os.fspath

2019-01-14 Thread Eric Wieser
This looks like a bug to me - can you file it on the issue tracker. Evidently I did not consider python 2 behavior when backporting `os.fspath` from python 3. Eric On Mon, 14 Jan 2019 at 16:28 Stuart Reynolds wrote: > After a recent upgrade of numpy (1.13.1 -> 1.6.0), my code is failing > wher

Re: [Numpy-discussion] Add guaranteed no-copy to array creation and reshape?

2019-01-12 Thread Eric Wieser
I don’t think a NEVERCOPY entry in arr.flags would make much sense. Is this really a sensible limitation to put on how data gets used? Isn’t it up to the algorithm to decide whether to copy its data, not the original owner of the data? It also leads to some tricky questions of precedence - would n

Re: [Numpy-discussion] Add guaranteed no-copy to array creation and reshape?

2019-01-09 Thread Eric Wieser
could go for the more heavy-handed np.CopyMode.NEVER, which still has a unique enough case for name clashes with functions never to be an issue. Eric ​ On Wed, 9 Jan 2019 at 22:25 Ralf Gommers wrote: > On Mon, Jan 7, 2019 at 11:30 AM Eric Wieser > wrote: > >> >> @Ralf &g

Re: [Numpy-discussion] timedelta64 remainder behavior with div by 0

2019-01-08 Thread Eric Wieser
If we consider it a bug, we could patch it in 1.16.1 (or are we still waiting on 1.16.0?), which would minimize the backwards compatibility cost. Eric On Tue, 8 Jan 2019 at 10:05 Stefan van der Walt wrote: > On Tue, 08 Jan 2019 09:57:03 -0800, Tyler Reddy wrote: > > np.timedelta64(5) % np.timed

Re: [Numpy-discussion] Add guaranteed no-copy to array creation and reshape?

2019-01-07 Thread Eric Wieser
@Matthias: Most of the time I would not assign to arr.shape, but in some rare occasions I find it very useful. And one of those rare occasions is when you want guaranteed no-copy behavior. Can you come up with any other example? The only real argument you seem to have here is “my code uses arr.s

[Numpy-discussion] Adding more detailed exception types to numpy

2019-01-04 Thread Eric Wieser
PR #12593 adds a handful of new exception types for . Some consequences of this change are that: 1. The string formatting is moved to python, allowing us to give better error messages without a lot of work 2. The formatting is dispatched lazily,

Re: [Numpy-discussion] Add guaranteed no-copy to array creation and reshape?

2018-12-27 Thread Eric Wieser
I think np.never_copy is really ugly. I’d much rather simply use ‘never’, I actually think np.never_copy is the better choice of the two. Arguments for it: - It’s consistent with np.newaxis in spelling (modulo the _) - If mistyped, it can be caught easily by IDEs. - It’s typeable with my

Re: [Numpy-discussion] three-dim array

2018-12-26 Thread Eric Wieser
; to get accepted this way. > > On Wed, Dec 26, 2018 at 1:59 AM Eric Wieser > wrote: > >> In the latest version of numpy, this runs without an error, although may >> or may not be what you want: >> >> In [1]: np.array([[1,2],[[1,2],[3,4]]]) >> Out[1]: >

Re: [Numpy-discussion] three-dim array

2018-12-25 Thread Eric Wieser
In the latest version of numpy, this runs without an error, although may or may not be what you want: In [1]: np.array([[1,2],[[1,2],[3,4]]]) Out[1]: array([[1, 2], [list([1, 2]), list([3, 4])]], dtype=object) Here the result is a 2x2 array, where some elements are numbers and others are l

Re: [Numpy-discussion] lstsq underdetermined behaviour

2018-11-18 Thread Eric Wieser
> In 1.15 the call is instead to `_umath_linalg.lstsq_m` and I'm not sure what this actually ends up doing - does this end up being the same as `dgelsd`? When the arguments are real, yes. What changed is that the dispatching now happens in C, which was done as a step towards the incomplete https:/

Re: [Numpy-discussion] Vectorized version of numpy.linspace

2018-11-14 Thread Eric Wieser
I too buy into axis=0 being the better default here for broadcasting reasons. Having it possible to specify explicitly would be handy though, for something like: x_ramped = np.linspace(x.min(axis=2), x.max(axis=2), 100, axis=2) ​ On Wed, 14 Nov 2018 at 15:20 Stephan Hoyer wrote: > On Wed, Nov

Re: [Numpy-discussion] asarray/anyarray; matrix/subclass

2018-11-10 Thread Eric Wieser
> If the only way MaskedArray violates Liskov is in terms of NA skipping aggregations by default, then this might be viable One of the ways to fix these liskov substitution problems is just to introduce more base classes - for instance, if we had an `NDContainer` base class with only slicing suppo

Re: [Numpy-discussion] numpy pprint?

2018-11-06 Thread Eric Wieser
Mark to show L R U P or combination of these >plus some numbers (similar to Pandas .head .tail) methods and then show the >rest by unicod 3dot > > P.S. I had no idea our university Microsoft services also offers Azure > Notebooks awesome :P > > F. > > On

Re: [Numpy-discussion] numpy pprint?

2018-11-06 Thread Eric Wieser
)) > for IPython/Jupyter using Markdown/LaTeX would be awesome > or even better using HTML to add sliders just like Pandas... > > F. > > On Tue, Nov 6, 2018 at 6:51 AM Eric Wieser > wrote: > >> Hijacking this thread while on the topic of pprint - we might want to >&

Re: [Numpy-discussion] numpy pprint?

2018-11-05 Thread Eric Wieser
Hijacking this thread while on the topic of pprint - we might want to look into a table-based `_html_repr_` or `_latex_repr_` for use in ipython - where we can print the full array and let scrollbars replace ellipses. Eric On Mon, 5 Nov 2018 at 21:11 Mark Harfouche wrote: > Foad, > > Visualizin

Re: [Numpy-discussion] Attribute hiding APIs for PyArrayObject

2018-10-30 Thread Eric Wieser
In NumPy 1.14 we changed UPDATEIFCOPY to WRITEBACKIFCOPY, and in 1.16 we would like to deprecate PyArray_SetNumericOps and PyArray_GetNumericOps. The strange warning when NPY_NO_DEPRICATED_API is annoying I’m not sure I make the connection here between hidden fields and API deprecation. You seem t

Re: [Numpy-discussion] asanyarray vs. asarray

2018-10-29 Thread Eric Wieser
The latter - changing the behavior of multiplication breaks the principle. But this is not the main reason for deprecating matrix - almost all of the problems I’ve seen have been caused by the way that matrices behave when sliced. The way that m[i][j] and m[i,j] are different is just one example o

Re: [Numpy-discussion] Depreciating asfortranarray and ascontiguousarray

2018-10-26 Thread Eric Wieser
in order to be used prior to calling C or Fortran code that expected at least a 1-d array I’d argue that the behavior for these functions should have just been to raise an error saying “this function does not support 0d arrays”, rather than silently inserting extra dimensions. As a bonus, that wou

Re: [Numpy-discussion] asanyarray vs. asarray

2018-10-19 Thread Eric Wieser
have or want such a concept as "soft support". We >>> intend to not break anything that now has asanyarray, i.e. it's supported >>> and ideally we have regression tests for all such functions. For anything >>> we transition over from asarray to asanyarray

Re: [Numpy-discussion] ndrange, like range but multidimensiontal

2018-10-11 Thread Eric Wieser
t; should they need to. > > I'm not sure if we ever want the `ndrange` object to return a full matrix. > It seems like we would be creating a custom tuple class just for this which > seems pretty niche. > > > On Thu, Oct 11, 2018 at 10:21 AM Eric Wieser > wrote: > &

Re: [Numpy-discussion] ndrange, like range but multidimensiontal

2018-10-11 Thread Eric Wieser
/18 12:34 AM, Eric Wieser wrote: > > One thing that worries me here - in python, |range(...)| in essence > > generates a lazy |list| - so I’d expect |ndrange| to generate a lazy > > |ndarray|. In practice, that means it would be a duck-type defining an > > |__array__| m

Re: [Numpy-discussion] ndrange, like range but multidimensiontal

2018-10-11 Thread Eric Wieser
valid indices >>> m[tuple(ind)] 1.0 On Wed, 10 Oct 2018 at 09:08 Stephan Hoyer sho...@gmail.com <http://mailto:sho...@gmail.com> wrote: On Tue, Oct 9, 2018 at 9:34 PM Eric Wieser > wrote: > >> One thing that worries me here - in python, range(...) in essence >> gene

Re: [Numpy-discussion] ndrange, like range but multidimensiontal

2018-10-09 Thread Eric Wieser
One thing that worries me here - in python, range(...) in essence generates a lazy list - so I’d expect ndrange to generate a lazy ndarray. In practice, that means it would be a duck-type defining an __array__ method to evaluate it, and only implement methods already present in numpy. It’s not cle

Re: [Numpy-discussion] Adding a hex version like PY_VERSION_HEX

2018-10-09 Thread Eric Wieser
+1 on Ralf's suggestion. I'm not sure there's any case where the C code should be using a hex version number - either it's using the C api, in which case it should just be looking at the C api version - or it's calling back into the python API, in which case it's probably not unreasonable to ask it

Re: [Numpy-discussion] C99

2018-09-07 Thread Eric Wieser
Thanks for the first step on this! Should we allow // style comments I don’t think it matters too much. I think it might be a little messy to have a mix of the two styles where // means “post py3” and /* */ means pre-py3 - but at the same time, I do slightly prefer the C++-style. For C contributo

Re: [Numpy-discussion] Possible Deprecation of np.ediff1d

2018-08-27 Thread Eric Wieser
There is already a patch to add such a feature to np.diff at https://github.com/numpy/numpy/pull/8206 ​ On Mon, 27 Aug 2018 at 10:47 Charles R Harris wrote: > On Mon, Aug 27, 2018 at 11:37 AM Robert Kern > wrote: > >> On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy >> wrote: >> >>> Chuck suggeste

Re: [Numpy-discussion] pytest, fixture and parametrize

2018-08-08 Thread Eric Wieser
Is another nice feature You forget that we already have that feature in our testing layer, with np.testing.assert_raises(AnException): pass ​ On Wed, 8 Aug 2018 at 09:08 Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > BTW: > > with pytest.raises(AnException): > > > I

Re: [Numpy-discussion] Polynomial evaluation inconsistencies

2018-07-01 Thread Eric Wieser
is sound like a good idea to anyone? > > Max > > > On Sat, Jun 30, 2018 at 6:47 PM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> >> >> On Sat, Jun 30, 2018 at 4:42 PM, Charles R Harris < >> charlesr.har...@gmail.com> wrote: &

Re: [Numpy-discussion] Polynomial evaluation inconsistencies

2018-06-30 Thread Eric Wieser
t shouldn't it call >> the constructor with Polynomial([0,1])? >> >> On Sat, Jun 30, 2018 at 5:41 PM, Eric Wieser > > wrote: >> >>> Since the one of the arguments for the decreasing order seems to just be >>> textual representation - do we want t

Re: [Numpy-discussion] Polynomial evaluation inconsistencies

2018-06-30 Thread Eric Wieser
Eric Wieser wrote: > “the intuitive way” is the decreasing powers. > > An argument against this is that accessing the ith power of x is spelt: > >- x.coeffs[i] for increasing powers >- x.coeffs[-i-1] for decreasing powers > > The former is far more natural than

Re: [Numpy-discussion] Polynomial evaluation inconsistencies

2018-06-30 Thread Eric Wieser
“the intuitive way” is the decreasing powers. An argument against this is that accessing the ith power of x is spelt: - x.coeffs[i] for increasing powers - x.coeffs[-i-1] for decreasing powers The former is far more natural than the latter, and avoids a potential off-by-one error If I ask

Re: [Numpy-discussion] Polynomial evaluation inconsistencies

2018-06-30 Thread Eric Wieser
more sense for np.polyval() to use conventional indexing (c[0] > * x^0 + c[1] * x^1 + c[2] * x^2). np.polyval() can be convenient when a > polynomial object is just not needed, but if a single program uses both > np.polyval() and np.polynomail.Polynomial, it seems bound to cause > unnecessary c

Re: [Numpy-discussion] Polynomial evaluation inconsistencies

2018-06-29 Thread Eric Wieser
Here's my take on this, but it may not be an accurate summary of the history. `np.poly` is part of the original matlab-style API, built around `poly1d` objects. This isn't a great design, because they represent: p(x) = c[0] * x^2 + c[1] * x^1 + c[2] * x^0 For this reason, among others, the `

Re: [Numpy-discussion] Revised NEP-18, __array_function__ protocol

2018-06-29 Thread Eric Wieser
- irrespective of the resolution of the discussion on python-dev. Eric ​ On Fri, 29 Jun 2018 at 18:24 Stephan Hoyer wrote: > On Thu, Jun 28, 2018 at 5:36 PM Eric Wieser > wrote: > >> Another option would be to directly compare the methods against known >> ones: >> &

Re: [Numpy-discussion] Revised NEP-18, __array_function__ protocol

2018-06-28 Thread Eric Wieser
Another option would be to directly compare the methods against known ones: obj = func.__self__ if isinstance(obj, np.ufunc): if func is obj.reduce: got_reduction() Eric ​ On Thu, 28 Jun 2018 at 17:19 Stephan Hoyer wrote: > On Thu, Jun 28, 2018 at 1:12 PM Marten van Kerkwijk < > m.

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

2018-06-26 Thread Eric Wieser
> Hameer Abbasi > > Sent from Astro for Mac > > > > > On 26. Jun 2018 at 09:46, Robert Kern > > > wrote: > > > > > > On Tue, Jun 26, 2018 at 12:13 AM Eric Wieser > > il.com> wrote: > > > > > I don't think it shoul

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

2018-06-26 Thread Eric Wieser
Another thing I’d say is arr.?index should be replaced with arr.?idx. Or perhaps arr.o_[] and arr.v_[], to match the style of our existing np.r_, np.c_, np.s_, etc? ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mai

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

2018-06-26 Thread Eric Wieser
> I don't think it should be relegated to the "officially discouraged" ghetto of `.legacy_index` The way I read it, the new spelling lof that would be the explicit but not discouraged `image.vindex[rr, cc]`. > I would reserve warnings for the cases where the current behavior is something no one r

Re: [Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

2018-06-25 Thread Eric Wieser
Generally +1 on this, but I don’t think we need To ensure that existing subclasses of ndarray that override indexing do not inadvertently revert to default behavior for indexing attributes, these attribute should have explicit checks that disable them if __getitem__ or __setitem__ has been overrid

Re: [Numpy-discussion] Remove sctypeNA and typeNA from numpy core

2018-06-21 Thread Eric Wieser
> I bet the NA is for the missing value support thatnever happened Nope - NA stands for NumArray Eric On Thu, 21 Jun 2018 at 11:07 Sebastian Berg wrote: > On Thu, 2018-06-21 at 09:25 -0700, Matti Picus wrote: > > numpy.core has many ways to catalogue dtype names: sctypeDict, > > typeDict > > (

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

2018-06-12 Thread Eric Wieser
I don’t understand your alternative here. If we overload np.matmul using *array_function*, then it would not use *ether* of these options for writing the operation in terms of other gufuncs. It would simply look for an *array_function* attribute, and call that method instead. Let me explain that s

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

2018-06-11 Thread Eric Wieser
Frozen dimensions: I started with just making every 3-vector and 3x3-matrix structured arrays with the relevant single sub-array entry I was actually suggesting omitting the structured dtype (ie, field names) altogether, and just using the subarray dtypes (which exist alone, but not in arrays).

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

2018-06-10 Thread Eric Wieser
g a reduction with a broadcasting ufunc? Eric On Sun, 10 Jun 2018 at 16:02 Eric Wieser wrote: Rendered here: > https://github.com/mhvk/numpy/blob/nep-gufunc-signature-enhancement/doc/neps/nep-0020-gufunc-signature-enhancement.rst > > > Eric > > On Sun, 10 Jun 2018 at 09:3

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

2018-06-10 Thread Eric Wieser
Rendered here: https://github.com/mhvk/numpy/blob/nep-gufunc-signature-enhancement/doc/neps/nep-0020-gufunc-signature-enhancement.rst Eric On Sun, 10 Jun 2018 at 09:37 Marten van Kerkwijk wrote: > OK, I spent my Sunday morning writing a NEP. I hope this can lead to some > closure... > See http

Re: [Numpy-discussion] NEP: Random Number Generator Policy

2018-06-03 Thread Eric Wieser
You make a bunch of good points refuting reproducible research as an argument for not changing the random number streams. However, there’s a second use-case you don’t address - unit tests. For better or worse, downstream, or even our own

[Numpy-discussion] Adding take_along_axis and put_along_axis functions

2018-05-28 Thread Eric Wieser
These functions provide a vectorized way of using one array to look up items in another. In particular, they extend the 1d: a = np.array([4, 5, 6, 1, 2, 3]) b = np.array(["four", "five", "six", "one", "two", "three"]) i = a.argsort() b_sorted = b[i] To work for higher-dimensions: a = np.array([[

Re: [Numpy-discussion] matmul as a ufunc

2018-05-28 Thread Eric Wieser
which ensure that it is still well defined (as the identity) on 1d arrays. This strikes me as a bad idea. There’s already enough confusion from beginners that array_1d.T is a no-op. If we introduce a matrix-transpose, it should either error on <1d inputs with a useful message, or insert the extra

Re: [Numpy-discussion] Matrix opreation

2018-05-22 Thread Eric Wieser
I’d recommend asking this kind of question on stackoverflow in future, but you can do that with: b = (a .reshape((2, 2, 4, 4)) # split up the (4,) axis into (2, 2) .transpose((2, 0, 3, 1)) # reorder to (4, 2, 4, 2) .reshape((8, 8)) # collapse adjacent dimensions ) ​ On Tue,

Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions

2018-04-30 Thread Eric Wieser
I think I’m -1 on this - this just makes things harder on the implementers of _array_ufunc__ who now might have to work out which signature matches. I’d prefer the solution where np.matmul is a wrapper around one of three gufuncs (or maybe just around one with axis insertion) - this is similar to h

Re: [Numpy-discussion] numpy.pad -- problem?

2018-04-29 Thread Eric Wieser
I would consider this a bug, and think we should fix this. On Sun, 29 Apr 2018 at 13:48 Virgil Stokes wrote: > Here is a python code snippet: > > # python vers. 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC > v.1900 64 bit (AMD64)] > import numpy as np # numpy vers. 1.14.3 > #import mat

Re: [Numpy-discussion] Changing the return type of np.histogramdd

2018-04-27 Thread Eric Wieser
Apr 2018 at 22:26 Ralf Gommers wrote: > On Wed, Apr 25, 2018 at 11:00 PM, Eric Wieser > wrote: > >> For precision loss of the order of float64 eps, I disagree. >> >> I was thinking more about precision loss on the order of 1, for large >> 64-bit i

Re: [Numpy-discussion] Changing the return type of np.histogramdd

2018-04-25 Thread Eric Wieser
scipy functions from the basic numpy ones? ​ On Wed, 25 Apr 2018 at 22:51 Ralf Gommers wrote: > On Wed, Apr 25, 2018 at 10:07 PM, Eric Wieser > wrote: > >> what does that gain over having the user do something like result.astype() >> >> It means that the user can use integ

Re: [Numpy-discussion] Changing the return type of np.histogramdd

2018-04-25 Thread Eric Wieser
aw too, but I suppose that can be dealt with separately. Eric ​ On Wed, 25 Apr 2018 at 21:57 Ralf Gommers wrote: > On Mon, Apr 9, 2018 at 10:24 PM, Eric Wieser > wrote: > >> Numpy has three histogram functions - histogram, histogram2d, and >> histogramdd. >> >> h

Re: [Numpy-discussion] Adding a return value to np.random.shuffle

2018-04-12 Thread Eric Wieser
I’m against this change, because it: - Is inconsistent with the builtin random.shuffle - Makes it easy to fall into the trap of assuming that np.random.shuffle does not mutate it’s input Eric ​ On Thu, 12 Apr 2018 at 10:37 Joseph Fox-Rabinovitz wrote: > Would it break backwards compat

  1   2   >