[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
Maybe I wasn't clear, I'm talking about the 1-dimensional vector product, but applied to N-D arrays of vectors. Certainly dot products can be realized as matrix products, and often are in mathematics for convenience, but matrices and vectors are not the same thing, theoretically or coding wise. If I have two (M, N, k) arrays a and b where k is the vector dimension, to dot product them using matrix notation I have to do: (a[:, :, np.newaxis, :] @ b[:, :, :, np.newaxis])[:, :, 0, 0] Which I certainly don't find readable (I always have to scratch my head a little bit to figure out whether the newaxis's are in the right places). If this is a common operation in larger expressions, then it basically has to be written as a separate function, which then someone reading the code may have to look at for the semantics. It also breaks down if you want to write generic vector functions that may be applied along different axes; then you need to do something like np.squeeze(np.expand_dims(a, axis=axis) @ np.expand_dims(b, axis=axis+1), (axis, axis+1)) (after normalizing the axis; if it's negative you'd need to do axis-1 and axis instead). Compare this to the simplicity, composability and consistency of: a.dot(b, axis=-1) * np.cross(c, d, axis=-1).dot(e, axis=-1) / np.linalg.norm(f, axis=-1) (the cross and norm operators already support an axis parameter) ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
On Tue, Jul 5, 2022 at 5:49 PM Aaron Meurer wrote: > > The vecdot() function in the array API should be what you are looking > for (note that the current implementation in numpy.array_api is > incorrect, which I'm fixing at > https://github.com/numpy/numpy/pull/21928). It works like dot() but it > always applies the 1-D dot product case with broadcasting, and lets > you specify the axis. We'd want this function to be added to the main > numpy namespace as well. See https://data-apis.org/array-api/latest/API_specification/generated/signatures.linear_algebra_functions.vecdot.html Aaron Meurer > > Aaron Meurer > > On Tue, Jul 5, 2022 at 5:39 PM wrote: > > > > Maybe I wasn't clear, I'm talking about the 1-dimensional vector product, > > but applied to N-D arrays of vectors. Certainly dot products can be > > realized as matrix products, and often are in mathematics for convenience, > > but matrices and vectors are not the same thing, theoretically or coding > > wise. If I have two (M, N, k) arrays a and b where k is the vector > > dimension, to dot product them using matrix notation I have to do: > > > > (a[:, :, np.newaxis, :] @ b[:, :, :, np.newaxis])[:, :, 0, 0] > > > > Which I certainly don't find readable (I always have to scratch my head a > > little bit to figure out whether the newaxis's are in the right places). If > > this is a common operation in larger expressions, then it basically has to > > be written as a separate function, which then someone reading the code may > > have to look at for the semantics. It also breaks down if you want to write > > generic vector functions that may be applied along different axes; then you > > need to do something like > > > > np.squeeze(np.expand_dims(a, axis=axis) @ np.expand_dims(b, axis=axis+1), > > (axis, axis+1)) > > > > (after normalizing the axis; if it's negative you'd need to do axis-1 and > > axis instead). > > > > Compare this to the simplicity, composability and consistency of: > > > > a.dot(b, axis=-1) * np.cross(c, d, axis=-1).dot(e, axis=-1) / > > np.linalg.norm(f, axis=-1) > > > > (the cross and norm operators already support an axis parameter) > > ___ > > NumPy-Discussion mailing list -- numpy-discussion@python.org > > To unsubscribe send an email to numpy-discussion-le...@python.org > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > Member address: asmeu...@gmail.com ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
The vecdot() function in the array API should be what you are looking for (note that the current implementation in numpy.array_api is incorrect, which I'm fixing at https://github.com/numpy/numpy/pull/21928). It works like dot() but it always applies the 1-D dot product case with broadcasting, and lets you specify the axis. We'd want this function to be added to the main numpy namespace as well. Aaron Meurer On Tue, Jul 5, 2022 at 5:39 PM wrote: > > Maybe I wasn't clear, I'm talking about the 1-dimensional vector product, but > applied to N-D arrays of vectors. Certainly dot products can be realized as > matrix products, and often are in mathematics for convenience, but matrices > and vectors are not the same thing, theoretically or coding wise. If I have > two (M, N, k) arrays a and b where k is the vector dimension, to dot product > them using matrix notation I have to do: > > (a[:, :, np.newaxis, :] @ b[:, :, :, np.newaxis])[:, :, 0, 0] > > Which I certainly don't find readable (I always have to scratch my head a > little bit to figure out whether the newaxis's are in the right places). If > this is a common operation in larger expressions, then it basically has to be > written as a separate function, which then someone reading the code may have > to look at for the semantics. It also breaks down if you want to write > generic vector functions that may be applied along different axes; then you > need to do something like > > np.squeeze(np.expand_dims(a, axis=axis) @ np.expand_dims(b, axis=axis+1), > (axis, axis+1)) > > (after normalizing the axis; if it's negative you'd need to do axis-1 and > axis instead). > > Compare this to the simplicity, composability and consistency of: > > a.dot(b, axis=-1) * np.cross(c, d, axis=-1).dot(e, axis=-1) / > np.linalg.norm(f, axis=-1) > > (the cross and norm operators already support an axis parameter) > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: asmeu...@gmail.com ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
It might be just me, that @ product is way more readable than chaining different operators below that I don't find readable at all but anyways that's taste I guess. Also if you are going to do this, for a better performance code, you shouldn't bend the ops but you should wrangle the array to the correct type so that you end up straightforward array ops. Anyways, nevermind my noise if you are happy with it. On Wed, Jul 6, 2022 at 1:36 AM wrote: > Maybe I wasn't clear, I'm talking about the 1-dimensional vector product, > but applied to N-D arrays of vectors. Certainly dot products can be > realized as matrix products, and often are in mathematics for convenience, > but matrices and vectors are not the same thing, theoretically or coding > wise. If I have two (M, N, k) arrays a and b where k is the vector > dimension, to dot product them using matrix notation I have to do: > > (a[:, :, np.newaxis, :] @ b[:, :, :, np.newaxis])[:, :, 0, 0] > > Which I certainly don't find readable (I always have to scratch my head a > little bit to figure out whether the newaxis's are in the right places). If > this is a common operation in larger expressions, then it basically has to > be written as a separate function, which then someone reading the code may > have to look at for the semantics. It also breaks down if you want to write > generic vector functions that may be applied along different axes; then you > need to do something like > > np.squeeze(np.expand_dims(a, axis=axis) @ np.expand_dims(b, axis=axis+1), > (axis, axis+1)) > > (after normalizing the axis; if it's negative you'd need to do axis-1 and > axis instead). > > Compare this to the simplicity, composability and consistency of: > > a.dot(b, axis=-1) * np.cross(c, d, axis=-1).dot(e, axis=-1) / > np.linalg.norm(f, axis=-1) > > (the cross and norm operators already support an axis parameter) > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: ilhanpo...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
On Tue, 2022-07-05 at 23:36 +, rmccampbe...@gmail.com wrote: > Maybe I wasn't clear, I'm talking about the 1-dimensional vector > product, but applied to N-D arrays of vectors. Certainly dot products > can be realized as matrix products, and often are in mathematics for > convenience, but matrices and vectors are not the same thing, > theoretically or coding wise. If I have two (M, N, k) arrays a and b > where k is the vector dimension, to dot product them using matrix > notation I have to do: > > (a[:, :, np.newaxis, :] @ b[:, :, :, np.newaxis])[:, :, 0, 0] > You can make it more readable for example with: res = a[..., np.newaxis, :] @ b[..., :, np.newaxis] res = res[..., 0, 0] (could remove the `:`). Maybe even more tricks like: rowmat = np.s_[..., np.newaxis, :] colmat = np.s_[..., :, np.newaxis] res = a[rowmat] @ a[colmat] > Which I certainly don't find readable (I always have to scratch my > head a little bit to figure out whether the newaxis's are in the > right places). If this is a common operation in larger expressions, > then it basically has to be written as a separate function, which > then someone reading the code may have to look at for the semantics. > It also breaks down if you want to write generic vector functions > that may be applied along different axes; then you need to do > something like > I would suggest using `np.moveaxis` to implement a helper. Now of course there may be a point to put that helper into NumPy as `np.vecdot` (or similar), even if it is probably a 3 line function if implemented in terms of `matmul`. Cheers, Sebastian > np.squeeze(np.expand_dims(a, axis=axis) @ np.expand_dims(b, > axis=axis+1), (axis, axis+1)) > > (after normalizing the axis; if it's negative you'd need to do axis-1 > and axis instead). > > Compare this to the simplicity, composability and consistency of: > > a.dot(b, axis=-1) * np.cross(c, d, axis=-1).dot(e, axis=-1) / > np.linalg.norm(f, axis=-1) > > (the cross and norm operators already support an axis parameter) > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: sebast...@sipsolutions.net > signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
I'm unaware of the context here, is this a specification for functions that it is hoped will eventually be made consistent across numpy/tensorflow/etc? If that's the idea then yeah, I'm all for it, but I would suggest also adding a keepdim parameter (as I mentioned above it helps with broadcasting, i.e. vec_array1.dot(vec_array2, keepdims=True) * vec_array3 would work as expected). But is there an active effort to incorporate these APIs back into numpy? ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
Yes, if I am doing this more than once in some code I would make a helper. But it's much better I think to have a common function that people can learn and use consistently instead of having to roll their own functions all the time. Especially because numpy otherwise usually just works when you write the algorithm how you write it on paper. There shouldn't need to be too much thinking involved in the translation. Also yeah using fancy index tricks like `rowmat` and `colmat` simplifies the code a bit, but is still obscure and esoteric looking to somebody who's not intimately familiar with numpy indexing. ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
Oh nevermind, I see that this is added as an experimental module in the latest numpy version. It would be nice to not have to have another whole set of APIs, but on the other hand the numpy API is so messy and inconsistent that maybe it is a good thing :) But it does mean now we have at least 9 different functions/methods/operators that can compute dot products 😢 (not even including the other array_api functions...) I see that currently the vecdot function returns a 2x2 array from two 2x3 APIs, which matches np.inner but is not what I would expect. Does your fix make it instead return a 1-d length-2 array? ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Feature request: dot product along arbitrary axes
On Tue, Jul 5, 2022 at 9:10 PM wrote: > > Oh nevermind, I see that this is added as an experimental module in the > latest numpy version. It would be nice to not have to have another whole set > of APIs, but on the other hand the numpy API is so messy and inconsistent > that maybe it is a good thing :) But it does mean now we have at least 9 > different functions/methods/operators that can compute dot products 😢 (not > even including the other array_api functions...) The idea of the array API is to have a standard API across all Python array libraries. numpy.array_api is currently implemented as a fully conformant version of that API, but the plan is to eventually make NumPy itself conform as well (so vecdot should be added to numpy at some point). > > I see that currently the vecdot function returns a 2x2 array from two 2x3 > APIs, which matches np.inner but is not what I would expect. Does your fix > make it instead return a 1-d length-2 array? Yes. I incorrectly implemented vecdot using tensordot, but with my PR this would return a shape (2,) array (with the default axis=-1). Aaron Meurer > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: asmeu...@gmail.com ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
