Re: [Numpy-discussion] Using the C-API iterator with object arrays
On Mon, Feb 12, 2018 at 6:13 AM, Eugen Wintersberger <
eugen.wintersber...@gmail.com> wrote:
> Hi there,
> I have a question concerning the numpy iterator C-API. I want to create a
> numpy
> string array using NPY_OBJECT as a datatype for creating the array (the
> reason I am going for this
> approach is that I do not know the length of the individual strings at the
> time I construct
> the array, I only know its shape). The original strings are stored
> in a std::vector instance. The approach I took was something like
> this
>
> std::vector buffer = ;
> NpyIter *iter = NpyIter_New(numpy_array,
> NPY_ITER_READWRITE | NPY_ITER_C_INDEX |
> NPY_ITER_REFS_OK,
> NPY_CORDER , NPY_NO_CASTING,nullptr);
> if(iter==NULL)
> {
> return;
> }
> NpyIter_IterNextFunc *iternext = NpyIter_GetIterNext(iter,nullptr);
> if(iternext == NULL)
> {
> std::cerr<<"Could not instantiate next iterator function"< return;
> }
> PyObject **dataptr = (PyObject**)NpyIter_GetDataPtrArray(iter);
> for(auto string: buffer)
> {
> dataptr[0] = PyString_FromSting(string); // this string construction
> seem to work
> iternext(iter);
> }
> NpyIter_Deallocate(iter);
>
>
> This code snippet is a bit stripped down with all the safety checks
> removed to make
> things more readable.
> However, the array I get back still contains only a None instance. Does
> anyone have an idea
> what I am doing wrong here?
> Thanks in advance.
>
> best regards
>Eugen
>
>
Honestly, given that you're iterating over a single 1D array you've just
constructed, I don't think I would even bother trying to use the iterator.
For this case, a simple loop will be both concise and clear.
Eric
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] asanyarray vs. asarray
On Tue, Oct 30, 2018 at 12:49 AM Eric Wieser wrote: > The latter - changing the behavior of multiplication breaks the principle. > > But this is not the main reason for deprecating matrix - almost all of the > problems I’ve seen have been caused by the way that matrices behave when > sliced. The way that m[i][j] and m[i,j] are different is just one example > of this, the fact that they must be 2d is another. > > Matrices behaving differently on multiplication isn’t super different in > my mind to how string arrays fail to multiply at all. > The difference is that string arrays are not numeric. This is an issue since people want to pass a matrix Into places that want to multiple element wise but that then breaks that code unless special provisions are taken. Numerical codes don’t work on string arrays anyway. Eric Eric > > On Mon, 29 Oct 2018 at 20:54 Ralf Gommers wrote: > > On Mon, Oct 29, 2018 at 4:31 PM Chris Barker >> wrote: >> >>> On Fri, Oct 26, 2018 at 7:12 PM, Travis Oliphant >>> wrote: >>> >>> agree that we can stop bashing subclasses in general. The problem with numpy subclasses is that they were made without adherence to SOLID: https://en.wikipedia.org/wiki/SOLID. In particular the Liskov substitution principle: https://en.wikipedia.org/wiki/Liskov_substitution_principle . >>> >>> ... >>> >>> did not properly apply them in creating np.matrix which clearly violates the substitution principle. >>> >>> So -- could a matrix subclass be made "properly"? or is that an example >>> of something that should not have been a subclass? >>> >> >> The latter - changing the behavior of multiplication breaks the principle. >> >> Ralf >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] round / set_printoptions discrepancy
See the notes section here. https://numpy.org/devdocs/reference/generated/numpy.around.html. This note was recently added in https://github.com/numpy/numpy/pull/14392 Eric On Fri, Sep 13, 2019 at 9:20 AM Andras Deak wrote: > On Fri, Sep 13, 2019 at 2:59 PM Philip Hodge wrote: > > > > On 9/13/19 8:45 AM, Irvin Probst wrote: > > > On 13/09/2019 14:05, Philip Hodge wrote: > > >> > > >> Isn't that just for consistency with Python 3 round()? I agree that > > >> the discrepancy with np.set_printoptions is not necessarily expected, > > >> except possibly for backwards compatibility. > > >> > > >> > > > > > > I've just checked and np.set_printoptions behaves as python's round: > > > > > > >>> round(16.055,2) > > > 16.05 > > > >>> np.round(16.055,2) > > > 16.06 > > > > > > I don't know why round and np.round do not behave the same, actually I > > > would even dare to say that I don't care :-) > > > However np.round and np.set_printoptions should provide the same > > > output, shouldn't they ? This discrepancy is really disturbing whereas > > > consistency with python's round looks like the icing on the cake but > > > in no way a required feature. > > > > > > > Python round() is supposed to round to the nearest even value, if the > > two closest values are equally close. So round(16.055, 2) returning > > 16.05 was a surprise to me. I checked the documentation and found a > > note that explained that this was because "most decimal fractions can't > > be represented exactly as a float." round(16.55) returns 16.6. > > > > Phil > > > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > Ah, of course, endless double-precision shenanigans... > >>> format(16.055, '.30f') > '16.054715782905695960' > > >>> format(16.55, '.30f') > '16.550710542735760100' > > András > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Handle type convertion in C API
Hi Benoit, Since you have a function that takes two scalars to one scalar, it sounds to me as though you would be best off creating a ufunc. This will then handle the conversion to and looping over the arrays, etc for you. The documentation is available here: https://numpy.org/doc/1.18/user/c-info.ufunc-tutorial.html. Regards, Eric On Tue, Mar 10, 2020 at 12:28 PM Benoit Gschwind wrote: > Hello, > > I writing a python binding of one of our library. The binding intend to > vectorize the function call. for exemple: > > double foo(double, double) will be bound to a python: > > module.foo(, ) > > and the function foo will be called like : > > for (int i = 0; i < size; ++i) > outarr[i] = foo(inarr0[i], inarr[1]); > > My question is about how can I handle type conversion of input array, > preferably in efficient manner, given that each input array may require > different input type. > > Currently I basically enforce the type and no type conversion is > performed. But I would like relax it. I thought of several possibility > starting from the obvious solution consisting on recasting in the inner > loop that would give: > > for (int i = 0; i < size; ++i) > if (inarr[i] need recast) > in0 = > recast(inarr[i]) > else > in0 = inarr[i] > [... same for all > inputs parameter ...] > outarr[i] = foo(in0, in1, ...); > > This solution is memory efficient, but not actually computationally > efficient. > > The second solution is to copy&recast the entire inputs arrays, but in > that case it's not memory efficient. And my final thought is to mix the > first and the second by chunking the second method, i.e. converting N > input in a raw, then applying the function to them en so on until all > the array is processed. > > Thus my questions are: > - there is another way to do what I want? > - there is an existing or recommended way to do it? > > And a side question, I use the PyArray_FROM_OTF, but I do not > understand well it's semantic. If I pass a python list, it is converted > to the desired type and requirement; when I pass a non-continuous array > it is converted to the continuous one; but when I pass a numpy array of > another type than the one specified I do not get the conversion. There > is a function that do the conversion unconditionally? Did I missed > something ? > > Thank you by advance for your help > > Best regards > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Handle type convertion in C API
There are a variety of ways to resolve your issues. You can try using the optional arguments casting, dtype or signature that work for all ufuncs see https://numpy.org/doc/1.18/reference/ufuncs.html#optional-keyword-arguments This will allow you to override the default type checks. What you'll find for many ufuncs in, for instance, scipy.special, is that the underlying function in cython, C or Fortran is only defined for doubles, but the function has a signature for f->f, which just casts the input and output in the loop. Generally, downcasting is not permitted without explicit opt in, e.g. int64 to int32, since this is not a safe cast as there are many values that an int64 can hold that an int32 cannot. Generally speaking, you do have to manage the types of your arrays when the defaults aren't what you want. There really isn't anyway around it. Eric On Wed, Mar 11, 2020 at 4:43 AM Benoit Gschwind wrote: > Hello Eric, > > Thank you for pointing out ufunc, I implemented my binding using it, > it's working and it's simpler than my previous implementation, but I > still not have the flexibility to dynamically recast input array, i.e. > using a int64 array as input for a int32. For instance for testing my > code I use numpy.random that provide int64 array, I have to convert the > array manually before calling my ufunc, which is somehow annoying. > > For function that have 1 or 2 parameters it's practical to have 4 > variant of the function, but in case of 6-8 parameters it's becoming > more difficult. > > Best regards > > Le mardi 10 mars 2020 à 13:13 -0400, Eric Moore a écrit : > > Hi Benoit, > > > > Since you have a function that takes two scalars to one scalar, it > > sounds to me as though you would be best off creating a ufunc. This > > will then handle the conversion to and looping over the arrays, etc > > for you. The documentation is available here: > > https://numpy.org/doc/1.18/user/c-info.ufunc-tutorial.html. > > > > Regards, > > > > Eric > > > > On Tue, Mar 10, 2020 at 12:28 PM Benoit Gschwind < > > gschw...@gnu-log.net> wrote: > > > Hello, > > > > > > I writing a python binding of one of our library. The binding > > > intend to > > > vectorize the function call. for exemple: > > > > > > double foo(double, double) will be bound to a python: > > > > > > module.foo(, ) > > > > > > and the function foo will be called like : > > > > > > for (int i = 0; i < size; ++i) > > > outarr[i] = foo(inarr0[i], inarr[1]); > > > > > > My question is about how can I handle type conversion of input > > > array, > > > preferably in efficient manner, given that each input array may > > > require > > > different input type. > > > > > > Currently I basically enforce the type and no type conversion is > > > performed. But I would like relax it. I thought of several > > > possibility > > > starting from the obvious solution consisting on recasting in the > > > inner > > > loop that would give: > > > > > > for (int i = 0; i < size; ++i) > > > if (inarr[i] need recast) > > > in0 = > > > recast(inarr[i]) > > > else > > > in0 = inarr[i] > > > [... same for all > > > inputs parameter ...] > > > outarr[i] = foo(in0, in1, ...); > > > > > > This solution is memory efficient, but not actually computationally > > > efficient. > > > > > > The second solution is to copy&recast the entire inputs arrays, but > > > in > > > that case it's not memory efficient. And my final thought is to mix > > > the > > > first and the second by chunking the second method, i.e. converting > > > N > > > input in a raw, then applying the function to them en so on until > > > all > > > the array is processed. > > > > > > Thus my questions are: > > > - there is another way to do what I want? > > > - there is an existing or recommended way to do it? > > > > > > And a side question, I use the PyArray_FROM_OTF, but I do not > > > understand well it's semantic. If I pass a python list, it is > > > converted > > > to the desired type and requirement; when I pass a non-continuous > > > array > > > it is converted to the continuous one; but when I pass a numpy > > > array of > > > another type than the one specified I do
Re: [Numpy-discussion] Type declaration to include all valid numerical NumPy types for Cython
If that is really all you need, then the version in python is:
def convert_one(a):
"""
Converts input with arbitrary layout and dtype to a blas/lapack
compatible dtype with either C or F order. Acceptable objects are
passed
through without making copies.
"""
a_arr = np.asarray(a)
dtype = np.result_type(a_arr, 1.0)
# need to handle these separately
if dtype == np.longdouble:
dtype = np.dtype('d')
elif dtype == np.clongdouble:
dtype = np.dtype('D')
elif dtype == np.float16:
dtype = np.dtype('f')
# explicitly force a copy if a_arr isn't one segment
return np.array(a_arr, dtype, copy=not a_arr.flags.forc, order='K')
In Cython, you could just run exactly this code and it's probably fine.
The could also be rewritten using the C calls if you really wanted.
You need to either provide your own or use a casting table and the copy /
conversion routines from somewhere. Cython, to my knowledge, doesn't
provide these things, but Numpy does.
Eric
On Sun, Aug 9, 2020 at 6:16 PM Ilhan Polat wrote:
> Hi all,
>
> As you might have seen my recent mails in Cython list, I'm trying to cook
> up an input validator for the linalg.solve() function. The machinery of
> SciPy linalg is as follows:
>
> Some input comes in passes through np.asarray() then depending on the
> resulting dtype of the numpy array we choose a LAPACK flavor (s,d,c,z) and
> off it goes through f2py to lalaland and comes back with some result.
>
> For the backslash polyalgorithm I need the arrays to be contiguous (C- or
> F- doesn't matter) and any of the four (possibly via making new copies)
> float, double, float complex, double complex after the intake because we
> are using wrapped fortran code (LAPACK) in SciPy. So my difficulty is how
> to type such function input, say,
>
> ctypedef fused numeric_numpy_t:
> bint
> cnp.npy_bool
> cnp.int_t
> cnp.intp_t
> cnp.int8_t
> cnp.int16_t
> cnp.int32_t
> cnp.int64_t
> cnp.uint8_t
> cnp.uint16_t
> cnp.uint32_t
> cnp.uint64_t
> cnp.float32_t
> cnp.float64_t
> cnp.complex64_t
> cnp.complex128_t
>
> Is this acceptable or something else needs to be used? Then there is the
> storyof np.complex256 and mysterious np.float16. Then there is the Linux vs
> Windows platform dependence issue and possibly some more that I can't
> comprehend. Then there are datetime, str, unicode etc. that need to be
> rejected. So this is quickly getting out of hand for my small brain.
>
> To be honest, I am a bit running out of steam working with this issue even
> though I managed to finish the actual difficult algorithmic part but got
> stuck here. I am quite surprised how fantastically complicated and
> confusing both NumPy and Cython docs about this stuff. Shouldn't we keep a
> generic fused type for such usage? Or maybe there already exists but I
> don't know and would be really grateful for pointers.
>
> Here I wrote a dummy typed Cython function just for type checking:
>
> cpdef inline bint ncc( numeric_numpy_t[:, :] a):
> print(a.is_f_contig())
> print(a.is_c_contig())
>
> return a.is_f_contig() or a.is_c_contig()
>
> And this is a dummy loop (with aliases) just to check whether fused type
> is working or not (on windows I couldn't make it work for float16).
>
> for x in (np.uint, np.uintc, np.uintp, np.uint0, np.uint8, np.uint16,
> np.uint32,
> np.uint64, np.int, np.intc, np.intp, np.int0, np.int8, np.int16,
> np.int32,np.int64, np.float, np.float32, np.float64, np.float_,
> np.complex, np.complex64, np.complex128, np.complex_):
> print(x)
> C = np.arange(25., dtype=x).reshape(5, 5)
> ncc(C)
>
>
> Thanks in advance,
> ilhan
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy Feature Request: Function to wrap angles to range [ 0, 2*pi] or [ -pi, pi ]
On Tue, Nov 24, 2020 at 6:38 AM Daniele Nicolodi wrote: > On 24/11/2020 10:25, Thomas wrote: > > Like Nathaniel said, it would not improve much when compared to the > > modulo operator. > > > > It could handle the edge cases better, but really the biggest benefit > > would be that it is more convenient. > > Which edge cases? Better how? > > > And as the "unwrap" function already exists, > > The unwrap() function exists because it is not as trivial. > > > people would expect that > > and look for a function for the inverse operation (at least I did). > > What is your use of a wrap() function? I cannot think of any. > > Cheers, > Dan > For what it’s worth, this kind of range reduction can be extremely nontrivial depending on the needs of your application. Look at the efforts needed to ensure that the trigonometric functions give good results. This is discussed in this paper: https://www.csee.umbc.edu/~phatak/645/supl/Ng-ArgReduction.pdf. I don’t think that this belongs in Numpy, but it certainly isn’t a one liner. Best, Eric > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Re: representation of valid float type range
On Mon, Jan 10, 2022 at 7:15 AM Hameer Abbasi wrote: > Hello all. > > I believe that over the years there were multiple proposals to replace the > linspace formula start + n *(stop - start) / (npoints - 1) with a * start + > b * end with a, b linearly spaced between 0 and 1 with npoints. Concretely, > a = n / (npoints - 1), b = 1 - a. Here, 0 <= n < npoints. > > I believe this would fix the issue here among many others. However, it was > always rejected due to backward-incompatibility concerns. Perhaps it’s time > to revisit this issue. Is there an actual use case for the version of this from the original email? I’m not sure how a linspace with range covering 10^600 or so is something that has an actual use. Just because this fails doesn’t mean that it actually needs to be made to work. Eric > > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
