Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Robert Kern
On Mon, Feb 8, 2021 at 12:10 PM Sebastian Berg wrote: > > This type of change should be in the release notes undoubtedly and > likely a `.. versionchanged::` directive in the docstring. > > Maybe the best thing would be to create a single, prominent but brief, > changelog listing all (or almost a

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Charles R Harris
On Sat, Feb 6, 2021 at 2:32 AM wrote: > I tried to implement a different implementation of the ziggurat method for > generating standard normal distributions that is about twice as fast and > uses 2/3 of the memory than the old one. > I tested the implementation separately and am very confident i

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Sebastian Berg
ruary 8, 2021 4:06 PM > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Question about optimizing > random_standard_normal >   > On Mon, Feb 8, 2021 at 10:53 AM Kevin Sheppard < > kevin.k.shepp...@gmail.com> wrote: > > My reading is that the first 4 are p

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Robert Kern
On Mon, Feb 8, 2021 at 11:38 AM Kevin Sheppard wrote: > That is good news indeed. Seems like a good upgrade, especially given the > breadth of application of normals and the multiple appearances within > distributions.c (e.g., Cauchy). Is there a deprecation for a change like > this? Or is it ju

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Kevin Sheppard
this is the first time a substantially new algo has replaced an existing one. Kevin  From: Robert KernSent: Monday, February 8, 2021 4:06 PMTo: Discussion of Numerical PythonSubject: Re: [Numpy-discussion] Question about optimizing random_standard_normal On Mon, Feb 8, 2021 at 10:53 AM Kevin Sheppard

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Robert Kern
On Mon, Feb 8, 2021 at 10:53 AM Kevin Sheppard wrote: > My reading is that the first 4 are pure C, presumably using the standard > practice of inclining so as to make the tightest loop possible, and to > allow the compiler to make other optimizations. The final line is what > happens when you re

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Kevin Sheppard
PythonSubject: Re: [Numpy-discussion] Question about optimizing random_standard_normal On Mon, Feb 8, 2021 at 3:05 AM Kevin Sheppard <kevin.k.shepp...@gmail.com> wrote:If I understand correctly, there is no gain when applying this patch to Generator.  I'm not that surprised that this

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Robert Kern
On Mon, Feb 8, 2021 at 3:05 AM Kevin Sheppard wrote: > If I understand correctly, there is no gain when applying this patch to > Generator. I'm not that surprised that this is the case since the compiler > is much more limited in when it can do in Generator than what it can when > filling a larg

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Kevin Sheppard
If I understand correctly, there is no gain when applying this patch to Generator. I'm not that surprised that this is the case since the compiler is much more limited in when it can do in Generator than what it can when filling a large array directly with functions available for inlining and unro

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread camel-cdr
‐‐‐ Original Message ‐‐‐ On Saturday, February 6, 2021 3:29 PM, Robert Kern wrote: > On Sat, Feb 6, 2021 at 7:27 AM wrote: > >>> Well, I can tell you why it needs to be backward compatible! I use random >>> numbers fairly frequently, and to unit test them I set a specific seed and >>>

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread camel-cdr
> Well, I can tell you why it needs to be backward compatible! I use random > numbers fairly frequently, and to unit test them I set a specific seed and > then make sure I get the same answers. Hmm, I guess that makes sense. I tried to adjust my algorithms to do the same thing with the same bit

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread Robert Kern
On Sat, Feb 6, 2021 at 7:27 AM wrote: > Well, I can tell you why it needs to be backward compatible! I use random > numbers fairly frequently, and to unit test them I set a specific seed and > then make sure I get the same answers. > > Hmm, I guess that makes sense. I tried to adjust my algorith

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread Charles R Harris
On Sat, Feb 6, 2021 at 5:27 AM wrote: > Well, I can tell you why it needs to be backward compatible! I use random > numbers fairly frequently, and to unit test them I set a specific seed and > then make sure I get the same answers. > > Hmm, I guess that makes sense. I tried to adjust my algorith

[Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread camel-cdr
I tried to implement a different implementation of the ziggurat method for generating standard normal distributions that is about twice as fast and uses 2/3 of the memory than the old one. I tested the implementation separately and am very confident it's correct, but it does fail 28 test in cove

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread Kevin Sheppard
Have you benchmarked it using the generator interface? The structure of this as a no monolithic generator makes it a good deal slower than generating in straight C (with everything inline). While I'm not sure a factor of 2 is enough to justify a change (for me 10x, 1.2x is not but I don't know whe

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread Tom Swirly
Well, I can tell you why it needs to be backward compatible! I use random numbers fairly frequently, and to unit test them I set a specific seed and then make sure I get the same answers. If your change went in (and I were using numpy normal distributions, which I am not) then my tests would brea