Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-11 Thread Ajo Fod
Ted: Thanks for the pointer. I've been looking for a replacement for cold for a while because it doesn't allow the merge operation for QuantileBin1D and as you pointed out it is a generally messy library. stream-lib could be the replacement I"m looking for ... if it passes other requirements. Will

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Ted Dunning
On Sat, Aug 10, 2013 at 8:59 AM, Ajo Fod wrote: > If the data doesn't fit, you probably need a StorelessQuantile estimator > like QuantileBin1D from the colt libraries. Then pick a resolution and do > the single pass search. > Peripheral to the actual topic, but the Colt libraries are out of dat

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Phil Steitz
On 8/10/13 11:00 AM, Phil Steitz wrote: > On 8/10/13 10:41 AM, Ajo Fod wrote: >> This paper provides some approximations and situations where you could use >> exact computations: >> http://www.iro.umontreal.ca/~lecuyer/myftp/papers/ksdist.pdf >> ... they even refer to a java program at the end of p

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Phil Steitz
On 8/10/13 10:41 AM, Ajo Fod wrote: > This paper provides some approximations and situations where you could use > exact computations: > http://www.iro.umontreal.ca/~lecuyer/myftp/papers/ksdist.pdf > ... they even refer to a java program at the end of page 11. He he. This is the reference that ou

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Ajo Fod
This paper provides some approximations and situations where you could use exact computations: http://www.iro.umontreal.ca/~lecuyer/myftp/papers/ksdist.pdf ... they even refer to a java program at the end of page 11. Cheers, Ajo. On Sat, Aug 10, 2013 at 10:06 AM, Phil Steitz wrote: > On 8/10/1

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Phil Steitz
On 8/10/13 9:35 AM, Ajo Fod wrote: > In: > http://ocw.mit.edu/courses/mathematics/18-443-statistics-for-applications-fall-2006/lecture-notes/lecture14.pdf > > Take a look at 2 sample KS stats and the relationship to the 1 sample ... > page 88. You already have the 1 sample table. The problem is th

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Ajo Fod
In: http://ocw.mit.edu/courses/mathematics/18-443-statistics-for-applications-fall-2006/lecture-notes/lecture14.pdf Take a look at 2 sample KS stats and the relationship to the 1 sample ... page 88. You already have the 1 sample table. Cheers, Ajo On Sat, Aug 10, 2013 at 9:16 AM, Phil Steitz w

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Phil Steitz
On 8/10/13 8:59 AM, Ajo Fod wrote: > This depends on data size. If it fits in memory, a single pass through the > sorted array to find the biggest differences would suffice. > > If the data doesn't fit, you probably need a StorelessQuantile estimator > like QuantileBin1D from the colt libraries. Th

Re: [math] Kolmogorov-Smirnov 2-sample test

2013-08-10 Thread Ajo Fod
This depends on data size. If it fits in memory, a single pass through the sorted array to find the biggest differences would suffice. If the data doesn't fit, you probably need a StorelessQuantile estimator like QuantileBin1D from the colt libraries. Then pick a resolution and do the single pass

[math] Kolmogorov-Smirnov 2-sample test

2013-07-20 Thread Phil Steitz
I am working on MATH-437 (turning K-S distribution into a proper K-S test impl) and have to decide how to implement 2-sample tests. Asymptotically, the 2-sample D_n,m test statistic (see [1]) has a K-S distribution, so for large samples just using the cdf we already have is appropriate. For small