I believe this is correct behavior for computing the p-value, though the wording is awkward in that it implies that R is not implementing the continuity correction in this situation, when in fact, this behavior is part of how the continuity correction is defined. The correction simply treats the normal approximation as appropriately discrete, so (to translate to a binomial variable) computes P(X > 12) using P(X > 11.5). The case the documentation discusses is simply the case where the null hypothesis falls within the discrete band corresponding to the observed value; in this only enough of the correction is used so that the test statistic is appropriately zero and the p-value is 1.
However, this is not correct behavior for the confidence interval. There is nothing in any of the listed documentation that would support such behavior, and additionally, it doesn't make sense for a confidence interval to depend on a null parameter. If continuity correction is desired, the edges of the confidence bound should still be fully adjusted even when the observed proportion is close to the null parameter. What's currently happening is that it's not adjusted at all when the observed proportion equals the null proportion, and in cases where it is not equal but still close enough that the correction is adjusted, the confidence intervals are neither "with" correction" or "without" correction but instead somewhere in between! An additional confusing matter is how R reports whether the test was performed "with" or "without" continuity correction; this is determined in code by whether or not the adjusted correction is zero or not. This happens when the observed proportion equals the null proportion, so whenever that happens, it's reported "without" continuity correction, so this "flips" on the user in this case. Though oddly (to the user), changing the null p by a tiny amount gives only tiny changes to the result but then it is reported "with" correction. This behavior has presumably been in R for a long time (though I haven't checked the code history), so I would love to have feedback from the R-help community about: * does the current behavior really make sense, and I've just misunderstood something? * is there documentation or discussions about this behavior out there somewhere that I've missed? * if this really is a "new" discovery, how best to bring it to the attention of those who can decide what to do about it? Thanks! On Mon, Oct 21, 2019 at 11:33 AM Aaron Rendahl <rend0...@umn.edu> wrote: > Why does prop.test use continuity correction "only if it does not exceed > the difference between sample and null proportions in absolute value"? I'm > referring here to the single group method, though I believe there is a > similar issue with the two group method. > > What this means in practice is that the confidence interval changes > depending on the null hypothesis; see examples below. This is unexpected, > and I have been unable to find any documentation explaining why this is > done (see links below examples). > > ## when the null proportion is equal to the sample proportion, it does not > ## use the continuity correction, even when one is asked for > > > prop.test(30,60,p=0.5, correct=TRUE) > > 1-sample proportions test without continuity correction > > data: 30 out of 60, null probability 0.5 > X-squared = 0, df = 1, p-value = 1 > alternative hypothesis: true p is not equal to 0.5 > 95 percent confidence interval: > 0.3773502 0.6226498 > sample estimates: > p > 0.5 > > ## however, when the null proportion is not equal to the sample > proportion, > ## it does use the continuity correction when it is asked for. > > > prop.test(30,60,p=0.499, correct=TRUE) > > 1-sample proportions test with continuity correction > > data: 30 out of 60, null probability 0.499 > X-squared = 0, df = 1, p-value = 1 > alternative hypothesis: true p is not equal to 0.499 > 95 percent confidence interval: > 0.3764106 0.6235894 > sample estimates: > p > 0.5 > > > The documentation refers to Newcombe's 1998 Statistics in Medicine > article; I read through this and found nothing about not using the > continuity correction in this situation. > > https://doi.org/10.1002/(SICI)1097-0258(19980430)17:8%3C857::AID-SIM777%3E3.0.CO;2-E > > On this mailing list, there was a 2013 post "prop.test correct true and > false gives same answer", which was answered only with the quote from the > help page: https://stat.ethz.ch/pipermail/r-help/2013-March/350386.html > > I also found several questions asking which Newcombe method is > implemented, which didn't elicit specific answers; here's one from 2011: > https://stat.ethz.ch/pipermail/r-help/2011-April/274086.html > -- Aaron Rendahl, Ph.D. Assistant Professor of Statistics and Informatics College of Veterinary Medicine, University of Minnesota 295L AS/VM, 612-301-2161 [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
