Hi Chris, Thanks for sharing your thoughts.
The reviewer used the heterogeneity that I observed in my study for the power analysis. I understand what you have descried. And I agree that with the sample size I have, I do not have enough power to detect the heterogeneity that I observed with significance. But if let's say I have enough sample size as calculated by the power analysis, then I will have 80% power to detect the heterogeneity, would it be true that I will almost very unlikely to declare homogeneity among study sites, so that I would almost never be able to combine study sites? This goes to the general thinking that if you have a sample size large enough, you will always be able to make any difference statistically significant... On the the hand, making a statistical inference using any statistical test (including Mantel Haenszel test), I though, is always valid regardless of sample size. For the heterogeneity test, I am just doing that -- making a statistical inference with the p value from Mantel Haenszel test. I am not sure if it is correct that it is mandatory to perform a power analysis before attempting a statistical test. Please share your thoughts... Thanks John ________________________________ From: Christopher W. Ryan <cr...@binghamton.edu> Sent: Tuesday, November 12, 2013 6:53 PM Subject: Re: [R] power analysis is applicable or not John-- Well, my simple-minded way of thinking about these issues goes something like this: You want to know if there is heterogeneity. You gather some data and do your MH analysis. You never know *for sure* whether there is *really* heterogeneity in your population; all you know is whether there is any in your sample--you concluded there was not. Your reviewer calculated that with the sample size you used, *even if there was heterogeneity in your population* (unknowable by you or anyone else) then your sample size only had a 50% probability of detecting it (a 50% probability of coming up with a p < 0.05). Meaning there *could have been* heterogeneity there, at a 0.05 signficance level, and you *would* have seen it, *if* your sample size was larger. It's when you come up with a "non-significant result" that the issue of power is most relevant. If you already have a "significant" result, then yes, your sample size was large enough to show a significant result. An important question is: what *magnitude* of heterogeneity did your reviewer assume he/she was looking for when he/she did the power calculation? And is that magnitude meaningful? All this being said, power calculations are best done before recruiting subjects or gathering data. --Chris Ryan SUNY Upstate Medical University Binghamton, NY array chip wrote: > Hi, this is a statistical question rather than a pure R question. I have got > many help from R mailing list in the past, so would like to try here and > appreciate any input: > > I conducted Mantel-Haenszel test to show that the performance of a diagnostic > test did not show heterogeneity among 4 study sites, i.e. Mantel Haenszel > test p value > 0.05, so that I could conduct a meta-analysis by combining > data of all 4 study sites. > > Now one of the reviewers for the manuscript did a powering analysis for > Mantel Haneszel test showing that with the sample sizes I have, the power for > Mantel Haeszel test was only 50%. So he argued that I did not have enough > power for Mantel Haenszel test. > > My usage of Mantel Haenszel was NOT to show a significant p value, instead a > non-sginificant p value was what I was looking for because non-significant p > value indicate NO heterogeneity among study sites. Powering analysis in > general is to show whether you have enough sample size to ensure a > statistical significant difference can be seen with certain likelihood. But > this is not how I used Mantel Haenszel test. So I think in my scenario, the > power analysis is NOT applicable because I am simply using the test to > demonstrate a non-significant p value. > > Am I correct on this view? > > Thanks and appreciate any thoughts. > > John > [[alternative HTML version deleted]] > > > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
