Dear all,
I am acting as the stats advisor for my unit in the psychology department of
the University of Edinburgh, UK. Last week a colleague of mine presented me
with the following issue, and I am not quite sure how to respond:
She is running a psychological experiment, in which she a priori specified
her sample size as 200 people. She has already sampled 40 participants and a
preliminary effect size (ES) analysis suggests an almost zero effect. Based
on previous research, she was expecting a "detectable" effect even with 40
subjects - though I suspect she was not expecting enough power to get a
significant result at that stage. In addition, it appears that the reason
the ES she gets is nowhere close to the expected figure may be because of a
design flaw. So she asked me whether it is justified to go up to, say, 100
participants, check again her ES and if it's still near zero, stop sampling,
or whether she had to sample all 200 people because she had says so in her
protocol?
I do think it would be foolish to keep sampling when one has grounds to
believe that there is "no" effect or that there is a flaw in the study. I
believe that if the plot of subjects versus power, suggested that the power
curve levelled after a given sample size, that would be enough justification
to stop sampling (needless to say that participants that satisfy her
protocol are "precious" and hard to find). Her query though sounds to me
more like a methodological (if not ethical) one, rather than a "true"
statistical problem, and thus this bottom-up justification may not suffice.
However, I have failed to think of a top-down, theoretical approach to this
problem, and since she will try to publish the results of this study
regardless, she would like references that back up my position (or in that
sense falsify it).
I would really appreciate your comments on this.
Best
Niko Tiliopoulos
Department of Psychology
The University of Edinburgh
7 George Square
Edinburgh, EH8 9JZ
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
