In article <[EMAIL PROTECTED]>,
John Hendrickx <[EMAIL PROTECTED]> wrote:
>In article <ye1C4.14$[EMAIL PROTECTED]>,
>[EMAIL PROTECTED] says...
>> I have been trying to explain to some co-workers that a sample
>> can be too big.
>> That is not very easy because it is contratictory to what
>> intuition says.
>> Can someone point me to some good arguments or literature?
>> Or correct me if my assumption is wrong?
>If the sample is too large, effects could be significant although trivial
>in magnitude. As for references, see
>Raftery, A.E. (1986) "Choosing modles for cross-classifications: comment
>on Grusky and Hauser". American Sociological Review 51: 145-146.
>Raftery, A.E. (1995) "Bayesian model selection in social research".
>Sociological Methodology 25: 111-163.
>Raftery's BIC is often used in loglinear models of occupational mobility,
>where large sample sizes often lead to overly complex models.
All this shows is that the current use of statistical
significance is what is WRONG. Even if one has a point
null hypothesis, with a larger sample, some of the
reduced risk of a type 2 error should go into reducing
the risk of a type 1 error. And the real problem, is
the "null" hypothesis close enough to being true to
ignore the difference, is another problem.
In one problem, the appropriate significance level
should be 50%, and in another, 1/10^10 or less. It
changes not just with the problem, but with the
sample size as well, and faster than one would think.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
[EMAIL PROTECTED] Phone: (765)494-6054 FAX: (765)494-0558
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
