Hi Bert, thank you for your thoughtful and humorous comments, :-) It is scientifically meaningful to do those comparisons, and the results of these comparisons actually make sense to our hypothesis, i.e. one is significant at B2 level while the other is not at B1 level. Just unfortunately, the overall F test for interaction is not significant. I understand "formally" one should not do these post-hoc comparisons under non-significant interaction term. But should I really stop comparing under this situation, especially when these comparisons conform to our hypothesis? I am encouraged to see that you said "For exploratory purposes, such post hoc comparisons might lead to great science". However, my concern is these results may not pass reviewers when sent out for publication.
BTW, I am non-US reader, so I did google "never inhaled". :-) John ________________________________ From: Bert Gunter <gunter.ber...@gene.com> Cc: r-h...@stat.math.ethz.ch Sent: Mon, March 7, 2011 9:20:11 PM Subject: Re: [R] ok to use glht() when interaction is NOT significant? Inline below > Hi, let's say I have a simple ANOVA model with 2 factors A (level A1 and A2) >and > B (level B1 and B2) and their interaction: > > aov(y~A*B, data=dat) > > It turns out that the interaction term is not significant (e.g. P value = 0.2), > but if I used glht() to compare A1 vs. A2 within each level of B, I found that > the comparison is not significant when B=B1, but is very significant (P<0.01) > when B=B2. > > My question is whether it's legal to do this post-hoc comparison when the > interaction is NOT significant? Can I still make the claim that there is a > significant difference between A1 and A2 when B=B2? (I am serious here). Don't know what "legal" means. Why do you want to make the claim? When does it **ever** mean anything scientifically meaningful to make it? What is the **scientific** question of interest? Are the data unbalanced? Have you plotted the data to tell you what's going on? Warning: I come from the school (maybe I'm the only student...) that believes all such formal post hoc comparisons are pointless, silly, wastes of effort. Note the word: "formal" -- that is pretending the P values mean anything, For exploratory purposes, which can certainly include producing P values as well as graphs, such post hoc comparisons might lead to great science. It's the "formal" part that I reject and that you seem to be hung up on. Note also: If you're a Bayesian and can put priors on everything, you can spit out posteriors and Bayes factors to your heart's content. Really! -- no need to sweat multiplicity even. Of course, I speak here only as an observer, having never actually inhaled myself.* Cheers, Bert *Apologies to all non-US and younger readers. This is a smart-aleck reference to an infamous dumb remark from a recent famous, smart former U.S. president. Google "never inhaled" for details. > > Thanks > > John > > > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Bert Gunter Genentech Nonclinical Biostatistics 467-7374 http://devo.gene.com/groups/devo/depts/ncb/home.shtml [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
