Thanks a lot for your last letter, you're right, I wasn't clear enough. I've already tried AIC before, but I thought that comparing models based on this criterion would be applicable if I had good models and I wanted to find the best fitting one. However, I only have poor models with high AIC, and I thought that it would make sense to also look for models with relatively good fit. So probably I should try to take into account the significance level of the residual deviance (thanks for the correction) somehow. I may be wrong, but simply minimising the deviance would not necessarily maximise the significance level as it also depends on the degrees of freedom, which varies with each model (with different variables and interaction terms). And yes, simply maximising the significance level would make the model very complex, so some penalty is necessary for additional variables. All in all, I'm not really sure how to balance AIC and model fit in this context. Thanks again for your comments.
Best, Mano Gabor TOTH On 1 March 2011 23:39, <bill.venab...@csiro.au> wrote: > The "probability OF the residual deviance" is zero. The significance level > for the residual deviance according to its asymptotic Chi-squared > distribution is a possible criterion, but a silly one. If you want to > minimise that, just fit no variables at all. That's the best you can do. If > you want to maximise it, just minimise the deviance itself, which means > include all possible variables in the regression, together with as many > interactions as you can as well. (Incidently R doesn't have restrictions on > how many interaction terms it can handle, those are imposed my your > computer.) > > I suggest you think again about what criterion you really want to use. > Somehow you need to balance fit in the training sample against some > complexity measure. AIC and BIC are commonly used criteria, but not the > only ones. I suggest you start with these and see if either does the kind > of job you want. > > Stepwise regression with interaction terms can be a bit tricky if you want > to impose the marginality constraints, but that is a bigger issue. > > Bill Venables. > > -----Original Message----- > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] > On Behalf Of Mano Gabor Toth > Sent: Wednesday, 2 March 2011 6:44 AM > To: r-help@r-project.org > Subject: [R] Logistic Stepwise Criterion > > Dear R-help members, > > I'd like to run a binomial logistic stepwise regression with ten > explanatory > variables and as many interaction terms as R can handle. I'll come up with > the right R command sooner or later, but my real question is whether and > how > the criterion for the evaluation of the different models can be set to be > the probability of the residual deviance in the Chi-Square distribution > (which would be more informative of overall model fit than AIC). > > Thanks in advance for all your help. > > Kind regards, > > Mano Gabor TOTH > MA Political Science > Central European University > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
