Re: [R] ROCR package question for evaluating two regression models

2011-09-04 Thread RockO
Hi Andra, I have been doing some ROC analysis for a new diagnosis test. I used the pROC package to assess thresholds and compare different diagnosis tests to a "gold standard". In your case, let say the gold standard are the observed values y0. Here is an example: y0 <- sample(0:1,50,replace=TRUE

Re: [R] ROCR package question for evaluating two regression models

2011-09-03 Thread Frank Harrell
It is not possible to have one cutoff point unless you have a very strange utility function. Nor is there a need for a cutoff when using a probability model. It is not advisable to compare models based on ROC area as this loses power. A likelihood-based approach is recommended. Frank Andra Isan

[R] ROCR package question for evaluating two regression models

2011-09-02 Thread Andra Isan
Hello All,  I have used logistic regression glm in R and I am evaluating two models both learned with glm but with different predictors. model1 <- glm (Y ~ x4+ x5+ x6+ x7, data = dat, family = binomial(link=logit))model2 <- glm (Y~ x1 + x2 +x3 , data = dat, family = binomial(link=logit))  and I