Thats great thanks, I suppose it is hard to move away from a more "traditional" measure of performance such a percentage correct, at least for the relatively amateur statisticians among us who have been graded on such a system.
The difficulty comes in reporting the effectiveness of the model to my peers. I have a c-value of 0.71 and a Bieber score of 0.199. So when it comes to predicting the response of newdata (using the estimated mean Y, which i am more comfortable understanding) i.e species 1 - mean = 2.12 therefore on the response scale this is 2 species 2 - mean = 2.98 therefore on the response scale this is 3 etc (on a side note, using this approach no species had a mean of 6, which is the last ordinal category?) A common question is how confident are you that the species has that response. What you are saying is using bootsrapping, and quoting "more proper scoring rules" such as c-values and Bieber score explains the confidence sufficiently? Chris On 22 Sep 2010, at 12:36, Frank Harrell wrote: % correct is an improper scoring rule and a discontinuous one to boot. So it will not always agree with more proper scoring rules. When you have a more difficult task, e.g., discriminating more categories, indexes such as the generalized c-index that utilize all the categories will recognize the difficulty of the task and give a lower value. No cause for alarm. Frank ----- Frank Harrell Department of Biostatistics, Vanderbilt University -- View this message in context: http://r.789695.n4.nabble.com/predict-lrm-Design-package-tp2546894p2550118.html Sent from the R help mailing list archive at Nabble.com. ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
