Tobias Sing wrote:
Waverley,
you can also use p...@y.values to access the slot (see
help(performance-class) for a description of the slots).
You might also want have a look at the code for demo(ROCR) and at this
slide deck:
http://rocr.bioinf.mpi-sb.mpg.de/ROCR_Talk_Tobias_Sing.ppt
HTH,
Tobias
Tobias,
In my view there is one significant omission from your handout: high
resolution calibration curves. There is a need to show that predictive
models predict accurately. See for example the val.prob function in the
Design package. The many graphs related to cumulative probabilities
are nice, but in some ways they get in the way of the fundamental
elements of absolute accuracy (calibration curves) and predictive
descrimination (simple histogram of predicted probabilities ignoring Y).
I go into this in my 1996 Stat in Med paper. In my view the
continuous accuracy measures need to be examined first, because
dichotomizations provide only crude approximations to be plugged into
decision making. Dichotomizations (classifiers) may provide good
decisions for a group of subjects but not so good decisions for every
individual member of the group. For one thing, different group members
have different loss/utility functions. For another, a predicted
probability of 0.5 may often best be summarized as "collect another
predictor variable for this subject."
Related to this is that ROC-type measures result in a decision rule for
one subject that is a function of all the data of all the subjects in
the sample. This violates a basic principle of optimum Bayes decisions.
A related reference is below.
Just my $.02.
Frank
@Article{bri08ski,
author = {Briggs, William M. and Zaretzki, Russell},
title = {The skill plot: {A} graphical technique for evaluating
continuous diagnostic tests (with discussion)},
journal = Biometrics,
year = 2008,
volume = 63,
pages = {250-261},
annote = {ROC curve;sensitivity;skill plot;skill
score;specificity;diagnostic accuracy;diagnosis;``statistics such as the
AUC are not especially relevant to someone who must make a decision
about a particular $x_{c}$. \ldots ROC curves lack or obscure several
quantities that are necessary for evaluating the operational
effectiveness of diagnostic tests. \ldots ROC curves were first used to
check how radio \emph{receivers} (like radar receivers) operated over a
range of frequencies. \ldots This is not how must ROC curves are used
now, particularly in medicine. The receiver of a diagnostic measurement
\ldots wants to make a decision based on some $x_{c}$, and is not
especially interested in how well he would have done had he used some
different cutoff.''; in the discussion David Hand states ``when
integrating to yield the overall AUC measure, it is necessary to decide
what weight to give each value in the integration. The AUC implicitly
does this using a weighting derived empirically from the data. This is
nonsensical. The relative importance of misclassifying a case as a
noncase, compared to the reverse, cannot come from the data itself. It
must come externally, from considerations of the severity one attaches
to the different kinds of misclassifications.''}
}
On Sat, Feb 7, 2009 at 10:40 PM, Jorge Ivan Velez
<jorgeivanve...@gmail.com> wrote:
Hi Waverley,
I forgot to tell you that "perf" is your performance object. Here is an
example from the ROCR package:
## computing a simple ROC curve (x-axis: fpr, y-axis: tpr)
library(ROCR)
data(ROCR.simple)
pred <- prediction( ROCR.simple$predictions, ROCR.simple$labels)
perf <- performance(pred,"tpr","fpr")
# y.values
unlist(slot(perf,"y.values"))
HTH,
Jorge
On Sat, Feb 7, 2009 at 3:17 PM, Waverley <waverley.paloa...@gmail.com>wrote:
Hi,
I have a question about ROCR package. I got the ROC curve plotted
without any problem following the manual. However, I don't know to
extract the values, e.g. y.values ( I think it is the area under the
curve auc measure). The return is an object of class "performance"
which have Slots and one of the slot is "y.values". I type the object
and I can see them in screen. But I want to extract the value for
further programming and computation. I did a summary of the object
and it is a "S4" mode which I don't understand.
Can someone help?
Thanks a lot in advance.
--
Waverley @ Palo Alto
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
