N. Lapidus wrote:
Hi Pierre-Jean,
Sensitivity (Se) and specificity (Sp) are calculated for cutoffs stored in
the "performance" x.values of your prediction for Se and Sp:
For example, let's generate the performance for Se and Sp:
sens <- performance(pred,"sens")
spec <- performance(pred,"spec")
Now, you can have acces to:
[EMAIL PROTECTED] # (or [EMAIL PROTECTED]), which is the list of cutoffs
[EMAIL PROTECTED] # for the corresponding Se
[EMAIL PROTECTED] # for the corresponding Sp
You can for example sum up this information in a table:
(SeSp <- cbind ([EMAIL PROTECTED], [EMAIL PROTECTED],
[EMAIL PROTECTED]))
You can also write a function to give Se and Sp for a specific cutoff, but
you will have to define what to do for cutoffs not stored in the list. For
example, the following function keeps the closest stored cutoff to give
corresponding Se and Sp (but this is not always the best solution, you may
want to define your own way to interpolate):
se.sp <- function (cutoff, performance) {
sens <- performance(pred,"sens")
spec <- performance(pred,"spec")
num.cutoff <- which.min(abs([EMAIL PROTECTED] - cutoff))
return(list([EMAIL PROTECTED],
[EMAIL PROTECTED], [EMAIL PROTECTED]
[[1]][num.cutoff]))
That is a biased procedure (like how stepwise regression results in
overfitting). It also uses a strange loss function. The bootstrap
would need to be used to penalize for the uncertainty in the cutoff.
You are also assuming that a cutoff exists, which is a major assumption.
Frank
}
se.sp(.5, pred)
Hope this helps,
Nael
On Thu, Nov 13, 2008 at 5:59 PM,
<[EMAIL PROTECTED]>wrote:
Hi Frank,
Thank you for your answer.
In fact, I don't use this for clinical research practice.
I am currently testing several scoring methods and I'd like
to know which one is the most effective and which threshold
value I should apply to discriminate positives and negatives.
So, any idea for my problem ?
Pierre-Jean
-----Original Message-----
From: Frank E Harrell Jr [mailto:[EMAIL PROTECTED]
Sent: Thursday, November 13, 2008 5:00 PM
To: Breton, Pierre-Jean-EXT R&D/FR
Cc: r-help@r-project.org
Subject: Re: [R] Calculate Specificity and Sensitivity for a given
threshold value
Kaliss wrote:
Hi list,
I'm new to R and I'm currently using ROCR package.
Data in input look like this:
DIAGNOSIS SCORE
1 0.387945
1 0.50405
1 0.435667
1 0.358057
1 0.583512
1 0.387945
1 0.531795
1 0.527148
0 0.526397
0 0.372935
1 0.861097
And I run the following simple code:
d <- read.table("inputFile", header=TRUE); pred <- prediction(d$SCORE,
d$DIAGNOSIS); perf <- performance( pred, "tpr", "fpr");
plot(perf)
So building the curve works easily.
My question is: can I have the specificity and the sensitivity for a
score threshold = 0.5 (for example)? How do I compute this ?
Thank you in advance
Beware of the utility/loss function you are implicitly assuming with
this approach. It is quite oversimplified. In clinical practice the
cost of a false positive or false negative (which comes from a cost
function and the simple forward probability of a positive diagnosis,
e.g., from a basic logistic regression model if you start with a cohort
study) vary with the type of patient being diagnosed.
Frank
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt
University
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
