Hi Max and Andrew, Thanks so much for your reply. Indeed I found your link last night using steps shown below.
My first question is if the following two steps are right and AUC is 51% as shown below. My seond question is that currently I am using cost parameter=1 (the default in R-SVM; http://www.stanford.edu/group/wonglab/RSVMpage/R-SVM.html). To improve the AUC from ROC curve, can I can optimize the SVM cost function (instead of keeping it fixed a 1) to get the mimimum LOO error in training cross.validation and then draw the ROC from decision.values as Step2 below using a cost parameter that gave mimimum cross.validation error in the training data. Is that right? Many thanks. ---------- *Step 1: For obtaining ROC curve of test data I turned on "prob=T" option:* > svmres.prob <- svm(traindx[,resrsvm$SelInd], as.factor(traindy), decision.values = TRUE) > svmpred.prob <- predict(svmres.prob, testdx[,resrsvm$SelInd], decision.values = TRUE) > print(confusionMatrix(svmpred.prob,testdy)) Confusion Matrix and Statistics Reference Prediction Resistant Sensitive Resistant 5 13 Sensitive 37 88 Accuracy : 0.6503 *Step 2: Actual ROC plot command using output from above and plot attached as well as pdf (I am assuming the following says the AUC is 51.4):* > library(ROCR) > svm.roc <- prediction(attributes(svmpred.prob)$decision.values, testdy) > svm.auc <- performance(svm.roc, 'tpr', 'fpr') > aucsvm <- performance(svm.roc, 'auc') > pdf(file="roc_curve_rsvm_decval.pdf") > plot(svm.auc) > print(str(aucsvm)) > print(str(aucsvm)) Formal class 'performance' [package "ROCR"] with 6 slots ..@ x.name : chr "None" ..@ y.name : chr "Area under the ROC curve" ..@ alpha.name : chr "none" ..@ x.values : list() ..@ y.values :List of 1 .. ..$ : num 0.514 ..@ alpha.values: list() --------------------------- On Tue, Feb 22, 2011 at 4:23 PM, Andrew Ziem <az...@us.ci.org> wrote: > In addition's to Max's suggestion about caret, look at ROCR which > visualizes ROC charts for any binary classifier. I have an example of > e1071::SVN and ROCR here > > > https://heuristically.wordpress.com/2009/12/23/compare-performance-machine-learning-classifiers-r/ > > > > -----Original Message----- > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] > On Behalf Of Angel Russo > Sent: Monday, February 21, 2011 3:34 PM > To: r-help@r-project.org > Subject: [R] ROC from R-SVM? > > *Hi, > > *Does anyone know how can I show an *ROC curve for R-SVM*? I understand in > R-SVM we are not optimizing over SVM cost parameter. Any example ROC for > R-SVM code or guidance can be really useful. > > Thanks, Angel. > > > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
