I am trying to get statistics of prediction for binary classification
problems and for various training models with caret. Below is an example
that illustrates my need:
------------------------------------------
library(caret)
# ... Get X and Y for training a binary classification problem. X is input
(2000, 5) Y is output (2000,1) ...
tmp <- createDataPartition(Y, p = 3/4, times = 3, list = TRUE, groups =
min(5, length(Y)))
myCtrl <- trainControl(method = "boot", index = tmp, timingSamps = 2,
classProbs = TRUE, summaryFunction = twoClassSummary)
RFmodel <- train(X,Y,method='rf',trControl=myCtrl,tuneLength=1,
metric="ROC")
SVMmodel <- train(X,Y,method='svmRadial',trControl=myCtrl,tuneLength=3,
metric="ROC")
KNNmodel <- train(X,Y,method='knn',trControl=myCtrl,tuneLength=10,
metric="ROC")
NNmodel <- train(X,Y,method='nnet',trControl=myCtrl,tuneLength=3, trace =
FALSE, metric="ROC")
# resamps reports ROC, Sens, Spec for all models
resamps <- resamples(list(RF = RFmodel, KNN = KNNmodel, NN = NNmodel, SVM =
SVMmodel))
# Prediction:
# ... Assume I have X_pred and Y_pred for evaluating prediction, with
X_pred (7000, 5) Y_pred (7000,1) ...
testPred <- predict(list(RF = RFmodel, KNN = KNNmodel, NN = NNmodel, SVM =
SVMmodel), Xtst, type="prob")
------------------------------------------
How can I get the statistics of prediction (ROC, etc.) from X_pred and
Y_pred for *all 4 models*?
Thanks!
James
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
