Re: [R] Cross validation multivariate kernel regression

2019-11-19 Thread Abby Spurdle
> I am planning to implement Nadaraya-Watson regression model, with I'm not sure what you mean by "implement". Write a package, fit a model, or something else... Reading your whole post, I get the impression you want mid-level "building blocks", so you customize the model fitting process, in some

Re: [R] cross validation in random forest using rfcv functin

2017-08-24 Thread David Winsemius
> On Aug 23, 2017, at 10:59 AM, Elahe chalabi via R-help > wrote: > > Any responds?! When I look at the original post a I see a question about a function named `rfcv` but do not see a `library` call to load such a function. I also see a reference to a help page or vignette, perhaps?, from th

Re: [R] Cross-Validation for Zero-Inflated Models

2017-06-21 Thread Jeff Newmiller
1) Helpdesk implies people whose job it is to provide support. R-help is a mailing list in which users help each other when they have spare time. 2) You sent an email to the R-help mailing list, not to Lara, whoever that is. I suggest you figure out what her email address is and send your quest

Re: [R] Cross-Validation for Zero-Inflated Models

2017-06-21 Thread Eric Weine
Lara: I see you sent this email to the R helpdesk a really long time ago, but I was just wondering if you ever got an answer to this question. I was just thinking that I would build my own cross validation function, but if you figured out a way to do this automatically, could you let me know?

Re: [R] Cross validation in R

2013-07-02 Thread Max Kuhn
> How do i make a loop so that the process could be repeated several time, > producing randomly ROC curve and under ROC values? Using the caret package http://caret.r-forge.r-project.org/ -- Max __ R-help@r-project.org mailing list https://stat.ethz

Re: [R] Cross validation in R

2013-07-02 Thread Adams, Jean
This code is untested, since you did not provide any example data. But it may help you get started. Jean mydata <- read.csv(file.choose(), header=TRUE) library(ROCR) # ROC curve and assessment of my prediction plot(0:1, 0:1, type="n", xlab="False positive rate", ylab="True positive rate") abli

Re: [R] Cross validation for Naive Bayes and Bayes Networks

2013-04-15 Thread Marco Scutari
Hi Guilherme, On Sun, Apr 14, 2013 at 11:48 PM, Guilherme Ferraz de Arruda wrote: > Hi, > I need to classify, using Naive Bayes and Bayes Networks, and estimate > their performance using cross validation. > How can I do this? > I tried the bnlearn package for Bayes Networks, althought I need to

Re: [R] cross validation in rvm not working? (kernlab package)

2012-02-14 Thread Uwe Ligges
Please report bugs in packages to the corresponding package maintainer (perhaps suggesting a fix if you have an idea how to do that). Uwe Ligges On 14.02.2012 12:42, Martin Batholdy wrote: Hi, according to ?rvm the relevance vector machine function as implemented in the kernlab-package has a

Re: [R] Cross-validation error with tune and with rpart

2011-12-31 Thread Prof Brian Ripley
On 31/12/2011 12:34, Israel Saeta Pérez wrote: Hello list, I'm trying to generate classifiers for a certain task using several methods, one of them being decision trees. The doubts come when I want to estimate the cross-validation error of the generated tree: tree<- rpart(y~., data=data.frame(x

Re: [R] cross-validation in rpart

2011-03-19 Thread Prof Brian Ripley
On Sat, 19 Mar 2011, Penny B wrote: I am trying to find out what type of sampling scheme is used to select the 10 subsets in 10-fold cross-validation process used in rpart to choose the best tree. Is it simple random sampling? Is there any documentation available on this? Not SRS (and least in

Re: [R] cross-validation in rpart

2011-03-19 Thread Allan Engelhardt
I assume you mean rpart::xpred.rpart ? The beauty of R means that you can look at the source. For the simple case (where xval is a single number) the code does indeed do simple random sampling xgroups<- sample(rep(1:xval, length = nobs), nobs, replace = FALSE) If you want another sampling,

Re: [R] Cross validation for Ordinary Kriging

2011-01-07 Thread Jon Olav Skoien
On 1/7/2011 12:40 PM, Jon Olav Skoien wrote: Pearl, The error suggests that there is something wrong with x2, and that there is a difference between the row names of the coordinates and the data. If you call str(x2) see if the first element of @coords is different from NULL, as this can caus

Re: [R] Cross validation for Ordinary Kriging

2011-01-07 Thread Jon Olav Skoien
Pearl, The error suggests that there is something wrong with x2, and that there is a difference between the row names of the coordinates and the data. If you call str(x2) see if the first element of @coords is different from NULL, as this can cause some problems when cross-validating. If it i

Re: [R] cross validation using e1071:SVM

2010-11-24 Thread Neeti
thank you so much for your help. if i am not wrong then createDataPartition can be used to create stratified random splits of a data set. is there other way to do that? Thank you -- View this message in context: http://r.789695.n4.nabble.com/cross-validation-using-e1071-SVM-tp3055335p3057684.

Re: [R] cross validation using e1071:SVM

2010-11-23 Thread Max Kuhn
Neeti, I'm pretty sure that the error is related to the confusionMAtrix call, which is in the caret package, not e1071. The error message is pretty clear: you need to pas in two factor objects that have the same levels. You can check by running the commands: str(pred_true1) str(species_tes

Re: [R] cross validation using e1071:SVM

2010-11-23 Thread Neeti
could anyone help me with my last problem. if the question is not clear please let me know thank you Hi everyone I am trying to do cross validation (10 fold CV) by using e1071:svm method. I know that there is an option (“cross”) for cross validation but still I wanted to make a funct

Re: [R] cross validation using e1071:SVM

2010-11-23 Thread Neeti
@Francial Giscard LIBENGUE please post your query again so that with different subject -- View this message in context: http://r.789695.n4.nabble.com/cross-validation-using-e1071-SVM-tp3055335p3055831.html Sent from the R help mailing list archive at Nabble.com. _

Re: [R] cross validation using e1071:SVM

2010-11-23 Thread Francial Giscard LIBENGUE
Hi everyone, Can you help me to plot Gamma(x/h+1) and Beta(x/h+1,(1-x)/h+1)?I want write x<-seq(0,3,0.1) thank 2010/11/23 Neeti > > Hi everyone > > I am trying to do cross validation (10 fold CV) by using e1071:svm method. > I > know that there is an option (“cross”) for cross validation but sti

Re: [R] cross-validation for choosing regression trees

2010-11-04 Thread Jonathan P Daily
Forgive me if I misunderstand your goals but I have no idea what you are trying to determine or what your data is. I can say, however, that setting mindev to 0 has always overfit data for me, and that you are more than likely looking at a situation in which that 1 node tree is more accurate. Al

Re: [R] cross validation of SVM

2010-06-16 Thread Uwe Ligges
From ?svm: cross if a integer value k>0 is specified, a k-fold cross validation on the training data is performed to assess the quality of the model: the accuracy rate for classification and the Mean Squared Error for regression Uwe Ligges On 15.06.2010 23:14, Amy Hessen wrote: hi, cou

Re: [R] cross-validation

2010-06-08 Thread Max Kuhn
Install the caret package and see ?train. There is also: http://cran.r-project.org/web/packages/caret/vignettes/caretTrain.pdf http://www.jstatsoft.org/v28/i05/paper Max On Tue, Jun 8, 2010 at 5:34 AM, azam jaafari wrote: > Hi > > I want to do leave-one-out cross-validation for multinom

Re: [R] cross-validation

2010-06-08 Thread Joris Meys
As far as my knowledge goes, nnet doesn't have a built-in function for crossvalidation. Coding it yourself is not hard though. Nnet is used in this book : http://www.stats.ox.ac.uk/pub/MASS4/ , which contains enough examples on how to do so. See also the crossval function in the bootstrap package.

Re: [R] Cross-validation for parameter selection (glm/logit)

2010-04-02 Thread Bert Gunter
Inline below: Bert Gunter Genentech Nonclinical Statistics -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Steve Lianoglou Sent: Friday, April 02, 2010 2:34 PM To: Jay Cc: r-help@r-project.org Subject: Re: [R] Cross-validation for

Re: [R] Cross-validation for parameter selection (glm/logit)

2010-04-02 Thread Steve Lianoglou
Hi, On Fri, Apr 2, 2010 at 9:14 AM, Jay wrote: > If my aim is to select a good subset of parameters for my final logit > model built using glm(). What is the best way to cross-validate the > results so that they are reliable? > > Let's say that I have a large dataset of 1000's of observations. I

Re: [R] Cross-validation for parameter selection (glm/logit)

2010-04-02 Thread JLucke
Jay Unless I have misunderstood some statistical subtleties, you can use the AIC in place of actual cross-validation, as the AIC is asymptotically equivalent to leave-out-one cross-validation under MLE. Joe Stone, M. An asymptotic equivalence of choice of model by cross-validation and Akaike's

Re: [R] cross-validation in plsr package

2010-02-22 Thread Max Kuhn
> The cross-validation in the pls package does not propose a number of > factors as optimum, you have to select this yourself.  (The reason for > this is that there is AFAIK no theoretically founded and widely accepted > way of doing this automatically.  I'd be happy to learn otherwise.) The caret

Re: [R] cross-validation in plsr package

2010-02-22 Thread Peter Tillmann
Kjaere Bjørn-Helge, > > > can anyone give an example how to use cross-validation in the plsr > package. > > There are examples in the references cited on > http://mevik.net/work/software/pls.html > > > I miss to find the number of factors proposed by cross-validation as > > optimum. > > T

Re: [R] cross-validation in plsr package

2010-02-22 Thread Bjørn-Helge Mevik
Peter Tillmann writes: > can anyone give an example how to use cross-validation in the plsr package. There are examples in the references cited on http://mevik.net/work/software/pls.html > I miss to find the number of factors proposed by cross-validation as > optimum. The cross-validation in t

Re: [R] cross validation function translated from stata

2010-01-21 Thread zhu yao
Thanks Frank and Steve. I rewrite the R code as follows. # m is the number of fold to split sample, n is the loop number of cross validation library(caret) calcvnb<-function(formula,dat,m,n) { cvnb<-rep(0,2) dim(cvnb)<-c(200,100) for (i in 1:n) { group<-rep(0,length=110) sg<-createFolds(d

Re: [R] cross validation function translated from stata

2010-01-21 Thread Frank E Harrell Jr
Take a look at the validate.lrm function in the rms package. Note that the use of threshold probabilities results in an improper scoring rule which will mislead you. Also note that you need to repeat 10-fold CV 50-100 times for precision, and that at each repeat you have to start from zero in

Re: [R] cross validation function translated from stata

2010-01-21 Thread Steve Lianoglou
Hi, On Thu, Jan 21, 2010 at 8:55 AM, zhu yao wrote: > Hi, everyone: > > I ask for help about translating a stata program into R. > > The program perform cross validation as it stated. > > #1. Randomly divide the data set into 10 sets of equal size, ensuring equal > numbers of events in each set >

Re: [R] cross validation for species distribution

2010-01-01 Thread Max Kuhn
Elaine, That's a fair answer, but completely not what I meant. I was hoping that you would elaborate on "the species data of species distribution models". What types of inputs and output for this particular modeling application etc. > Is it the same with the function inside caret, ipred, and e107

Re: [R] cross validation for species distribution

2010-01-01 Thread elaine kuo
Dear, Thanks for the warmful help on New Year's EVE. Cross-validation is used to validate the predictive quality of the training data with testing data. As for the amount, the cross-validation (cv) is supposed to be based on k-fold cross-validation, k-1 for the training and 1 for the testing.

Re: [R] cross validation for species distribution

2009-12-31 Thread Max Kuhn
You might want to be more specific about what you (exactly) intend to do. Reading the posting guide might help you get better answers. There are a few packages and functions to do what (I think) you desire. There is the train function in the caret package, the errorest function in ipred and

Re: [R] cross-validation in rpart

2009-05-26 Thread Paolo Radaelli
Dear R users, I know cross-validation does not work in rpart with user defined split functions. As Terry Therneau suggested, one can use the xpred.rpart function and then summarize the matrix of the predicted values into a single "goodness" value. I need only a confirmation: set for example xva

Re: [R] Cross-validation -> lift curve

2009-03-13 Thread Gene Leynes
This may be somewhat useful, but I might have more later. http://florence.acadiau.ca/collab/hugh_public/index.php?title=R:CheckBinFit (the code below is copied from the URL above) CheckBinFit <- function(y,phat,nq=20,new=T,...) { if(is.factor(y)) y <- as.double(y) y <- y-mean(y) y[y>0

Re: [R] Cross Validation output

2008-09-26 Thread Donald Catanzaro, PhD
Good Day All, I have a negative binomial model that I created using the function glm.nb() with the MASS library and I am performing a cross-validation using the function cv.glm() from the boot library. I am really interested in determining the performance of this model so I can have confiden

Re: [R] cross-validation in rpart

2008-07-07 Thread Terry Therneau
-- begin included message I'm having a problem with custom functions in rpart, and before I tear my hair out trying to fix it, I want to make sure it's actually a problem. It seems that, when you write custom functions for rpart (init, split and eval) then rpart no longer cross-validates the resul

Re: [R] Cross-validation in R

2008-06-10 Thread Prof Brian Ripley
1) cv.glm is not 'in R', it is part of contributed package 'boot'. Please give credit where it is due. 2) There is nothing 'cross' about your 'home-made cross validation'. cv.glm is support software for a book, so please consult it for the definition used of cross-validation, or MASS (the boo

Re: [R] Cross Validation

2008-03-08 Thread Uwe Ligges
JStainer wrote: > an example from my R table will calculating the average LOOCV for two > treatments ALL and AML > > table > ALL AML > 11.2 .3 > 2.87.3 > 31.1.5 > 41.2.7 > 53.21.2 > 61.11.1 > 7.90 .99 > 81.1.32 > 92.

Re: [R] Cross Validation

2008-03-08 Thread JStainer
JStainer wrote: > > Hi, > > I am trying to find out the best way to calculate the average LOOCV in R > for several classifier for, KNN, centroid classification, DLDA and SVM. > > I have four types of diseases and 62 samples. > > Is there a R code available to do this? > > > -- View thi

Re: [R] Cross Validation

2008-03-08 Thread JStainer
an example from my R table will calculating the average LOOCV for two treatments ALL and AML table ALL AML 11.2 .3 2.87.3 31.1.5 41.2.7 53.21.2 61.11.1 7.90 .99 81.1.32 92.1 1.2 JStainer wrote: > > Hi, > >

Re: [R] Cross Validation

2008-03-08 Thread JStainer
an example from my R table will calculating the average LOOCV for two treatments ALL and AML table ALL AML 11.2 .3 2.87.3 31.1.5 41.2.7 53.21.2 61.11.1 7.90 .99 81.1.32 92.1 1.2 JStainer wrote: > > Hi, > >

Re: [R] Cross Validation

2008-02-27 Thread Patrick Burns
http://www.burns-stat.com/pages/Tutor/bootstrap_resampling.html may be of some use to you. Patrick Burns [EMAIL PROTECTED] +44 (0)20 8525 0696 http://www.burns-stat.com (home of S Poetry and "A Guide for the Unwilling S User") Carla Rebelo wrote: >Hello, > >How can I do a cross validation in R