Thanks a lot. I have changed the calculation method by using optimism defined by Efron. The results from using boot and rms packages are quite close now.
Jim --- On Tue, 6/3/12, Frank Harrell <f.harr...@vanderbilt.edu> wrote: > From: Frank Harrell <f.harr...@vanderbilt.edu> > Subject: Re: [R] 632 estimator using boot package > To: r-help@r-project.org > Date: Tuesday, 6 March, 2012, 3:23 > Jim if I understand you, you are > using the ordinary bootstrap instead of the > Efron-Gong "optimism bootstrap". I don't think the > ordinary bootstrap is > applicable here. > Frank > > Jin Minming wrote > > > > Hello Frank, > > > > Thanks for your advice. > > > > I also used the validation using rms package. The > results from rms > > indicate that the linear model used in my project is > overfitted around 0.1 > > in terms of R2. But, the results from using ordinary > boot method in boot > > package suggest the mean of R2 is even higher than the > original one. Then > > I guess maybe direclty using Boot package may not be a > good idea for > > assessing the overfitting in the regression. > > > > Jim > > > > > > --- On Mon, 5/3/12, Frank Harrell > <f.harrell@> wrote: > > > >> From: Frank Harrell <f.harrell@> > >> Subject: Re: [R] 632 estimator using boot package > >> To: r-help@ > >> Date: Monday, 5 March, 2012, 17:06 > >> Bootstrapping does not leave one > >> out. As for .632 this is implemented in the > >> rms package's validate and calibrate functions. > Note > >> however that any > >> claimed advantages of .632 over the ordinary > optimism > >> bootstrap seem to be a > >> result only of the use of a discontinuous improper > scoring > >> role (proportion > >> classified correctly). The advantage may vanish > when > >> better scoring rules > >> are used. Simulations showing this may be found > at > >> http://biostat.mc.vanderbilt.edu/rms > >> > >> Frank > >> > >> Jin Minming wrote > >> > > >> > Dear All, > >> > > >> > Anyone has some idea how to implement 632 > estimator and > >> leave-one out > >> > bootstraping method by using boot package. I > know > >> the bootstrap package > >> > has this function, but it sounds not very > flexible for > >> my project. > >> > > >> > Thanks, > >> > > >> > Jim > >> > > >> > > ______________________________________________ > >> > R-help@ mailing list > >> > https://stat.ethz.ch/mailman/listinfo/r-help > >> > PLEASE do read the posting guide > >> > http://www.R-project.org/posting-guide.html > >> > and provide commented, minimal, > self-contained, > >> reproducible code. > >> > > >> > >> > >> ----- > >> Frank Harrell > >> Department of Biostatistics, Vanderbilt University > >> -- > >> View this message in context: > >> http://r.789695.n4.nabble.com/632-estimator-using-boot-package-tp4446720p4446736.html > >> Sent from the R help mailing list archive at > Nabble.com. > >> > >> ______________________________________________ > >> R-help@ > >> mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide > >> http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, > reproducible > >> code. > >> > > > > ______________________________________________ > > R-help@ mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, > reproducible code. > > > > > ----- > Frank Harrell > Department of Biostatistics, Vanderbilt University > -- > View this message in context: > http://r.789695.n4.nabble.com/632-estimator-using-boot-package-tp4446720p4448633.html > Sent from the R help mailing list archive at Nabble.com. > > ______________________________________________ > R-help@r-project.org > mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible > code. > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
