Thanks a lot, Martin and William. Looks like, we can't apply prediction on lsfit and lm.fit objects. Because, i am trying to use lm object to predict the values for new data frame.
Thanks & Regards Sasi On Tue, Nov 17, 2015 at 9:49 AM, Martin Maechler <maech...@stat.math.ethz.ch > wrote: > >>>>> William Dunlap <wdun...@tibco.com> > >>>>> on Mon, 16 Nov 2015 16:01:42 -0800 writes: > > > If a quick running time is important and your models involve only > > numeric data with no missing values and you are willing to spend more > > programming time setting things up, the lsfit() function may work > > better for you. > > > Bill Dunlap > > TIBCO Software > > wdunlap tibco.com > > or even faster is the extra-simple but fast .lm.fit() function > (in R >= 3.1.0). > > I've written a small demo about it and published it here, > http://rpubs.com/maechler/fast_lm > > Martin Maechler, ETH Zurich (and R Core) > > > > On Mon, Nov 16, 2015 at 3:25 PM, Sasikumar Kandhasamy < > ckms...@gmail.com> wrote: > >> Thanks a lot Bill & Bert. > >> > >> Hi Bill, > >> > >> Sorry i was wrong on number of records, actually, i am using two > dimensional > >> data of 250K records each. And regarding CPU usage, it was the > elapsed time. > >> Infact, i have pined one core to run R. > >> > >> Thanks & Regards > >> Sasi > >> > >> On Mon, Nov 16, 2015 at 2:04 PM, William Dunlap <wdun...@tibco.com> > wrote: > >>> > >>> You cannot do a linear regression with one column of data - there > must > >>> be at least one response column and one predictor. By default, lm > >>> throws in a constant term which gives you a second predictor. If > your > >>> predictor is categorical, you get a new column for all but the > first > >>> unique value in it. > >>> > >>> lm() deals only with double precision data, at 8 bytes/number. > Thus > >>> 250k numbers occupies 2 million bytes. Your three columns (in the > >>> non-categorical-predictor case) take up 6 million bytes, > >>> > >>> lm()'s output contains several columns the size of the response > >>> variable: residuals, effects, and fitted.values. It also contains > the > >>> QR decomposition of the design matrix (the size of all the > predictor > >>> columns together). > >>> > >>> There are also some temporary variables generated in the course of > the > >>> computation. > >>> > >>> So your observed 40 MB memory usage seems reasonable. > >>> > >>> Use the object.size() function to see how big objects are and > str() to > >>> look at their structure. > >>> > >>> My laptop with a 2.5 GHz Intel i7 processor takes a quarter > second to > >>> fit a simple linear model with one numeric predictor and a constant > >>> term. 6 seconds sounds slow. Is that cpu or elapsed time (use > >>> system.time() to see)? > >>> > >>> > >>> > >>> Bill Dunlap > >>> TIBCO Software > >>> wdunlap tibco.com > >>> > >>> > >>> On Mon, Nov 16, 2015 at 12:25 PM, Sasikumar Kandhasamy > >>> <ckms...@gmail.com> wrote: > >>> > Hi All, > >>> > > >>> > I have couple of clarifications on R run-time performance. I have > >>> > R-3.2.2 > >>> > package compiled for MIPS64 and am running it on my linux > machine with > >>> > mips64 processor (core speed 1.5GHz) and observing the following > >>> > behaviors, > >>> > > >>> > 1. Applying "linear regression model" (lm) on 1MB of data > (contains 1 > >>> > column of 250K records) takes ~6 seconds to complete. Anyidea, > is it an > >>> > expected behavior or not? If not, can you please the suggestions > or > >>> > options > >>> > to improve if we have any? > >>> > > >>> > 2. Also, the R process runtime virtual memory is increased by > 40MB after > >>> > applying the linear model on 1MB data. Is it also expected > behavior? If > >>> > it > >>> > is expected, can you please share the insight of memory usage? > >>> > > >>> > Thanks in advance. > >>> > > >>> > Regards > >>> > Sasi > >>> > > >>> > [[alternative HTML version deleted]] > >>> > > >>> > ______________________________________________ > >>> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, > see > >>> > https://stat.ethz.ch/mailman/listinfo/r-help > >>> > PLEASE do read the posting guide > >>> > http://www.R-project.org/posting-guide.html > >>> > and provide commented, minimal, self-contained, reproducible > code. > >> > >> > > > ______________________________________________ > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
