Hello, I need help with the details of loess prediction algorithm. I would like to get it implemented as a part of a measurement system programmed in LabView. My job is provide a detailed description of the algorithm. This is a simple one-dimensional problem - smoothing an (x, y) data set.
I found quite a detailed description of the fitting procedure in the "white book". It is also described in great detail at the NIST site in the Engineering Statistics Handbook. It provides an example of Loess computations. I managed to reproduce their example exactly in R. At each data point I compute a weighted local linear fit using the number of points based of span. Then I predict the values from these local fits. This matches R "loess" predictions exactly. The problem starts when I try to predict at x values not in the data set. The "white book" does not talk about predictions at all. In the NIST handbook in the "Final note on Loess Computations" they mention this type of predictions but just say that the same steps are used for predictions as for fitting. When I try to use "the same steps" I get predictions that are quite different that the predictions obtained by fitting R loess model to a data set and running predict(<model object>, newdata=<grid of x values>). They match quite well at the lowest and highest ends of the x grid but in the middle are different and much less smooth. I can provide details but basically what I do to create the predictions at x0 is this: 1. I append c(x0, NA) to the data frame of (x, y) data. 2. I calculate abs(xi-x0), i.e., absolute deviations of the x values in the data set and a given x0 value. 3. I sort the data set according to these deviations. This way the first row has the (x0, NA) value. 4. I drop the first row. 5. I divide all the deviations by the m-th one, where m is the number of points used in local fitting - m = floor(n*span) where n is the number of points. 6. I calculate the "tricube" weights and assign 0's to the negative ones. This eliminates all the points except the m points of interest. 7. I fit a linear weighted regression using lm. 8. I predict y(x0) from this linear model. This is basically the same procedure I use to predict at the x values from the data set, except for point 4. I got the R sources for loess but it looks to me like most of the work is done in a bunch of Fortran modules. They are very difficult to read and understand, especially since they handle multiple x values. A couple of things that worry me are parameters in loess.control such as surface and cell. They seem to have something to do with predictions but I do not account for them in my simple procedure above. Could anyone shed a light on this problem? Any comment will be appreciated. I apologize in advance if this should have been posted in r-help. I figured that I have a better chance asking people who read the r-devel group, since they are likely to know more details about inner workings of R. Thanks in advance, Andy __________________________________ Andy Jaworski 518-1-01 Process Laboratory 3M Corporate Research Laboratory ----- E-mail: [EMAIL PROTECTED] Tel: (651) 733-6092 Fax: (651) 736-3122 ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
