I see the same problem on Windows XP. But if I run loess with surface='direct' then the results are correct. So it looks like the problem comes from the smoothing/interpolating, not the main loess algorithm.
-- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 > -----Original Message----- > From: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r- > project.org] On Behalf Of Ryan Hafen > Sent: Thursday, March 05, 2009 7:43 AM > To: Prof Brian Ripley > Cc: Uwe Ligges; Berwin A Turlach; r-de...@stat.math.ethz.ch; Peter > Dalgaard > Subject: Re: [Rd] bug (PR#13570) > > > On Mar 5, 2009, at 7:59 AM, Prof Brian Ripley wrote: > > > On Thu, 5 Mar 2009, Peter Dalgaard wrote: > > > >> Prof Brian Ripley wrote: > >>> Undortunately the example is random, so not really reproducible > >>> (and I > >>> see nothing wrong on my Mac). However, Linux valgrind on R-devel is > >>> showing a problem: > >>> > >>> ==3973== Conditional jump or move depends on uninitialised value(s) > >>> ==3973== at 0xD76017B: ehg141_ (loessf.f:532) > >>> ==3973== by 0xD761600: lowesa_ (loessf.f:769) > >>> ==3973== by 0xD736E47: loess_raw (loessc.c:117) > >>> > >>> (The uninitiialized value is in someone else's code and I suspect > >>> it was > >>> either never intended to work or never tested.) No essential > >>> change has > >>> been made to the loess code for many years. > >>> > >>> I would not have read the documentation to say that degree = 0 was > a > >>> reasonable value. It is not to my mind 'a polynomial surface', and > >>> loess() is described as a 'local regression' for degree 1 or 2 in > >>> the > >>> reference. So unless anyone wants to bury their heads in that > >>> code I > >>> think a perfectly adequate fix would be to disallow degree = 0. > >>> (I vaguely recall debating allowing in the code ca 10 years ago.) > >> > >> The code itself has > >> > >> if (!match(degree, 0:2, 0)) > >> stop("'degree' must be 0, 1 or 2") > >> > >> though. "Local fitting of a constant" essentially becomes kernel > >> smoothing, right? > > > > I do know the R code allows it: the question is whether it is worth > > the effort of finding the problem(s) in the underlying c/dloess > > code, whose manual (and our reference) is entirely about 1 or 2. I > > am concerned that there may be other things lurking in the degree=0 > > case if it was never tested (in the netlib version: I am sure it was > > only minmally tested through my R interface). > > > > I checked the original documentation on netlib and that says > > > > 29 DIM dimension of local regression > > 1 constant > > d+1 linear (default) > > (d+2)(d+1)/2 quadratic > > Modified by ehg127 if cdeg<tdeg. > > > > which seems to confirm that degree = 0 was intended to be allowed, > > and what I dimly recall from ca 1998 is debating whether the R code > > should allow that or not. > > > > If left to me I would say I did not wish to continue to support > > degree = 0. > > True. There are plenty of reasons why one wouldn't want to use > degree=0 anyway. And I'm sure there are plenty of other simple ways > to achieve the same effect. > > I ran into the problem because some code I'm planning on distributing > as part of a paper submission "blends" partway down to degree 0 > smoothing at the endpoints to reduce the variance. The only bad > effect of disallowing degree 0 is for anyone with code depending on > it, although there are probably few that use it and better to disallow > than to give an incorrect computation. I got around the problem by > installing a modified loess by one of Cleveland's former students: > https://centauri.stat.purdue.edu:98/loess/ > (but don't want to require others who use my code to do so as well). > > What is very strange to me is that it has been working fine in > previous R versions (tested on 2.7.1 and 2.6.1) and nothing has > changed in the loess source but yet it is having problems on 2.8.1. > Would this suggest it not being a problem with the netlib code? > > Also strange that it reportedly works on Linux but not on Mac or > Windows. On the mac, the effect was much smaller. With windows, it > was predicting values like 2e215 whereas on the mac, you would almost > believe the results were legitimate if you didn't think about the fact > that a weighted moving average involving half the data shouldn't > oscillate so much. > > If the consensus is to keep degree=0, I'd be happy to help try to find > the problem or provide a test case or something. Thanks for looking > into this. > > Ryan > > > > >> > >> > >>> On Thu, 5 Mar 2009, Uwe Ligges wrote: > >>> > >>>> Berwin A Turlach wrote: > >>>>> G'day Peter, > >>>>> > >>>>> On Thu, 05 Mar 2009 09:09:27 +0100 > >>>>> Peter Dalgaard <p.dalga...@biostat.ku.dk> wrote: > >>>>> > >>>>>> rha...@stat.purdue.edu wrote: > >>>>>>> <<insert bug report here>> > >>>>>>> > >>>>>>> This is a CRITICAL bug!!! I have verified it in R 2.8.1 for > mac > >>>>>>> and for windows. The problem is with loess degree=0 smoothing. > >>>>>>> For example, try the following: > >>>>>>> > >>>>>>> x <- 1:100 > >>>>>>> y <- rnorm(100) > >>>>>>> plot(x, y) > >>>>>>> lines(predict(loess(y ~ x, degree=0, span=0.5))) > >>>>>>> > >>>>>>> This is obviously wrong. > >>>>>> Obvious? How? I don't see anything particularly odd (on Linux). > >>>>> > >>>>> Neither did I on linux; but the OP mentioned mac and windows. On > >>>>> windows, on running that code, the lines() command added a lot of > >>>>> vertical lines; most spanning the complete window but some only > >>>>> part. > >>>>> Executing the code a second time (or in steps) gave sensible > >>>>> results. My guess would be that some memory is not correctly > >>>>> allocated or > >>>>> initialised. Or is it something like an object with storage mode > >>>>> "integer" being passed to a double? But then, why doesn't it > >>>>> show on > >>>>> linux? > >>>>> > >>>>> Happy bug hunting. If my guess is correct, then I have no idea > >>>>> how to > >>>>> track down such things under windows..... > >>>>> > >>>>> Cheers, > >>>>> > >>>>> Berwin > >>>>> > >>>>> ______________________________________________ > >>>>> R-devel@r-project.org mailing list > >>>>> https://stat.ethz.ch/mailman/listinfo/r-devel > >>>> > >>>> > >>>> Please can you folks try under R-devel (to be R-2.9.0 in a couple > >>>> of > >>>> weeks) and report if you still see it. I do not under R-devel > >>>> (but do > >>>> under R-release), so my guess is that something called by loess() > >>>> has > >>>> been fixed in the meantime. > >>>> > >>>> Moreover it is not the plot stuff that was wrong under R-2.8.1 > >>>> (release) but the loess computations. > >>>> > >>>> Uwe Ligges > >>>> > >>>> ______________________________________________ > >>>> R-devel@r-project.org mailing list > >>>> https://stat.ethz.ch/mailman/listinfo/r-devel > >>>> > >>> > >> > >> > >> -- > >> O__ ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B > >> c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K > >> (*) \(*) -- University of Copenhagen Denmark Ph: (+45) > >> 35327918 > >> ~~~~~~~~~~ - (p.dalga...@biostat.ku.dk) FAX: (+45) > >> 35327907 > >> > >> > > > > -- > > Brian D. Ripley, rip...@stats.ox.ac.uk > > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > > University of Oxford, Tel: +44 1865 272861 (self) > > 1 South Parks Road, +44 1865 272866 (PA) > > Oxford OX1 3TG, UK Fax: +44 1865 272595 > > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
