Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Greg Dropkin
Dear Prof Ripley yes, but if the estimate is biased it's good to know what the bias is. The problem illustrated in the simulations has nothing to do with ML, though, as the default fitting method in mgcv when scale is unknown is "GCV" and that is what was used, by default, here. The point about

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Greg Dropkin
hi Simon yes, I also got the right shape of the mean-variance relation but the wrong estimate of the parameter. thanks very much Greg > Hi Greg, > > Yes, this sounds right - with quasipoisson gam uses `extended > quasi-likelihood' (see McCullagh and Nelder's GLM book) to allow > estimation of t

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Prof Brian Ripley
On 05/02/2014 12:56, Greg Dropkin wrote: thanks Simon also, it appears at least with ML that the default scale estimate with quasipoisson (i.e. using dev) is the scale which minimises the ML value of the fitted model. So it is the "best" model but doesn't actually give the correct mean-variance

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Greg Dropkin
thanks Simon also, it appears at least with ML that the default scale estimate with quasipoisson (i.e. using dev) is the scale which minimises the ML value of the fitted model. So it is the "best" model but doesn't actually give the correct mean-variance relation. Is that right? thanks again Gre

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Simon Wood
Hi Greg, Yes, this sounds right - with quasipoisson gam uses `extended quasi-likelihood' (see McCullagh and Nelder's GLM book) to allow estimation of the scale parameter along with the smoothing parameters via (RE)ML, and it could well be that this gives a biased scale estimate with low count

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Simon Wood
Greg, The deviance being chi^2 distributed on the residual degrees of freedom works in some cases (mostly where the response itself can be reasonably approximated as Gaussian), but rather poorly in others (noteably low count data). This is what you are seeing in your simulations - in the firs

[R] mgcv: distribution of dev with Poisson data

2014-02-04 Thread Greg Dropkin
mgcv: distribution of dev hi I can't tell if this is a simple error. I'm puzzled by the distribution of dev when fitting a gam to Poisson generated data. I expected dev to be approximately chi-squared on residual d.f., i.e. about 1000 in each case below. In particular, the low values in the 3rd a