Re: [R] mgcv: relative risk from GAM with distributed lag

2022-07-23 Thread jade.shodan--- via R-help
Hi Simon and all, I've corrected some mistakes in setting up the prediction data frame (sorry, very stressed and sleep deprived due to closing in deadlines), but am having problems getting the perspective plot using persp(). I've calculated relative risk (RR) but am having trouble getting the per

Re: [R] mgcv: relative risk from GAM with distributed lag

2022-07-22 Thread jade.shodan--- via R-help
Hi Simon, Thanks for the pointers! But I'm not sure what to do with the 'time' variable. (I don't want to make predictions for specific points in time). I coded as follows (full reproducible example at bottom of email), but get a warning and error: N <- 1000 # number of points for smooth to

Re: [R] mgcv: relative risk from GAM with distributed lag

2022-07-22 Thread Simon Wood
On 21/07/2022 15:19, jade.shodan--- via R-help wrote: Hello everyone (incl. Simon Wood?), I'm not sure that my original question (see below, including reproducible example) was as clear as it could have been. To clarify, what I would to like to get is: 1) a perspective plot of temperature x l

Re: [R] mgcv: relative risk from GAM with distributed lag

2022-07-21 Thread jade.shodan--- via R-help
Hello everyone (incl. Simon Wood?), I'm not sure that my original question (see below, including reproducible example) was as clear as it could have been. To clarify, what I would to like to get is: 1) a perspective plot of temperature x lag x relative risk. I know how to use plot.gam and vis.ga

[R] mgcv: relative risk from GAM with distributed lag

2022-07-19 Thread jade.shodan--- via R-help
Dear list members, Does anyone know how to obtain a relative risk/ risk ratio from a GAM with a distributed lag model implemented in mgcv? I have a GAM predicting daily deaths from time series data consisting of daily temperature, humidity and rainfall. The GAM includes a distributed lag model bec

[R] mgcv: bam warning messages and non-convergence

2021-04-20 Thread Williamson, Michael via R-help
I have a large dataset of 118225 observations from 16 columns and as such I’ve been using bam, rather than gam, for my analyses. The response variable I’m using is count data but it’s overdispersed, and as such, I thought I’d use a negative binomial model. I have 5 explanatory variables, which

Re: [R] mgcv::gam() scale parameter estimates for quasibinomial error models

2021-04-15 Thread Simon Wood
Thanks John. It's a bug in weights handling. mgcv will give wrong scale parameter estimates for weighted models where the scale parameter is unknown, (except Gaussian, fortunately). quasibinomial with trials > 1 is one such case, because the weights are used to store the number of trials. Other

[R] mgcv::gam() scale parameter estimates for quasibinomial error models

2021-04-14 Thread John Maindonald
For both glm() and mgcv::gam() quasibinomial error models, the summary object has a dispersion value that has the same role as sigma^2 in the summary object for lm() model fits. Where some fitted probabilities are small, the `gam()` default scale parameter estimates, returned as `scale` (and `sig

[R] [mgcv] Memory issues with bam() on computer cluster

2019-03-20 Thread Frank van Berkum
@r-project.org You can reach the person managing the list at r-help-ow...@r-project.org When replying, please edit your Subject line so it is more specific than "Re: Contents of R-help digest..." Date: Fri, 15 Mar 2019 12:31:31 + From: Simon Wood To: r-help@r-project.org Sub

Re: [R] [mgcv] Memory issues with bam() on computer cluster

2019-03-15 Thread Simon Wood
Can you supply the results of sessionInfo() please, and the full bam call that causes this. best, Simon (mgcv maintainer) On 15/03/2019 09:09, Frank van Berkum wrote: > Dear Community, > > In our current research we are trying to fit Generalized Additive Models to a > large dataset. We are usi

[R] [mgcv] Memory issues with bam() on computer cluster

2019-03-15 Thread Frank van Berkum
Dear Community, In our current research we are trying to fit Generalized Additive Models to a large dataset. We are using the package mgcv in R. Our dataset contains about 22 million records with less than 20 risk factors for each observation, so in our case n>>p. The dataset covers the period

[R] mgcv::bam with monotonic constraints

2019-01-13 Thread Axel Urbiz
Dear List, I need to fit a GAM to a large dataset (`mgcv::bam` does this), but ensuring that some covariates have a monotonic relation with the response. `mgcv::mono.con` with `mgcv::pcls` seem to do this, but only for `mgcv::gam` (not mgcv::bam)? I'd really appreciate any pointers! Regards, Ax

Re: [R] mgcv : 3-way interaction and 3D-plots ?

2019-01-12 Thread varin sacha via R-help
David, Richard, Many thanks for your responses. Le mardi 8 janvier 2019 à 04:25:19 UTC+1, Richard M. Heiberger a écrit : ## Here is an example using the 3-way interaction plot from the HH package install.packages("HH") ## if necessary ## The HH package supports the book ## Statistic

Re: [R] mgcv : 3-way interaction and 3D-plots ?

2019-01-07 Thread Richard M. Heiberger
## Here is an example using the 3-way interaction plot from the HH package install.packages("HH") ## if necessary ## The HH package supports the book ## Statistical Analysis and Data Display ##Richard M. Heiberger and Burt Holland ## http://www.springer.com/us/book/9781493921218 library(HH)

Re: [R] mgcv : 3-way interaction and 3D-plots ?

2019-01-07 Thread David Winsemius
On 1/7/19 3:35 PM, varin sacha via R-help wrote: Dear R-experts, I have fitted a model with 2-way and 3-way interactions. I would like, for the 3-way interaction (year,age,by=education), to obtain 3D-plots. How could I do that ? Forget ggplot2. It has ignored this sort of visualization effo

[R] mgcv : 3-way interaction and 3D-plots ?

2019-01-07 Thread varin sacha via R-help
Dear R-experts, I have fitted a model with 2-way and 3-way interactions. I would like, for the 3-way interaction (year,age,by=education), to obtain 3D-plots. How could I do that ? Many thanks for your response. Here is the reproducible example: # install.packages("ISLR") library

Re: [R] MGCV:: boundary conditions in gam

2018-11-13 Thread Mark R Payne
Perfect! This might be a good example to add to the documentation of mgcv somewhere Thanks. Mark On Thu, 8 Nov 2018 at 22:08, Simon Wood wrote: > This first derivative penalty spline will do it, but the price paid is > that the curves are often quite wiggly. > > > library(mgcv); set.seed(

Re: [R] MGCV:: boundary conditions in gam

2018-11-08 Thread Simon Wood
This first derivative penalty spline will do it, but the price paid is that the curves are often quite wiggly. library(mgcv); set.seed(5) x <- runif(100); y <- x^4 + rnorm(100)*.1 b <- gam(y~s(x,m=1)) pd <- data.frame(x=seq(-.5,1.5,length=200)) ff <- predict(b,pd,se=TRUE) plot(x,y,xlim=c(-

[R] MGCV:: boundary conditions in gam

2018-11-08 Thread Mark R Payne
Dear R-help, I have a problem where I am using the mgcv package to in a situation where I am fitting a gam model with a 1-D spline smoother model over a domain [a,b] but then need to make predictions and extrapolate beyond b. Is there anyway where I force the first derivative of the spline to be z

[R] mgcv::gamm error when combining random smooths and correlation/autoregressive term

2018-04-18 Thread Mathew Guilfoyle
I am having difficulty fitting a mgcv::gamm model that includes both a random smooth term (i.e. 'fs' smooth) and autoregressive errors. Standard smooth terms with a factor interaction using the 'by=' option work fine. Both on my actual data and a toy example (below) I am getting the same error

[R] mgcv::gam is it possible to have a 'simple' product of 1-d smooths?

2018-01-17 Thread Mathew Guilfoyle
I am trying to test out several mgcv::gam models in a scalar-on-function regression analysis. The following is the 'hierarchy' of models I would like to test: (1) Y_i = a + integral[ X_i(t)*Beta(t) dt ] (2) Y_i = a + integral[ F{X_i(t)}*Beta(t) dt ] (3) Y_i = a + integral[ F{X_i(t),t} dt ]

[R] mgcv::gam is it possible to have a 'simple' product of 1-d smooths?

2018-01-17 Thread Mathew Guilfoyle
I am trying to test out several mgcv::gam models in a scalar-on-function regression analysis. The following is the 'hierarchy' of models I would like to test: (1) Y_i = a + integral[ X_i(t)*Beta(t) dt ] (2) Y_i = a + integral[ F{X_i(t)}*Beta(t) dt ] (3) Y_i = a + integral[ F{X_i(t),t} dt ]

[R] mgcv gam/bam model selection with random effects and AR terms

2017-04-08 Thread Mathew Guilfoyle
Would be grateful for advice on gam/bam model selection incorporating random effects and autoregressive terms. I have a multivariate time series recorded on ~500 subjects at ~100 time points. One of the variables (A) is the dependent and four others (B to E) are predictors. My basic formula i

Re: [R] mgcv: bam(), error in models with random intercepts and random slopes

2016-09-22 Thread Fotis Fotiadis
Dear Professor Wood, Thank you for taking the time to fix the problem. Best, Fotis On Thu, Sep 22, 2016 at 4:25 PM, Simon Wood wrote: > Hi Fotis, > > Thanks for the report, and sending me the data and code (off list). The > problem is triggered by 'ctrial' being a (one column) matrix. An imm

Re: [R] mgcv: bam(), error in models with random intercepts and random slopes

2016-09-22 Thread Simon Wood
Hi Fotis, Thanks for the report, and sending me the data and code (off list). The problem is triggered by 'ctrial' being a (one column) matrix. An immediate fix is data_a$ctrial <- as.numeric(data_a$ctrial) - mgcv 1.8-16 will catch the problem automatically internally. best, Simon On 20/09

Re: [R] mgcv: bam(), error in models with random intercepts and random slopes

2016-09-21 Thread Simon Wood
Any chance you could send me the data and exact code that produces this (I'll only use the data for investigating this issue of course - often data with the predictor replaced by noise will produce the same error, if sending the raw data is a problem)? best, Simon (mgcv maintainer) On 20/09/16

[R] mgcv: bam(), error in models with random intercepts and random slopes

2016-09-20 Thread Fotis Fotiadis
Hi all I am using the bam function of the mgcv package to model behavioral data of a learning experiment. To model individual variation in learning rate, I am testing models with (a) by-participant random intercepts of trial, (b) by-participant random slopes and random intercepts of trial, and (c)

Re: [R] mgcv::gam(): NA parametric coefficient in a model with two categorical variables + model interpretation

2016-05-24 Thread Fotis Fotiadis
Dear Prof. Wood Thank you, again, for your immediate response. Best, Fotis On Mon, May 23, 2016 at 4:32 PM, Simon Wood wrote: > Q1: It looks like the model is not fully identifiably given the data and > as a result igcCAT.ideo has been set to zero - there is no sensible test to > conduct with

Re: [R] mgcv::gam(): NA parametric coefficient in a model with two categorical variables + model interpretation

2016-05-23 Thread Simon Wood
Q1: It looks like the model is not fully identifiably given the data and as a result igcCAT.ideo has been set to zero - there is no sensible test to conduct with such a term, hence the NAs in the test stat an p-value fields. Q2: A separate (centred) smooth is estimated for each level of igc. I

[R] mgcv::gam(): NA parametric coefficient in a model with two categorical variables + model interpretation

2016-05-22 Thread Fotis Fotiadis
Hallo all I am using a gam model for my data. m2.4<-bam(acc~ 1 + igc + s(ctrial, by=igc) + shape + s(ctrial, by=shape) + s(ctrial, sbj, bs = "fs", m = 1) , data=data, family=binomial) igc codes condition and there are four levels (CAT.pseudo, CAT.ideo,PA.pseudo, PA.ideo), and shape is a factor (

Re: [R] mgcv: BAM convergence conflicting messages

2014-06-20 Thread Trevor Davies
Hi Simon, Thanks for getting back to me. Would the opposite be true as well then? I have other models where I don't get a warning message at the end of the model fitting but the $converged slow indicates a 'FALSE'. Thank you ~Trevor On Wed, Jun 18, 2014 at 2:07 AM, Simon Wood wrote: > I thin

Re: [R] mgcv: I can't manually reconstruct a P-spline from a GAM's coefficients

2014-06-18 Thread Alexander Engelhardt
Hello Simon, thanks for your response! I incorporated the intercept, i.e. coef(my_gam)[1] into the spline. That is, my B-spline coefficients are the intercept for the first one, and for all others I added the intercept to them (see the code below if I'm not clear) Now I *almost* reproduce t

Re: [R] mgcv: BAM convergence conflicting messages

2014-06-18 Thread Simon Wood
I think it didn't converge. The warning message is for the whole fitting iteration, whereas mod07.3.BASE.bam5.1$converged indicates whether smoothing parameter selection converged at the final step of the iteration. best, Simon On 17/06/14 19:10, Trevor Davies wrote: I'm running some spatial

Re: [R] mgcv: I can't manually reconstruct a P-spline from a GAM's coefficients

2014-06-18 Thread Simon Wood
mgcv:gam automatically imposes sum-to-zero constraints on the smooths in a model (even if there is only one smooth). This is to avoid lack of identifiability with the intercept. The constraint removes one coefficient and shifts the curve... best, Simon On 17/06/14 15:40, Alexander Engelhardt

[R] mgcv: BAM convergence conflicting messages

2014-06-17 Thread Trevor Davies
I'm running some spatial GAMs and am using the bam call (negbin family) and am getting conflicting information on whether the model is converging or not. When the model completes its run, I get a warning message that the model did not converge. When I look at the object itself, I'm told that it d

[R] mgcv: I can't manually reconstruct a P-spline from a GAM's coefficients

2014-06-17 Thread Alexander Engelhardt
Hello R-helpers, I am working through Simon Wood's GAM book and want to specify my own knot locations (on even tens, i.e. 10, 20, 30, etc.). Then, I want to compute a GAM on that area, and given the coefficients, reconstruct the same P-spline that is drawn in plot(my_gam). I'm failing. Here

[R] mgcv gam bs=re questions

2014-05-03 Thread William Shadish
Dear R-helpers, I am working on a project assessing the prevalence and variance (random effects) of linear and nonlinear trends in a data set that has short time series (each time series identified as PID 1 through 5). I am using mgcv (gam) with the bs=”re” option (more on why not gamm or gamm

Re: [R] mgcv::gam.check qq plot residuals are not standardized?

2014-04-25 Thread Simon Wood
Hi Michael, You seem to have a quite recent mgcv if you are using qq.gam, so a help file for version 1.3 is probably not going to be much help (gam.fit2 no longer exists, for example). By default qq.gam plots deviance residuals (see ?qq.gam). So the default is standardization. When possible

[R] mgcv::gam.check qq plot residuals are not standardized?

2014-04-25 Thread Folkes, Michael
Hello all, I looking for confirmation of what I'm seeing. The qq plot in gam.check and qq.gam is not standardizing the residuals. The current help doesn't suggest they're standardized. Somehow I found, online, a help for gam.check from version [Package mgcv version 1.3-23 Index]: "If the fit

[R] mgcv, should include a intercept for the 'by' varying coefficient model, which is unconstrained

2014-03-17 Thread Xing Zhao
Dear Dr. Wood and other mgcv experts In ?gam.models, it says that the numeric "by" variable is genrally not subjected to an identifiability constraint, and I used the example in ?gam.models, finding some differences (code below). I think the the problem might become serious when several varying

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Greg Dropkin
Dear Prof Ripley yes, but if the estimate is biased it's good to know what the bias is. The problem illustrated in the simulations has nothing to do with ML, though, as the default fitting method in mgcv when scale is unknown is "GCV" and that is what was used, by default, here. The point about

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Greg Dropkin
hi Simon yes, I also got the right shape of the mean-variance relation but the wrong estimate of the parameter. thanks very much Greg > Hi Greg, > > Yes, this sounds right - with quasipoisson gam uses `extended > quasi-likelihood' (see McCullagh and Nelder's GLM book) to allow > estimation of t

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Prof Brian Ripley
On 05/02/2014 12:56, Greg Dropkin wrote: thanks Simon also, it appears at least with ML that the default scale estimate with quasipoisson (i.e. using dev) is the scale which minimises the ML value of the fitted model. So it is the "best" model but doesn't actually give the correct mean-variance

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Greg Dropkin
thanks Simon also, it appears at least with ML that the default scale estimate with quasipoisson (i.e. using dev) is the scale which minimises the ML value of the fitted model. So it is the "best" model but doesn't actually give the correct mean-variance relation. Is that right? thanks again Gre

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Simon Wood
Hi Greg, Yes, this sounds right - with quasipoisson gam uses `extended quasi-likelihood' (see McCullagh and Nelder's GLM book) to allow estimation of the scale parameter along with the smoothing parameters via (RE)ML, and it could well be that this gives a biased scale estimate with low count

Re: [R] mgcv: distribution of dev with Poisson data

2014-02-05 Thread Simon Wood
Greg, The deviance being chi^2 distributed on the residual degrees of freedom works in some cases (mostly where the response itself can be reasonably approximated as Gaussian), but rather poorly in others (noteably low count data). This is what you are seeing in your simulations - in the firs

[R] mgcv: distribution of dev with Poisson data

2014-02-04 Thread Greg Dropkin
mgcv: distribution of dev hi I can't tell if this is a simple error. I'm puzzled by the distribution of dev when fitting a gam to Poisson generated data. I expected dev to be approximately chi-squared on residual d.f., i.e. about 1000 in each case below. In particular, the low values in the 3rd a

[R] mgcv - markeov random field option

2014-01-01 Thread Helena Baptista
Dear R-users, Happy new year to all! I have been using the mgcv package, and I have run some models using the option mrf, for saptial data. But I have found quite hard to interpret the results. I could not find a lot of documentation on that, examples and so on, so I was wondering if anyone can h

[R] mgcv gam modeling trend variation over cases

2013-12-04 Thread William Shadish
Dear R-Helpers, I posted two days ago on testing significance of random effects in mgcv, but realize I did not make my overall purpose clear. I have a series of N short time series, where N might range from 3-10 and short means a median of 20 time points. The sample data below (PCP) has N = 4

[R] mgcv predict.bam strange results

2013-08-08 Thread fwickler
Dear useR, I don't understand the results of the predict.bam function of mgcv package when constucting a varying-coefficient model with bam instead of gam: library("mgcv") dat <- gamSim(4) b <- gam(y ~ fac+s(x2,by=fac)+s(x0), data=dat) predict(b, dat[1,], type = "terms") with gam everything is

Re: [R] MGCV: Degrees of freedom of smooth terms

2013-07-25 Thread Gavin Simpson
On Tue, 2013-07-23 at 11:16 +0200, Christoph Scherber wrote: > Dear all, > > This is just a quick question regarding degrees of freedom in GAM > models fit by MGCV (using select=T): > > Can I roughly interpret them as: > > 1 df: linear effect of x on y > 2 df: approximately quadratic of x on y >

Re: [R] mgcv: pcls() makes everything linear

2013-07-23 Thread Simon Wood
I can't see anything immediately wrong except: 1. presumably there are repeated values in 'road_quiet' aren't there? If so then your inequality constraint matrix will contain constraints that are *exact* copies of each other. I'm not sure, and don't immediately have time to try it out, but it c

[R] mgcv: pcls() makes everything linear

2013-07-23 Thread Kathrine Veie
Dear R helplisters, I am trying to implement a mononicity constraint on a smooth term in my generalized additive model with the mgcv package (v. 1.7-18). I adapted the code from an example given in the help file for pcls(). The example runs just as one would expect, but when I adapt it and use

[R] MGCV: Degrees of freedom of smooth terms

2013-07-23 Thread Christoph Scherber
Dear all, This is just a quick question regarding degrees of freedom in GAM models fit by MGCV (using select=T): Can I roughly interpret them as: 1 df: linear effect of x on y 2 df: approximately quadratic of x on y 3 df: approximately cubic effect of x on y? 1 df for a spatial term s(x,y): bil

[R] mgcv: Impose monotonicity constraint on single or more smooth terms

2013-07-19 Thread Kathrine Veie
Dear R help list, This is a long post so apologies in advance. I am estimating a model with the mgcv package, which has several covariates both linear and smooth terms. For 1 or 2 of these smooth terms, I "know" that the truth is monotonic and downward sloping. I am aware that a new package "s

Re: [R] MGCV: overlay fitted (marginal) curves over a plot of the original data

2013-07-16 Thread Christoph Scherber
Meanwhile, I found the solution myself: using plot.gam() with shift=intercept and trans=exp (for a poisson model) does the job. I can then add the original data using points() Thanks again for your help, which is greatly appreciated! Best wishes Christoph Am 16/07/2013 11:04, schrieb Simon Woo

Re: [R] MGCV: overlay fitted (marginal) curves over a plot of the original data

2013-07-16 Thread Christoph Scherber
Thanks, the sequence of x0 values was clearly too short. However, is there a way to overlay the (marginal) curve from plot.gam() over a plot of (x,y) values? Best wishes Christoph Am 16/07/2013 11:04, schrieb Simon Wood: > Probably you didn't want to set x0=0:1? Here is some code, to do what

Re: [R] MGCV: overlay fitted (marginal) curves over a plot of the original data

2013-07-16 Thread Simon Wood
Probably you didn't want to set x0=0:1? Here is some code, to do what you want. (The CI shape is not identical to the plot(b) version as the uncertainty includes the uncertainty in the other smooths and the intercept now.) library(mgcv) set.seed(2) dat <- gamSim(1,n=400,dist="normal",scale=2) b

[R] MGCV: overlay fitted (marginal) curves over a plot of the original data

2013-07-16 Thread Christoph Scherber
Dear R users, I´ve stumbled over a problem that can be easily seen from the R code below: - When I use plot.gam() on a fitted model object, I get a nice and well-looking smooth curve for all terms in the model. - However, when I use predict(model) for a given predictor, with values of all othe

Re: [R] mgcv: GAM with clustered standard errors

2013-07-11 Thread Simon Wood
I think it's going to be a problem to have different sized groups in your second model. ?corSymm says that a general correlation matrix is being estimated (i.e. the correlation between each pair of observations is being estimated - for this to be meaningful across groups you need the jth price

[R] mgcv: GAM with clustered standard errors

2013-07-11 Thread Kathrine Veie
Dear Help list, I am relatively new to the mgcv package, which I am using to model prices of housing transactions as a function of the characteristics of a home and a neighborhood. I have several smooth terms to capture price evolution over time, but also to non-parametrically fit the functiona

[R] mgcv summary.gam

2013-06-21 Thread Greg Dropkin
hi I'm trying to understand (a little) the code behind summary.gam, and have the Biometrika article referred to in the Help. But am stuck early on. In the code, starting at line 167: if (est.disp) rdf <- residual.df else rdf <- -1 res <- testStat(p, Xt, V, df[i], type = p.type, res.df = rdf)

Re: [R] mgcv summary.gam

2013-06-21 Thread Simon Wood
Hi Greg On 21/06/13 08:02, Greg Dropkin wrote: > hi > > I'm trying to understand (a little) the code behind summary.gam, and have > the Biometrika article referred to in the Help. But am stuck early on. > > In the code, starting at line 167: > > if (est.disp) >rdf <- residual.df > else rdf <-

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-25 Thread Jan Holstein
Juliet, for you the diagnostic plots: just to recall: the first model was this: fit<-gam(target ~s(mgs)+s(gsd)+s(mud)+s(ssCmax),family=quasi(link=log),data=wspe1,method="REML",select=F) > summary(fit) Parametric coefficients: Estimate Std. Error t value Pr(>|t|)

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-24 Thread Juliet Hannah
Hi Jan and Simon, If possible, could you attach the diagnostic plots. I would be curious to see them. Thanks, Juliet On Fri, Apr 19, 2013 at 4:39 AM, jholstei wrote: > Simon, > > that was very instructive—very special thanks to you. > I already noticed that the model was bad, but it was not

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-19 Thread jholstei
Simon, that was very instructive—very special thanks to you. I already noticed that the model was bad, but it was not clear to me that transformation of predictors to, say a more centered distribution is helpful here. And thanks for pointing out Tweedie, I noticed that the error structure is fa

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-18 Thread Simon Wood
Jan, Thanks for the data (off list). The p-value computations are based on the approximation that things are approximately normal on the linear predictor scale, but actually they are no where close to normal in this case, which is why the p-values look inconsistent. The reason that the approx

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-18 Thread Simon Wood
Jan, Thanks for this. Is there any chance that you could send me the data off list and I'll try to figure out what is happening? (Under the understanding that I'll only use the data for investigating this issue, of course). best, Simon on 18/04/13 11:11, Jan Holstein wrote: Simon, thanks

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-18 Thread Jan Holstein
Simon, thanks for the reply, I guess I'm pretty much up to date using mgcv 1.7-22. Upgrading to R 3.0.0 also didn't do any change. Unfortunately using method="REML" does not make any difference: ### first with "select=FALSE" > fit<-gam(target > ~s(mgs)+s(gsd)+s(mud)+s(ssCmax),family=quasi(

Re: [R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-17 Thread Simon Wood
Jan, What mgcv version are you using, please? (Older versions have a poor p-value approximation when select=TRUE, but of course it's possible that you've managed to break the newer approximation as well) The 'select=TRUE' option adds a penalty to each smooth, to allow it to be penalized out

[R] mgcv: how select significant predictor vars when using gam(...select=TRUE) using automatic optimization

2013-04-17 Thread Jan Holstein
I have 11 possible predictor variables and use them to model quite a few target variables. In search for a consistent manner and possibly non-manual manner to identify the significant predictor vars out of the eleven I thought the option "select=T" might do. Example: (here only 4 pedictors) firs

Re: [R] [mgcv][gam] Odd error: Error in PredictMat(object$smooth[[k]], data) : , `by' variable must be same dimension as smooth arguments

2013-03-22 Thread Andrew Crane-Droesch
Solved my own problem. If interested: http://stackoverflow.com/questions/15563589/odd-error-error-in-predictmatobjectsmoothk-data-by-variable-must On 03/21/2013 02:14 PM, Andrew Crane-Droesch wrote: Dear List, I'm getting an error in mgcv, and I can't figure out where it comes from. The se

[R] [mgcv][gam] Odd error: Error in PredictMat(object$smooth[[k]], data) : , `by' variable must be same dimension as smooth arguments

2013-03-21 Thread Andrew Crane-Droesch
Dear List, I'm getting an error in mgcv, and I can't figure out where it comes from. The setup is the following: I've got a fitted GAM object called "MI", and a vector of "prediction data" (with default values for predictors). I feed this into predict.gam(object, newdata = whatever) via th

[R] mgcv: Plotting probabilities for binomial GAM with crossed random intercepts and factor by variable

2013-01-10 Thread Jan Vanhove
mgcv: Constructing probabilities for binomial GAM with crossed random intercepts and factor by variable Hello, (I'm sorry if this has been discussed elsewhere; I may not have been looking in the right places.) I ran a binomial GAM in which "Correct" is modelled in terms of the participant's

Re: [R] [mgcv][gam] Problem defining axis labels for non-smooth terms (via termplot)

2012-12-20 Thread Simon Wood
plot(fit,all.terms=TRUE,select=3,xlabs="???") On 20/12/12 02:06, Andrew Crane-Droesch wrote: Dear List, plot.gam appears to be having trouble communicating its xlab and ylab information to termplot. A simple example: library(mgcv) x = 1:1000 y = runif(1000)*x^.5 z = rnorm(1000)*y other = sin(

[R] [mgcv][gam] Problem defining axis labels for non-smooth terms (via termplot)

2012-12-19 Thread Andrew Crane-Droesch
Dear List, plot.gam appears to be having trouble communicating its xlab and ylab information to termplot. A simple example: library(mgcv) x = 1:1000 y = runif(1000)*x^.5 z = rnorm(1000)*y other = sin(z) fit = gam(y~s(x)+s(z)+other) plot(fit,all.terms=TRUE) plot(fit,select=3,xlab="???") de

Re: [R] [mgcv][gam] Manually defining my own knots?

2012-12-03 Thread Simon Wood
Andrew, I think you mean dumb.example2$coefficients = averaged.models ~~~ currently your code adds a new vector 'coeff' to the gam object, rather than modifying 'coefficients'. (A good example of why forgiving languages like R are dangerous, and you should really wr

Re: [R] [mgcv][gam] Manually defining my own knots?

2012-12-02 Thread Andrew Crane-Droesch
Hi Simon, Thanks for your help. I've got another question if you don't mind -- is it possible to "swap out" a set of coefficients of a gamObject in order to change the results when that gamObject is plotted? The (silly) example below illustrates that this is possible with the Vp matrix. But

Re: [R] [mgcv][gam] Manually defining my own knots?

2012-12-02 Thread Simon Wood
Hi Andrew, mgcv matches the knots to the smooth arguments by name. If an element of 'knots' has no name it will be ignored. The following will do what you want... dumb.example = gam(y~s(x,k=3),knots=list(x=dumb.knots)) best, Simon On 29/11/12 23:44, Andrew Crane-Droesch wrote: Dear List, I'

[R] [mgcv][gam] Manually defining my own knots?

2012-11-29 Thread Andrew Crane-Droesch
Dear List, I'm using GAMs in a multiple imputation project, and I want to be able to combine the parameter estimates and covariance matrices from each completed dataset's fitted model in the end. In order to do this, I need the knots to be uniform for each model with partially-imputed data.

Re: [R] mgcv package, problems with NAs in gam

2012-08-25 Thread grace
Hi Simon, I'm trying to fit a negative binomial gam with no covariates, that therefore looks at the detection/non-detection data and nothing else. I thought having 1~1 as formula would allow the model to just estimate occurrence without looking at relationship between variables. On 25 Aug 20

Re: [R] mgcv package, problems with NAs in gam

2012-08-25 Thread Simon Wood
'gam' doesn't know what to do with the model formula '1 ~ 1' (i.e. "one tilde one"). What is it supposed to mean? 'glm' also does nothing meaningful in this case... ## code to load you data file into 'dat' omitted > model<-glm(1~1, data=dat) > model Call: glm(formula = 1 ~ 1, data = dat) Co

Re: [R] mgcv package, problems with NAs in gam

2012-08-25 Thread grace
I've attached the data to this message. http://r.789695.n4.nabble.com/file/n4641291/2005pip.csv 2005pip.csv -- View this message in context: http://r.789695.n4.nabble.com/mgcv-package-problems-with-NAs-in-gam-tp4641253p4641291.html Sent from the R help mailing list archive at Nabble.com.

Re: [R] mgcv package, problems with NAs in gam

2012-08-24 Thread Bert Gunter
Grace: Confession: I loved that error message! -- and it seems pretty clear to me. What does "to no avail" mean -- in particular, what happened when you changed your NA's to 0? Presumably you did not get the same error message, again, but something else, right? What else? Modulo the above vague

[R] mgcv package, problems with NAs in gam

2012-08-24 Thread grace
Hi there, I'm using presence-absence data in a gam (i.e. 0 or 1 as values) I am trying to run a gam with 'dummy covariates' i.e. 1~1 unfortunately my model: * model<-gam(1~1, data=bats, family=negbin)* keeps putting out: * Error in gam(1 ~ 1, data = bats, family = negbin) : Not enough (non-NA

Re: [R] mgcv and gamm4: REML, GCV, and AIC

2012-08-09 Thread Ben Bolker
john benson hotmail.com> writes: > I've been using gamm4 to build GAMMs for exploring environmental > influences on genetic ancestry. Things have gone well and I have 2 > very straightforward questions: 1. I've used method=REML. Am I > correct that this is an alternative method for estimating

[R] mgcv and gamm4: REML, GCV, and AIC

2012-08-08 Thread john benson
Hi, I've been using gamm4 to build GAMMs for exploring environmental influences on genetic ancestry. Things have gone well and I have 2 very straightforward questions: 1. I've used method=REML. Am I correct that this is an alternative method for estimating the smooth functions in GAMMs rather

Re: [R] mgcv 1.7-19, vis.gam(): "invalid 'z' limits'

2012-08-01 Thread janvanhove
I somehow solved the problem - kind of. The data set on which I ran the GAM model contains many more variables than are needed in the model, so I created a new data set in R and reran the GAM model on the slimmed down data set. Same problem: The GAM can be computed, but the tensor product cannot be

Re: [R] mgcv 1.7-19, vis.gam(): "invalid 'z' limits'

2012-07-31 Thread janvanhove
Hi Simon, Thanks for your reply. m <- bam(Correct ~ cEnglishTotal + te(WSTResid, RavenResid) + s(Stimulus, bs="re") + s(Subject, bs="re"), data = dat, family = "binomial") # cEnglishTotal, WSTResid and RavenResid are continuous variables; Correct, Stimulus and Subject are factors. > vis.gam(m, v

Re: [R] mgcv 1.7-19, vis.gam(): "invalid 'z' limits'

2012-07-30 Thread Simon Wood
Jan, Could you send the exact gam call and exact vis.gam call that did this please? Also, if 'm' denotes your fitted model, what result does 'fitted(m)' give? and what is the output from print(m)? best, Simon On 07/30/2012 09:19 PM, janvanhove wrote: Hi everyone, I ran a binomial GAM consi

[R] mgcv 1.7-19, vis.gam(): "invalid 'z' limits'

2012-07-30 Thread janvanhove
Hi everyone, I ran a binomial GAM consisting of a tensor product of two continuous variables, a continuous parametric term and crossed random intercepts on a data set with 13,042 rows. When trying to plot the tensor product with vis.gam(), I get the following error message: Error in persp.default

Re: [R] mgcv: Extract random effects from gam model

2012-07-23 Thread Simon Wood
Hi Jan , coef(mod) will extract the coefficients, including for a. They are labelled and for 'a' are in the same order as levels(a). best, Simon On 23/07/12 10:08, janvanhove wrote: Hi everyone, I can't figure out how to extract by-factor random effect adjustments from a gam model (mgcv p

[R] mgcv: Extract random effects from gam model

2012-07-23 Thread janvanhove
Hi everyone, I can't figure out how to extract by-factor random effect adjustments from a gam model (mgcv package). Example (from ?gam.vcomp): library(mgcv) set.seed(3) dat <- gamSim(1,n=400,dist="normal",scale=2) a <- factor(sample(1:10,400,replace=TRUE)) b <- factor(sample(1:7,400,replace=TRUE)

Re: [R] mgcv: inclusion of random intercept in model - based on p-value of smooth or anova?

2012-07-19 Thread Simon Wood
Martin, I've just submitted mgcv_1.7-19 to CRAN, which includes a major upgrade of the p-value computation for random effect terms (and any other smooth term which can be penalized to zero as part of estimation). The new p-values are still conditional on the smoothing parameter/variance compo

Re: [R] mgcv: inclusion of random intercept in model - based on p-value of smooth or anova?

2012-06-25 Thread Simon Wood
Martin, I had a nagging feeling that there must be more to this than my previous reply suggested, and have been digging a bit further. Basically I would expect these p-values to not be great if you had nested random effects (such as main effects + their interaction), but your case looked rathe

[R] mgcv

2012-06-23 Thread ywh123
Hi,Dear Professor Wood, I am studying the generalized additive model(GAM) and I have read your related papers,especially the book named "Generalized Additive Models:An Introduction with R".I am benefit a lots.However,I also have some questions about GAM models. First,Is there some restriction

Re: [R] MGCV: Use of irls.reg option

2012-06-22 Thread r-help . 20 . trevva
Hi Simon, Thanks for taking the time to reply. Please let me explain a few more details. The problem that I am working on is essentially the same as the Bristol Channel Sole Egg distribution example in your book and in the "soap" paper but instead it is Herring Larvae in the English Channel - sam

Re: [R] MGCV: Use of irls.reg option

2012-06-21 Thread Simon Wood
Hi Mark, irls.reg is kind of `legacy code'. Does model fitting actually fail for your example, or is it just that the estimated spatial smooth looks unpleasant? best, Simon On 06/21/2012 01:28 AM, r-help.20.tre...@spamgourmet.com wrote: Hi, In the help files in the mgcv package for the ga

[R] MGCV: Use of irls.reg option

2012-06-20 Thread r-help . 20 . trevva
Hi, In the help files in the  mgcv package for the gam.control() function, there is an option irls.reg. The help files describe this option as: For most models this should be 0. The iteratively re-weighted least squares method by which GAMs are fitted can fail to converge in some circumstances. F

Re: [R] mgcv: inclusion of random intercept in model - based on p-value of smooth or anova?

2012-06-11 Thread Simon Wood
Hi Martijn, Irrespective of the p-value, 'bam' and 'lmer' agree that the variance component for 'Placename' is practically zero. In the 'bam' output see the 'edf' for s(Placename), or for a more direct comparison call gam.vcomp(m1). As mentioned in ?summary.gam the p-values for "re" terms ar

  1   2   >