Thanks, Thierry and other R users.
I estimate the model using the factor rather than the dummy variables I used
previously. It still takes forever for the function "lme" to run. But "lmer"
is much better with my large data size (about 60,000 observations).
The interesting part is that the results from the model using factor are
slightly different from what I got from the model using dummy variables,
especially for the variables at random level.
The estimated random effects by using dummy variable are like this (each
dummy got one intercept):
Random effects:
Groups Name Variance Std.Dev. Corr
TAZ (Intercept) 0.059160 0.24323
MH_D 0.215210 0.46391 -0.583
TAZ (Intercept) 0.212061 0.46050
APT_D 0.205028 0.45280 -0.992
TAZ (Intercept) 0.086223 0.29364
ResOth_D 0.305678 0.55288 0.665
TAZ (Intercept) 0.161892 0.40236
NonRes_D 0.537284 0.73300 -0.874
TAZ (Intercept) 0.088684 0.29780
Vacant_noimp_D 0.501495 0.70816 -0.570
TAZ (Intercept) 0.136630 0.36964
Vacant_imp_D 0.368722 0.60722 -0.850
Residual 0.382439 0.61842
Number of obs: 55762, groups: TAZ, 739
The estimated random effects by using factor are like this (one intercept
for all):
Random effects:
Groups Name Variance Std.Dev. Corr
TAZ (Intercept) 0.83894 0.91594
HousingType1MH_D 0.23214 0.48181 -0.375
HousingType1APT_D 0.28850 0.53712 -0.827 0.630
HousingType1ResOth_D 0.29392 0.54214 0.156 -0.251 -0.165
HousingType1NonRes_D 0.58169 0.76269 -0.572 0.155 0.656
-0.030
HousingType1Vacant_imp_D 0.45349 0.67342 -0.522 0.203 0.265
0.101 0.611
HousingType1Vacant_noimp_D 0.54146 0.73584 -0.286 0.251 0.265
0.390 0.313 0.475
Residual 0.38228 0.61829
Number of obs: 55762, groups: TAZ, 739
The fixed coefficients for each group are also slightly different. I'm
wondering which one makes more sense.
Thanks.
Harry
Harry
R still report error
On Wed, Aug 5, 2009 at 1:22 AM, ONKELINX, Thierry
<thierry.onkel...@inbo.be>wrote:
> Harry,
>
> I you use dummy variables, then you can only use (n-1) dummy variables
> if your variable has n levels. Otherwise you introduce
> multicollinearity! If you use n dummy variable then you can express one
> dummy variable as a linear combination of the others.
>
> Make use of a factor variable. That is much easier to work with that
> dummy variables. The model itself will create the necessary dummy
> variables.
>
> lusdrdata$HousingType <- factor(lusdrdata$HousingType, levels = 1:6,
> labels = c("Reference", "MH_D", "APT_D", "ResOth_D", "NonRes_D",
> "Vacant_D"))
> lme(fixed = LN_unitlandval ~ HousingType +
> access_emp1+pct_vacant+transit_D +park_dum,data=lusdrdata, random = ~
> HousingType | TAZ)
>
> HTH,
>
> Thierry
>
> ------------------------------------------------------------------------
> ----
> ir. Thierry Onkelinx
> Instituut voor natuur- en bosonderzoek / Research Institute for Nature
> and Forest
> Cel biometrie, methodologie en kwaliteitszorg / Section biometrics,
> methodology and quality assurance
> Gaverstraat 4
> 9500 Geraardsbergen
> Belgium
> tel. + 32 54/436 185
> thierry.onkel...@inbo.be
> www.inbo.be
>
> To call in the statistician after the experiment is done may be no more
> than asking him to perform a post-mortem examination: he may be able to
> say what the experiment died of.
> ~ Sir Ronald Aylmer Fisher
>
> The plural of anecdote is not data.
> ~ Roger Brinner
>
> The combination of some data and an aching desire for an answer does not
> ensure that a reasonable answer can be extracted from a given body of
> data.
> ~ John Tukey
>
> -----Oorspronkelijk bericht-----
> Van: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> Namens Hongwei Dong
> Verzonden: woensdag 5 augustus 2009 1:49
> Aan: r-help@r-project.org
> Onderwerp: Re: [R] lme funcion in R
>
> Yeah, I have a very large sample size, about 60,000 observations.
> Multicollinearity should not be a problem here. The weird thing is that
> SPSS can converge very quickly and gives out reasonable results.
> The only problem I can think of is that, my first level (random)
> variables are dummy variables: 6 housing types, and I used five dummies
> in model and one as the reference. I also tried to combine them into two
> groups and use only dummy at random level, but it does not work either.
>
> is there any one here has similar experience with the LME function in R?
>
> Thanks.
>
> Harry
>
>
>
> On Tue, Aug 4, 2009 at 1:28 AM, ONKELINX, Thierry
> <thierry.onkel...@inbo.be>wrote:
>
> > Dear Harry,
> >
> > Your model seems rather complex. Do you have enough data to support
> it?
> > Did you check for multicollinearity between the variables?
> >
> > HTH,
> >
> > Thierry
> >
> >
> >
> > ----------------------------------------------------------------------
> > --
> > ----
> > ir. Thierry Onkelinx
> > Instituut voor natuur- en bosonderzoek / Research Institute for Nature
>
> > and Forest Cel biometrie, methodologie en kwaliteitszorg / Section
> > biometrics, methodology and quality assurance Gaverstraat 4 9500
> > Geraardsbergen Belgium tel. + 32 54/436 185 thierry.onkel...@inbo.be
> > www.inbo.be
> >
> > To call in the statistician after the experiment is done may be no
> > more than asking him to perform a post-mortem examination: he may be
> > able to say what the experiment died of.
> > ~ Sir Ronald Aylmer Fisher
> >
> > The plural of anecdote is not data.
> > ~ Roger Brinner
> >
> > The combination of some data and an aching desire for an answer does
> > not ensure that a reasonable answer can be extracted from a given body
>
> > of data.
> > ~ John Tukey
> >
> > -----Oorspronkelijk bericht-----
> > Van: r-help-boun...@r-project.org
> > [mailto:r-help-boun...@r-project.org]
> > Namens Hongwei Dong
> > Verzonden: maandag 3 augustus 2009 19:45
> > Aan: r-help@r-project.org
> > Onderwerp: Re: [R] lme funcion in R
> >
> > Thanks for the replies above. Here are my script and data structure:
> > library(nlme)
> > tlevel<-lme(fixed = LN_unitlandval ~
> > MH_D+APT_D+ResOth_D+NonRes_D+Vacant_D+access_emp1+pct_vacant+transit_D
> > +p
> > ark_dum,data=lusdrdata,random
> > = ~MH_D+APT_D+ResOth_D+NonRes_D+Vacant_D | TAZ)
> >
> > str:
> >
> > $ TAZ : int 100 100 100 100 100 100 100 100 100 100 ...
> > $ MH_D : num 0 0 0 0 0 0 0 0 0 0 ...
> > $ APT_D : num 0 0 0 0 0 0 0 0 0 0 ... $ ResOth_D : num 0 0 0 0 0 0 0 0
>
> > 0 0 ... $ NonRes_D : num 0 0 0 0 0 0 0 0 0 1 ...
> > $ Vacant_D : num 1 1 1 0 0 1 1 1 1 0 ...
> > $ access_emp1 : num 45.8 45.8 45.8 45.8 45.8 ...
> > $ pct_vacant : num 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 ... $
> > transit_D :
> > num 0 0 0 0 0 0 0 0 0 0 ... $ park_dum : num 0 0 0 0 0 0 0 0 0 0 ...
> >
> >
> > Thanks.
> >
> > Harry
> >
> >
> >
> > On Mon, Aug 3, 2009 at 10:36 AM, Jason Morgan <jwm-r-h...@skepsi.net>
> > wrote:
> >
> > > On 2009.08.03 10:15:46, Hongwei Dong wrote:
> > > > Hi, R users,
> > > > I'm using the "lme" function in R to estimate a 2 level mixed
> > > > effects model, in which the size of the subject groups are
> > > > different. It turned
> > > out
> > > > that It takes forever for R to converge. I also tried the same
> > > > thing
> >
> > > > in
> > > SPSS
> > > > and SPSS can give the results out within 20 minutes. Anyone can
> > > > give
> >
> > > > me
> > > some
> > > > advice on the lme function in R, especially why R does not
> converge?
> > > Thanks.
> > > >
> > > > Harry
> > >
> > > Hello Harry,
> > >
> > > As Chuck mentions, providing some more information on the model and
> > > the data you are using would be helpful. Also, be sure to compare
> > > the optimization methods used in SPSS to that used in R. You can
> > > change the optimization method in R if the default seems to be
> > > causing issues. See help(lmeControl) for numerous setting options.
> > >
> > > ~Jason
> > >
> > > --
> > > Jason W. Morgan
> > > Graduate Student
> > > Department of Political Science
> > > *The Ohio State University*
> > > 154 North Oval Mall
> > > Columbus, Ohio 43210
> > >
> > >
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
> > Dit bericht en eventuele bijlagen geven enkel de visie van de
> > schrijver weer en binden het INBO onder geen enkel beding, zolang dit
> > bericht niet bevestigd is door een geldig ondertekend document. The
> > views expressed in this message and any annex are purely those of the
>
> > writer and may not be regarded as stating an official position of
> > INBO, as long as the message is not confirmed by a duly signed
> > document.
> >
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> Dit bericht en eventuele bijlagen geven enkel de visie van de schrijver
> weer
> en binden het INBO onder geen enkel beding, zolang dit bericht niet
> bevestigd is
> door een geldig ondertekend document. The views expressed in this message
> and any annex are purely those of the writer and may not be regarded as
> stating
> an official position of INBO, as long as the message is not confirmed by a
> duly
> signed document.
>
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
