Dear Tim,
I think that in this paper you would find a suite of different metrics
to evaluate your hurdle model:
Potts, Joanne M., and Jane Elith. "Comparing species abundance models."
Ecological Modelling 199.2 (2006): 153-163.
Best regards,
David March
El 12/03/2014 18:55, Tim Marcella escribió:
Hi,
My data is characterized by many zeros (82%) and overdispersion. I have
chosen to model with hurdle regression (pscl package) with a negative
binomial distribution for the count data. In an effort to validate the
model I would like to calculate the RMSE of the predicted vs. the observed
values. From my reading I understand that this is the calculated on the raw
residuals generated from the model output. This is the formula I used
H1.RMSE <- sqrt(mean(H1$residuals^2)) # Where H1 is my fitted hurdle
model
I get 46.7 as the RMSE. This seems high to me based on the model results.
Assuming my formula and my understanding of RMSE is correct (and please
correct me if I am wrong) I question whether this is an appropriate use of
validation for this particular structure of model. The hurdle model
correctly predicts all of my zeros. The predictions I get from the fitted
model are all values greater than zero. From my readings I understand that
the predictions from the fitted hurdle model are means generated for the
particular covariate environment based on the model coefficients. If this
is truly the case it does not make sense to compare these means to the
observations. This will generate large residuals (only 18% of the
observations contain counts greater than 0, while the predicted counts all
exceed 0). It seems like comparing apples to oranges. Other correlative
tests (Pearson's r, Spearman's p) would seem to be comparing the mean
predicted value for particular covariate to the observed which again is
heavily dominated by zeros.
Any tips on how best to validate hurdle models in R?
Thanks
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
David March Morlà
Spatial Ecologist
Email: da...@imedea.uib-csic.es
IMEDEA
Instituto Mediterraneo de Estudios Avanzados (UIB-CSIC)
C/Miquel Marquès 21, 07190 Esporles, Balearic Islands. Spain
www.imedea.uib.es
SOCIB
Balearic Islands Coastal Observing and Forecasting System
Strategic Issues and Applications for Society (SIAS Division)
Parc Bit, Naorte, Bloc A 2ºp. pta. 3, 07121 Palma de Mallorca. Spain
Tel: +034 971 43 97 64
www.socib.es
SOCIAL MEDIA
Google Scholar: http://scholar.google.es/citations?user=xABsDpAAAAAJ
Research Gate: https://www.researchgate.net/profile/David_March3/
Linked In: http://www.linkedin.com/in/dmarch
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
