Nestor Fernandez
2005-Apr-17 13:39 UTC
[R] generalized linear mixed models - how to compare?
Dear all, I want to evaluate several generalized linear mixed models, including the null model, and select the best approximating one. I have tried glmmPQL (MASS library) and GLMM (lme4) to fit the models. Both result in similar parameter estimates but fairly different likelihood estimates. My questions: 1- Is it correct to calculate AIC for comparing my models, given that they use quasi-likelihood estimates? If not, how can I compare them? 2- Why the large differences in likelihood estimates between the two procedures? Thanks, Nestor
Deepayan Sarkar
2005-Apr-17 16:31 UTC
[R] generalized linear mixed models - how to compare?
On Sunday 17 April 2005 08:39, Nestor Fernandez wrote:> Dear all, > > I want to evaluate several generalized linear mixed models, including > the null model, and select the best approximating one. I have tried > glmmPQL (MASS library) and GLMM (lme4) to fit the models. Both result > in similar parameter estimates but fairly different likelihood > estimates. > My questions: > 1- Is it correct to calculate AIC for comparing my models, given that > they use quasi-likelihood estimates? If not, how can I compare them? > 2- Why the large differences in likelihood estimates between the two > procedures?The likelihood reported by glmmPQL is wrong, as it's the likelihood of an incorrect model (namely, an lme model that approximates the correct glmm model). GLMM uses (mostly) the same procedure to get parameter estimates, but as a final step calculates the likelihood for the correct model for those estimates (so the likelihood reported by it should be fairly reliable). Deepayan
Andrew Criswell
2005-Apr-20 01:08 UTC
[R] generalized linear mixed models - how to compare?
Hello All: Should I conclude from this discussion that there is no practical means by which nested generalized mixed models can be compared from output produced through glmmPQL or GLMM? What is one then to do??? Andrew On Sun, 17 Apr 2005, Deepayan Sarkar wrote:> On Sunday 17 April 2005 08:39, Nestor Fernandez wrote:>> I want to evaluate several generalized linear mixed models, including >> the null model, and select the best approximating one. I have tried >> glmmPQL (MASS library) and GLMM (lme4) to fit the models. Both result >> in similar parameter estimates but fairly different likelihood >> estimates. >> My questions: >> 1- Is it correct to calculate AIC for comparing my models, given that >> they use quasi-likelihood estimates? If not, how can I compare them? >> 2- Why the large differences in likelihood estimates between the two >> procedures? > > > The likelihood reported by glmmPQL is wrong, as it's the likelihood of > an incorrect model (namely, an lme model that approximates the correct > glmm model).Actually glmmPQL does not report a likelihood. It returns an object of class "lme", but you need to refer to the reference for how to interpret that. It *is* support software for a book.> GLMM uses (mostly) the same procedure to get parameter estimates, but as a final step calculates the likelihood for the correct model for those estimates (so the likelihood reported by it should be fairly reliable).Well, perhaps but I need more convincing. The likelihood involves many high-dimensional non-analytic integrations, so I do not see how GLMM can do those integrals -- it might approximate them, but that would not be `calculates the likelihood for the correct model'. It would be helpful to have a clarification of this claim. (Our experiments show that finding an accurate value of the log-likelihood is difficult and many available pieces of software differ in their values by large amounts.) Further, since neither procedure does ML fitting, this is not a maximized likelihood as required to calculate an AIC value. And even if it were, you need to be careful as often one GLMM is a boundary value for another, in which case the theory behind AIC needs adjustment. -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html -- Andrew R. Criswell, Ph.D. Graduate School, Bangkok University
Bill.Venables@csiro.au
2005-Apr-20 01:46 UTC
[R] generalized linear mixed models - how to compare?
Andrew Criswell asks: Hello All: Should I conclude from this discussion that there is no practical means by which nested generalized mixed models can be compared from output produced through glmmPQL or GLMM? [WNV] The picture is, in my view, not as bleak as this, but there are are certainly many open questions in this area and much research left to do. What is one then to do??? [WNV] Research in statistics, perhaps? Every little bit helps. It is a mistake to assume that everything is known about even the common approximations used in statistical practice, and this area is still opening up. Andrew On Sun, 17 Apr 2005, Deepayan Sarkar wrote: > On Sunday 17 April 2005 08:39, Nestor Fernandez wrote: >> I want to evaluate several generalized linear mixed models, including >> the null model, and select the best approximating one. I have tried >> glmmPQL (MASS library) and GLMM (lme4) to fit the models. Both result >> in similar parameter estimates but fairly different likelihood >> estimates. >> My questions: >> 1- Is it correct to calculate AIC for comparing my models, given that >> they use quasi-likelihood estimates? If not, how can I compare them? >> 2- Why the large differences in likelihood estimates between the two >> procedures? > > > The likelihood reported by glmmPQL is wrong, as it's the likelihood of > an incorrect model (namely, an lme model that approximates the correct > glmm model). Actually glmmPQL does not report a likelihood. It returns an object of class "lme", but you need to refer to the reference for how to interpret that. It *is* support software for a book. > GLMM uses (mostly) the same procedure to get parameter estimates, but as a final step calculates the likelihood for the correct model for those estimates (so the likelihood reported by it should be fairly reliable). Well, perhaps but I need more convincing. The likelihood involves many high-dimensional non-analytic integrations, so I do not see how GLMM can do those integrals -- it might approximate them, but that would not be `calculates the likelihood for the correct model'. It would be helpful to have a clarification of this claim. (Our experiments show that finding an accurate value of the log-likelihood is difficult and many available pieces of software differ in their values by large amounts.) Further, since neither procedure does ML fitting, this is not a maximized likelihood as required to calculate an AIC value. And even if it were, you need to be careful as often one GLMM is a boundary value for another, in which case the theory behind AIC needs adjustment. -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html -- Andrew R. Criswell, Ph.D. Graduate School, Bangkok University ______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Seemingly Similar Threads
- summary: Generalized linear mixed model software
- generalized linear mixed model by ML
- How to do generalized linear mixed effects models
- R vs Stata on generalized linear mixed models: glmer and xtmelogit
- glmmADMB: Generalized Linear Mixed Models using AD Model Builder