On Friday 15 December 2006 22:38, BRENDAN KLICK wrote:> For summary(GAM) in the mgcv package smooth the degrees of freedom for
> the F value for test of smooth terms are the rank of covariance matrix
> of \hat{beta} and the residuals df. I've noticed that in a lot of GAMs
> I've fit the rank of the covariance turns out to be 9. In Simon
Wood's
> book, the rank of covariance matrix is usually either 9 or 99 (pages
> 239-230 and 259).
>
> Can anyone comment on why so many smooth terms have a denominator
> degree of freedom involving 9. Simon Wood writes "r is usually
> determinted numerically, while forming the pseudoinverse of the
> covariance matrix, or with reference to the effective degrees of freedom
> of the term" which doesn't really clarify the issue for me at
least.
The rank used for the covariance matrix is often the number of free
coefficients associated with the term (i.e. k-1, the maximum EDF for the term
less the identifiability constraint). The idea is to base the test statistic
on the parts of the model space that are not completley supressed by the
penalization of the terms, so if penalization is not very high then this may
mean the whole space. 9 occurs frequently because by default k=10 for a 1-D
smooth. Where 99 occurs it's because a basis dimension (k) of 100 was being
employed. The rank used is less than k-1 when some subspace of the model
space has been very heavily penalized, so that it should not contribute
anything to the test statistic.
Finally... these tests are not great, and only provide a rough guide to
significance: the worst failing is the neglect of smoothing parameter
uncertainty. See the final example in ?summary.gam to get an indication of
how well/badly the p-values perform in practice.
best,
Simon
>
> Thanks.
>
> Brendan Klick
> Johns Hopkins University School of Medicine.
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html and provide commented, minimal,
> self-contained, reproducible code.
-- > Simon Wood, Mathematical Sciences, University of Bath, Bath, BA2 7AY UK
> +44 1225 386603 www.maths.bath.ac.uk/~sw283