On Sunday 30 January 2011 09:10:30 Sturla Molden wrote: > Den 29.01.2011 12:40, skrev Algis Kabaila: > > So my question is: how can one reliably detect singularity > > (or near singularity) and raise an exception? > > Use an SVD, examine the singular values. I gather that SVD is the Singular Value Decomposition, but I have no idea how to perform such decomposition. Would you care to refer me to some simple source material? I have been advised to watch the condition numbers. No doubt, SVD and condition numbers are related. The references about condition numbers are very interesting and I intend to follow them in the first instance.
> In statistics we sometimes see ill-conditioning of covariance > matrices. Another way to deal with multicollinearity besides > SVD/PCA is regularisation. Simply adding a small bias k*I to > the diagonal might fix the problem (cf. ridge regression). > In the Levenberg-Marquardt algorithm used to fit non-linear > least squares models (cf. > scipy.optimize.leastsq), the bias k to the diagonal of the > Jacobian is changed adaptively. One might also know in > advance if a covariance matrix could be ill-conditioned (the > number of samples is small compared to the number of > dimensions) or singular (less data than parameters). That > is, sometimes we don't even need to look at the matrix to > give the correct diagnosis. Another widely used strategy is > to use Cholesky factorization on covariance matrices. It is > always stable unless there is a singularity, for which it > will fail (NumPy will raise a LinAlgError exception). > Cholesky is therefore safer to use for inverting covariance > matrices than LU (as well as faster). If Cholesky fails one > might fallback to SVD or regularisation to correct the > problem. > > Sturla > My knowledge of statistics is rather limited, though our son Dr. Paul Kabaila is a specialist in that area. My interests lie in the area of Analysis of Engineering Structures - it saves my brain from falling to a permafrost like sleep :) Thank you for your reply - greatly appreciated. Al. PS: Paul, I thought there is a minuscule chance that this is of some interest to you. Tete. -- Algis http://akabaila.pcug.org.au/StructuralAnalysis.pdf _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
