Re: [R] SVD on very large data matrix

On 8 Apr 2013, at 23:21, Andy Cooper wrote: > So, no one has direct experience running irlba on a data matrix as large as > 500,000 x 1,000 or larger? I haven't used irlba in production code, but ran a few benchmarks on much smaller matrices. My impression was (also from the documentation, I

Re: [R] SVD on very large data matrix

Cc: "r-help@R-project.org" Sent: Monday, 8 April 2013, 20:31 Subject: Re: [R] SVD on very large data matrix > > > Dear All, > > I need to perform a SVD on a very large data matrix, of dimension ~ 500,000 x > 1,000 , and I am looking > for an efficient algorithm th

Re: [R] SVD on very large data matrix

On 08-04-2013, at 16:44, Andy Cooper wrote: > > > Dear All, > > I need to perform a SVD on a very large data matrix, of dimension ~ 500,000 x > 1,000 , and I am looking > for an efficient algorithm that can perform an approximate (partial) SVD to > extract on the order of the top 50 > right

Re: [R] SVD on very large data matrix

Hi Andy, On Mon, Apr 8, 2013 at 7:44 AM, Andy Cooper wrote: > > > > Dear All, > > I need to perform a SVD on a very large data matrix, of dimension ~ 500,000 x 1,000 , and I am looking > for an efficient algorithm that can perform an approximate (partial) SVD to extract on the order of the top 50

Re: [R] SVD on very large data matrix

No answer, but first obvious question" Is the matrix sparse? Next obvious question: what's your ram, OS, etc. (Reply to list, as I can't help further). -- Bert On Mon, Apr 8, 2013 at 7:44 AM, Andy Cooper wrote: > > > Dear All, > > I need to perform a SVD on a very large data matrix, of dimensio

[R] SVD on very large data matrix

Dear All, I need to perform a SVD on a very large data matrix, of dimension ~ 500,000 x 1,000 , and I am looking for an efficient algorithm that can perform an approximate (partial) SVD to extract on the order of the top 50 right and left singular vectors. Would be very grateful for any advic

[R] svd (via MFA): All coordinates fall on straight lines?

Applying the svd function to my data by way of the FactoMineR package's MFA function: dfmfa <- MFA(df, group=c(2,96), type=c("n","c")) the result is that all my data points fall on one of 8 straight parallel lines when projected onto any two axes, e.g., points(dfmfa$ind$coord[, c(1, 2)]) Furt

Re: [R] SVD Memory Issue

Hi, An SVD on a 771x5677 matrix should be fine, it took 30 seconds and no memory on my workstation. The problem is most likely when you transform the array tdm2 to a matrix. The array tdm2 has a much greater size than 771x5677, so does tdm_matrix. Without a reproducible example we cannot help you

[R] SVD Memory Issue

I am trying to perform Singular Value Decomposition (SVD) on a Term Document Matrix I created using the 'tm' package. Eventually I want to do a Latent Semantic Analysis (LSA). There are 5677 documents with 771 terms (the DTM is 771 x 5677). When I try to do the SVD, it runs out of memory. I am us

Re: [R] svd

Thanks juan, I got that, but what I have two matrices A and B, How can an svd be performed on the two together. Is it correct to get the covariance matrix and then perform the svd on the covariance matrix. If that is the case I have another doubt. I understand the covariance of A and B is t(A)%*

[R] svd

Dear list, I searched the libraries but could not find means to compute the svd of a coupled field. Is it possible in R Thanks nuncio -- Nuncio.M Research Scientist National Center for Antarctic and Ocean research Head land Sada Vasco da Gamma Goa-403804 [[alternative HTML ve

[R] SVD, UV-Decomposition and NMF

I am reading the Mining of Massive Datasets Book by Rajaraman and Ullman. It has a good explanation of Recommendation System at Chapter 9. But what are the relationship between 1) SVD (Singular Decomposition) 2) UV-Decomposition 3) NMF (Non-negative Matrix Factorization) In particular, it

[R] SVD in microarray

Hi, I want to use singular value decompositions (SVD) to remove some artifacts in my microarray data. what i do is replacing the first eigenvalue to zero: library(MASS) data <- as.matrix(read.table("data.txt", header=TRUE,row.names=1, sep = "\t", as.is = TRUE)) a.svd <- svd(data) length(a.svd$d)

Re: [R] SVD on a matix

the variance is the eigen values of the correlation matrix of yoru matrix X.cor <- cor(X) X.e <- eigen(X.cor) X.e$values# Eigenvalues of cor(X) = variances you're asking about kayj wrote: > > Hi All, > > I performed an svd on a matrix X and saved the first three column of the > left singular

[R] SVD on a matix

Hi All, I performed an svd on a matrix X and saved the first three column of the left singular matrix U. ( I assume that they correspond to the projection of the matrix on the first three eigen vectors that corresponds to the first three largest eigenvalues). I would like to know how much varian

Re: [R] SVD for a 500, 000* 500, 000 matrix (singular value decomposition)

Use blzpack, it could work it out. Aimin At 02:44 AM 5/8/2008, Uwe Ligges wrote: >kayj wrote: >>Hi, >> >>I tried to run SVD on a 500,000* 500,000 matrix and i get a message that it >>can not allocate a vector of length 270 mb > > >Well, you will obviously need >> 1Tera(!)bytes of RAM just in

Re: [R] SVD for a 500, 000* 500, 000 matrix (singular value decomposition)

kayj wrote: Hi, I tried to run SVD on a 500,000* 500,000 matrix and i get a message that it can not allocate a vector of length 270 mb Well, you will obviously need >> 1Tera(!)bytes of RAM just in order to store the matrix (or is it some sparse one?). I wonder how you managed that iss

[R] SVD for a 500, 000* 500, 000 matrix (singular value decomposition)

Hi, I tried to run SVD on a 500,000* 500,000 matrix and i get a message that it can not allocate a vector of length 270 mb doe snayone know how to solve this problem? any ideas on other softwares where I can do this? I appreciate your help thanks -- View this message in context: http://w

Re: [R] SVD of a variance matrix

ssage- From: Giovanni Petris [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 15, 2008 8:07 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; r-help@r-project.org Subject: Re: [R] SVD of a variance matrix Hi Ravi, Thank you for your useful reply. Does the result also hold for variance-covariance matrice

Re: [R] SVD of a variance matrix

-- > > > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > Behalf Of Ravi Varadhan > Sent: Tuesday, April 15, 2008 6:03 PM > To: 'Giovanni Petris'; r-help@r-project.o

Re: [R] SVD of a variance matrix

etris'; r-help@r-project.org Subject: Re: [R] SVD of a variance matrix Yes. SVD of any symmetric (which is, of course, also square) matrix will always have U = V. Also, SVD is the same as spectral decomposition, and the columns of U and V are the eigenvectors, but the singular values will be th

Re: [R] SVD of a variance matrix

-project.org Subject: [R] SVD of a variance matrix Hello! I suppose this is more a matrix theory question than a question on R, but I will give it a try... I am using La.svd to compute the singular value decomposition (SVD) of a variance matrix, i.e., a symmetric nonnegative definite square matrix

[R] SVD of a variance matrix

Hello! I suppose this is more a matrix theory question than a question on R, but I will give it a try... I am using La.svd to compute the singular value decomposition (SVD) of a variance matrix, i.e., a symmetric nonnegative definite square matrix. Let S be my variance matrix, and S = U D V' be

[R] SVD least squares sub-space projection

Hi all, A good new year for everybody. Could somebody help me on a question? The Singular Value Decomposition of a matrix A gives A = U * D * t(V) I A is a M X N matrix, U is the left singular matrix (M X N), D is a diagonal singular values matrix (N X N) and V is the transpose right singular