Dear All,
I'm dealing with a case, where 'manhattan' distance of each of 100
vectors is calculated from 10000 other vectors. For achieving this,
following 4 scenarios are tested:
1) scenario 1:
> x<-read.table("query.vec")
> v<-read.table("query.vec2")
> d<-matrix(nrow=nrow(v),ncol=nrow(x))
> for (i in 1:nrow(v)){
+ d[i,]<-
sapply(1:nrow(x),function(z){dist(rbind(v[i,],x[z,]),method="manhattan")})
+ }
> print(d[1,1:10])
time taken for running the code is :
real 1m33.088s
user 1m32.287s
sys 0m0.036s
2) scenario2:
> x<-read.table("query.vec")
> v<-read.table("query.vec2")
> v<-as.matrix(v)
> d<-matrix(nrow=nrow(v),ncol=nrow(x))
> for (i in 1:nrow(v)){
+ tmp_m<-matrix(rep(v[i,],nrow(x)),nrow=nrow(x),byrow=T)
+ d[i,]<- rowSums(abs(tmp_m - x))
+ }
> print(d[1,1:10])
time taken for running the code is:
real 0m0.882s
user 0m0.854s
sys 0m0.025s
3) scenario3:
> x<-read.table("query.vec")
> v<-read.table("query.vec2")
> v<-as.matrix(v)
> d<-matrix(nrow=nrow(v),ncol=nrow(x))
> for (i in 1:nrow(v)){
+
d[i,]<-sapply(1:nrow(x),function(z){dist(rbind(v[i,],x[z,]),method="manhattan")})
+ }
> print(d[1,1:10])
time taken for running the code is:
real 1m3.817s
user 1m3.543s
sys 0m0.031s
4) scenario4:
> x<-read.table("query.vec")
> v<-read.table("query.vec2")
> v<-as.matrix(v)
> d<-dist(rbind(v,x),method="manhattan")
> m<-as.matrix(d)
> m2<-m[1:nrow(v),(nrow(v)+1):nrow(x)]
> print(m2[1,1:10])
time taken for running the code:
real 0m0.445s
user 0m0.401s
sys 0m0.041s
Queries:
1) Though scenario 4 is optimum, this scenario failed when matrix 'v'
having more no. of rows. An error occurred while converting distance
object 'd' to a matrix 'm'.
For E.g: > m<-as.matrix(d)
the above command resulted in error: "Error: cannot allocate
vector of size 922.7 MB".
So, what can be done to convert a larger dist object into a matrix or
how allocation size can be increased?
2) Here I observed that 'dist()' function calculates the distances
across all vectors present in a given matrix or dataframe. Is it not
possible to calculate distances of specific vectors from other vectors
present in a matrix using 'dist()' function? Which means, suppose if a
matrix 'x' having 20 rows, is it not possible using 'dist()' to
calculate only distance of 1st row vector from other 19 vectors.
3) Any other ideas to optimize the problem i'm facing with.
Regards,
Purnachander
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
