Re: [R] Replace split with regex for speed ?

2011-03-18 Thread rex.dwyer
That's a good solution, but if you're really, really sure that the timestamps are in the format you gave, it's quite a bit faster to use substr and paste, because you don't have to do any searching in the string. HTH Rex > x = rep("09:30:00.000.633",100) > system.time(y<-paste(substr(x,1,12)

Re: [R] Singularity problem

2011-03-16 Thread rex.dwyer
Feng, Your matrix is *not* (practically) singular; its inverse is. The message said that the *system* was singular, not the matrix. Remember Cramer's Rule: xi = |Ai| / |A| The really, really large determinant of your matrix is going to appear in the denominator of your solutions, so, essentially,

Re: [R] Does R have a "const object"?

2011-03-15 Thread rex.dwyer
Cheer up! R is a step closer to that concept than the old FORTRAN compilers that couldn't even guarantee that 37 was a constant if used repeatedly in a subroutine call. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Uwe Ligges Se

Re: [R] How to read only specified columns from a data file

2011-03-15 Thread rex.dwyer
I think you need to read an introduction to R. For starters, read.table returns its results as a value, which you are not saving. The probable answer to your question: Read the whole file with read.table, and select columns you need, e.g.: tab <- read.table(myfile, skip=2)[,1:5] -Original Mes

Re: [R] (no subject)

2011-03-15 Thread rex.dwyer
Hi Jon, I read your question differently. Is the answer? - Rex > ch=scan(stdin(),what=character(0),n=1) 1: f Read 1 item > ch [1] "f" > -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Bert Gunter Sent: Tuesday, March 15, 2011 10

Re: [R] *Building* a covariance matrix efficiently

2011-03-14 Thread rex.dwyer
Tsjerk, It seems to me that memory and not time is your big efficiency problem, and I've showed you how to avoid storing your entire input. If you want to avoid doing each multiplication twice, you can replace the "outer" with a function that computes each product only once and accumulate sums o

Re: [R] *Building* a covariance matrix efficiently

2011-03-14 Thread rex.dwyer
Tjerk, This is just a pseudo code outline of what you need to do: M = matrix(0, number of variables, number of variables) V = rep(0, number of variables) N = 0 While (more observations to read) { X <- next observation V <- V + X M <- M + outer(X,X) N <- N+1 } Compute covariance matrix

Re: [R] minimum distance between line segments

2011-03-11 Thread rex.dwyer
I think I need to retract the part about 3 iterations... not true if, e.g., the segments intersect and the angle is small. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of rex.dw...@syngenta.com Sent: Friday, March 11, 2011 2:37 PM

Re: [R] minimum distance between line segments

2011-03-11 Thread rex.dwyer
I like Thomas's idea as a quick practical solution. Here is one more little variation just in case you really do have millions of these distances. Pick point P1 on line segment L1 (e.g., an endpoint). Pick 101 evenly spaced points on line segment L2. Find the nearest to P1 and call it P2. N

Re: [R] using lapply

2011-03-10 Thread rex.dwyer
But no one answered Kushan's question about performance implications of for-loop vs lapply. With apologies to George Orwell: "for-loops BAAD, no loops GOOD." -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Uwe Ligges Sent:

Re: [R] Complex sampling?

2011-03-09 Thread rex.dwyer
It sounds like you want a bunch of random permutations of 1:7. Try order(runif(7)) If you need, say, 10 of them: as.vector(sapply(1:10,function(i) order(runif(7 Is it more complicated than that? -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org

Re: [R] Extracting only odd columns from a matrix

2011-03-09 Thread rex.dwyer
Or, if X1 Y1 X2 Y2... are really your column names m[, grep("X",colnames(m)) ] or m[, grepl("X",colnames(m)) ] -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Dimitris Rizopoulos Sent: Wednesday, March 09, 2011 9:36 AM To: Nixon, Ma

Re: [R] generate 3 distinct random samples without replacement

2011-03-07 Thread rex.dwyer
Cesar, I think your basic misconception is that you believe 'sample' returns a list of indices into the original vector. It does not; it returns actual elements of the vector: > sample(runif(100),3) [1] 0.4492988 0.0336069 0.6948440 I'm not sure why you keep resetting the seed, but if it's imp

Re: [R] rowSums - am I getting something wrong?

2011-03-07 Thread rex.dwyer
Hi Thomas, Several of us explained this in different ways just last week, so you might search the archive. Floating point numbers are an approximate representation of real numbers. Things that can be expressed exactly in powers of 10 can't be expressed exactly in powers of 2. So the sum 0.6+0

Re: [R] R usage survey

2011-03-04 Thread rex.dwyer
Harsh, not to worry, but you were wrong to assert that I engaged in any name calling, let alone constant name calling. I also didn't and don't claim to be an authority on survey design. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf

Re: [R] R usage survey

2011-03-04 Thread rex.dwyer
You still don't say what organization you are associated with. Your domain name and e-mail address give no hint. How do we know that "Harsh Singhal" is even a real person? An e-mail address at a university (for example) would go a long way to establish that. Gmail doesn't cut it for me. The p

Re: [R] questions about using loop, while and next

2011-03-04 Thread rex.dwyer
Carrie, If your while-loop condition depends only on dt, and you don't change dt in your loop, your loop won't terminate. The only thing inside your loop is "next". Perhaps you mean to write: temp=rep(NA, 10) for(i in 1:10) { dt=sum(rbinom(10, 5, 0.5)) while (dt<25) { dt=sum(rbinom(10, 5, 0.5))

Re: [R] Floating points and floor() ?

2011-03-03 Thread rex.dwyer
Hi Michael, In floating point calculation, 1.0-.9 is not exactly 0.1. This is easily seen by subtracting. > (1.0-.9)-0.1 [1] -2.775558e-17 > (1.0-.9)==0.1 [1] FALSE David is right, you can't "correct" this. You can only compensate by taking care that you never, ever test whether 2 FP numbers

Re: [R] R usage survey

2011-03-03 Thread rex.dwyer
Just out of curiosity : It is possible to get name, employer name, location, usage information and academic background details when searching for R users on LinkedIn and the many R related groups there. Does this also provide potential opportunities for misuse and "outrageous" analyses, since

Re: [R] Greek character and R

2011-03-03 Thread rex.dwyer
Eval it. This works at my house: plot(0) title(eval(parse(text=paste("expression(paste(delta^13,'C Station ',",i,"))" -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Filoche Sent: Thursday, March 03, 2011 9:39 AM To: r-help@r-

Re: [R] R usage survey

2011-03-03 Thread rex.dwyer
Harsh, "Suitably analyzed" for whose purposes? One man's "suitable" is another's "outrageous". That's why people want to see the gowns at the Oscars. Under what auspices are you conducting this survey? What do you intend to do with it? You don't give any assurance that the results you post w

Re: [R] Greek character and R

2011-03-03 Thread rex.dwyer
mytitle = parse(text=paste("expression(paste(delta^13,'C Station ',",i,"))")) title(mytitle) -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Filoche Sent: Thursday, March 03, 2011 8:16 AM To: r-help@r-project.org Subject: [R] Greek

Re: [R] Developing a web crawler

2011-03-03 Thread rex.dwyer
Perl seems like a 10x better choice for the task, but try looking at the examples in ?strsplit to get started. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of antujsrv Sent: Thursday, March 03, 2011 4:23 AM To: r-help@r-project.org

Re: [R] merge( , by='row.names') slowness

2011-03-02 Thread rex.dwyer
-Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of dms Sent: Wednesday, March 02, 2011 3:16 PM To: r-help@r-project.org Subject: [R] merge( , by='row.names') slowness I noticed that joining two data.frames in R using the "merge" fun

Re: [R] clustering problem

2011-03-02 Thread rex.dwyer
Don't you expect it to be a lot faster if you cluster 20 items instead of 25000? -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Maxim Sent: Wednesday, March 02, 2011 4:08 PM To: r-help@r-project.org Subject: [R] clustering problem

Re: [R] inefficient ifelse() ?

2011-03-02 Thread rex.dwyer
Hi Ivo, It might be useful for you to study the examples below. The key from a programming language point of view is that functions like ifelse are functions of whole vectors, not elements of vectors. You either evaluate an argument or you don't; you don't evaluate only part of argument. (Someb

Re: [R] Finding pairs with least magnitude difference from mean

2011-03-01 Thread rex.dwyer
No, that's not what I meant, but maybe I didn't understand the question. What I suggested would involve sorting y, not x: "sort the *distances*". If you want to minimize the sd of a subset of numbers, you sort the numbers and find a subset that is clumped together. If the numbers are a function of

Re: [R] Is there any Command showing correlation of all variables in a dataset?

2011-03-01 Thread rex.dwyer
?cor answers that question. If Housing is a dataframe, cor(Housing) should do it. Surprisingly, ??correlation doesn't point you to ?cor. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of JoonGi Sent: Tuesday, March 01, 2011 5:41 A

Re: [R] Explained variance for ICA

2011-03-01 Thread rex.dwyer
You determine the variance explained by *any* unit vector by taking its inner product with the data points, then finding the variance of the results. In the case of FastICA, the variance explained by the ICs collectively is exactly the same as the variance explained by the principal components

Re: [R] Finding pairs with least magnitude difference from mean

2011-02-28 Thread rex.dwyer
James, It seems the 2*mean(x) term is irrelevant if you are seeking to minimize sd. Then you want to sort the distances from smallest to largest. Then it seems clear that your five values will be adjacent in the list, since if you have a set of five adjacent values, exchanging any of them for

Re: [R] Error

2011-02-28 Thread rex.dwyer
I have to agree that it's pretty hard to take something that works and figure out why it doesn't work :) The only other suggestion is that sometimes I find that this sort of error goes away if I add "drop=FALSE" to the subsetting, and, if so, that usually lets me figure out why. -Original

Re: [R] help

2011-02-28 Thread rex.dwyer
Generally, you can save your excel spreadsheet as comma-separated values, and then read with read.csv function: ?read.csv Or, tab-separated values and use read.delim. Then look at ?barplot Possibly you would like to read the Intro to R on the CRAN website. Go to www.r-project.org , find Docume

Re: [R] Error

2011-02-25 Thread rex.dwyer
Does it work for FUN=mean? If yes, you need to print out the results of f before you return them to find the anomalous value. BTW "Error" is not a very good subject line. I don't see many posts from people reporting how well things are going :) -Original Message- From: r-help-boun...@

Re: [R] Calculate probabilty

2011-02-25 Thread rex.dwyer
Are you clear about the question you are asking? Do you want to know whether there are 6 balls or at least 6 balls? (It sounds like "at least".) Do you want to know whether there are at least 6 balls in the first box, or at least 6 balls in exactly one box or at least 6 balls in at least one

Re: [R] weighted Voronoi diagrams

2011-02-24 Thread rex.dwyer
One way to do Dirichlet triangulations is to map point (x,y) to point (x,y,x^2+y^2) (I think, it's been a while) and then find the convex hull of these points in 3 dimensions. You can do the Voronoi diagram of circles by mapping (x,y,r) to (x,y,x^2+y^2-r^2) I would try assigning an r to each

Re: [R] Running code sequentially from separate scripts (but not functions)

2011-02-24 Thread rex.dwyer
You don't need to write functions to "source" files: source("code1.R") source("code2.R") source("code3.R") When you source a file with a bunch of function definitions, the definitions are just assignment statements: f <- function (x)... g <- function (x,y,z) ... Did you think you would break y

Re: [R] Gaps in plotting temporal data.

2011-02-24 Thread rex.dwyer
If you're in a hurry, it's way easier than that: t <- c(1,2,3,7,8,9,11,12,13) x <- rnorm(length(t)) new.t <- min(t):max(t) new.x <- NULL new.x[t-min(t)+1] <- x plot(new.t, new.x, type='l') This is wastes max(t)-min(t)-length(t)+1 vector entries, but presumably you won't be wasting a lot of rea

Re: [R] How to find points of intersection between harmonic function and a line

2011-02-22 Thread rex.dwyer
How is the curve is represented? That's more important that its organ-of-origin. If you have values of y=f(x) at discrete time points, then y-(x+2) will change sign sometimes... the intersection point is at some time x' in between. Am I missing something subtle here? You could interpolate the

Re: [R] Discrepancies in run times

2011-02-22 Thread rex.dwyer
My surmise would be that you have not analyzed the situation correctly, and you are making a false assumption about your code. Since you can't show the code, it's pretty hard to figure out what that is. I think you're going to have to produce a simple example that you can share that has the sa

Re: [R] How to find points of intersection

2011-02-22 Thread rex.dwyer
How is the curve defined? If the curve is y=f(x) and the line is y=mx+b, you look for the roots of f(x)-mx-b. ?polyroot ?uniroot -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of FMH Sent: Tuesday, February 22, 2011 6:28 AM To: r-he

Re: [R] question about generics

2011-02-21 Thread rex.dwyer
?InternalMethods ?S3groupGeneric ?S4groupGeneric -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Erin Hodgess Sent: Friday, February 18, 2011 10:45 PM To: R help Subject: [R] question about generics Dear R People: Is there a way

Re: [R] Building an array from matrix blocks

2011-02-21 Thread rex.dwyer
Well, you can lose B by just adding to X in the first for-loop, can't you? For (...) X <- X + A[...] But if you want elegance, you could try: X = Reduce("+",lapply(1:(p+1), function(i) A[i:(n-p-1+i),i:(n-p-1+i)])) I imagine someone can be even more eleganter than this. rad -Original Messag

Re: [R] sort a 3 dimensional array across third dimension ?

2011-02-18 Thread rex.dwyer
I was going to say: The problem with for-loops (as best I understand it) is that the R code gets interpreted over and over; what you normally want to do is design the computation so that you jump into the internals of R and stay there. But the inner loop is in the R internals of the sort in th

Re: [R] sort a 3 dimensional array across third dimension ?

2011-02-18 Thread rex.dwyer
Although I suggested to someone else that for-loops be avoided, they are not in the inner loop in this code, and it's probably easier to understand than some sort of apply: a = array(round(100*runif(60)),dim=c(3,4,5)) a for (i in 1:dim(a)[1]) for (j in 1:dim(a)[2]) a[i,j,] = sort(a[i,j,]) a

Re: [R] speed up the code

2011-02-18 Thread rex.dwyer
Yes, remove the call to intersect, and rely on the results of match to tell you whether there is an overlap. If there are any matches, all(is.na(index)) will be false. Read help for match. ?match -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.

Re: [R] covar

2011-02-17 Thread rex.dwyer
I hate to sound like David "Have You Read The Posting Guide?" Winsemius, but there's no way for anyone to know what you are trying to accomplish here without a lot more information. You don't show us the output you expect and the output you got. I would expected "relatedness" to be on a scale

Re: [R] calling pairs of variables into a function

2011-02-17 Thread rex.dwyer
Try putting d,e,f in a list: Xxx = list(d,e,f) For (I in 1:length(xxx)) For (j in 1:length(xxx)) If (i!=j) bigfunction(xxx[[i]],xxx[[j]]) (bad indentation, caps thanks to outlook) -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of s

Re: [R] A Math question

2011-02-16 Thread rex.dwyer
If y'all want to discuss this more, do it somewhere else, please. This has little to do with R except that both depend on Peano's Axioms. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Tuesday, February

Re: [R] string parsing

2011-02-16 Thread rex.dwyer
It's only "awfully" inefficient if it's a bottleneck. You're not doing this more than once per item fetched from the network, and the time is insignificant relative to the fetch. If it were somehow in your inner loop, it would be worth worrying about, but your purpose is to eliminate Ms and Bs

Re: [R] String manipulation

2011-02-16 Thread rex.dwyer
A quick way to do this is to replace \d and \D with character classes [0-9.] and [^0-9.] . This assumes that there is no scientific notation and that there is nothing like 123.45.678 in the string. You did not account for a leading minus sign. The book Mastering Regular Expressions is probably

Re: [R] cycle in a directed graph

2011-02-11 Thread rex.dwyer
If the graph has n nodes and is represented by an adjacency matrix, you can square the matrix (log_2 n)+1 times. Then you can multiply the matrix element-wise by its transpose. The positive entries in the 7th row will tell you all nodes sharing a cycle with node 7. This assumes all edge weigh

Re: [R] Calculating rowMeans from different columns in each row?

2011-02-10 Thread rex.dwyer
-Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Marine Andersson Sent: Thursday, February 10, 2011 3:53 PM To: r-help@r-project.org Subject: [R] Calculating rowMeans from different columns in each row? Hello! I have a dataset lik

Re: [R] Generate multivariate normal data with a random correlation matrix

2011-02-10 Thread rex.dwyer
If you want a random correlation matrix, why not just generate random data and accept the correlation matrix that you get? The standard normal distribution in k dimensions is (hyper)spherically symmetric. If you generate k standard normal N(0,1) variates, you have a point in k-space with direc

Re: [R] A question on Duplicating

2011-02-09 Thread rex.dwyer
ab = paste(a,b,sep=";~;~;~") flag = length(ab)==length(unique(ab)) This should work unless you use 3 consecutive winking elephants in other places in your program. -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Nipesh Bajaj Sen

Re: [R] "strange" behavior of panel.abline inside a for-loop

2011-02-08 Thread rex.dwyer
Dear Marius, Try this: plot.list = lapply(1:10, function(i) xyplot(i~i,type="p",xlim=c(0,11),panel=function(...) { panel.xyplot(...); panel.abline(v=i)}) ) plot.list[[3]] I imagine it will work for Mr Luftjammer, too. Rex -Original Message- From: r-help-boun...@r-project.org [ma

Re: [R] pass nrow(x) to dots in function(x){plot(x,...)}

2011-02-04 Thread rex.dwyer
Hi Marianne, The quick-and-dirty solution is to add one character and make ns global: ns <<- nrow(x) Poor practice, but OK temporarily if you're just debugging. This is an issue of "scope". You are assuming dynamic scope, whereas R uses static scope. 'ns' was not defined when you said paste("n=