Re: [Rd] speeding up perception

2011-07-04 Thread Tim Hesterberg
I've written a "dataframe" package that replaces existing methods for data frame creation and subscripting with versions that use less memory. For example, as.data.frame(a vector) makes 4 copies of the data in R 2.9.2, and 1 copy with the package. There is a small speed gain. I and others have b

Re: [Rd] speeding up perception

2011-07-04 Thread Simon Urbanek
Timothée, On Jul 4, 2011, at 2:47 AM, Timothée Carayol wrote: > Hi -- > > It's my first post on this list; as a relatively new user with little > knowledge of R internals, I am a bit intimidated by the depth of some > of the discussions here, so please spare me if I say something > incredibly si

Re: [Rd] speeding up perception

2011-07-04 Thread Timothée Carayol
Hi -- It's my first post on this list; as a relatively new user with little knowledge of R internals, I am a bit intimidated by the depth of some of the discussions here, so please spare me if I say something incredibly silly. I feel that someone at this point should mention Matthew Dowle's excel

Re: [Rd] speeding up perception

2011-07-03 Thread ivo welch
thank you, simon.  this was very interesting indeed. I also now understand how far out of my depth I am here. fortunately, as an end user, obviously, *I* now know how to avoid the problem. I particularly like the as.list() transformation and back to as.data.frame() to speed things up without los

Re: [Rd] speeding up perception

2011-07-03 Thread Simon Urbanek
Robert, it's not the handling of row names per se that causes the slowdown, but my point was that if what you need is just matrix-like structure with different column types, you may want to use lists instead and for equal column types you're better of with a matrix. But to address your point,

Re: [Rd] speeding up perception

2011-07-03 Thread Robert Stojnic
Hi Simon, On 03/07/11 05:30, Simon Urbanek wrote: This is just a quick, incomplete response, but the main misconception is really the use of data.frames. If you don't use the elaborate mechanics of data frames that involve the management of row names, then they are definitely the wrong tool

Re: [Rd] speeding up perception

2011-07-02 Thread Simon Urbanek
This is just a quick, incomplete response, but the main misconception is really the use of data.frames. If you don't use the elaborate mechanics of data frames that involve the management of row names, then they are definitely the wrong tool to use, because most of the overhead is exactly to man

[Rd] speeding up perception

2011-07-02 Thread ivo welch
Dear R developers: R is supposed to be slow for iterative calculations. actually, it isn't. matrix operations are fast. it is data frame operations that are slow. R <- 1000 C <- 1000 example <- function(m) { cat("rows: "); cat(system.time( for (r in 1:R) m[r,20] <- sqrt(abs(m[r,20])) + rnorm