Thanks again for all the help!
Jon
Soli Deo Gloria
Jon Erik Ween, MD, MS
Scientist, Kunin-Lunenfeld Applied Research Unit
Director, Stroke Clinic, Brain Health Clinic, Baycrest Centre
Assistant Professor, Dept. of Medicine, Div. of Neurology
University of Toronto Faculty of Medicine
...code
################################################################################
## R script for automating stats crunching in large datasets ##
## Needs space separated list of variable names matching dataset column names ##
## You have to tinker with the code to customize for your application ##
##
##
## Jon Erik Ween MD, MSc, 26 Feb 2010 ##
################################################################################
library(ROCR) # Load stats package to use if not standard
varslist<-scan("/Users/jween/Desktop/INCASvars.txt","list") # Read variable list
results<-as.data.frame(array(,c(3,length(varslist)))) # Initialize results
array, one type of stat at a time for now
for (i in 1:length(varslist)){ # Loop throught the variables you want to
process. Determined by varslist
j<-noquote(varslist[i])
vars<-c(varslist[i],"Issue_class") # Variables to be analyzed
temp<-na.omit(incas[vars]) # Have to subset to get rid of NA values
causing ROCR to choke
n<-nrow(temp) # Record how many cases the analysis ios based on. Need
to figure out how to calc cases/controls
#.table<-table(temp$SubjClass) # Maybe for later figure out
cases/controls
results[1,i]<-j # Name particular results column
results[2,i]<-n # Number of subjects in analysis
test<-try(aucval(i,j),silent=TRUE) # Error handling in case procedure
craps oust so loop can continue. Supress annoying error messages
if(class(test)=="try-error") next else # Run procedure only if OK,
otherwise skip
pred<-prediction(incas[[j]],incas$Issue_class); # Procedure
perf<-performance(pred,"auc");
results[3,i]<-as.numeric(p...@y.values) # Enter result into appropriate
row
}
write.table(results,"/Users/jween/Desktop/IncasRres_
Issue_class.csv",sep=",",col.names=FALSE,row.names=FALSE) # Write results to table
rm(aucval,i,n,temp,vars,results,test,pred,perf,j,varslist) # Clean up
aucval<-function(i,j){ # Function to trap errors. Should be the same as real
procedure above
pred<-prediction(incas[[j]],incas$Issue_class) # Don't put any real
results here, they don't seem to be passed back
perf<-performance(pred,"auc")
}
...end
On 2010-02-24, at 9:19 PM, Dennis Murphy wrote:
Hi:
The plyr package may come in handy here, as it allows you to create functions
based on the variables (and their names) in the data frame. Here's a simple,
cooked-up example that shows a couple of ways to handle this class of problem:
(1) Create three simple data frames with the same set of variables,
coincidentally
in the same order, although that shouldn't really matter since we're
referencing
by name rather than position:
library(plyr)
a<- data.frame(x = sample(1:50, 10, replace = TRUE),
y = rpois(10, 30),
z = rnorm(10, 15, 5))
b<- data.frame(x = sample(1:50, 10, replace = TRUE),
y = rpois(10, 30),
z = rnorm(10, 15, 5))
d<- data.frame(x = sample(1:50, 10, replace = TRUE),
y = rpois(10, 30),
z = rnorm(10, 15, 5))
(2) rbind the three data frames and assign an indicator to differentiate the
individual data frames:
dd<- rbind(a, b, d)
dd$df<- rep(letters[c(1, 2, 4)], each = 10)
(3) Use the ddply() function: .(df) refers to the grouping variable, summarise
indicates that we want to compute a groupwise summary, and the
remaining code defines the desired summaries (by variable name).
ddply(dd, .(df), summarise, avgx = mean(x), avgz = mean(z))
df avgx avgz
1 a 28.3 17.27372
2 b 28.0 14.32962
3 d 20.3 13.26147
(4) If we create a list of data frames instead, we can accomplish the same
task by using ldply() [list to data frame as the first two characters]
instead.
Since we have a list as input, there's no need for a group indicator as the
list components comprise the 'groups'.
l<- list(a, b, d)
ldply(l, summarise, avgx = mean(x), avgz = mean(z))
avgx avgz
1 28.3 17.27372
2 28.0 14.32962
3 20.3 13.26147
These represent two ways that you can produce summaries by variable name
for multiple data frames. The rbind construct works if all of the data frames
have
the same variables in the same order; if not, the list approach in (4) is
better.
To see this,
e<- data.frame(y = rpois(10, 30), z = rnorm(10, 15, 5),
x = sample(1:50, 10, replace =TRUE))
l<- list(a, b, d, e)
ldply(l, summarise, avgx = mean(x), avgz = mean(z))
avgx avgz
1 28.3 17.27372
2 28.0 14.32962
3 20.3 13.26147
4 29.9 13.64617
plyr is not the only package you could use for this. The doBy package with
function summaryBy() would also work, and you could also use the aggregate()
function. The advantage of plyr and doBy is that the code is a bit tighter and
easier to understand.
On Wed, Feb 24, 2010 at 5:18 PM, Jon Erik Ween<jw...@klaru-baycrest.on.ca>
wrote:
Friends
I can't quite find a direct answer to this question from the lists, so here
goes:
I have several dataframes, 200+ columns 2000+ rows. I wish to script some operations to
perform on some of the variables (columns) in the data frames not knowing what the column
number is, hence have to refer by name. I have variable names in a text file
"varlist". So, something like this:
for (i in 1:length(varlist)){
j<-varlist[i]
v<-mean(Dataset[[j]])
print(v)
}
When you think of writing code like this, you should think "apply family". R
performs
vectorized operations, and you'll become more efficient when you start thinking
about
how to vectorize rather than how to loop...
Now, if I force it
j<-"Var1"
v<-mean(Dataset[[j]])
print(v)
then it works, but not if i read the varlist as above.
Looking at "j" I get:
print(j)
V1
1 Var1
Hence there is a lot of other stuff read into "j" that confuses "mean". I can't
figure out how to just get the value of the variable and nothing else. I've tried space separated,
comma separated, tab separated lists and all give the same error. I've tried get(), parse()... no
go.
Any suggestions?
Thanks a lot
Jon
Soli Deo Gloria
Jon Erik Ween, MD, MS
Scientist, Kunin-Lunenfeld Applied Research Unit
Director, Stroke Clinic, Brain Health Clinic, Baycrest Centre
Assistant Professor, Dept. of Medicine, Div. of Neurology
University of Toronto Faculty of Medicine
Kimel Family Building, 6th Floor, Room 644
Baycrest Centre
3560 Bathurst Street
Toronto, Ontario M6A 2E1
Canada
Phone: 416-785-2500 x3648
Fax: 416-785-2484
Email: jw...@klaru-baycrest.on.ca
Confidential: This communication and any attachment(s) may contain confidential
or privileged information and is intended solely for the address(es) or the
entity representing the recipient(s). If you have received this information in
error, you are hereby advised to destroy the document and any attachment(s),
make no copies of same and inform the sender immediately of the error. Any
unauthorized use or disclosure of this information is strictly prohibited.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
