Hi, I have trying to run a simple MR program using rmr2 in a single node Hadoop cluster. Here is the environment for the setup
Ubuntu 12.04 (32 bit) R (Ubuntu comes with 2.14.1, so updated to 3.0.2) Installed the latest rmr2 and rhdfs from here<https://github.com/RevolutionAnalytics/RHadoop/wiki/Downloads>and the corresponding dependencies Hadoop 1.2.1 Now I am trying to run a simple MR program as Sys.setenv(HADOOP_HOME="/home/training/Installations/hadoop-1.2.1") Sys.setenv(HADOOP_CMD="/home/training/Installations/hadoop-1.2.1/bin/hadoop") library(rmr2) library(rhdfs) ints = to.dfs(1:100) calc = mapreduce(input = ints, map = function(k, v) cbind(v, 2*v)) from.dfs(calc) The mapreduce job fails with the below error message in * hadoop-1.2.1/logs/userlogs/job_201310091055_0001/attempt_201310091055_0001_m_000000_0/stderr * Error in library(functional) : there is no package called ‘functional’ Execution halted java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) But, the sessionInfo() shows that functional package has been loaded sessionInfo() R version 3.0.2 (2013-09-25) Platform: i686-pc-linux-gnu (32-bit) locale: 1 LC_CTYPE=en_IN LC_NUMERIC=C LC_TIME=en_IN [4] LC_COLLATE=en_IN LC_MONETARY=en_IN LC_MESSAGES=en_IN [7] LC_PAPER=en_IN LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=en_IN LC_IDENTIFICATION=C attached base packages: 1 stats graphics grDevices utils datasets methods base other attached packages: 1 rhdfs_1.0.6 rJava_0.9-4 rmr2_2.3.0 reshape2_1.2.2 plyr_1.8 [6] stringr_0.6.2 *functional_0.4* digest_0.6.3 bitops_1.0-6 RJSONIO_1.0-3 [11] Rcpp_0.10.5 How to get around this problem? I have posted the same in StackOverflow also (http://goo.gl/KEKRVJ) Thanks, Praveen [[alternative HTML version deleted]]
______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
