Hello everyone, I'm running a small toy cluster (3 nodes) on EC2 configured as follows:
* one node as JT+NN * two nodes as DN+TT I use whirr to build such cluster on demand (config file here http://pastebin.com/JXHYvMNb). Since my jobs are memory intensive I'd like to exploit the 8GB of ram the m1.large instance offers. Thus I added mapred.map.child.java.opts=-Xmx1g and mapred.reduce.child.java.opts=-Xmx1g (adding just hadoop-mapreduce.mapred.child.java.opts=-Xmx1g produced no effect since the map tasks were allocated the default 200MB). The problem is that with these settings I cannot have any job running because I always get 1/12/11 18:14:24 INFO mapred.JobClient: Running job: job_201112111644_0002 11/12/11 18:14:25 INFO mapred.JobClient: map 0% reduce 0% 11/12/11 18:14:28 INFO mapred.JobClient: Task Id : attempt_201112111644_0002_m_000004_0, Status : FAILED java.lang.Throwable: Child Error at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242) Caused by: java.io.IOException: Task process exit with nonzero status of 1. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229) 11/12/11 18:14:28 WARN mapred.JobClient: Error reading task outputip-10-87-1-170.ec2.internal 11/12/11 18:14:28 WARN mapred.JobClient: Error reading task outputip-10-87-1-170.ec2.internal 11/12/11 18:14:30 INFO mapred.JobClient: Task Id : attempt_201112111644_0002_r_000001_0, Status : FAILED java.lang.Throwable: Child Error at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242) Caused by: java.io.IOException: Task process exit with nonzero status of 1. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229) And the only log I can get from the job_201112111644_0002 is the stdout and the stderr whose combined output is Could not create the Java virtual machine. Error occurred during initialization of VM Could not reserve enough space for object heap I really cannot understand why the jvm cannot allocate enough space: there's plenty of ram. I also tried to reduce the number of map slots to two: nothing changed. I'm out of ideas. I hope you can shed some light :) FYI I use cloudera distribution for hadoop, latest stable release available. Thanks for your attention. MD
