Raj,
Top from one datanode when i get error from that machine
top - 14:10:15 up 23:12, 1 user, load average: 13.45, 12.91, 8.31
Tasks: 187 total, 1 running, 186 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 0.4%sy, 0.0%ni, 0.0%id, 98.9%wa, 0.0%hi, 0.1%si,
0.0%st
Mem: 8061608k total, 7927124k used, 134484k free, 19316k buffers
Swap: 2097144k total, 384k used, 2096760k free, 6694656k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1622 hdfs 20 0 1619m 157m 11m S 2.0 2.0 33:55.42 java
14712 mapred 20 0 709m 119m 11m S 1.3 1.5 0:10.06 java
1706 mapred 20 0 1588m 126m 11m S 1.0 1.6 24:51.69 java
14663 mapred 20 0 708m 89m 11m S 1.0 1.1 0:11.23 java
14686 mapred 20 0 714m 106m 11m S 0.7 1.4 0:11.53 java
14762 mapred 20 0 710m 89m 11m S 0.7 1.1 0:10.05 java
14640 mapred 20 0 704m 119m 11m S 0.3 1.5 0:11.36 java
Error Message:
12/05/22 14:09:52 INFO mapred.JobClient: Task Id :
attempt_201205211504_0009_m_000002_0, Status : FAILED
java.io.IOException: All datanodes 10.0.24.175:50010 are bad. Aborting...
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:3181)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2100(DFSClient.java:2720)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2892)
attempt_201205211504_0009_m_000002_0: log4j:WARN No appenders could be
found for logger (org.apache.hadoop.hdfs.DFSClient).
attempt_201205211504_0009_m_000002_0: log4j:WARN Please initialize the
log4j system properly.
But other map tasks are running on the same datanode.
Thanks,
sandeep.
