Hi,
I have some junit tests for a MapReduce job that run fine from my IDE, but fail
with the following error when using Maven (mvn test):
2011-08-17 15:08:49,191 [main] INFO org.apache.hadoop.mapred.JobClient
- Task Id : attempt_20110817150608891_0003_m_000027_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)
....
2011-08-17 15:08:49,196 [32680187@qtp-19417347-2] WARN
org.apache.hadoop.mapred.TaskLog - Failed to retrieve stderr log for
task: attempt_20110817150608891_0003_m_000027_0
java.io.FileNotFoundException:
/home/cionescu/dev/newsearch/newsearch-deleter/../logs/userlogs/job_20110817150608891_0003/attempt_20110817150608891_0003_m_000027_0/log.index
(No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at java.io.FileReader.<init>(FileReader.java:55)
at org.apache.hadoop.mapred.TaskLog.getAllLogsFileDetails(TaskLog.java:110)
I have searched the archive and came over this:
http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201107.mbox/browser
but none of the suggestions there could solve my problem:
- the disk is not full
- the /userlogs directory doesn't contain too many files (also tried renaming
the directory and it got recreated but error is still there)
- permissions ok, /userlogs and its subdirectories are writeable (mkdir
successfully with the user running the job).
When examining the file system I could see that the stdout and stderr logs are
created but there is no log.index.
Stderr contains what seems to be a classpath error:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/logging/LogFactory
at org.apache.hadoop.mapred.Child.<clinit>(Child.java:61)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
...
Could not find the main class: org.apache.hadoop.mapred.Child. Program will
exit.
I'm guessing this is the actual problem here. Does anyone have any idea on why
this happens?
Thanks,
Cristina
