Hello Ravi,
Thanks for your response.
I got started by running Rumen and generating the required trace file.
However, while trying to run Gridmix with following command,
java -classpath $JAR_CLASSPATH org.apache.hadoop.mapred.gridmix.Gridmix
-generate 10m ~/Desktop/test_gridmix_output
/home/username/Desktop/test_rumen_output/job-trace.json
*I get the following error in starting Gridmix.
12/06/18 17:32:34 ERROR gridmix.Gridmix: Startup failed
*
Here, I suppose that:
~/Desktop/test_gridmix_output is the output data directory.
/home/username/Desktop/test_rumen_output/job-trace.json is the output file
of Rumen.
10m is the size of data to be generated
$JAR_CLASSPATH contains the path of Gridmix.jar among other dependencies.
I am running the test on a single node.
Please suggest if there are any other hadoop related settings that I have to
do.
My version of hadoop is 1.0.0.
Why do Gridmix raises this exception
"org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
/home/username/Desktop/test already exists"
I do not have this directory already.
Here is the complete Error trace...............
2/06/18 17:32:23 INFO gridmix.SubmitterUserResolver: Current user resolver
is SubmitterUserResolver
12/06/18 17:32:23 WARN gridmix.Gridmix: Resource null ignored
12/06/18 17:32:23 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
12/06/18 17:32:23 INFO gridmix.Gridmix: Submission policy is STRESS
12/06/18 17:32:24 INFO gridmix.Gridmix: Generating 1.0m of test data...
12/06/18 17:32:24 INFO mapred.JobClient: Cleaning up the staging area
file:/tmp/hadoop-username/mapred/staging/username446004824/.staging/job_local_0001
12/06/18 17:32:24 ERROR security.UserGroupInformation:
PriviledgedActionException as:username
cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
/home/username/Desktop/test already exists
12/06/18 17:32:24 ERROR security.UserGroupInformation:
PriviledgedActionException as:username
cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
/home/username/Desktop/test already exists
12/06/18 17:32:24 WARN gridmix.JobSubmitter: Failed to submit
GRIDMIX_GENDATA as username
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
/home/username/Desktop/test already exists
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:134)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:889)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
at
org.apache.hadoop.mapred.gridmix.GenerateData$1.run(GenerateData.java:116)
at
org.apache.hadoop.mapred.gridmix.GenerateData$1.run(GenerateData.java:101)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at
org.apache.hadoop.mapred.gridmix.GenerateData.call(GenerateData.java:101)
at
org.apache.hadoop.mapred.gridmix.GenerateData.call(GenerateData.java:57)
at
org.apache.hadoop.mapred.gridmix.JobSubmitter$SubmitTask.run(JobSubmitter.java:106)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
12/06/18 17:32:24 INFO gridmix.JobMonitor: Job submission failed notify if
anyone is waiting org.apache.hadoop.mapreduce.Job@7bcd107f
12/06/18 17:32:34 INFO mapred.JobClient: Cleaning up the staging area
file:/tmp/hadoop-username/mapred/staging/username-2025454801/.staging/job_local_0002
12/06/18 17:32:34 ERROR security.UserGroupInformation:
PriviledgedActionException as:username
cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
/home/username/Desktop/test already exists
12/06/18 17:32:34 ERROR gridmix.Gridmix: Startup failed
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
/home/username/Desktop/test already exists
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:134)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:889)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
at
org.apache.hadoop.mapred.gridmix.Gridmix.writeInputData(Gridmix.java:118)
at org.apache.hadoop.mapred.gridmix.Gridmix.start(Gridmix.java:283)
at org.apache.hadoop.mapred.gridmix.Gridmix.runJob(Gridmix.java:263)
at org.apache.hadoop.mapred.gridmix.Gridmix.access$000(Gridmix.java:55)
at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:217)
at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:215)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.gridmix.Gridmix.run(Gridmix.java:215)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.mapred.gridmix.Gridmix.main(Gridmix.java:390)
Thanks a lot!
Amit
-----
Sangroya
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-to-use-GridMix3-tp3989438p3990160.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
