Ravi, Thanks for the info.
Arun On Fri, Dec 16, 2011 at 12:27 PM, Ravi Gummadi <[email protected]> wrote: > Amar is working on this issue MAPREDUCE-3349. The patch is not comiited to > trunk yet. Feel free to try it out while it gets reviewed and committed. > > -Ravi > ________________________________________ > From: arun k [[email protected]] > Sent: Friday, December 16, 2011 12:22 PM > To: [email protected] > Subject: Generating job and topology traces from history folder of > multinode cluster using Rumen > > Hai guys ! > > I have set up 5 node cluster with each of them in different racks. > > I have hadoop-0.20.2 set up on my Eclipse Helios. So, i ran Tracebuilder > using > Main Class: org.apache.hadoop.tools.rumen.TraceBuilder > I ran some jobs on cluster and used copy of > /usr/local/hadoop/logs/history folder of master node as input to > TraceBuilder. > Arguments: /home/arun/job.json /home/arun/topology.json > /home/ubuntu/Documents/testlog > > But i get > 11/12/16 12:02:37 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 11/12/16 12:02:38 WARN rumen.TraceBuilder: TraceBuilder got an error while > processing the [possibly virtual] file > master_1324011575958_job_201112161029_0001_hduser_word+count within Path > file:/home/ubuntu/Documents/testlog/master_1324011575958_job_201112161029_0001_hduser_word+count > java.lang.NullPointerException > at > org.apache.hadoop.tools.rumen.JobBuilder.processTaskAttemptFinishedEvent(JobBuilder.java:492) > at > org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:149) > at > org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:310) > at > org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:264) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83) > at > org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:142) > 11/12/16 12:02:39 WARN rumen.TraceBuilder: TraceBuilder got an error while > processing the [possibly virtual] file > master_1324011575958_job_201112161029_0004_hduser_word+count within Path > file:/home/ubuntu/Documents/testlog/master_1324011575958_job_201112161029_0004_hduser_word+count > java.lang.NullPointerException > at > org.apache.hadoop.tools.rumen.JobBuilder.processTaskAttemptFinishedEvent(JobBuilder.java:492) > at org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilde > ......... > .................... > > It generates job trace json file but the topology trace json file doesn't > have 5 node's info and is like this : > > { > "name" : "<root>", > "children" : [ ] > } > > > Which history folder do i need to use ? masters or slaves ? how do i > use it ? > > I have a single node cluster from which i got logs and gave input > TraceBuilder and it works. > > How do i run tracebuilder from terminal and do i need to use only from > version 21 ? > > Can anyone help me out ? > > > Thanks, > Arun >
