When i submit a job using yarn its seems working only with oozie its failing i guess, not sure what is missing.
yarn jar /uap/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar pi 20 1000 Number of Maps = 20 Samples per Map = 1000 . . . Job Finished in 19.622 seconds Estimated value of Pi is 3.14280000000000000000 Ram On Fri, Aug 19, 2016 at 11:46 AM, rammohan ganapavarapu < [email protected]> wrote: > Ok, i have used yarn-utils.py to get the correct values for my cluster and > update those properties and restarted RM and NM but still no luck not sure > what i am missing, any other insights will help me. > > Below are my properties from yarn-site.xml and map-site.xml. > > python yarn-utils.py -c 24 -m 63 -d 3 -k False > Using cores=24 memory=63GB disks=3 hbase=False > Profile: cores=24 memory=63488MB reserved=1GB usableMem=62GB disks=3 > Num Container=6 > Container Ram=10240MB > Used Ram=60GB > Unused Ram=1GB > yarn.scheduler.minimum-allocation-mb=10240 > yarn.scheduler.maximum-allocation-mb=61440 > yarn.nodemanager.resource.memory-mb=61440 > mapreduce.map.memory.mb=5120 > mapreduce.map.java.opts=-Xmx4096m > mapreduce.reduce.memory.mb=10240 > mapreduce.reduce.java.opts=-Xmx8192m > yarn.app.mapreduce.am.resource.mb=5120 > yarn.app.mapreduce.am.command-opts=-Xmx4096m > mapreduce.task.io.sort.mb=1024 > > > <property> > <name>mapreduce.map.memory.mb</name> > <value>5120</value> > </property> > <property> > <name>mapreduce.map.java.opts</name> > <value>-Xmx4096m</value> > </property> > <property> > <name>mapreduce.reduce.memory.mb</name> > <value>10240</value> > </property> > <property> > <name>mapreduce.reduce.java.opts</name> > <value>-Xmx8192m</value> > </property> > <property> > <name>yarn.app.mapreduce.am.resource.mb</name> > <value>5120</value> > </property> > <property> > <name>yarn.app.mapreduce.am.command-opts</name> > <value>-Xmx4096m</value> > </property> > <property> > <name>mapreduce.task.io.sort.mb</name> > <value>1024</value> > </property> > > > > <property> > <name>yarn.scheduler.minimum-allocation-mb</name> > <value>10240</value> > </property> > > <property> > <name>yarn.scheduler.maximum-allocation-mb</name> > <value>61440</value> > </property> > > <property> > <name>yarn.nodemanager.resource.memory-mb</name> > <value>61440</value> > </property> > > > Ram > > On Thu, Aug 18, 2016 at 11:14 PM, tkg_cangkul <[email protected]> > wrote: > >> maybe this link can be some reference to tune up the cluster: >> >> http://jason4zhu.blogspot.co.id/2014/10/memory-configuration >> -in-hadoop.html >> >> >> On 19/08/16 11:13, rammohan ganapavarapu wrote: >> >> Do you know what properties to tune? >> >> Thanks, >> Ram >> >> On Thu, Aug 18, 2016 at 9:11 PM, tkg_cangkul <[email protected]> >> wrote: >> >>> i think that's because you don't have enough resource. u can tune your >>> cluster config to maximize your resource. >>> >>> >>> On 19/08/16 11:03, rammohan ganapavarapu wrote: >>> >>> I dont see any thing odd except this not sure if i have to worry about >>> it or not. >>> >>> 2016-08-19 03:29:26,621 INFO [main] org.apache.hadoop.yarn.client.RMProxy: >>> Connecting to ResourceManager at /0.0.0.0:8030 >>> 2016-08-19 03:29:27,646 INFO [main] org.apache.hadoop.ipc.Client: >>> Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 >>> time(s); retry policy is RetryUpToMaximumCo >>> untWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) >>> 2016-08-19 03:29:28,647 INFO [main] org.apache.hadoop.ipc.Client: >>> Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 >>> time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >>> sleepTime=1000 MILLISECONDS) >>> >>> >>> its keep printing this log ..in app container logs. >>> >>> On Thu, Aug 18, 2016 at 8:20 PM, tkg_cangkul <[email protected]> >>> wrote: >>> >>>> maybe u can check the logs from port 8088 on your browser. that was RM >>>> UI. just choose your job id and then check the logs. >>>> >>>> On 19/08/16 10:14, rammohan ganapavarapu wrote: >>>> >>>> Sunil, >>>> >>>> Thanks you for your input, below are my server metrics for RM. Also >>>> attached RM UI for capacity scheduler resources. How else i can find? >>>> >>>> { >>>> "name": "Hadoop:service=ResourceManage >>>> r,name=QueueMetrics,q0=root", >>>> "modelerType": "QueueMetrics,q0=root", >>>> "tag.Queue": "root", >>>> "tag.Context": "yarn", >>>> "tag.Hostname": "hadoop001", >>>> "running_0": 0, >>>> "running_60": 0, >>>> "running_300": 0, >>>> "running_1440": 0, >>>> "AppsSubmitted": 1, >>>> "AppsRunning": 0, >>>> "AppsPending": 0, >>>> "AppsCompleted": 0, >>>> "AppsKilled": 0, >>>> "AppsFailed": 1, >>>> "AllocatedMB": 0, >>>> "AllocatedVCores": 0, >>>> "AllocatedContainers": 0, >>>> "AggregateContainersAllocated": 2, >>>> "AggregateContainersReleased": 2, >>>> "AvailableMB": 64512, >>>> "AvailableVCores": 24, >>>> "PendingMB": 0, >>>> "PendingVCores": 0, >>>> "PendingContainers": 0, >>>> "ReservedMB": 0, >>>> "ReservedVCores": 0, >>>> "ReservedContainers": 0, >>>> "ActiveUsers": 0, >>>> "ActiveApplications": 0 >>>> }, >>>> >>>> On Thu, Aug 18, 2016 at 6:49 PM, Sunil Govind <[email protected]> >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> It could be because of many of reasons. Also I am not sure about which >>>>> scheduler your are using, pls share more details such as RM log etc. >>>>> >>>>> I could point out few reasons >>>>> - Such as "Not enough resource is cluster" can cause this >>>>> - If using Capacity Scheduler, if queue capacity is maxed out, such >>>>> case can happen. >>>>> - Similarly if max-am-resource-percent is crossed per queue level, >>>>> then also AM container may not be launched. >>>>> >>>>> you could check RM log to get more information if AM container is >>>>> laucnhed. >>>>> >>>>> Thanks >>>>> Sunil >>>>> >>>>> On Fri, Aug 19, 2016 at 5:37 AM rammohan ganapavarapu < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> When i submit a MR job, i am getting this from AM UI but it never get >>>>>> finished, what am i missing ? >>>>>> >>>>>> Thanks, >>>>>> Ram >>>>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>>> >>> >>> >> >> >
