resolution - the services application in TAP (MasterService.scala) creates a process that runs the AppSubmitter - providing a classPath, and GEARPUMP.HOME. If the AppSubmitter is run where there is no gear.conf but there is a geardefault.conf then gearpump.executor.vmargs is empty and passed as part of the submission config. Once it reaches the Worker - it will prioritize the submission config over it's own config when running the executor. This can happen when the services app is launched incorrectly - without having a gear.conf but having a geardefault.conf (in this case geardefault.conf came the core jar.
On Tue, Aug 30, 2016 at 5:36 AM, Kam Kasravi (JIRA) <[email protected]> wrote: > > [ https://issues.apache.org/jira/browse/GEARPUMP-185?page= > com.atlassian.jira.plugin.system.issuetabpanels:comment- > tabpanel&focusedCommentId=15448900#comment-15448900 ] > > Kam Kasravi commented on GEARPUMP-185: > -------------------------------------- > > i'm able to confirm that the missing vmargs is related to the user > argument. Under the applications section of the dashboard UI - the vcap > users application's config had no vmargs. Launching a new application > (which ran as yarn) had a config that contained a valid vmargs for the > executor > > > Yarn kills Apache Gearpump's worker if the memory usage is too high > > ------------------------------------------------------------------- > > > > Key: GEARPUMP-185 > > URL: https://issues.apache.org/jira/browse/GEARPUMP-185 > > Project: Apache Gearpump > > Issue Type: Bug > > Environment: yarn > > Reporter: Piotr Grabuszynski > > Assignee: Kam Kasravi > > Fix For: 0.8.0 > > > > > > h3. Description > > If the traffic is to big (memory usage is height) Yarn kills Gearpump's > worker. > > Clean YARN log (only the part with processes, bigger piece below): > > {code} > > Dump of the process-tree for container_e05_1468841606343_0128_01_000003 > : > > |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE > > |- 2702 2164 2159 2159 (java) 137198 34820 8656748544 199840 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=1 -Dgearpump.executorId=0 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true -Dgearpump.config.file=/tmp/ > gearpump6797336735445779094.conf ... > > |- 2646 2164 2159 2159 (java) 4268 464 5847887872 70626 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=1 -Dgearpump.executorId=-1 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true -Dgearpump.config.file=/tmp/ > gearpump941454978115257558.conf ... > > |- 2159 2157 2159 2159 (bash) 0 0 108654592 301 /bin/bash -c > /usr/java/jdk1.8.0_72/bin/java -Xmx512m -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/daemon/*:pack/gearpump-2. > 11-0.8.0/lib/*:/etc/hadoop/conf.cloudera.YARN:/var/run/ > cloudera-scm-agent/process/195-yarn-NODEMANAGER ... > > |- 2165 2159 2159 2159 (tee) 38 396 103337984 164 /usr/bin/tee -a > /var/log/hadoop-yarn/container/application_1468841606343_0128/container_ > e05_1468841606343_0128_01_000003/stderr > > |- 2164 2159 2159 2159 (java) 4915 815 2367729664 78249 > /usr/java/jdk1.8.0_72/bin/java -Xmx512m -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/daemon/*:pack/gearpump-2. > 11-0.8.0/lib/*:/etc/hadoop/conf.cloudera.YARN:/var/run/ > cloudera-scm-agent/process/195-yarn-NODEMANAGER ... > > |- 6417 2164 2159 2159 (java) 3900 1182 5893246976 125356 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=2 -Dgearpump.executorId=0 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true ... > > |- 6360 2164 2159 2159 (java) 966 58 5845979136 54720 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=2 -Dgearpump.executorId=-1 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true ... > > {code} > > *As we can see there are no "-Xmx" or "-Xsx" parameters near executors' > processes.* > > In standalone mode, we can see both: > > {code} > > // MASTER > > GER\pgr+ 48280 5.3 3.1 5406960 261356 pts/13 Sl+ 09:28 0:10 > /usr/lib/jvm/java-8-oracle/bin/java -server -Djava.net.preferIPv4Stack=true > -DlogFilename=master -Dgearpump.home=/home/local/GER/pgrabusz/Documents/ > gearpump-sources/gearpump-2.11-0.8.0 -Djava.rmi.server.hostname=localhost > ... > > // WORKER > > GER\pgr+ 48417 7.7 1.8 5392212 154728 pts/27 Sl+ 09:29 0:05 > /usr/lib/jvm/java-8-oracle/bin/java -server -Djava.net.preferIPv4Stack=true > -DlogFilename=worker -Dgearpump.home=/home/local/GER/pgrabusz/Documents/ > gearpump-sources/gearpump-2.11-0.8.0 -Djava.rmi.server.hostname=localhost > ... > > // APP MASTER > > GER\pgr+ 48707 15.0 3.1 4572132 255776 pts/27 Sl+ 09:33 0:11 > /usr/lib/jvm/java-8-oracle/jre/bin/java -server -Xms512M -Xmx1024M -Xss1M > -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -XX: > CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:NewRatio=3 > -Djava.rmi.server.hostname=localhost -Dgearpump.username=GER\pgrabusz > -Dgearpump.applicationId=1 -Dgearpump.executorId=-1 ... > > // APP EXECUTOR 0 > > GER\pgr+ 48761 132 5.2 7268592 427200 pts/27 Sl+ 09:33 2:31 > /usr/lib/jvm/java-8-oracle/jre/bin/java -server -Xms512M -Xmx1024M -Xss1M > -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -XX: > CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:NewRatio=3 [...] > -Dgearpump.applicationId=1 -Dgearpump.executorId=0 ... > > {code} > > h3. Steps to Reproduce > > To generate trafic I've used such application: > > {code} > > kafka source > > / \ > > kafka sink hbase sink > > {code} > > Where kafka source write to the same topic as kafka source reads. > > Send some message to topic. > > Worker will disappear after few minutes (1-40 min depends on how much > memory worker has); > > h3. Actual Results > > Worker has been killed > > h3. Additional information > > Increasing value of env WORKERS_MEMORY_LIMIT in gearpump-broker doesn't > work. It increases only running time. > > It might be a problem with some java properties (hip size or sth). It is > also possible there's some memory lick f.e. in hbase sink. > > Bigger piece of YARN log: > > {code} > > 2016-07-20 09:04:22,217 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 27737 for container-id > container_e05_1468841606343_0021_01_000002: > 285.7 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:22,270 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2046 for container-id > container_e05_1468841606343_0128_01_000001: > 356.7 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:22,317 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2159 for container-id > container_e05_1468841606343_0128_01_000003: > 2.0 GB of 2 GB physical memory used; 26.8 GB of 4.2 GB virtual memory used > > 2016-07-20 09:04:22,378 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2116 for container-id > container_e05_1468841606343_0128_01_000002: > 348.0 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:25,424 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 27737 for container-id > container_e05_1468841606343_0021_01_000002: > 285.7 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:25,494 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2046 for container-id > container_e05_1468841606343_0128_01_000001: > 356.7 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:25,538 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2159 for container-id > container_e05_1468841606343_0128_01_000003: > 2.0 GB of 2 GB physical memory used; 26.8 GB of 4.2 GB virtual memory used > > 2016-07-20 09:04:25,539 WARN org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Process tree > for container: container_e05_1468841606343_0128_01_000003 has processes > older than 1 iteration running over the configured limit. Limit=2147483648, > current usage = 2167832576 > > 2016-07-20 09:04:25,547 WARN org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Container > [pid=2159,containerID=container_e05_1468841606343_0128_01_000003] is > running beyond physical memory limits. Current usage: 2.0 GB of 2 GB > physical memory used; 26.8 GB of 4.2 GB virtual memory used. Killing > container. > > Dump of the process-tree for container_e05_1468841606343_0128_01_000003 > : > > |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE > > |- 2702 2164 2159 2159 (java) 137198 34820 8656748544 199840 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=1 -Dgearpump.executorId=0 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true -Dgearpump.config.file=/tmp/ > gearpump6797336735445779094.conf -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/netty-3.8.0.Final.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-streaming_2.11-0.8.0. > jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-guava-16. > 0.1.jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded- > metrics-graphite-3.0.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-reflect-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/slf4j- > log4j12-1.7.7.jar:pack/gearpump-2.11-0.8.0/lib/log4j- > 1.2.17.jar:pack/gearpump-2.11-0.8.0/lib/akka-slf4j_2.11-2.4. > 2.jar:pack/gearpump-2.11-0.8.0/lib/scala-parser- > combinators_2.11-1.0.4.jar:pack/gearpump-2.11-0.8.0/lib/ > akka-actor_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-java8-compat_2.11-0.7.0.jar:pack/gearpump-2.11-0.8.0/ > lib/slf4j-api-1.7.16.jar:pack/gearpump-2.11-0.8.0/lib/akka- > agent_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/scala- > library-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/commons-lang-2.6.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-remote_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/config-1.3.0.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-protobuf_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/gearpump-core_2.11-0.8.0.jar: > pack/gearpump-2.11-0.8.0/lib/akka-kernel_2.11-2.4.2.jar: > pack/gearpump-2.11-0.8.0/lib/scala-stm_2.11-0.7.jar:pack/ > gearpump-2.11-0.8.0/lib/uncommons-maths-1.2.2a.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-gs- > collections-6.2.0.jar:pack/gearpump-2.11-0.8.0/lib/ > gearpump-shaded-akka-kryo_2.11-0.3.2.jar:/etc/hadoop/conf. > cloudera.YARN:/var/run/cloudera-scm-agent/process/ > 195-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-aws-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-format.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common-tests.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-scrooge_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-encoding.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-format-javadoc.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-common-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-generator.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-jackson.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-column.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-protobuf.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-pig.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-annotations.jar:/opt/cloudera/ > parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet- > scala_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-test-hadoop2. > jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > hadoop/parquet-common.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-pig-bundle.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-annotations-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/ > CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-format- > sources.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-aws.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/hadoop-common-2.6.0- > cdh5.5.1-tests.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-nfs-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > > |- 2646 2164 2159 2159 (java) 4268 464 5847887872 70626 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=1 -Dgearpump.executorId=-1 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true -Dgearpump.config.file=/tmp/ > gearpump941454978115257558.conf -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/netty-3.8.0.Final.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-streaming_2.11-0.8.0. > jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-guava-16. > 0.1.jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded- > metrics-graphite-3.0.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-reflect-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/slf4j- > log4j12-1.7.7.jar:pack/gearpump-2.11-0.8.0/lib/log4j- > 1.2.17.jar:pack/gearpump-2.11-0.8.0/lib/akka-slf4j_2.11-2.4. > 2.jar:pack/gearpump-2.11-0.8.0/lib/scala-parser- > combinators_2.11-1.0.4.jar:pack/gearpump-2.11-0.8.0/lib/ > akka-actor_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-java8-compat_2.11-0.7.0.jar:pack/gearpump-2.11-0.8.0/ > lib/slf4j-api-1.7.16.jar:pack/gearpump-2.11-0.8.0/lib/akka- > agent_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/scala- > library-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/commons-lang-2.6.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-remote_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/config-1.3.0.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-protobuf_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/gearpump-core_2.11-0.8.0.jar: > pack/gearpump-2.11-0.8.0/lib/akka-kernel_2.11-2.4.2.jar: > pack/gearpump-2.11-0.8.0/lib/scala-stm_2.11-0.7.jar:pack/ > gearpump-2.11-0.8.0/lib/uncommons-maths-1.2.2a.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-gs- > collections-6.2.0.jar:pack/gearpump-2.11-0.8.0/lib/ > gearpump-shaded-akka-kryo_2.11-0.3.2.jar:/etc/hadoop/conf. > cloudera.YARN:/var/run/cloudera-scm-agent/process/ > 195-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-aws-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-format.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common-tests.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-scrooge_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-encoding.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-format-javadoc.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-common-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-generator.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-jackson.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-column.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-protobuf.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-pig.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-annotations.jar:/opt/cloudera/ > parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet- > scala_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-test-hadoop2. > jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > hadoop/parquet-common.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-pig-bundle.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-annotations-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/ > CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-format- > sources.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-aws.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/hadoop-common-2.6.0- > cdh5.5.1-tests.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-nfs-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > > |- 2159 2157 2159 2159 (bash) 0 0 108654592 301 /bin/bash -c > /usr/java/jdk1.8.0_72/bin/java -Xmx512m -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/daemon/*:pack/gearpump-2. > 11-0.8.0/lib/*:/etc/hadoop/conf.cloudera.YARN:/var/run/ > cloudera-scm-agent/process/195-yarn-NODEMANAGER:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/*:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > lib/*:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/ > lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop-hdfs/lib/*:/opt/cloudera/ > parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop-yarn/*:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop-yarn/lib/* > -Dgearpump.cluster.masters.0=cdh-worker-2.node.envname.consul:54946 > -Dgearpump.log.daemon.dir=/var/log/hadoop-yarn/container/ > application_1468841606343_0128/container_e05_1468841606343_0128_01_000003 > -Dgearpump.worker-resource-manager-container-id= > container_e05_1468841606343_0128_01_000003 -Dgearpump.home=/data1/hadoop/ > yarn/nm/usercache/cf/appcache/application_1468841606343_ > 0128/container_e05_1468841606343_0128_01_000003/pack/gearpump-2.11-0.8.0 > -Dgearpump.log.application.dir=/var/log/hadoop-yarn/container/application_ > 1468841606343_0128/container_e05_1468841606343_0128_01_000003 > -Dgearpump.hostname=cdh-worker-2.node.envname.consul > io.gearpump.cluster.main.Worker 2>&1 | /usr/bin/tee -a > /var/log/hadoop-yarn/container/application_1468841606343_0128/container_ > e05_1468841606343_0128_01_000003/stderr > > |- 2165 2159 2159 2159 (tee) 38 396 103337984 164 /usr/bin/tee -a > /var/log/hadoop-yarn/container/application_1468841606343_0128/container_ > e05_1468841606343_0128_01_000003/stderr > > |- 2164 2159 2159 2159 (java) 4915 815 2367729664 78249 > /usr/java/jdk1.8.0_72/bin/java -Xmx512m -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/daemon/*:pack/gearpump-2. > 11-0.8.0/lib/*:/etc/hadoop/conf.cloudera.YARN:/var/run/ > cloudera-scm-agent/process/195-yarn-NODEMANAGER:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/*:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > lib/*:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/ > lib/hadoop-hdfs/*:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop-hdfs/lib/*:/opt/cloudera/ > parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop-yarn/*:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop-yarn/lib/* > -Dgearpump.cluster.masters.0=cdh-worker-2.node.envname.consul:54946 > -Dgearpump.log.daemon.dir=/var/log/hadoop-yarn/container/ > application_1468841606343_0128/container_e05_1468841606343_0128_01_000003 > -Dgearpump.worker-resource-manager-container-id= > container_e05_1468841606343_0128_01_000003 -Dgearpump.home=/data1/hadoop/ > yarn/nm/usercache/cf/appcache/application_1468841606343_ > 0128/container_e05_1468841606343_0128_01_000003/pack/gearpump-2.11-0.8.0 > -Dgearpump.log.application.dir=/var/log/hadoop-yarn/container/application_ > 1468841606343_0128/container_e05_1468841606343_0128_01_000003 > -Dgearpump.hostname=cdh-worker-2.node.envname.consul > io.gearpump.cluster.main.Worker > > |- 6417 2164 2159 2159 (java) 3900 1182 5893246976 125356 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=2 -Dgearpump.executorId=0 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true -Dgearpump.config.file=/tmp/ > gearpump7648852439078424047.conf -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/netty-3.8.0.Final.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-streaming_2.11-0.8.0. > jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-guava-16. > 0.1.jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded- > metrics-graphite-3.0.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-reflect-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/slf4j- > log4j12-1.7.7.jar:pack/gearpump-2.11-0.8.0/lib/log4j- > 1.2.17.jar:pack/gearpump-2.11-0.8.0/lib/akka-slf4j_2.11-2.4. > 2.jar:pack/gearpump-2.11-0.8.0/lib/scala-parser- > combinators_2.11-1.0.4.jar:pack/gearpump-2.11-0.8.0/lib/ > akka-actor_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-java8-compat_2.11-0.7.0.jar:pack/gearpump-2.11-0.8.0/ > lib/slf4j-api-1.7.16.jar:pack/gearpump-2.11-0.8.0/lib/akka- > agent_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/scala- > library-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/commons-lang-2.6.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-remote_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/config-1.3.0.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-protobuf_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/gearpump-core_2.11-0.8.0.jar: > pack/gearpump-2.11-0.8.0/lib/akka-kernel_2.11-2.4.2.jar: > pack/gearpump-2.11-0.8.0/lib/scala-stm_2.11-0.7.jar:pack/ > gearpump-2.11-0.8.0/lib/uncommons-maths-1.2.2a.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-gs- > collections-6.2.0.jar:pack/gearpump-2.11-0.8.0/lib/ > gearpump-shaded-akka-kryo_2.11-0.3.2.jar:/etc/hadoop/conf. > cloudera.YARN:/var/run/cloudera-scm-agent/process/ > 195-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-aws-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-format.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common-tests.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-scrooge_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-encoding.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-format-javadoc.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-common-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-generator.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-jackson.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-column.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-protobuf.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-pig.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-annotations.jar:/opt/cloudera/ > parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet- > scala_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-test-hadoop2. > jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > hadoop/parquet-common.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-pig-bundle.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-annotations-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/ > CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-format- > sources.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-aws.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/hadoop-common-2.6.0- > cdh5.5.1-tests.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-nfs-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > > |- 6360 2164 2159 2159 (java) 966 58 5845979136 54720 > /usr/java/jdk1.8.0_72/jre/bin/java -Dgearpump.username=vcap > -Dgearpump.applicationId=2 -Dgearpump.executorId=-1 > -Dgearpump.master.starttime=2016-07-20-08-36 > -Dgearpump.log.application.dir=logs > -Djava.net.preferIPv4Stack=true -Dgearpump.config.file=/tmp/ > gearpump7311147404608349128.conf -cp conf:pack/gearpump-2.11-0.8.0/ > conf:pack/gearpump-2.11-0.8.0/lib/netty-3.8.0.Final.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-streaming_2.11-0.8.0. > jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-guava-16. > 0.1.jar:pack/gearpump-2.11-0.8.0/lib/gearpump-shaded- > metrics-graphite-3.0.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-reflect-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/slf4j- > log4j12-1.7.7.jar:pack/gearpump-2.11-0.8.0/lib/log4j- > 1.2.17.jar:pack/gearpump-2.11-0.8.0/lib/akka-slf4j_2.11-2.4. > 2.jar:pack/gearpump-2.11-0.8.0/lib/scala-parser- > combinators_2.11-1.0.4.jar:pack/gearpump-2.11-0.8.0/lib/ > akka-actor_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/ > scala-java8-compat_2.11-0.7.0.jar:pack/gearpump-2.11-0.8.0/ > lib/slf4j-api-1.7.16.jar:pack/gearpump-2.11-0.8.0/lib/akka- > agent_2.11-2.4.2.jar:pack/gearpump-2.11-0.8.0/lib/scala- > library-2.11.8.jar:pack/gearpump-2.11-0.8.0/lib/commons-lang-2.6.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-remote_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/config-1.3.0.jar:pack/ > gearpump-2.11-0.8.0/lib/akka-protobuf_2.11-2.4.2.jar:pack/ > gearpump-2.11-0.8.0/lib/gearpump-core_2.11-0.8.0.jar: > pack/gearpump-2.11-0.8.0/lib/akka-kernel_2.11-2.4.2.jar: > pack/gearpump-2.11-0.8.0/lib/scala-stm_2.11-0.7.jar:pack/ > gearpump-2.11-0.8.0/lib/uncommons-maths-1.2.2a.jar: > pack/gearpump-2.11-0.8.0/lib/gearpump-shaded-gs- > collections-6.2.0.jar:pack/gearpump-2.11-0.8.0/lib/ > gearpump-shaded-akka-kryo_2.11-0.3.2.jar:/etc/hadoop/conf. > cloudera.YARN:/var/run/cloudera-scm-agent/process/ > 195-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-aws-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-format.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common-tests.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-scrooge_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-encoding.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-format-javadoc.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-common-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-generator.jar:/ > opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/parquet-jackson.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-column.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-protobuf.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-common.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > parquet-pig.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-annotations.jar:/opt/cloudera/ > parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet- > scala_2.10.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-test-hadoop2. > jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/ > hadoop/parquet-common.jar:/opt/cloudera/parcels/CDH-5.5. > 1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-pig-bundle.jar:/opt/ > cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/ > hadoop-annotations-2.6.0-cdh5.5.1.jar:/opt/cloudera/parcels/ > CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/parquet-format- > sources.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1. > p0.11/lib/hadoop/hadoop-aws.jar:/opt/cloudera/parcels/CDH- > 5.5.1-1.cdh5.5.1.p0.11/lib/hadoop/hadoop-common-2.6.0- > cdh5.5.1-tests.jar:/opt/cloudera/parcels/CDH-5.5.1-1. > cdh5.5.1.p0.11/lib/hadoop/hadoop-nfs-2.6.0-cdh5.5.1.jar: > /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib > > 2016-07-20 09:04:25,548 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.container.Container: Container > container_e05_1468841606343_0128_01_000003 transitioned from RUNNING to > KILLING > > 2016-07-20 09:04:25,548 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up > container container_e05_1468841606343_0128_01_000003 > > 2016-07-20 09:04:25,548 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Removed > ProcessTree with root 2159 > > 2016-07-20 09:04:25,584 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2116 for container-id > container_e05_1468841606343_0128_01_000002: > 348.0 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:25,621 WARN > > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: > Exit code from container container_e05_1468841606343_0128_01_000003 is : > 143 > > 2016-07-20 09:04:25,655 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.container.Container: Container > container_e05_1468841606343_0128_01_000003 transitioned from KILLING to > CONTAINER_CLEANEDUP_AFTER_KILL > > 2016-07-20 09:04:25,655 INFO > > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: > Deleting absolute path : /data1/hadoop/yarn/nm/usercache/cf/appcache/ > application_1468841606343_0128/container_e05_1468841606343_0128_01_000003 > > 2016-07-20 09:04:25,656 INFO > > org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: > USER=cf OPERATION=Container Finished - Killed TARGET=ContainerImpl > RESULT=SUCCESS APPID=application_1468841606343_0128 > CONTAINERID=container_e05_1468841606343_0128_01_000003 > > 2016-07-20 09:04:25,656 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.container.Container: Container > container_e05_1468841606343_0128_01_000003 transitioned from > CONTAINER_CLEANEDUP_AFTER_KILL to DONE > > 2016-07-20 09:04:25,656 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.application.Application: Removing > container_e05_1468841606343_0128_01_000003 from application > application_1468841606343_0128 > > 2016-07-20 09:04:25,656 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: > Considering container container_e05_1468841606343_0128_01_000003 for > log-aggregation > > 2016-07-20 09:04:25,656 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for > appId application_1468841606343_0128 > > 2016-07-20 09:04:25,656 INFO > > org.apache.spark.network.yarn.YarnShuffleService: > Stopping container container_e05_1468841606343_0128_01_000003 > > 2016-07-20 09:04:27,703 INFO > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: > Removed completed containers from NM context: [container_e05_1468841606343_ > 0128_01_000003] > > 2016-07-20 09:04:28,584 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping > resource-monitoring for container_e05_1468841606343_0128_01_000003 > > 2016-07-20 09:04:28,594 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 27737 for container-id > container_e05_1468841606343_0021_01_000002: > 285.7 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:28,603 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2046 for container-id > container_e05_1468841606343_0128_01_000001: > 356.7 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:28,613 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 2116 for container-id > container_e05_1468841606343_0128_01_000002: > 348.1 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > 2016-07-20 09:04:31,623 INFO org.apache.hadoop.yarn.server. > nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage > of ProcessTree 27737 for container-id > container_e05_1468841606343_0021_01_000002: > 285.7 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used > > {code} > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) >
