I run it from IDE right now, but would like to create a command line app eventually.
I should clarify that the exception above is thrown on the YARN node, not in the IDE. On Fri, Jan 22, 2016 at 9:32 PM, Terence Yim <[email protected]> wrote: > Hi Kristoffer, > > The example itself shouldn't need any modification. However, how do > you run that class? Do you run it from IDE or from command line using > "java" command? > > Terence > > On Fri, Jan 22, 2016 at 12:16 PM, Kristoffer Sjögren <[email protected]> wrote: >> Hi Terence, >> >> I'm quite new to Twill and not sure how to do that exactly. Could you >> show me how to modify the following example to do the same? >> >> https://github.com/apache/incubator-twill/blob/master/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java >> >> On Fri, Jan 22, 2016 at 7:24 PM, Terence Yim <[email protected]> wrote: >>> Hi Kristoffer, >>> >>> Seems like the exception comes from the YARN class "ConverterUtils". I >>> believe need to start the application with the version 2.7.1 Hadoop >>> Jars. How to do start the twill application? Usually on a cluster with >>> hadoop installed, you can get all the hadoop jars in the classpath by >>> running this: >>> >>> export CP=`hadoop classpath` >>> java -cp .:$CP YourApp ... >>> >>> Assuming your app classes and Twill jars are in the current directory. >>> >>> Terence >>> >>> On Fri, Jan 22, 2016 at 4:54 AM, Kristoffer Sjögren <[email protected]> >>> wrote: >>>> Here's the full stacktrace. >>>> >>>> Exception in thread "main" java.lang.reflect.InvocationTargetException >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:497) >>>> at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:89) >>>> Caused by: java.lang.RuntimeException: >>>> java.lang.reflect.InvocationTargetException >>>> at com.google.common.base.Throwables.propagate(Throwables.java:160) >>>> at >>>> org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) >>>> at >>>> org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:77) >>>> ... 5 more >>>> Caused by: java.lang.reflect.InvocationTargetException >>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>>> at >>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >>>> at >>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) >>>> at >>>> org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) >>>> ... 6 more >>>> Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: >>>> container_e25_1453466340022_0004_01_000001 >>>> at >>>> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) >>>> at >>>> org.apache.twill.internal.yarn.AbstractYarnAMClient.<init>(AbstractYarnAMClient.java:83) >>>> at >>>> org.apache.twill.internal.yarn.Hadoop21YarnAMClient.<init>(Hadoop21YarnAMClient.java:65) >>>> at >>>> org.apache.twill.internal.yarn.Hadoop22YarnAMClient.<init>(Hadoop22YarnAMClient.java:34) >>>> ... 11 more >>>> Caused by: java.lang.NumberFormatException: For input string: "e25" >>>> at >>>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) >>>> at java.lang.Long.parseLong(Long.java:589) >>>> at java.lang.Long.parseLong(Long.java:631) >>>> at >>>> org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) >>>> at >>>> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) >>>> ... 14 more >>>> >>>> On Thu, Jan 21, 2016 at 10:59 PM, Kristoffer Sjögren <[email protected]> >>>> wrote: >>>>> Hi >>>>> >>>>> I'm trying the basic example [1] on yarn 2.7.1 but get an exception as >>>>> soon as the application starts on the resource manager that tells me >>>>> the container id cannot be parsed. >>>>> >>>>> java.lang.IllegalArgumentException: Invalid containerId: >>>>> container_e04_1427159778706_0002_01_000001 >>>>> >>>>> I don't have the exact stacktrace but I recall it failing in >>>>> ConverterUtils.toContainerId because it assumes that that the first >>>>> token is an application attempt to be parsed as an integer. This class >>>>> resides in hadoop-yarn-common 2.3.0. >>>>> >>>>> Is there any way to either tweak the container id or make twill use >>>>> the 2.7.1 jar instead? >>>>> >>>>> Cheers, >>>>> -Kristoffer >>>>> >>>>> >>>>> [1] >>>>> https://github.com/apache/incubator-twill/blob/master/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java
