HI Terence, Yes, this hack should be OK for the time-being. My user definitely available on the cluster and I have been running map reduce jobs regularly.
Sam > On Feb 22, 2016, at 3:06 PM, Terence Yim <[email protected]> wrote: > > Is your user available in the cluster? In a non-secure cluster, you can > impersonate a user when creating the FileSystem object. > > FileSystem fs = UserGroupInformation.createRemoteUser("some_user").doAs( > ... > ) > > > On Mon, Feb 22, 2016 at 2:39 PM, Sam William <[email protected]> wrote: > >> Yeah, current user is ‘yarn’, and not the user that launched the job. >> >> 14:35:25.400 [TwillContainerService] INFO >> o.a.twill.example.yarn.HelloWorld - Current User: yarn (auth:SIMPLE) >> >>> On Feb 22, 2016, at 1:52 PM, Terence Yim <[email protected]> wrote: >>> >>> Hi Sam, >>> >>> I see. In that case, try to create the Filesystem object with a >>> UserGroupInformation. First, you might want to verify you have the right >>> user in the UGI: >>> >>> LOG.info("Current User: {}", UserGroupInformation.getCurrentUser()); >>> FileSystem fs = UserGroupInformation.getCurrentUser().doAs(new >>> PrivilegedExceptionAction<FileSystem>() { >>> @Override >>> public FileSystem run() throws Exception { >>> return FileSystem.get(new Configuration()); >>> } >>> }); >>> >>> Terence >>> >>> >>> On Mon, Feb 22, 2016 at 1:33 PM, Sam William <[email protected]> >> wrote: >>> >>>> Terence, >>>> Im running it on a non-secure cluster. Im launching the job as my own >> user >>>> (say.. sam_william), I can see the job on the web UI, and it shows the >>>> correct user id (sam_william). >>>> That brings us to another point, that I forgot to add earlier. Im >>>> actually trying to create a file on HDFS from inside a TwillRunnable. I >>>> thought I could use getContext() to get a Configuration() object, but >> it >>>> looks like the context does not contain a reference to Configuration. I >>>> just went ahead and did a Filesystem.get(new Configuration()) and it >>>> worked. I was able create and write into the file. When I check the >> file >>>> after the job is completed, I see that the file is owned by user >> ‘yarn’ . >>>> >>>> Sam >>>> >>>> >>>>> On Feb 22, 2016, at 11:54 AM, Terence Yim <[email protected]> wrote: >>>>> >>>>> Hi Sam, >>>>> >>>>> Are you running a non-secure cluster or a Kerberos cluster? And what >> user >>>>> you used to launch the job on the client side? >>>>> In non-secure cluster, the container user would be the same as the >> login >>>>> user you used to launch the job. For secure cluster, the current user >> in >>>>> UserGroupInformation will be used (usually the Kerberos authenticated >>>> user >>>>> you are using to launch the job). >>>>> >>>>> Terence >>>>> >>>>> On Mon, Feb 22, 2016 at 11:29 AM, Sam Prince Daniel William < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> I couldn't find user mailing list, hence asking this question here. >> I >>>>>> have been playing around with twill for the last few days and >> wondering >>>> if >>>>>> there is a way to prevent the container tasks running as the superuser >>>>>> (yarn in my case) ? I tried using the preparer.setUser() method, but >>>> that >>>>>> didn't have any effect. >>>>>> >>>>>> Sam >>>>>> >>>> >>>> >> >>
