Is your user available in the cluster? In a non-secure cluster, you can
impersonate a user when creating the FileSystem object.
FileSystem fs = UserGroupInformation.createRemoteUser("some_user").doAs(
...
)
On Mon, Feb 22, 2016 at 2:39 PM, Sam William <[email protected]> wrote:
> Yeah, current user is ‘yarn’, and not the user that launched the job.
>
> 14:35:25.400 [TwillContainerService] INFO
> o.a.twill.example.yarn.HelloWorld - Current User: yarn (auth:SIMPLE)
>
> > On Feb 22, 2016, at 1:52 PM, Terence Yim <[email protected]> wrote:
> >
> > Hi Sam,
> >
> > I see. In that case, try to create the Filesystem object with a
> > UserGroupInformation. First, you might want to verify you have the right
> > user in the UGI:
> >
> > LOG.info("Current User: {}", UserGroupInformation.getCurrentUser());
> > FileSystem fs = UserGroupInformation.getCurrentUser().doAs(new
> > PrivilegedExceptionAction<FileSystem>() {
> > @Override
> > public FileSystem run() throws Exception {
> > return FileSystem.get(new Configuration());
> > }
> > });
> >
> > Terence
> >
> >
> > On Mon, Feb 22, 2016 at 1:33 PM, Sam William <[email protected]>
> wrote:
> >
> >> Terence,
> >> Im running it on a non-secure cluster. Im launching the job as my own
> user
> >> (say.. sam_william), I can see the job on the web UI, and it shows the
> >> correct user id (sam_william).
> >> That brings us to another point, that I forgot to add earlier. Im
> >> actually trying to create a file on HDFS from inside a TwillRunnable. I
> >> thought I could use getContext() to get a Configuration() object, but
> it
> >> looks like the context does not contain a reference to Configuration. I
> >> just went ahead and did a Filesystem.get(new Configuration()) and it
> >> worked. I was able create and write into the file. When I check the
> file
> >> after the job is completed, I see that the file is owned by user
> ‘yarn’ .
> >>
> >> Sam
> >>
> >>
> >>> On Feb 22, 2016, at 11:54 AM, Terence Yim <[email protected]> wrote:
> >>>
> >>> Hi Sam,
> >>>
> >>> Are you running a non-secure cluster or a Kerberos cluster? And what
> user
> >>> you used to launch the job on the client side?
> >>> In non-secure cluster, the container user would be the same as the
> login
> >>> user you used to launch the job. For secure cluster, the current user
> in
> >>> UserGroupInformation will be used (usually the Kerberos authenticated
> >> user
> >>> you are using to launch the job).
> >>>
> >>> Terence
> >>>
> >>> On Mon, Feb 22, 2016 at 11:29 AM, Sam Prince Daniel William <
> >>> [email protected]> wrote:
> >>>
> >>>> Hi,
> >>>> I couldn't find user mailing list, hence asking this question here.
> I
> >>>> have been playing around with twill for the last few days and
> wondering
> >> if
> >>>> there is a way to prevent the container tasks running as the superuser
> >>>> (yarn in my case) ? I tried using the preparer.setUser() method, but
> >> that
> >>>> didn't have any effect.
> >>>>
> >>>> Sam
> >>>>
> >>
> >>
>
>
