Hi Sam,
I see. In that case, try to create the Filesystem object with a
UserGroupInformation. First, you might want to verify you have the right
user in the UGI:
LOG.info("Current User: {}", UserGroupInformation.getCurrentUser());
FileSystem fs = UserGroupInformation.getCurrentUser().doAs(new
PrivilegedExceptionAction<FileSystem>() {
@Override
public FileSystem run() throws Exception {
return FileSystem.get(new Configuration());
}
});
Terence
On Mon, Feb 22, 2016 at 1:33 PM, Sam William <[email protected]> wrote:
> Terence,
> Im running it on a non-secure cluster. Im launching the job as my own user
> (say.. sam_william), I can see the job on the web UI, and it shows the
> correct user id (sam_william).
> That brings us to another point, that I forgot to add earlier. Im
> actually trying to create a file on HDFS from inside a TwillRunnable. I
> thought I could use getContext() to get a Configuration() object, but it
> looks like the context does not contain a reference to Configuration. I
> just went ahead and did a Filesystem.get(new Configuration()) and it
> worked. I was able create and write into the file. When I check the file
> after the job is completed, I see that the file is owned by user ‘yarn’ .
>
> Sam
>
>
> > On Feb 22, 2016, at 11:54 AM, Terence Yim <[email protected]> wrote:
> >
> > Hi Sam,
> >
> > Are you running a non-secure cluster or a Kerberos cluster? And what user
> > you used to launch the job on the client side?
> > In non-secure cluster, the container user would be the same as the login
> > user you used to launch the job. For secure cluster, the current user in
> > UserGroupInformation will be used (usually the Kerberos authenticated
> user
> > you are using to launch the job).
> >
> > Terence
> >
> > On Mon, Feb 22, 2016 at 11:29 AM, Sam Prince Daniel William <
> > [email protected]> wrote:
> >
> >> Hi,
> >> I couldn't find user mailing list, hence asking this question here. I
> >> have been playing around with twill for the last few days and wondering
> if
> >> there is a way to prevent the container tasks running as the superuser
> >> (yarn in my case) ? I tried using the preparer.setUser() method, but
> that
> >> didn't have any effect.
> >>
> >> Sam
> >>
>
>
