Alexandre Linte created PIG-5042:
------------------------------------
Summary: Pig with Tez requires a staging directory with submitter
ownership
Key: PIG-5042
URL: https://issues.apache.org/jira/browse/PIG-5042
Project: Pig
Issue Type: Bug
Components: tez
Affects Versions: 0.16.0
Environment: Hadoop 2.7.2, Pig 0.16.0, Tez 0.8.4
Reporter: Alexandre Linte
With Pig 0.15.0, I have been able to work using the Tez execution engine
without issues. After upgrading Pig to 0.16.0, I haven't been able to submit my
jobs.
Since the beginning, the tez-site.xml is configured as below:
{noformat}
<property>
<description>String value to a file path. The location of the Tez libraries
which will be localized for DAGs. This follows the following
semantics</description>
<name>tez.lib.uris</name>
<value>hdfs:///Products/TEZ/tez-0.8.4.tar.gz</value>
</property>
<property>
<description>String value. Specifies a directory where Tez can create
temporary job artifacts.</description>
<name>tez.staging-dir</name>
<value>/Products/TEZ/staging/</value>
</property>
{noformat}
The staging directory was set up with 777 rights, including a sticky bit:
{noformat}
[root@spark01 ~]# hadoop fs -ls /Products/TEZ/
Found 3 items
drwxrwxrwt+ - hdfs hadoop 0 2014-11-27 10:31 /Products/TEZ/staging
-rw-r--r-- 3 hdfs hadoop 46518370 2016-09-02 11:06
/Products/TEZ/tez-0.8.4.tar.gz
{noformat}
Pig 0.16.0 seems to need a staging directory with submitter directory, even if
the staging directory has full access to "other" like my previous
configuration.
This issue looks like a regression to me, compared to Pig 0.15.0. Can you check
that point?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
