zzzzming95 opened a new pull request, #9944:
URL: https://github.com/apache/iceberg/pull/9944
### What changes were proposed in this pull request?
When we are in kerberos environment, use hive mr to insert data into iceberg
table, we will get Exception :
```
Caused by: MetaException(message:Could not connect to meta store using any
of the URIs provided. Most recent failure:
org.apache.thrift.transport.TTransportException: GSS initiate failed at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
at
org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
```
hive can not connect HMS in yarn container beacause it has no kerberos
ticket. So we need to send delegation token to the yarn container.
To resolve this issue, i define a hive hook and get delegation before
mapreduce runging. Then store the delegation token to config . When
`OutputCommitter#setupJob` , it would get the token from the config then add it
to UGI . So hive can connect to HMS in the yarn container.
### Why are the changes needed?
fix issue.
### Does this PR introduce any user-facing change?
add hive hook:
```
<property>
<name>hive.exec.pre.hooks</name>
<value>org.apache.iceberg.mr.hive.HiveIcebergGenTokenHook</value>
</property>
```
### How was this patch tested?
Manual testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
