Hi there,
I am having issues to get oozie spark action to read from hdfs on one secured
cluster and write it back to an hdfs in a remote secured cluster with kerberos
. However ooze/spark can read from the hdfs in a local cluster and write back
to hdfs in the same cluster . the oozie job properties file has the following :
sparkOpts=--conf
spark.yarn.am.extraJavaOptions=-Diop.version={{iop_full_version}} --conf
spark.yarn.access.namenodes=hdfs://namenode1:8020,hdfs://namenode2:8020
It seems that oozie is not attempting to get the delegation token for the
second namenode which a remote namenode although the abov option is set. Is it
any other way to force oozie to request the HDFS delegation token for both name
nodes? please help.
Also notice a regular spark can be able to perform the above
