Dénes Bodó created KNOX-2991:
--------------------------------
Summary: Sanitise Oozie rewrite rules
Key: KNOX-2991
URL: https://issues.apache.org/jira/browse/KNOX-2991
Project: Apache Knox
Issue Type: Bug
Components: Server
Affects Versions: 1.6.0
Reporter: Dénes Bodó
Assignee: Dénes Bodó
Testing Oozie through Knox proxy I found that there are some strange rewrite
rule which seems outdated:
{noformat}
<detect path="name" value="inputDir">
<apply path="value" rule="OOZIE/oozie/hdfs-path"/>
</detect>
<detect path="name" value="outputDir">
<apply path="value" rule="OOZIE/oozie/hdfs-path"/>
</detect>
{noformat}
This ticket is intended to track the work removing them.
*inputDir* and *outputDir* are frequently used in Oozie's job.properties as a
single directory name instead of a full HDFS path so in these cases the Oozie
workflow fails running due to incorrect variable resolution:
Configuration in job.properties:
{noformat}
nameNode=WILL_BE_UPDATED_BY_KNOX
outputDir=my_custom_output_dir {noformat}
workflow.xml:
{code:xml}
<prepare>
<delete
path="${nameNode}/user/${wf:user()}/examples/output-data/${outputDir}"/>
</prepare>{code}
Error in Oozie launcher:
{noformat}
Launcher AM execution failed
java.lang.IllegalArgumentException: java.net.URISyntaxException: Expected
scheme-specific part at index 5: hdfs:
at org.apache.hadoop.fs.Path.initialize(Path.java:259)
at org.apache.hadoop.fs.Path.<init>(Path.java:217)
at org.apache.hadoop.fs.Path.<init>(Path.java:125)
at org.apache.hadoop.fs.Globber.doGlob(Globber.java:285)
at org.apache.hadoop.fs.Globber.glob(Globber.java:202)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2107)
at
org.apache.oozie.action.hadoop.FSLauncherURIHandler.delete(FSLauncherURIHandler.java:59)
at
org.apache.oozie.action.hadoop.PrepareActionsHandler.execute(PrepareActionsHandler.java:83)
at
org.apache.oozie.action.hadoop.PrepareActionsHandler.prepareAction(PrepareActionsHandler.java:74)
at
org.apache.oozie.action.hadoop.LauncherAM.executePrepare(LauncherAM.java:378)
at
org.apache.oozie.action.hadoop.LauncherAM.access$100(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:229)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:226)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
Caused by: java.net.URISyntaxException: Expected scheme-specific part at index
5: hdfs:
at java.net.URI$Parser.fail(URI.java:2847)
at java.net.URI$Parser.failExpecting(URI.java:2853)
at java.net.URI$Parser.parse(URI.java:3056)
at java.net.URI.<init>(URI.java:746)
at org.apache.hadoop.fs.Path.initialize(Path.java:256)
... 20 more {noformat}
Found the real HDFS path after debugging the Oozie action:
{noformat}
hdfs://a.b.c.d:8020/user/test/examples/output-data/hdfs://a.b.c.d:8020/my_custom_output_dir{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
