[
https://issues.apache.org/jira/browse/PIG-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-4908:
------------------------------------
Attachment: PIG-4908-3-fixtest.patch
When absolute path is used, the nameInJar finally has the leading / removed
and added to jar. For eg: /tmp/sleep.py will be added to jar as tmp/sleep.py
so that it is accessible through
ScriptEngine.class.getResourceAsStream(scriptPath) when the jar is in
classpath.
Problem was we were looking for the absolute path first before trying the jar.
So additionally fixed that code to look for local file first if frontend else
look at the last in PIG-4908-3-fixtest.patch.
Saw that during compilation we were initializing the function too many times
with the different visitors. Will file a separate jira to cut down on that.
> JythonFunction refers to Oozie launcher script absolute path
> ------------------------------------------------------------
>
> Key: PIG-4908
> URL: https://issues.apache.org/jira/browse/PIG-4908
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.16.0
>
> Attachments: PIG-4908-1.patch, PIG-4908-2-fixtest.patch,
> PIG-4908-3-fixtest.patch
>
>
> We had a scenario where a user had multiple udfs all named udfs.py.
> JythonFunction was referring to the absolute localized path of udfs.py in the
> Oozie launcher. Tasks which ran on a node and which had a different version
> of udfs.py localized to same path (hashcode conflict) as the Oozie launcher
> failed. We should be referring to relative path of the files. The current
> code check is the canonical path starts with cwd, but that does not work as
> the files are downloaded to a different location and symlinked to the current
> working directory of the task attempt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
