[
https://issues.apache.org/jira/browse/PIG-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387614#comment-14387614
]
Mohit Sabharwal commented on PIG-4269:
--------------------------------------
[~kellyzly], looks like the POSort is being removed from MR plan as part of
SecondaryKeyOptimizerMR:
https://github.com/apache/pig/blob/8294883c9f6798be2871ca513a266f51cc1e9614/src/org/apache/pig/backend/hadoop/executionengine/util/SecondaryKeyOptimizerUtil.java#L352
If the extra POSort is indeed the root cause of
TestAccumulator#testAccumAfterNestedOp failure, then we may need to implement
SecondaryKeyOptimizer for Spark.
It's not clear to me how your PIG-4438 patch is addressing the failure.
> Enable unit test "TestAccumulator" for spark
> --------------------------------------------
>
> Key: PIG-4269
> URL: https://issues.apache.org/jira/browse/PIG-4269
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: PIG-4269.patch, PIG-4269_1.patch, PIG-4269_Jekins.png,
> TEST-org.apache.pig.test.TestAccumulator.txt
>
>
> error log is attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
