This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b3e6d97a06f0 [SPARK-55385][CORE][SQL][FOLLOW-UP]
`getAncestorWithSamePartitionSizes` should stop at checkpointed ancestor
b3e6d97a06f0 is described below
commit b3e6d97a06f02df822744add3b9f95148ea0ea65
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Fri Feb 13 11:52:31 2026 -0800
[SPARK-55385][CORE][SQL][FOLLOW-UP] `getAncestorWithSamePartitionSizes`
should stop at checkpointed ancestor
### What changes were proposed in this pull request?
`getAncestorWithSamePartitionSizes` should stop at checkpointed ancestor
### Why are the changes needed?
if the ancestor is already checkpointed, no matter reliably or locally, it
should be returned
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
ci
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #54311 from zhengruifeng/zip_cp.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
core/src/main/scala/org/apache/spark/rdd/ZippedWithIndexRDD.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/core/src/main/scala/org/apache/spark/rdd/ZippedWithIndexRDD.scala
b/core/src/main/scala/org/apache/spark/rdd/ZippedWithIndexRDD.scala
index 98b1f60dbbef..60638282668d 100644
--- a/core/src/main/scala/org/apache/spark/rdd/ZippedWithIndexRDD.scala
+++ b/core/src/main/scala/org/apache/spark/rdd/ZippedWithIndexRDD.scala
@@ -42,7 +42,7 @@ class ZippedWithIndexRDD[T: ClassTag](prev: RDD[T]) extends
RDD[(T, Long)](prev)
private def getAncestorWithSamePartitionSizes(rdd: RDD[_]): RDD[_] = {
rdd match {
- case c: RDD[_] if c.getStorageLevel != StorageLevel.NONE => c
+ case c: RDD[_] if c.isCheckpointed || c.getStorageLevel !=
StorageLevel.NONE => c
case m: MapPartitionsRDD[_, _] if m.preservesPartitionSizes =>
getAncestorWithSamePartitionSizes(m.prev)
case m: MapPartitionsWithEvaluatorRDD[_, _] if m.preservesPartitionSizes
=>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
