This is an automated email from the ASF dual-hosted git repository.
gengliang pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 6c6d36c8f912 [SPARK-52521][SQL] `Right#replacement` should not access
SQLConf dynamically
6c6d36c8f912 is described below
commit 6c6d36c8f9123dded2a44ebf16d4368b6088ebd0
Author: Wenchen Fan <[email protected]>
AuthorDate: Wed Jun 18 09:12:32 2025 -0700
[SPARK-52521][SQL] `Right#replacement` should not access SQLConf dynamically
### What changes were proposed in this pull request?
`Right#replacement` is a lazy val that has non-deterministic initialization
timing. It's fragile to access SQLConf there as the conf value can be different
with different lazy val initialization timing. For example, if we initialize
the lazy val during view plan resolution, it will use the ANSI conf from the
view's recorded SQL confs, which can be different from the current session's
SQL conf. If the `Right` expression appears more than once in the query plan,
and their lazy val initi [...]
This PR fixes `Right#replacement` to always create `Substring` with
`failOnError = false`. The length parameter is guaranteed to be positive here,
so the `failOnError` flag doesn't matter. Setting it to false makes the
generated java code simpler.
### Why are the changes needed?
bug fix
### Does this PR introduce _any_ user-facing change?
yes, the query failed to analysis can now work
### How was this patch tested?
a new test
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #51210 from cloud-fan/ansi.
Lead-authored-by: Wenchen Fan <[email protected]>
Co-authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Gengliang Wang <[email protected]>
(cherry picked from commit 18faa83903b127a96472d5fb23ec4eb730c123fa)
Signed-off-by: Gengliang Wang <[email protected]>
---
.../sql/catalyst/expressions/stringExpressions.scala | 2 +-
.../org/apache/spark/sql/execution/SQLViewSuite.scala | 17 +++++++++++++++++
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
index b534d3fb4dfb..a0852298aaa5 100755
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
@@ -2354,7 +2354,7 @@ case class Right(str: Expression, len: Expression)
extends RuntimeReplaceable
If(
LessThanOrEqual(len, Literal(0)),
Literal(UTF8String.EMPTY_UTF8, str.dataType),
- new Substring(str, UnaryMinus(len))
+ new Substring(str, UnaryMinus(len, failOnError = false))
)
)
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
index b26cdfaeb756..77a988f340e9 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
@@ -1323,4 +1323,21 @@ abstract class SQLViewSuite extends QueryTest with
SQLTestUtils {
assert(ts1._1.getTime < ts2._1.getTime)
}
}
+
+ test("SPARK-52521: view with ANSI expressions") {
+ withView("v1") {
+ withSQLConf(ANSI_ENABLED.key -> "true") {
+ sql(
+ """
+ |CREATE VIEW v1 AS
+ |SELECT RIGHT(CAST(id AS STRING), 1) AS c
+ |FROM range(1)
+ |GROUP BY RIGHT(CAST(id AS STRING), 1)
+ |""".stripMargin)
+ }
+ withSQLConf(ANSI_ENABLED.key -> "false") {
+ checkAnswer(sql("SELECT * FROM v1"), Row("0"))
+ }
+ }
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
