This is an automated email from the ASF dual-hosted git repository.
gengliang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 18faa83903b1 [SPARK-52521][SQL] `Right#replacement` should not access
SQLConf dynamically
18faa83903b1 is described below
commit 18faa83903b127a96472d5fb23ec4eb730c123fa
Author: Wenchen Fan <[email protected]>
AuthorDate: Wed Jun 18 09:12:32 2025 -0700
[SPARK-52521][SQL] `Right#replacement` should not access SQLConf dynamically
### What changes were proposed in this pull request?
`Right#replacement` is a lazy val that has non-deterministic initialization
timing. It's fragile to access SQLConf there as the conf value can be different
with different lazy val initialization timing. For example, if we initialize
the lazy val during view plan resolution, it will use the ANSI conf from the
view's recorded SQL confs, which can be different from the current session's
SQL conf. If the `Right` expression appears more than once in the query plan,
and their lazy val initi [...]
This PR fixes `Right#replacement` to always create `Substring` with
`failOnError = false`. The length parameter is guaranteed to be positive here,
so the `failOnError` flag doesn't matter. Setting it to false makes the
generated java code simpler.
### Why are the changes needed?
bug fix
### Does this PR introduce _any_ user-facing change?
yes, the query failed to analysis can now work
### How was this patch tested?
a new test
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #51210 from cloud-fan/ansi.
Lead-authored-by: Wenchen Fan <[email protected]>
Co-authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Gengliang Wang <[email protected]>
---
.../sql/catalyst/expressions/stringExpressions.scala | 2 +-
.../org/apache/spark/sql/execution/SQLViewSuite.scala | 17 +++++++++++++++++
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
index 55cc030a9953..18984dce34e9 100755
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
@@ -2354,7 +2354,7 @@ case class Right(str: Expression, len: Expression)
extends RuntimeReplaceable
If(
LessThanOrEqual(len, Literal(0)),
Literal(UTF8String.EMPTY_UTF8, str.dataType),
- new Substring(str, UnaryMinus(len))
+ new Substring(str, UnaryMinus(len, failOnError = false))
)
)
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
index 4fd96eadfac7..f715353fd431 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
@@ -1316,4 +1316,21 @@ abstract class SQLViewSuite extends QueryTest with
SQLTestUtils {
assert(ts1._1.getTime < ts2._1.getTime)
}
}
+
+ test("SPARK-52521: view with ANSI expressions") {
+ withView("v1") {
+ withSQLConf(ANSI_ENABLED.key -> "true") {
+ sql(
+ """
+ |CREATE VIEW v1 AS
+ |SELECT RIGHT(CAST(id AS STRING), 1) AS c
+ |FROM range(1)
+ |GROUP BY RIGHT(CAST(id AS STRING), 1)
+ |""".stripMargin)
+ }
+ withSQLConf(ANSI_ENABLED.key -> "false") {
+ checkAnswer(sql("SELECT * FROM v1"), Row("0"))
+ }
+ }
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
