sunchao commented on PR #5457: URL: https://github.com/apache/datafusion-comet/pull/5457#issuecomment-5447229825
Fixed both review issues. [10a06cc64](https://github.com/apache/datafusion-comet/pull/5457/commits/10a06cc64c0942468f3ecdccef7e0f9242f363da) keeps unbounded non-TRY `DATE` to `TIMESTAMP_NTZ` casts on Spark row execution in both native and JVM-dispatch admission. This preserves LIMIT and semi/anti join short circuiting, including nested/enclosing casts, while still throwing when Spark consumes an overflowing value. Safe literals and byte/short-derived dates remain eligible for native execution; nullable scalar `TRY_CAST` and fixed-zone `DATE` to `TIMESTAMP` retain their existing routes. Arbitrary date columns, including ordinary dates, now incur operator fallback. The checked Rust arithmetic is unchanged. [129cc1646](https://github.com/apache/datafusion-comet/pull/5457/commits/129cc16468768364e1a9009b4451c7ccb9bc813e) removes the redundant interpolator that failed Scalafix. I reproduced that failure against the unchanged-head test file and verified that the full patched Spark source passes CI's exact syntactic check. The full Spark 4.1.3 JVM reactor passed **201 tests**, with **8 existing ignored tests** and no failures, including the full cast/error suites and ten NTZ SQL configurations. The actual loaded native library matches this PR's unchanged native source. Spotless and Scalastyle also pass. The PR description now records the fallback cost and validation limits; CI for the new commit is pending. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
