Gabriel39 opened a new pull request, #70219:
URL: https://github.com/apache/doris/pull/70219
### What problem does this PR solve?
Problem Summary:
An asynchronous MV defined with `non_nullable(vec) AS vec` cannot rewrite
queries selecting the original nullable `vec`. This prevents transparent use of
NOT NULL materialized vector columns and their ANN indexes. The missing value
mapping is in the shared async materialization context, so it affects external
catalogs generally.
Keep the existing `non_nullable(slot)` mapping and add one equivalent
raw-slot mapping to a same-type NOT NULL MV output. Preserve query nullability
and ExprIds through existing output alignment. Prefer direct outputs and avoid
multiplying equivalent mapping alternatives. Also recognize value-preserving
nullable wrappers in ANN TopN virtual expressions, and request early MV
rewriting for vector TopN queries so ANN pushdown runs after discovering the MV
index under the default strategy.
### Release note
Allow nullable source-column queries to use asynchronous MVs materialized
with `non_nullable(column)`, including ANN TopN queries, without adding the
assertion to the query.
### Check List (For Author)
- Test:
- [x] Unit Test: 23 focused FE tests pass, with
failing-before/passing-after checks for the mapping, ANN wrapper and
rewrite-stage regressions.
- [x] Regression test: native scalar/array projection, filters, ANN plan
and execution counter, outer-join NULLs, failed refresh/fallback and partial
partition UNION; loopback JDBC source comparison.
- Added gated external suites for Hive, Iceberg, Paimon, Hudi, JDBC,
ADBC, Elasticsearch, Fluss, Trino connector and MaxCompute. Hive ORC/Parquet
and Iceberg include array columns and ANN queries.
- Remote source suites require their respective fixtures and were not
executed in the local environment. Local SQL validation uses the modified FE
with an earlier master ASAN BE build; this PR has no BE code changes.
- FE build, focused incremental compilation and FE Checkstyle.
- Behavior changed:
- [x] Yes. More queries can use equivalent MV outputs. Source schema,
NULL assertion failures and freshness checks are preserved.
- Does this need documentation?
- [x] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
