zclllyybb commented on issue #65909: URL: https://github.com/apache/doris/issues/65909#issuecomment-5044727959
Breakwater-GitHub-Analysis-Slot: slot_047e3257d370 This content is generated by AI for reference only. I checked the report against the issue commit `a4dae0b64e9e7c6db5983b566656442f1ae2bc5d` and refreshed `upstream/master` (`ab043a433cef0246bd81fbe76011a83b9a13d17b`). The reported diagnosis looks valid. The current BE scan path can evaluate the `region = 'east'` predicate too early: - `OlapScanner::_init_tablet_reader_params()` copies slot predicates into `_tablet_reader_params.predicates`, copies function filters, and keeps common-expression pushdown before the row-binlog change rows are materialized. - `TabletReader::_init_conditions_param()` classifies a non-key column predicate such as `region = 'east'` as a value predicate. - For a UNIQUE KEY merge-on-write table, `BetaRowsetReader::_should_push_down_value_predicates()` can return true through `enable_unique_key_merge_on_write`, so `BetaRowsetReader::get_segment_iterators()` installs that value predicate into `StorageReadOptions.column_predicates`. - `SegmentIterator` applies those storage predicates to the raw row-binlog row. At that point the normal `region` column is the after image (`north`), while the historical value is still in `__BEFORE__region__`. - `BlockReader::_min_delta_next_block()` reconstructs `UPDATE_BEFORE` only later by resolving regular columns to their `__BEFORE__*` mirrors when emitting the before half of an update. So `WHERE region = 'east'` is evaluated against the raw after-image row and can discard the row before `UPDATE_BEFORE(east)` is constructed. This is a row-binlog scan correctness bug, not an IVM refresh-only issue and not a stream-consumption-offset issue. Recommended fix direction: - For row-binlog scans that materialize change rows (`MIN_DELTA`, and the same class of risk applies to `DETAIL`), do not push mutable user value-column predicates, function filters, or common-expression filters into the storage/segment layer before reconstruction. - Keep only predicates that are proven safe before reconstruction, such as the TSO range filter and any other before/after-invariant predicate after explicit validation. - Ensure the fenced predicates are still evaluated after `_min_delta_next_block()` / `_detail_change_next_block()` has produced the final change rows. If any planner/scan path currently treats a pushed predicate as storage-only, it should be kept or reintroduced as an upper filter for table streams. - Add a regression test using the direct stream reproduction from this issue, preferably covering both `MIN_DELTA` and `DETAIL`: update a non-key value column from `east` to `north`, then verify `WHERE region = 'east'` returns the reconstructed `UPDATE_BEFORE` row and `WHERE region = 'north'` returns the `UPDATE_AFTER` row. Missing information is small for root-cause triage because the issue already includes a minimal SQL reproduction and observed output. For release/backport decisions, maintainers still need to identify the first affected commit or affected release branches. If the regression test behaves differently in CI, the useful follow-up artifacts would be `EXPLAIN VERBOSE` for the filtered stream query and BE scan logs showing which predicates were pushed into the row-binlog tablet reader. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
