github-actions[bot] commented on code in PR #68128:
URL: https://github.com/apache/doris/pull/68128#discussion_r4089283397
##########
be/src/exec/sink/writer/iceberg/viceberg_partition_writer.cpp:
##########
@@ -107,7 +110,7 @@ Status VIcebergPartitionWriter::open(RuntimeState* state,
RuntimeProfile* profil
.enable_int96_timestamps =
false};
_file_format_transformer = std::make_unique<VIcebergParquetWriter>(
state, _file_writer.get(), _write_output_expr_ctxs,
_write_column_names, false,
- parquet_options, _iceberg_schema_json, _schema);
+ parquet_options, _iceberg_schema_json, _schema,
_nan_count_field_ids);
Review Comment:
[P2] Preserve range pruning for Doris-written ORC files
The predicate change adds an `OR isNaN(...)` arm to FLOAT/DOUBLE range
filters for every Iceberg file format, but this branch only gives
`nan_count_field_ids` to the Parquet transformer.
`VOrcTransformer::collect_file_statistics_after_close` still emits no
`nan_value_counts` (the unchanged ORC half of `test_iceberg_write_stats2.out`
shows those counts remain null), so Iceberg must treat every NaN-free Doris ORC
file as possibly containing NaN. For example, a file bounded at `[1,2]` can no
longer be pruned for `d > 100`; the new Parquet-only regression test misses
this. Please count and publish the requested fields on the ORC path too, or
share the block-level counter across both transformers, and cover ORC pruning
end to end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
