github-actions[bot] commented on code in PR #67573:
URL: https://github.com/apache/doris/pull/67573#discussion_r3950070563
##########
be/src/format_v2/parquet/reader/native/column_chunk_reader.cpp:
##########
@@ -192,7 +192,12 @@ ParquetReaderCompat parquet_reader_compat(const
std::string& created_by) {
return {.parquet_816_padding =
version.VersionLt(::parquet::ApplicationVersion::PARQUET_816_FIXED_VERSION()),
.data_page_v2_always_compressed = version.VersionLt(
-
::parquet::ApplicationVersion::PARQUET_CPP_10353_FIXED_VERSION())};
+
::parquet::ApplicationVersion::PARQUET_CPP_10353_FIXED_VERSION()),
+ .null_count_trusted =
+ !version.VersionLt(
Review Comment:
[P1] Use Arrow's actual null-count fix boundary.
[ARROW-12513/#18632](https://github.com/apache/arrow/issues/18632) reproduces
PyArrow 1.0.1-3.0.0 writing `null_count=0` for nullable dictionary columns, and
the fix landed only for Arrow 6. Affected Arrow 3 files identify as
`parquet-cpp version 1.5.1-SNAPSHOT`, while Arrow 4/5 use `parquet-cpp-arrow`;
this expression trusts both because 1.5.1 is above the unrelated 1.3.0
min/max-statistics cutoff and `VersionLt` returns false for a different
application name. Format-v2 `IS NULL` pruning can therefore still turn the bad
zero into `has_null=false` and discard the row group. Please classify both
pre-6 Arrow identities using the null-count fix boundary and add an
affected-writer fixture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
