github-actions[bot] commented on code in PR #69016:
URL: https://github.com/apache/doris/pull/69016#discussion_r4257982859
##########
be/src/format_v2/parquet/parquet_reader.cpp:
##########
@@ -74,104 +74,149 @@ const ParquetColumnSchema* schema_child_by_name(const
ParquetColumnSchema& schem
return child_it == schema.children.end() ? nullptr : child_it->get();
}
-bool collect_variant_residual_leaf_ids(const ParquetColumnSchema& schema,
- const format::LocalColumnIndex&
projection,
- std::vector<int>* residual_leaf_ids) {
+bool collect_variant_terminal_fallback_leaf_ids(const ParquetColumnSchema&
schema,
+ const
format::LocalColumnIndex& projection,
+ std::vector<int>*
residual_leaf_ids) {
DORIS_CHECK(residual_leaf_ids != nullptr);
const auto* value = schema_child_by_name(schema, "value");
const auto* typed_value = schema_child_by_name(schema, "typed_value");
- if (value != nullptr && typed_value != nullptr) {
- if (value->kind != ParquetColumnSchemaKind::PRIMITIVE ||
value->leaf_column_id < 0) {
+ if (value != nullptr && value->kind == ParquetColumnSchemaKind::PRIMITIVE
&&
+ typed_value != nullptr) {
+ if (value->leaf_column_id < 0) {
return false;
}
+ const auto typed_projection_it =
+ std::ranges::find_if(projection.children, [&](const auto&
child_projection) {
+ return child_projection.local_id() ==
typed_value->local_id;
+ });
+ if (typed_projection_it == projection.children.end()) {
+ return false;
+ }
+ if (typed_value->kind != ParquetColumnSchemaKind::PRIMITIVE) {
+ return collect_variant_terminal_fallback_leaf_ids(*typed_value,
*typed_projection_it,
Review Comment:
[P1] Preserve legacy generator plans during rolling upgrades
Skipping the ancestor `value` here relies on the new FE container-preserving
access paths, but an upgraded BE still accepts plans from the pre-PR FE:
```text
Project(x['k'])
Generate(explode(v) AS x)
FileScan(v)
```
That FE sends `[v,k]` for `SELECT x['k'] FROM t LATERAL VIEW explode(v) e AS
x`. With a conforming shredded Parquet schema containing object field `k`, a
row can store `v = [{"k":1}]` in the root residual while `k.value` is all NULL.
The new guard therefore retains a partial `v` and omits the array.
`VExplodeV2TableFunction::_process_init_variant` then calls
`cast_variant_to_array(nullptr, ...)`, whose `get_value_ref()` throws "A
projected Parquet Variant can only serve its validated shredded leaves" instead
of returning `1`. The previous ancestor check restored full `v`.
Both sides still accept execution version 17, and scanner selection does not
negotiate this new path contract. Keep the conservative ancestor check for
legacy requests, or gate terminal-only projection on an explicit producer
capability, and cover this old-FE request in a compatibility test. This remains
distinct from thread 4256992660: that current-FE variadic fix cannot protect an
older FE's unary plan during BE-first upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
