github-actions[bot] commented on code in PR #68452:
URL: https://github.com/apache/doris/pull/68452#discussion_r4245679552
##########
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##########
@@ -2951,6 +2954,13 @@ public Map<String, Action> getForceEagerAggHintMap() {
+ "The value set belongs to the fluss connector, which
rejects anything else")
public String flussUnionReadMode = "";
+ @VarAttrDef.VarAttr(name = ENABLE_PARTITION_COLUMN_VALUE_ONLY_OPTIMIZATION,
Review Comment:
[P2] Forward this optimization setting to the planning FE. After SET
enable_partition_column_value_only_optimization=false on a follower, a SELECT
MAX(p) FROM hive_table forwarded via force_forward_all_queries (or because the
follower cannot read) is replanned on the master with the default true. This
annotation has neither needForward nor an affectQueryResult flag, so
getForwardVariables() omits it and setForwardedSessionVariables() also skips
it. The client's opt-out is therefore lost and PARTITION_VALUE is still
selected. Add needForward=true and cover the session-map round trip/forwarded
plan.
##########
regression-test/suites/external_table_p2/hudi/test_hudi_runtime_filter_partition_pruning.groovy:
##########
@@ -202,6 +202,30 @@ suite("test_hudi_runtime_filter_partition_pruning",
"p2,external") {
sql """ set enable_runtime_filter_partition_prune = true; """
test_runtime_filter_partition_pruning()
+ // A min/max over only partition columns is answered from partition
metadata, on Hudi just as
+ // on Hive. Assert it is planned AND that it actually ran: the plan
can report
+ // pushdown agg=PARTITION_VALUE while the reader declines the range
and scans normally.
+ sql """ set enable_profile=true """
+ def totalRows = sql("select count(*) from int_partition_tb")[0][0] as
long
+ explain {
+ sql "select max(part1) from int_partition_tb"
+ contains "pushdown agg=PARTITION_VALUE"
+ }
+ profile("hudi_partition_value_input_rows") {
+ run {
+ sql """/* hudi_partition_value_input_rows */
+ select max(part1) from int_partition_tb"""
+ }
+ check { profileString, exception ->
+ assert exception == null
+ def scanRows = (profileString =~ /InputRows:\s+sum\s+(\d+)/)
+ .collect { it[1] as long }.max()
+ assertTrue(scanRows < totalRows,
+ "PARTITION_VALUE must not materialize every row: the
scan read " +
+ "${scanRows} rows for a ${totalRows}-row table")
Review Comment:
[P2] Pin native Hudi scanning for this row-reduction assertion. External
fuzzy mode randomizes force_jni_scanner, and the suite never overrides it. When
it is true, HudiScanPlanProvider routes even this COW table to JNI;
PARTITION_VALUE correctly falls back to reading all source rows, so MAX(part1)
feeds the aggregate the same four rows as totalRows and this assertion fails.
Save/set/restore force_jni_scanner=false for the native profile check, and
cover the JNI fallback with a result comparison that allows normal input
cardinality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
