yashmayya opened a new pull request, #19003: URL: https://github.com/apache/pinot/pull/19003
This fixes two bugs in `PinotImplicitTableHintRule` (the rule that infers `tableOptions` partition hints when `inferPartitionHint` is enabled via the `pinot.broker.multistage.infer.partition.hint` broker config or the `inferPartitionHint` query option): ### 1. Explicit non-partition hint options (e.g. `is_replicated`) were dropped during inference `withNewTableOptions()` rebuilt the `tableOptions` hint with only the partition options (`partition_key` / `partition_function` / `partition_size` / `partition_parallelism`), silently discarding any other explicitly supplied options — notably `is_replicated='true'`. As a result, a replicated local join like: ```sql SELECT /*+ joinOptions(left_distribution_type='local', right_distribution_type='local') */ a.col1, b.col2 FROM a JOIN b /*+ tableOptions(is_replicated='true') */ ON a.col1 = b.col1 ``` fails during worker assignment with `Found multiple local exchanges in the children` when partition hint inference is enabled, because the replicated leaf loses its `is_replicated` flag and its `SINGLETON` send is counted as a second local exchange in `WorkerManager`. The rebuilt hint's kv-options are now seeded from the explicit hint so that options not modeled by `TableOptions` are carried over, and the merged (inferred + explicitly overridden) partition options overwrite the explicitly supplied ones. ### 2. Explicit `partition_key` / `partition_function` overrides of inferred options were never applied `overridePartitionKey()` and `overridePartitionFunction()` did a double map lookup — `kvOptions.get(kvOptions.get(PARTITION_KEY))` — which looks up the option *value* as a key and therefore always resolves to `null`, so partial explicit overrides of the inferred partition key/function were silently ignored (the documented contract is "any explicit hint will override the implicit hint"). **Note on behavior:** this is a user-visible behavior change for deployments running with partition hint inference enabled — a partial `tableOptions` hint carrying an incorrect `partition_key` / `partition_function` was previously ignored (the query planned with the inferred values), and will now be honored and fail planning with a validation error (e.g. `Partition key: col1 does not match partition column: col2`) until the hint is corrected or removed. ### Tests Planner regression tests (all fail without the fix): - `JoinPlans.json`: the replicated local join above with `SET inferPartitionHint=true`, plus a variant where the hint also carries `partition_parallelism`, both asserting the plan and successful worker assignment. - `PinotHintablePlans.json`: partial-hint override cases with `SET inferPartitionHint=true` — a wrong `partition_key` / `partition_function` now fails planning with the mismatch error, and a matching `partition_key` plans successfully. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
