yashmayya opened a new pull request, #18510: URL: https://github.com/apache/pinot/pull/18510
## Summary Follow-up to #18035. Adds a broker-level config to opt a cluster into the MSE streaming group-by leaf-stage operator by default, while leaving per-query control to operators and users. - New broker config `pinot.broker.mse.streaming.group.by.flush.threshold` (default `0`, disabled). - When set to a positive value, the broker injects it as the `streamingGroupByFlushThreshold` query option for MSE queries that don't already specify it. - Per-query `SET streamingGroupByFlushThreshold = N` always wins via `putIfAbsent`, including `SET streamingGroupByFlushThreshold = 0` to disable streaming group-by for a specific query when the cluster default is non-zero. - Injection happens once after compilation but before the EXPLAIN/execute branch, so `EXPLAIN PLAN FOR …` reflects the same options the executed query would see. ## Rollout / backwards compatibility - Default is `0` → no behavior change for existing deployments. Existing query option still works exactly as before. - Recommended starting value mirrors the PR #18035 guidance (a few thousand groups); operators should tune based on observed leaf-stage cardinality. - Mixed-broker rollouts are safe: brokers without the config simply don't inject the option, and servers handle the option the same way either way. - The config takes effect on broker restart (resolved once in the constructor, like the other MSE broker defaults). ## Test plan - [x] New unit tests in `MultiStageBrokerRequestHandlerTest` covering: injection when option absent, per-query SET=0 / SET=N overriding the broker default, no injection when config unset. - [x] `./mvnw spotless:apply checkstyle:check license:check -pl pinot-spi,pinot-broker` clean. - [x] `./mvnw -pl pinot-broker -Dtest=MultiStageBrokerRequestHandlerTest test` — 4/4 passing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
