gortiz opened a new pull request, #12704:
URL: https://github.com/apache/pinot/pull/12704
At this moment this PR is a draft where I'm trying to modify the way stats
are collected and encoded in multi-stage engine.
The plan is to:
1. Use enums as stat keys
2. Collect stats per stage.
* Each OpChain will collect a partial view for its stage and the stages
upstream.
* OpChains that receive data from more than one OpChain of the same stage
will merge them.
* The root stage (on the broker), where there is only one OpChain, will
collect the full view.
3. Change the way stats are encoded. Instead of JSON we will use a binary
format equal or more effective than the one used in DataTable.
4. Create new stats for specific multi-stage operators where it makes sense
5. Export these stats to broker clients
Current state:
[*] Define the StatMap that will map from enums to values.
[*] Define the encoding
[*] Define the StageStats
[ ] Substitute OpChainStats with StageStats in OpChain
[ ] Merge StageStats in receiving mailboxes.
[ ] Return the StageStats to broker clients
[ ] Create new stats for specific multi-stage operators where it makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
