mayankshriv opened a new issue #5490: URL: https://github.com/apache/incubator-pinot/issues/5490
With the Pinot community growing, as well as Pinot evolving for the better, we anticipate a ton of features that would go into performance critical parts of the code (e.g. query execution, RT segment generation, etc). Given that Pinot's forte is its blazingly fast speed, we would like to ensure that the rapid feature development does not compromise on its performance. We need a framework using which any PR can be evaluated for performance. Some ideas: Query Performance: 1. Create a few synthetic data sets with different characteristics that represent existing Pinot deployments across the community. We can probably enhance the existing data generator tool to server these requirements. 2. Generate a query set for a given data set to ensure desired level of selectivity. 3. Framework that runs a desired workload (qps) on these data sets with and without the changes from a PR, and reports the performance changes. Realtime: 1. Consumption rate 2. Segment generation time Ability to check on a variety of metrics, for example: 1. Latency p95/p99 2. Throughput 3. GC ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
