shwin opened a new issue, #10076: URL: https://github.com/apache/pinot/issues/10076
We have observed the following behavior: 1. We have an OFFLINE table 2. We load 1000 segments via the SegmentGenerationAndPush minion-based task; they are about 500MB each 3. The minions finish all the subtasks, and the `SegmentGenerationAndPushTask` is complete. 4. We run a `select COUNT(*) from table;` query, and continually get larger and larger numbers, as we start to hit more and more segments (as seen in the `numSegmentsQueried`; that is, we hit fewer than the idealState of segments, and so are getting partial results until all segments are fully available. We expect this to only return correct results for `numSegmentsQueried: 1000`, since the broker can't optimize hitting fewer than `idealstate` segments, as we understand it. 5. There is no indication in the query response that we are querying fewer than the idealState of segments; this is unlike if a server has just gone down, in which we get an `exception:` in the query response. We'd either like a something in the query response indicating that fewer-than-ideal segments are available, or some flag to fail queries in this case (I guess the former is more flexible for clients to decide how to handle). This may be a dupe of https://github.com/apache/pinot/issues/7130, as perhaps this is the same case as (2) there ("Routing table doesn't contain full segments, e.g. all replicas are down for a segment in idealStates.")/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
