tibrewalpratik17 opened a new issue, #12667: URL: https://github.com/apache/pinot/issues/12667
We have an partial-upsert table where the number of updates per key is pretty high in a short period of time (within an hour we get 1000s of updates for a key). Between this, if we are querying for that particular primary key, we see no response from Pinot intermittently. I saw, this coincides with an update received for that primary key (query is received within 1 second of a new record for that key). After few seconds, the record comes up again in the query response and everything works fine until there is another overlap of query-time and ingestion-time. I suspect it might be happening because we update DocID by removing it first and then adding it again. https://github.com/apache/pinot/blob/168408aa8d0f94de3004abe6e49b6263cef24186/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java#L303-L308 And it might be a race condition where query is received between these 2 actions. Is this expected behaviour? Is there a way we can guarantee atleast the older record (if not newer) during this time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
