somandal commented on PR #16931: URL: https://github.com/apache/pinot/pull/16931#issuecomment-3374309391
> > What will the behaviour be if server restarts after accepting the helix msg but before replacing the segment ? will server startup get the helix msgs again ? I guess server startup checks segment crc. But if not and helix msgs aren't resent, then this could be a problem ? > > AFAIK if the server goes down, the segments for it will no longer be in ONLINE / CONSUMING state in the EV. They'll be in OFFLINE state. When the server comes back online, Helix rebalancer will run and send out state transition messages for the segments assigned to the server since the EV won't match the IS. We have a check here to see if the segment CRC has changed, and if it has, we download a new copy and process that. > > My understanding of user defined messages is that if an error is thrown or it isn't processed then the message is lost (for error thrown i know for sure it's lost, but need to double check server restart). The above state transition messages should ensure we do replace the segments though. I'll try to test out what happens with user defined messages on restart and if the behavior is different than expected, will post an update here. > > cc @Jackie-Jiang in case I missed something in the above @krishan1390 i was able to verify locally that on restarting the server the user defined messages left in the queue are gone. so even though with this change, we'll be processing messages faster and storing them in the in-memory queue, the actual result is more or less the same - the refresh messages will be lost on restart. We will rely on the state transition messages to take the right action of downloading a new segment version if the CRC has changed. Hope this helps, let me know if you have any more questions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
