swaminathanmanish opened a new pull request, #14923: URL: https://github.com/apache/pinot/pull/14923
**Problem**: The race handling for parallel push protection does not take into account tier configs. See [the check](https://github.com/apache/pinot/blob/a29ecd7c27020a3ce598cfe3bc56f9c3016adb3a/pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/ZKOperator.java#L557) that looks for Zk version to determine if there's a race (i.e version 0 indicates no race and > 0 indicates a race). However [we have this update to Zk](https://github.com/apache/pinot/blob/a29ecd7c27020a3ce598cfe3bc56f9c3016adb3a/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java#L3640 ) in the presence of tiered config and this happens in the segment-server assignment path just after creation of Zk metadata which transitions the version from 0->1. The version check fails and we end up deleting the segment even when there's no race. **Solution:** As a workaround, disabling parallelProtection when pushParallelism is 1 . For batch load we dont need this feature (push parallelism) as we are uploading a bug batch of segments in one go. These uploads are also metadata pushes to the controller which is lightweight. Not changing the default at the API level, for backwards compatibility. The spark users of this feature. The existing integ test SegmentUploadIntegrationTest uses the default pushParallelism (1). Verified that parallel protection is set to false. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
