singhpk234 opened a new pull request, #13352: URL: https://github.com/apache/iceberg/pull/13352
### About the change presently, we retry on 502 / 504 as well [here](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/ExponentialHttpRequestRetryStrategy.java#L86), we have a spec definition stating [here](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L1070) that when these status are thrown the commit status can be unknown, so says the [RFC](https://datatracker.ietf.org/doc/html/rfc9110#name-502-bad-gateway) as something went wrong in the middle of processing the request. So we retry with 502 and 504 we can conflict with ourselves, as we don't know when we receive the status of 429 is it because we retried on 502 and then got 429 or something else happened, so it's better to throw the commit state unknown if we know the commit has been retried. The Iceberg users who complained this was with 504 : slack thread - https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1747992294134219 We have similar handling for glue as we did here https://github.com/apache/iceberg/pull/7198 as there are internal http retries over the glue SDK. So I Added a PR considering above https://github.com/apache/iceberg/pull/12818 But based on offline discussions with more folks, according to the spec its best to not retry on 502 / 504 as it clearly calls out that the status is unknown. The change attempts to : - revert https://github.com/apache/iceberg/pull/12818 [revert commit a5a58f0a14effa339caa66cdd17ef2dc2e34b7e8] - removes 502 and 504 from the retries. ### Testing Adding unit tests cc @amogh-jahagirdar @rdblue @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
