This is an automated email from the ASF dual-hosted git repository.
tustvold pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs-object-store.git
The following commit(s) were added to refs/heads/main by this push:
new a673ed6 Retry requests when status code is 429 (#410)
a673ed6 is described below
commit a673ed623a1b7ffe6c8acd962e5e0d0150d5d6c5
Author: Sebastián Galkin <[email protected]>
AuthorDate: Sun Jul 6 14:54:34 2025 -0300
Retry requests when status code is 429 (#410)
This code is what HTTP uses to indicate the client has sent too many
requests in a given amount of time. It may sound counter-intuitive to
retry in this situation, but that's why we use an exponential backoff
mechanism. It gives the server the opportunity to recover, without
failing the requests immediately.
The retry mechanism already works in object stores like S3 because they
return a server error. But without this change, we are not handling GCS
properly. GCS returns a client error `429 Too Many Requests` instead.
This change enables retries on this response too.
A more advanced retry mechanism would use the optional response header
`Retry-After`, but that is beyond the scope of this PR.
Closes: #309
---
src/client/retry.rs | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/client/retry.rs b/src/client/retry.rs
index 43ad3e0..d33e1e9 100644
--- a/src/client/retry.rs
+++ b/src/client/retry.rs
@@ -402,6 +402,7 @@ impl RetryableRequest {
let status = r.status();
if ctx.exhausted()
|| !(status.is_server_error()
+ || status == StatusCode::TOO_MANY_REQUESTS
|| (self.retry_on_conflict && status ==
StatusCode::CONFLICT))
{
let source = match status.is_client_error() {
