plusplusjiajia opened a new pull request, #892:
URL: https://github.com/apache/iceberg-cpp/pull/892
## What
Vended storage credentials are short-lived, so a long scan or a reused
table eventually fails on expired ones. This fills the `TODO(gangwu)` left in
`arrow_s3_file_io.cc` by #719.
## How
`ArrowS3FileIO` reads `s3.session-token-expires-at-ms` and, five minutes
before the earliest applied credential expires, calls a
`StorageCredentialRefresher` and rebuilds its per-prefix delegates.
`RestCatalog` supplies that callback from the `LoadCredentials` endpoint when
the server advertises it; `ResolvingFileIO`
passes it down. Aligned with Java's `VendedCredentialsProvider`: same lead
time, same lazy refresh rather than a background thread, same refusal of an
empty credential list.
- **A refresh never makes things worse.** On failure, or a result with
nothing this FileIO can serve, the current credentials stay and the next
attempt backs off 30s — installing an empty result would drop working
credentials for whatever ambient identity the AWS chain finds.
- **Rebuilding happens outside the credential lock**, since building an S3
client can reach out to discover a bucket region. An install counter lets a
refresh notice it has been superseded meanwhile.
- **An operation whose credentials already expired waits** for an
in-flight refresh instead of proceeding to a guaranteed auth error, bounded at
10s.
`SupportsStorageCredentials::credentials()` now returns by value: a
refresh can replace the vector concurrently. All in-tree callers are tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
