Hi all, I agree that identity federation needs more thought, especially from the perspective that we have a Polaris principal on one side and a corresponding IAM role/user on the other. This raises an important question around how we want to handle authorization, since the vended credentials themselves are effectively the outcome of an authorization decision.
For this effort, I’d recommend leveraging Polaris GRANTs as the source of truth for generating the policy string that will drive credential vending. This keeps authorization centralized and consistent with Polaris’ model. As a starting point, we could keep things simple by having the federated catalog use a storage integration of type S3Table. On top of that, we can introduce a dedicated AwsS3TablesStorageIntegration that is responsible for generating the appropriate STS policy string. The flow would then look like this: - Use SigV4-based authentication (as introduced for Glue) to communicate with the federated catalog - Fetch metadata via the catalog - Use the AwsS3TablesStorageIntegration to vend credentials based on the derived policy string. This approach keeps authentication and authorization concerns well separated while allowing us to iterate. Happy to discuss this further in the PR as it evolves. Best, Prashant Singh On Thu, Mar 26, 2026 at 5:07 PM Yufei Gu <[email protected]> wrote: > > Let me expand on my "channel" idea - Polaris could act as a delegator in > the OAuth2 sense and propagate the API user's identity to the Federated > catalog. Then use the credentials vended by the federated catalog for its > own access to storage within the context of the request and return them to > the client / engine. > > That'd be the first step, which handles the authN for principals. We still > need to handle the authorization part consistently between Polaris and the > remote catalogs. > > Yufei > > > On Thu, Mar 26, 2026 at 4:47 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yufei, > > > > Good point about authorization. > > > > Let me expand on my "channel" idea - Polaris could act as a delegator in > > the OAuth2 sense and propagate the API user's identity to the Federated > > catalog. Then use the credentials vended by the federated catalog for its > > own access to storage within the context of the request and return them > to > > the client / engine. > > > > I understand this is a rather big feature, so it's definitely out of > scope > > of the S3 tables PR... but I still wonder if there is interest in > > supporting this use case. > > > > Cheers, > > Dmitri. > > > > On Thu, Mar 26, 2026 at 7:28 PM Yufei Gu <[email protected]> wrote: > > > > > > should Polaris merely channel them to the client instead of minting > > > different credentials? > > > > > > Credential vending should be based on the authorization. If a principal > > has > > > read or write privileges on a table, Polaris should reflect those > > > privileges in vended credentials. AFAIK, the current federation doesn't > > > pass authorization for each individual principal. Authorization still > > > happens in Polaris, which credential vending must honor. Passing > > > credentials from a remote catalog bypasses the Polaris authorization, > > which > > > raises security concerns > > > > > > Both PRs (4052 and 3699) are still in draft. I will take a closer look > > when > > > they are ready for review. > > > > > > > > > Yufei > > > > > > > > > On Wed, Mar 25, 2026 at 2:04 PM Dmitri Bourlatchkov <[email protected]> > > > wrote: > > > > > > > Hi Aritra, > > > > > > > > Thanks for starting this discussion. It's a very interesting feature. > > > > > > > > From my POV I see two sub-topics, where I hope for broader community > > > > engagement. > > > > > > > > 1) Credential Vending for Federated IRC catalogs in principle. > > > > > > > > IIRC, this PR builds on top of fedetrating to AWS S3 tables via the > > > Iceberg > > > > REST Catalog API. > > > > > > > > Does AWS support credential vending in their IRC implementation? > > > > > > > > If yes, should Polaris merely channel them to the client instead of > > > minting > > > > different credentials? > > > > > > > > If no, should Polaris treat S3 tables as a sub-type of S3 storage? I > > see > > > > some similarities to normal S3 + STS use case, at the same time the > > ARNs > > > > scheme is different and policies are different. This creates too many > > > > if/else constructs in the existing storage integration code, so I > hope > > it > > > > might be possible to refactor that and leverate polymorphism to > reduce > > > > if/else branches. > > > > > > > > 2) As I noted in GH comments, the current POC code uses different > > > channels > > > > for passing S3 table locations and normal S3 bucket locations. I > > believe > > > it > > > > is time to refactor that part of Polaris to represent both use cases > on > > > the > > > > same call path leading to the storage integration code. > > > > > > > > Tornike has draft PR [3699], which seems to be moving in a generally > > > > compatible direction. I hope we can make progress on that > refactoring, > > > > benefiting the S3 tables too. > > > > > > > > [3699] https://github.com/apache/polaris/pull/3699 > > > > > > > > Cheers, > > > > Dmitri. > > > > > > > > On Tue, Mar 24, 2026 at 5:46 PM Gupta, Aritra <[email protected]> > > wrote: > > > > > > > > > Dear All, > > > > > > > > > > With Polaris 1.3, SigV4 auth support was added for federated > > catalogs, > > > > > which is great. I've been trying to use this to federate Polaris to > > > > Amazon > > > > > S3 Tables, but wasn't able to make it fully work. Metadata > operations > > > > like > > > > > listing namespaces and loading tables work fine through federation, > > but > > > > > credential vending breaks because S3 Tables uses a different IAM > > action > > > > > namespace (s3tables:) and ARN format than standard S3. The existing > > > > session > > > > > policy logic hardcodes s3: actions, so the policy intersection ends > > up > > > > > empty and clients get ACCESS_DENIED on data access. > > > > > > > > > > I've put together a directional first-cut draft PR that adds S3 > > Tables > > > > > credential vending support. The approach auto-detects S3 Tables > > > catalogs > > > > > from the signingName in the SigV4 connection config, so no > additional > > > > user > > > > > configuration is needed beyond what's already required for > > federation. > > > > > > > > > > This is very much a starting point, and there's more work needed > > around > > > > > integration testing, documentation, and code cleanup. I'd really > > > > appreciate > > > > > the community's input on whether the overall direction makes sense, > > and > > > > any > > > > > help or feedback from folks who have context on the credential > > vending > > > > flow > > > > > would be wonderful. > > > > > > > > > > Looking forward to the discussion! > > > > > > > > > > Links: > > > > > > > > > > Draft PR: https://github.com/apache/polaris/pull/4052 > > > > > Related Issue: https://github.com/apache/polaris/issues/577 > > > > > > > > > > Best, > > > > > Aritra Gupta > > > > > > > > > > > > > > > > > > > >
