After looking through the PR and the AWS docs, my read is that the delta here may be narrower than it first appeared.
IIUC, s3tables:GetTableData / s3tables:PutTableData are permission-only IAM actions, while the actual table data access still goes through standard S3 APIs. If so, the main difference here seems to be in how Polaris constructs the vending policy: s3tables:* actions and table-level ARN scoping, rather than the current s3:* / path-based model. Given that, I’m aligned with keeping Polaris GRANTs as the authz source of truth, and introducing a dedicated storage-integration path for S3 Tables to translate those decisions into the right IAM policy shape. My bias would be to keep that separation explicit rather than grow more branching in the existing S3 path. I’ve left a few implementation-specific comments on the PR, so I won’t repeat those here. -ej On Thu, Mar 26, 2026 at 6:07 PM Prashant Singh <[email protected]> wrote: > Hi all, > > I agree that identity federation needs more thought, especially from the > perspective that we have a Polaris principal on one side and a > corresponding IAM role/user on the other. This raises an important question > around how we want to handle authorization, since the vended credentials > themselves are effectively the outcome of an authorization decision. > > For this effort, I’d recommend leveraging Polaris GRANTs as the source of > truth for generating the policy string that will drive credential vending. > This keeps authorization centralized and consistent with Polaris’ model. > > As a starting point, we could keep things simple by having the federated > catalog use a storage integration of type S3Table. On top of that, we can > introduce a dedicated AwsS3TablesStorageIntegration that is responsible for > generating the appropriate STS policy string. > > The flow would then look like this: > > - Use SigV4-based authentication (as introduced for Glue) to communicate > with the federated catalog > - Fetch metadata via the catalog > - Use the AwsS3TablesStorageIntegration to vend credentials based on the > derived policy string. > > This approach keeps authentication and authorization concerns well > separated while allowing us to iterate. > > Happy to discuss this further in the PR as it evolves. > > Best, > Prashant Singh > > On Thu, Mar 26, 2026 at 5:07 PM Yufei Gu <[email protected]> wrote: > > > > Let me expand on my "channel" idea - Polaris could act as a delegator > in > > the OAuth2 sense and propagate the API user's identity to the Federated > > catalog. Then use the credentials vended by the federated catalog for its > > own access to storage within the context of the request and return them > to > > the client / engine. > > > > That'd be the first step, which handles the authN for principals. We > still > > need to handle the authorization part consistently between Polaris and > the > > remote catalogs. > > > > Yufei > > > > > > On Thu, Mar 26, 2026 at 4:47 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > Hi Yufei, > > > > > > Good point about authorization. > > > > > > Let me expand on my "channel" idea - Polaris could act as a delegator > in > > > the OAuth2 sense and propagate the API user's identity to the Federated > > > catalog. Then use the credentials vended by the federated catalog for > its > > > own access to storage within the context of the request and return them > > to > > > the client / engine. > > > > > > I understand this is a rather big feature, so it's definitely out of > > scope > > > of the S3 tables PR... but I still wonder if there is interest in > > > supporting this use case. > > > > > > Cheers, > > > Dmitri. > > > > > > On Thu, Mar 26, 2026 at 7:28 PM Yufei Gu <[email protected]> wrote: > > > > > > > > should Polaris merely channel them to the client instead of minting > > > > different credentials? > > > > > > > > Credential vending should be based on the authorization. If a > principal > > > has > > > > read or write privileges on a table, Polaris should reflect those > > > > privileges in vended credentials. AFAIK, the current federation > doesn't > > > > pass authorization for each individual principal. Authorization still > > > > happens in Polaris, which credential vending must honor. Passing > > > > credentials from a remote catalog bypasses the Polaris authorization, > > > which > > > > raises security concerns > > > > > > > > Both PRs (4052 and 3699) are still in draft. I will take a closer > look > > > when > > > > they are ready for review. > > > > > > > > > > > > Yufei > > > > > > > > > > > > On Wed, Mar 25, 2026 at 2:04 PM Dmitri Bourlatchkov < > [email protected]> > > > > wrote: > > > > > > > > > Hi Aritra, > > > > > > > > > > Thanks for starting this discussion. It's a very interesting > feature. > > > > > > > > > > From my POV I see two sub-topics, where I hope for broader > community > > > > > engagement. > > > > > > > > > > 1) Credential Vending for Federated IRC catalogs in principle. > > > > > > > > > > IIRC, this PR builds on top of fedetrating to AWS S3 tables via the > > > > Iceberg > > > > > REST Catalog API. > > > > > > > > > > Does AWS support credential vending in their IRC implementation? > > > > > > > > > > If yes, should Polaris merely channel them to the client instead of > > > > minting > > > > > different credentials? > > > > > > > > > > If no, should Polaris treat S3 tables as a sub-type of S3 storage? > I > > > see > > > > > some similarities to normal S3 + STS use case, at the same time the > > > ARNs > > > > > scheme is different and policies are different. This creates too > many > > > > > if/else constructs in the existing storage integration code, so I > > hope > > > it > > > > > might be possible to refactor that and leverate polymorphism to > > reduce > > > > > if/else branches. > > > > > > > > > > 2) As I noted in GH comments, the current POC code uses different > > > > channels > > > > > for passing S3 table locations and normal S3 bucket locations. I > > > believe > > > > it > > > > > is time to refactor that part of Polaris to represent both use > cases > > on > > > > the > > > > > same call path leading to the storage integration code. > > > > > > > > > > Tornike has draft PR [3699], which seems to be moving in a > generally > > > > > compatible direction. I hope we can make progress on that > > refactoring, > > > > > benefiting the S3 tables too. > > > > > > > > > > [3699] https://github.com/apache/polaris/pull/3699 > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > On Tue, Mar 24, 2026 at 5:46 PM Gupta, Aritra <[email protected]> > > > wrote: > > > > > > > > > > > Dear All, > > > > > > > > > > > > With Polaris 1.3, SigV4 auth support was added for federated > > > catalogs, > > > > > > which is great. I've been trying to use this to federate Polaris > to > > > > > Amazon > > > > > > S3 Tables, but wasn't able to make it fully work. Metadata > > operations > > > > > like > > > > > > listing namespaces and loading tables work fine through > federation, > > > but > > > > > > credential vending breaks because S3 Tables uses a different IAM > > > action > > > > > > namespace (s3tables:) and ARN format than standard S3. The > existing > > > > > session > > > > > > policy logic hardcodes s3: actions, so the policy intersection > ends > > > up > > > > > > empty and clients get ACCESS_DENIED on data access. > > > > > > > > > > > > I've put together a directional first-cut draft PR that adds S3 > > > Tables > > > > > > credential vending support. The approach auto-detects S3 Tables > > > > catalogs > > > > > > from the signingName in the SigV4 connection config, so no > > additional > > > > > user > > > > > > configuration is needed beyond what's already required for > > > federation. > > > > > > > > > > > > This is very much a starting point, and there's more work needed > > > around > > > > > > integration testing, documentation, and code cleanup. I'd really > > > > > appreciate > > > > > > the community's input on whether the overall direction makes > sense, > > > and > > > > > any > > > > > > help or feedback from folks who have context on the credential > > > vending > > > > > flow > > > > > > would be wonderful. > > > > > > > > > > > > Looking forward to the discussion! > > > > > > > > > > > > Links: > > > > > > > > > > > > Draft PR: https://github.com/apache/polaris/pull/4052 > > > > > > Related Issue: https://github.com/apache/polaris/issues/577 > > > > > > > > > > > > Best, > > > > > > Aritra Gupta > > > > > > > > > > > > > > > > > > > > > > > > > > >
