jimmystewpot commented on code in PR #2396: URL: https://github.com/apache/iceberg-rust/pull/2396#discussion_r4605817673
########## crates/catalog/glue/src/catalog.rs: ########## Review Comment: G'day @blackmwk, thanks for the review! Splitting this into the three PRs (`iceberg-aws`, `glue`, and `s3tables`) and tidying up `iceberg-aws` with `AwsProperties` (modeled on `TableProperties`), separate modules (`properties`, `credentials`, `config`), `publish = true`, and a `README.md` makes complete sense. We have had a think about your comment on adopting a `SessionCatalog` design similar to `RestSessionCatalog`, and these are the designs that came to mind based on the example you provided: **Option 1: Dedicated `SessionCatalog` Crate (e.g. `crates/catalog/session` or `iceberg-catalog-session`)** - Extract the reusable `SessionCatalog` plumbing — the `SessionBoundCatalog<S: SessionCatalog>` façade (which binds a `SessionContext` and implements the 15 `Catalog` trait methods), `SessionCatalogBuilder`, contextual session caching (keyed by `SessionContext::session_id()`), and pluggable `AuthManager` / `AuthSession` traits — into its own crate. - `iceberg-catalog-rest`, `iceberg-catalog-glue`, `iceberg-catalog-s3tables`, and future catalogs can all depend on this crate instead of each re-implementing the `Catalog` ↔ `SessionCatalog` wrapper and session cache boilerplate. **Option 2: Generic `SessionBoundCatalog<S>` in Core `iceberg` + Shared AWS Session Layer in `iceberg-aws`** - Put the generic `Catalog` façade over any `S: SessionCatalog` directly in `iceberg::catalog::session` (alongside the existing `SessionCatalog` trait and `SessionContext` struct, similar to what #2999 explored). - Put the AWS-specific session/credential lifecycle (`AwsAuthSession` / `AwsSessionManager`, managing `SdkConfig`, STS `AssumeRole` refresh, `FileIO` credential bridging, and per-`SessionContext` role/identity overrides) in `iceberg-aws`, so `GlueSessionCatalog` and `S3TablesSessionCatalog` share the same AWS session engine. **Option 3: `GlueSessionCatalog` & `S3TablesSessionCatalog` Inside Each Catalog Crate (Matching `iceberg-catalog-rest` Today)** - Mirror `RestSessionCatalog` / `RestCatalog` directly inside `iceberg-catalog-glue` (`GlueSessionCatalog` implementing `SessionCatalog`, with `GlueCatalog` as the `Catalog` façade binding a `SessionContext`) and `iceberg-catalog-s3tables`, while delegating the shared AWS STS / `FileIO` session state to `iceberg-aws`. We can either land the initial 3-PR split (`iceberg-aws`, `glue`, `s3tables`) first and follow up immediately with the `SessionCatalog` PR, or fold the `SessionCatalog` change into the Glue / S3Tables PRs once `iceberg-aws` lands. What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
