Hi all, I'd like to bring up a refactoring effort around credential vending that I've been working on in PR #3699 <https://github.com/apache/polaris/pull/3699>. Dmitri has been providing feedback and helping a lot along the way, but I wanted to open this up for broader discussion before iterating further.
Motivation The current credential vending flow is deeply entangled with the persistence layer. When a client requests scoped credentials (e.g. for S3, GCS, or Azure), the request goes through: StorageCredentialsVendor -> PolarisCredentialVendor -> MetaStoreManager -> persistence layer -> back out through PolarisStorageIntegrationProvider. This means credential vending re-loads entities from persistence even though the caller already has them, and MetaStoreManager implementations are burdened with credential vending logic that doesn't belong in persistence. Overall, the sheer amount of complexity and the amount of layers that credential vending flow goes through makes further changes particularly challenging as evidenced by some recent efforts around cache key generation, storage info resolution, additional storage backends and so on. What the PR does 1. Removes credential vending from MetaStoreManager. The PolarisCredentialVendor interface, StorageCredentialsVendor, and getSubscopedCredsForEntity() implementations are removed from MetaStoreManager. This cleans up both the transactional and NoSQL backends. 2. Moves orchestration into StorageAccessConfigProvider. This application-scoped bean now directly resolves the storage integration and delegates to it, cutting out the persistence round-trip. 3. Moves caching into storage integrations. Each PolarisStorageIntegration subclass (AWS, GCP, Azure) now owns its StorageCredentialCache interaction and builds cloud-specific cache keys, rather than using a one-size-fits-all key. I'd appreciate any feedback on the overall direction, concerns about API compatibility in polaris-core, or suggestions for how to best land these changes. Thanks, Tornike
