babumahesh commented on PR #10732:
URL: https://github.com/apache/gravitino/pull/10732#issuecomment-4224233187
> > > Thanks for your design document. I am pleased to communicate about
this topic. I took a short look. There are a few questions!
> > >
> > > 1. How did the existed catalog migrate the sensitive properties?
> > > 2. Should we configure a KMS for a Gravitino server or serveral KMS
for a Gravitino server?
> >
> >
> > For Existing catalog migrate the sensitive properties
> >
> > * In-House Encryption: Migration script makes sense
> >
> > * Script transforms plain-text → enc:AES-GCM:v1:... in the Gravitino DB
> > * Safe to automate because it's a deterministic transformation
> > * Its also upto users whether they wanted to migrate or not
> > * External Secret Management
> >
> > * Manual migration preference is reasonable
> >
> > * Store secrets in their secret manager (AWS Secrets Manager/Vault)
> > * Update catalog properties manually : plain-text →
${secret:path/to/secret}
> > * Hard to automate because secret paths/organization vary by company
>
> We need a migration document to help users to migrate the sensitive
properties when required.
>
> > Should we configure a KMS for a Gravitino server or serveral KMS for a
Gravitino server:
> >
> > * Both patterns supported via hybrid syntax:
> >
> > * Global default: One secret provider configured in gravitino.conf,
catalogs use simple syntax `${secret:mysql/password}`
> > * Multi-tenant: Multiple secret providers enabled, catalogs explicitly
specify provider
> >
> > * ${secret:vault:team-a/mysql/password} → uses HashiCorp Vault
> > * ${secret:aws-sm:team-b/postgres#password} → uses AWS Secrets
Manager
> >
> > **`Key principle: Either with in-house encryption or with KMS; Both
Plain-text and secured catalogs can coexist`**
>
> Is it necessary to have multiple KMS for one server? Which situations
require this feature?
Multiple KMS support is not strictly necessary, but provides flexibility
for specific enterprise scenarios:
- Multi-cloud/Hybrid deployments
- AWS catalogs → AWS Secrets Manager
- On-premise catalogs → HashiCorp Vault
- Example: ${secret:aws-sm:rds/password} vs
${secret:vault:mysql/password}
- Multi-tenant environments
- Team A uses their corporate Vault instance
- Team B uses a different Vault/secret manager
Though we could always start with opne KMS for simplicity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
