Hi everyone, I’ve been exploring the implementation of a feature in Apache Gravitino to securely store credentials in HashiCorp Vault. During my investigation, a few design-related questions arose that I’d like to clarify.
1. Catalog Creation & Test Connection I noticed that when creating a catalog, there doesn’t appear to be a test connection performed using the provided credentials. For example, Hadoop catalogs simply return null without validation. Is this an intentional design choice, or am I overlooking something? Ideally, should the credentials be extracted and validated at the CatalogManager layer before persisting them (along with other properties) to the database? Similarly, when creating schemas or entities (tables, filesets, etc.), should these configurations be fetched separately and verified? 2. Externally Managed Entities & Secure Storage For externally managed entities, I’m considering extracting sensitive credentials and storing them securely in Vault (using its key-value store), while keeping the remaining properties in the database. Does this approach align with Gravitino’s design principles? Are there existing patterns or constraints I should be aware of? I’d appreciate any insights or guidance on these points, especially regarding the intended behavior for credential validation and the preferred integration strategy with HashiCorp Vault. Thanks in advance for your help!
