GitHub user krishnakeshan created a discussion: Should
`IcebergCatalogProvider::try_new` not short-circuit on the first failed
`IcebergSchemaProvider::try_new`?
I've been working with a GCS bucket that has some invalid schemas that cause
`IcebergSchemaProvider::try_new` to fail. When building these schemas with
`IcebergCatalogProvider::try_new`, I'm unable to get the catalog provider for
valid schemas due to some corrupted schemas. I modified the method like below
and it works fine for me:
```
impl IcebergCatalogProvider {
pub async fn try_new(client: Arc<dyn Catalog>) -> Result<Self> {
let schema_names: Vec<_> = client
.list_namespaces(None)
.await?
.iter()
.flat_map(|ns| ns.as_ref().clone())
.collect();
// changed try_join_all to join_all and manually collected them into valid
IcebergSchemaProviders
let schema_providers = join_all(
schema_names
.iter()
.map(|name| async {
(
name.clone(),
IcebergSchemaProvider::try_new(
client.clone(),
NamespaceIdent::new(name.clone()),
)
.await,
)
})
.collect::<Vec<_>>(),
)
.await;
let mut schemas = HashMap::new();
for (name, provider) in schema_providers {
if let Ok(provider) = provider {
schemas.insert(name, Arc::new(provider) as Arc<dyn SchemaProvider>);
}
}
Ok(IcebergCatalogProvider { schemas })
}
}
```
GitHub link: https://github.com/apache/iceberg-rust/discussions/1651
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
