manuzhang opened a new issue, #17718:
URL: https://github.com/apache/iceberg/issues/17718
### Problem
`SparkCatalog.listTables` delegates directly to the underlying Iceberg
catalog:
```java
return icebergCatalog.listTables(Namespace.of(namespace)).stream()
.map(ident -> Identifier.of(ident.namespace().levels(), ident.name()))
.toArray(Identifier[]::new);
```
Most Iceberg catalogs return only tables. However, `HiveCatalog` with
`list-all-tables=true` returns every matching metastore entry, including views.
As a result, `SparkCatalog.listTables` may expose views even though Spark's
`TableCatalog.listTables` contract requires table identifiers only.
Spark 4.2 makes the distinction explicit through `RelationCatalog`:
`listTableSummaries` returns tables only, while `listRelationSummaries`
combines table and view summaries. `SparkCatalog.listTableSummaries` already
subtracts identifiers returned by `listViews`, but `listTables` remains
unfiltered.
This behavior predates Spark 4.2 and is not a regression in the Spark 4.2
support PR.
### Proposed fix
- Filter view identifiers from `SparkCatalog.listTables`, including the
`HiveCatalog` `list-all-tables=true` configuration.
- Apply the correction consistently to supported Spark versions where the
same implementation is present.
- Add coverage that creates both a table and a view in the same namespace
and verifies that `listTables` returns only the table.
### Context
Recorded during review of Spark 4.2 support in #14984:
https://github.com/apache/iceberg/pull/14984#discussion_r3788259677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
