iting0321 commented on code in PR #3820:
URL: https://github.com/apache/polaris/pull/3820#discussion_r2895903449
##########
plugins/spark/v3.5/spark/src/main/java/org/apache/polaris/spark/SparkCatalog.java:
##########
@@ -183,33 +234,78 @@ public Table createTable(
@Override
public Table alterTable(Identifier ident, TableChange... changes) throws
NoSuchTableException {
+ // First, try to alter as an Iceberg table
try {
return this.icebergsSparkCatalog.alterTable(ident, changes);
} catch (NoSuchTableException e) {
- Table table = this.polarisSparkCatalog.loadTable(ident);
- String provider =
table.properties().get(PolarisCatalogUtils.TABLE_PROVIDER_KEY);
- if (PolarisCatalogUtils.useDelta(provider)) {
- // For delta table, most of the alter operations is a delta log
manipulation,
- // we load the delta catalog to help handling the alter table
operation.
- // NOTE: This currently doesn't work for changing file location and
file format
- // using ALTER TABLE ...SET LOCATION, and ALTER TABLE ... SET
FILEFORMAT.
- TableCatalog deltaCatalog =
deltaHelper.loadDeltaCatalog(this.polarisSparkCatalog);
- return deltaCatalog.alterTable(ident, changes);
- } else if (PolarisCatalogUtils.useHudi(provider)) {
- TableCatalog hudiCatalog =
hudiHelper.loadHudiCatalog(this.polarisSparkCatalog);
- return hudiCatalog.alterTable(ident, changes);
- } else if (PolarisCatalogUtils.usePaimon(provider)) {
- TableCatalog paimonCatalog =
paimonHelper.loadPaimonCatalog(this.polarisSparkCatalog);
+ // Not an Iceberg table, fall through to handle as generic table
+ }
+
+ // For generic tables, check the format/provider to decide delegation
+ // Use getTableFormat to avoid triggering Spark DataSource resolution for
routing decisions
+ String provider = this.polarisSparkCatalog.getTableFormat(ident);
Review Comment:
In this scenario, we just need to get the format (e.g. "paimon").
For the previous one will triggers full Spark DataSource resolution which
need lots of times.
But `getTableFormat` gives the exact same routing information at a fraction
of the cost.
We can extract format in
```java
public record GenericTable(
String name,
String format,
String baseLocation,
String doc,
Map<String, String> properties) { ... }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
