pan3793 opened a new pull request, #7732:
URL: https://github.com/apache/iceberg/pull/7732
This PR aims to make `IcebergSource extends SessionConfigSupport` to improve
the Spark DataSource v2 API coverage.
```
/**
* A mix-in interface for {@link TableProvider}. Data sources can implement
this interface to
* propagate session configs with the specified key-prefix to all data
source operations in this
* session.
*
* @since 3.0.0
*/
@Evolving
public interface SessionConfigSupport extends TableProvider {
/**
* Key prefix of the session configs to propagate, which is usually the
data source name. Spark
* will extract all session configs that starts with
`spark.datasource.$keyPrefix`, turn
* `spark.datasource.$keyPrefix.xxx -> yyy` into `xxx -> yyy`, and
propagate them to all
* data source operations in this session.
*/
String keyPrefix();
}
```
It allows to set read/write options by setting Spark session configuration
when using the DataFrame API to read/write tables. For examples,
```
// set write option through session configuration
spark.sql("set spark.datasource.iceberg.<write-opt-key>=<value>")
spark.write
.format("iceberg")
// .option("<write-opt-key>", "<value>") // equivalent w/ the above SET
statement
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
