advancedxy commented on PR #7733: URL: https://github.com/apache/iceberg/pull/7733#issuecomment-1567910949
> In my experience, allowing the user to control some behaviors by using `SET xxx=yyy` is fantastic Yes, it gives users great flexibility. > are there principles for which kind of configurations should be exposed to session conf? If #7732 is merged, then all the options in SparkReadOptions and SparkWriteOptions would be exposed to session conf? > since we are building a DataSource upon Spark DSv2 API, the API does have such limitations to allow overwrite options using SQL syntax, I would rather respect API design or promote the API change upstream than do such a hackly thing. I don't think there are any limitations to allow overwrite options using SQL syntax, it's just that Spark doesn't pass any options when load table through `CatalogPlugin`. It would be great that such change is promoted at Spark side. However it may take some time to accept that kind of change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
