nastra commented on code in PR #10926:
URL: https://github.com/apache/iceberg/pull/10926#discussion_r1718243019
##########
core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java:
##########
@@ -63,7 +63,11 @@ public class HadoopFileIO implements HadoopConfigurable,
DelegateFileIO {
* <p>{@link Configuration Hadoop configuration} must be set through {@link
* HadoopFileIO#setConf(Configuration)}
*/
- public HadoopFileIO() {}
+ public HadoopFileIO() {
+ // Create a default hadoopConf as it is required for the object to be
valid.
+ // E.g. newInputFile would throw NPE with hadoopConf.get() otherwise.
+ this.hadoopConf = new SerializableConfiguration(new Configuration())::get;
Review Comment:
ah you're right that this will add a Hadoop dependency and we don't want
that.
Rather than setting a conf in the constructor, maybe we should have some
handling in `conf()`, such as
```
public Configuration conf() {
- return hadoopConf.get();
+ return
Optional.ofNullable(hadoopConf).map(Supplier::get).orElseGet(Configuration::new);
}
```
We would need to adjust all the places that call `hadoopConf.get()` to call
`conf()` after this change. That way we wouldn't need to unnecessarily
instantiate a conf in the constructor when we don't have to, wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
