adutra commented on code in PR #9884:
URL: https://github.com/apache/iceberg/pull/9884#discussion_r1514756075
##########
core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java:
##########
@@ -53,20 +53,22 @@ public class ResolvingFileIO implements HadoopConfigurable,
DelegateFileIO {
private static final String S3_FILE_IO_IMPL =
"org.apache.iceberg.aws.s3.S3FileIO";
private static final String GCS_FILE_IO_IMPL =
"org.apache.iceberg.gcp.gcs.GCSFileIO";
private static final String ADLS_FILE_IO_IMPL =
"org.apache.iceberg.azure.adlsv2.ADLSFileIO";
- private static final Map<String, String> SCHEME_TO_FILE_IO =
+ private static final Map<String, String> DEFAULT_SCHEME_TO_FILE_IO =
ImmutableMap.of(
"s3", S3_FILE_IO_IMPL,
"s3a", S3_FILE_IO_IMPL,
"s3n", S3_FILE_IO_IMPL,
"gs", GCS_FILE_IO_IMPL,
"abfs", ADLS_FILE_IO_IMPL,
"abfss", ADLS_FILE_IO_IMPL);
+ private static final String SCHEME_PROPERTY_PREFIX = "resolving-io.schemes.";
private final Map<String, DelegateFileIO> ioInstances =
Maps.newConcurrentMap();
private final AtomicBoolean isClosed = new AtomicBoolean(false);
private final transient StackTraceElement[] createStack;
private SerializableMap<String, String> properties;
private SerializableSupplier<Configuration> hadoopConf;
+ private final Map<String, String> schemeToFileIo = Maps.newHashMap();
Review Comment:
Maybe switch to `Maps.newConcurrentMap()`?
##########
core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java:
##########
@@ -122,6 +124,15 @@ public void initialize(Map<String, String> newProperties) {
close(); // close and discard any existing FileIO instances
Review Comment:
This method can be called multiple times. You should probably clear
`schemeToFileIo`, either here or better yet, inside the `close()` method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
