HonahX commented on code in PR #12299:
URL: https://github.com/apache/iceberg/pull/12299#discussion_r1990498449
##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3InputFile.java:
##########
@@ -36,20 +37,53 @@ public static S3InputFile fromLocation(
MetricsContext metrics) {
return new S3InputFile(
client,
+ null,
new S3URI(location, s3FileIOProperties.bucketToAccessPointMapping()),
null,
s3FileIOProperties,
metrics);
}
+ public static S3InputFile fromLocation(
+ String location,
+ S3Client client,
+ S3AsyncClient asyncClient,
Review Comment:
It seems `S3Client` is always required to create an S3InputFile? Just
curious why we choose to not have a `fromLocation` like this
``` java
public static S3InputFile fromLocation(
String location,
long length,
S3AsyncClient asyncClient,
S3FileIOProperties s3FileIOProperties,
MetricsContext metrics)
```
Will there be any future scenarios where having both the synchronous
(S3Client) and asynchronous (S3AsyncClient) clients is necessary for
S3InputFile?
##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java:
##########
@@ -121,23 +136,49 @@ public S3FileIO(SerializableSupplier<S3Client> s3) {
* @param s3FileIOProperties S3 FileIO properties
*/
public S3FileIO(SerializableSupplier<S3Client> s3, S3FileIOProperties
s3FileIOProperties) {
+ this(s3, null, s3FileIOProperties);
+ }
+
+ /**
+ * Constructor with custom s3 supplier, s3Async supplier and S3FileIO
properties.
+ *
+ * <p>Calling {@link S3FileIO#initialize(Map)} will overwrite information
set in this constructor.
+ *
+ * @param s3 s3 supplier
+ * @param s3Async s3Async supplier
+ * @param s3FileIOProperties S3 FileIO properties
+ */
+ public S3FileIO(
+ SerializableSupplier<S3Client> s3,
+ SerializableSupplier<S3AsyncClient> s3Async,
+ S3FileIOProperties s3FileIOProperties) {
this.s3 = s3;
+ this.s3Async = s3Async;
this.s3FileIOProperties = s3FileIOProperties;
this.createStack = Thread.currentThread().getStackTrace();
}
@Override
public InputFile newInputFile(String path) {
+ if (shouldUseAsyncClient()) {
+ return S3InputFile.fromLocation(path, client(), asyncClient(),
s3FileIOProperties, metrics);
+ }
return S3InputFile.fromLocation(path, client(), s3FileIOProperties,
metrics);
}
@Override
public InputFile newInputFile(String path, long length) {
+ if (shouldUseAsyncClient()) {
+ return S3InputFile.fromLocation(path, client(), asyncClient(),
s3FileIOProperties, metrics);
Review Comment:
```suggestion
return S3InputFile.fromLocation(path, length, client(), asyncClient(),
s3FileIOProperties, metrics);
```
I think we miss the `length` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
