plusplusjiajia commented on code in PR #15242:
URL: https://github.com/apache/iceberg/pull/15242#discussion_r3036231052
##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java:
##########
@@ -295,6 +295,18 @@ public class S3FileIOProperties implements Serializable {
public static final boolean REMOTE_SIGNING_ENABLED_DEFAULT = false;
+ /**
+ * Enables or disables chunked encoding for S3 requests.
+ *
+ * <p>This feature is disabled by default.
+ *
+ * <p>For more details see:
+ *
https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.html#chunkedEncodingEnabled()
+ */
+ public static final String CHUNKED_ENCODING_ENABLED =
"s3.chunked-encoding-enabled";
+
+ public static final boolean CHUNKED_ENCODING_ENABLED_DEFAULT = false;
Review Comment:
> I agree for those scenarios we can prompt user to disable it themselves.
The way I am approaching is lets say iceberg v 1.10 shipped with chunked
encoding true, 1.11 will be shipping ti for false ? this can be regression for
no signing use cases ? Hence the recommendation to keep the sdk default but at
the same time give ways to override these, wdyt
Thanks for the feedback @singhpk234, that's a fair point. Changing the
default to false would indeed be a regression for users who don't use the REST
catalog signer.
@steveloughran I agree the CatalogSigner incompatibility is a real concern,
but I think we can address it without changing the SDK default for everyone.
Users who rely on CatalogSigner can explicitly set
s3.chunked-encoding-enabled=false, and we can document this clearly. This
way we avoid a silent performance regression for the majority of users while
still providing a clear path for signing use cases.
I'll update the PR to keep the default as true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
