mohamedtarek132 commented on issue #3346: URL: https://github.com/apache/polaris/issues/3346#issuecomment-3981155917
For anyone blocked by this, you can work around this issue immediately by passing a JVM argument to disable the AWS SDK's default checksum calculations. Add the following to your JAVA_OPTS: `-Daws.requestChecksumCalculation=WHEN_REQUIRED ` Why this works: Polaris 1.2.0 embeds AWS SDK for Java 2.35.0. There is a known bug in the AWS SDK (from 2.30.0 up to 2.36.x) where the S3AsyncClient ignores chunkedEncodingEnabled(false) because the new default checksum logic overrides it and forces aws-chunked encoding anyway (see aws/aws-sdk-java-v2#6387 and aws/aws-sdk-java-v2#5802). By setting the checksum calculation to WHEN_REQUIRED, it stops the SDK from forcing the chunked encoding, which resolves the rejection from Alibaba Cloud OSS and other third-party stores. (Note: I also saw `-Daws.responseChecksumValidation=WHEN_REQUIRED` recommended for complete V1 compatibility, but I found that just requestChecksumCalculation was enough to get this working.) AWS fixed the underlying bug in SDK 2.37.0. Looking at the gradle/libs.versions.toml file, Polaris 1.3 bumps the awssdk-bom dependency to 2.39.2 (and main is currently on 2.42.0). The solution for Polaris 1.3+: This means upgrading to Polaris 1.3+ should resolve this natively. Once on 1.3, you should be able to simply set "s3.chunked-encoding-enabled": "false" in your catalog's properties and have it actually respected by the async client, completely removing the need for the JAVA_OPTS workaround. Note that I haven't personally tried testing this on version 1.3 yet, but based on the AWS SDK version bump, the standard configuration should work out of the box. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
