steveloughran commented on PR #7882:
URL: https://github.com/apache/hadoop/pull/7882#issuecomment-3412290514
@ahmarsuhail
I'm handling the retries now by requiring the md5 plugin to be explicitly
requested (i.e. third party stores); also making it easier to switch checksum
generation from ALWAYS to WHEN_REQUESTED. So for AWS S3: stricter checksums, no
md5. Other stores: configure it as needed.
Still wondering if we should make this more automated, but not in a way
which causes problems later.
---
I am now seeing failings against s3 express
```
org.opentest4j.AssertionFailedError: [Counter named audit_request_execution
with expected value 4]
Expecting:
<11L>
to be equal to:
<4L>
but was not.
Expected :4
Actual :11
<Click to see difference>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
org.apache.hadoop.fs.statistics.IOStatisticAssertions.verifyStatisticValue(IOStatisticAssertions.java:274)
at
org.apache.hadoop.fs.statistics.IOStatisticAssertions.verifyStatisticCounterValue(IOStatisticAssertions.java:175)
at
org.apache.hadoop.fs.s3a.ITestS3AAnalyticsAcceleratorStreamReading.testMultiRowGroupParquet(ITestS3AAnalyticsAcceleratorStreamReading.java:186)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at java.util.ArrayList.forEach(ArrayList.java:1259)
```
I'm changing this test to measure the # of audited requests before the file
opening begins and then assert on the difference between them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
