[
https://issues.apache.org/jira/browse/HADOOP-18186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584501#comment-17584501
]
ASF GitHub Bot commented on HADOOP-18186:
-----------------------------------------
virajjasani commented on PR #4796:
URL: https://github.com/apache/hadoop/pull/4796#issuecomment-1226578101
@steveloughran i also tested the entire test suite by enabling prefatch.
Tests in `ITestS3AContractVectoredRead` failed. One example:
```
[ERROR] testMinSeekAndMaxSizeDefaultValues[Buffer type :
direct](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead) Time
elapsed: 13.025 s <<< FAILURE!
org.junit.ComparisonFailure: [Mismatch in default s3a min seek for vectored
reads] expected:<4[8]96> but was:<4[0]96>
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at org.apache.hadoop.test.MoreAsserts.assertEqual(MoreAsserts.java:99)
at
org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead.testMinSeekAndMaxSizeDefaultValues(ITestS3AContractVectoredRead.java:111)
```
However, the failures don't seem relevant to the change on this PR. As per
the test logic, it seems`ITestS3AContractVectoredRead` failures are expected
when prefetch is enabled.
> s3a prefetching to use SemaphoredDelegatingExecutor for submitting work
> -----------------------------------------------------------------------
>
> Key: HADOOP-18186
> URL: https://issues.apache.org/jira/browse/HADOOP-18186
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
>
> Use SemaphoredDelegatingExecutor for each to stream to submit work, if
> possible, for better fairness in processes with many streams.
> this also takes a DurationTrackerFactory to count how long was spent in the
> queue, something we would want to know
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
