[
https://issues.apache.org/jira/browse/HADOOP-19291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17887611#comment-17887611
]
Mukund Thakur commented on HADOOP-19291:
----------------------------------------
ChecksumFileSystem creates the chunked ranges based on the checksum chunk size
and then calls
readVectored on Raw Local which may lead to overlapping ranges even if initial
ranges are disjoint in some cases.
For more details see comments in [https://github.com/apache/hadoop/pull/7079]
> `CombinedFileRange.merge` should not convert disjoint ranges into overlapped
> ones
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-19291
> URL: https://issues.apache.org/jira/browse/HADOOP-19291
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.3.9, 3.5.0, 3.4.1
> Reporter: Dongjoon Hyun
> Priority: Major
> Attachments: Screenshot 2024-09-28 at 22.02.01.png
>
>
> Currently, Hadoop has a bug to convert disjoint ranges into overlapped ones
> and eventually fails by itself.
> {code:java}
> + public void testMergeSortedRanges() {
> + List<FileRange> input = asList(
> + createFileRange(13816220, 24, null),
> + createFileRange(13816244, 7423960, null)
> + );
> + assertIsNotOrderedDisjoint(input, 100, 800);
> + final List<CombinedFileRange> outputList = mergeSortedRanges(
> + sortRangeList(input), 100, 1001, 2500);
> +
> + assertRangeListSize(outputList, 1);
> + assertFileRange(outputList.get(0), 13816200, 7424100);
> + }
> {code}
> !Screenshot 2024-09-28 at 22.02.01.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
