[
https://issues.apache.org/jira/browse/HADOOP-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17601502#comment-17601502
]
ASF GitHub Bot commented on HADOOP-18439:
-----------------------------------------
mukund-thakur commented on code in PR #4862:
URL: https://github.com/apache/hadoop/pull/4862#discussion_r965289294
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:
##########
@@ -396,11 +407,33 @@ static ByteBuffer checkBytes(ByteBuffer sumsBytes,
return data;
}
+ /**
+ * Validates range parameters.
+ * In case of CheckSum FS, we already have calculated
+ * fileLength so failing fast here.
+ * @param ranges requested ranges.
+ * @param fileLen length of file.
+ * @throws EOFException end of file exception.
+ */
+ private void validateRangeRequest(List<? extends FileRange> ranges, long
fileLen) throws EOFException {
+ for (FileRange range : ranges) {
+ VectoredReadUtils.validateRangeRequest(range);
+ if (range.getOffset() + range.getLength() > fileLen) {
+ LOG.warn("Requested range [{}, {}) is beyond EOF for path {}",
+ range.getOffset(), range.getLength(), file);
+ throw new EOFException("Requested range [" + range.getOffset() + ", "
+ + range.getLength() + ") is beyond EOF for path " + file);
+ }
+ }
+ }
+
@Override
public void readVectored(List<? extends FileRange> ranges,
IntFunction<ByteBuffer> allocate) throws
IOException {
+ long length = fs.getFileStatus(file).getLen();
Review Comment:
Are you talking about FSDataBoundedInputStream#getFileLength() ? It is
private method in that inner class and can't/shouldn't be used by
ChecksumFSInputChecker? Should we create the same method here with just one
line?
> Fix VectoredIO for LocalFileSystem when checksum is enabled.
> ------------------------------------------------------------
>
> Key: HADOOP-18439
> URL: https://issues.apache.org/jira/browse/HADOOP-18439
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common
> Affects Versions: 3.3.9
> Reporter: Mukund Thakur
> Assignee: Mukund Thakur
> Priority: Major
> Labels: pull-request-available
>
> While merging the ranges in CheckSumFs, they are rounded up based on the
> value of checksum bytes size
> which leads to some ranges crossing the EOF thus they need to be fixed else
> it will cause EOFException during actual reads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
