kevinjqliu commented on code in PR #17284:
URL: https://github.com/apache/iceberg/pull/17284#discussion_r3865404155
##########
parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java:
##########
@@ -1352,7 +1356,12 @@ public ReaderFunction withSchema(Schema expectedSchema) {
}
private ReadBuilder(InputFile file) {
- this.file = file;
+ long fileLength = file.getLength();
+ this.file = canEagerFetch(fileLength) ? EagerInputFile.of(file,
fileLength) : file;
Review Comment:
Since all the other FileIO implementations already trust the supplied
length, I think its reasonable for HadoopFileIO to do this.
I think its worth calling that out, perhaps in the PR description. And
perhaps with a unit test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
