amogh-jahagirdar commented on code in PR #8925:
URL: https://github.com/apache/iceberg/pull/8925#discussion_r1375117854
##########
core/src/test/java/org/apache/iceberg/TestSplitPlanning.java:
##########
@@ -216,6 +217,34 @@ public void testSplitPlanningWithOffsets() {
"We should get one task per row group", 32,
Iterables.size(scan.planTasks()));
}
+ @Test
+ public void testSplitPlanningWithCorruptedOffsets() throws IOException {
Review Comment:
Discussed offline, this test exercises the path but has poor test hardness
since the offsets end up being null anyways.
For proper test hardness we can go through TableScanUtil.planTaskGroups
which will call FileScanTask.split and exercise the split offset array path.
The new test will fail without the fix since the number of tasks will be
different (without the fix, there will be 2 tasks, 1 per split offset but the
last task is a bad task. With the fix, the split size of 1 byte will be used
for the 10 byte file, and there will be 10 tasks)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
