kevinjqliu commented on code in PR #13445:
URL: https://github.com/apache/iceberg/pull/13445#discussion_r2299412893
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetWriters.java:
##########
@@ -613,13 +617,32 @@ private static class InternalRowWriter extends
ParquetValueWriters.StructWriter<
private final DataType[] types;
private InternalRowWriter(List<ParquetValueWriter<?>> writers,
List<DataType> types) {
- super(writers);
+ super(writerToFieldIndex(types, writers.size()), writers);
this.types = types.toArray(new DataType[0]);
}
@Override
protected Object get(InternalRow struct, int index) {
return struct.get(index, types[index]);
}
+
+ /** Returns a mapping from writer index to field index, skipping Unknown
columns. */
+ private static int[] writerToFieldIndex(List<DataType> types, int
numWriters) {
+ if (null == types) {
+ return IntStream.rangeClosed(0, numWriters).toArray();
Review Comment:
created tracking issue for range vs rangeClosed
https://github.com/apache/iceberg/issues/13921
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
