zeroshade commented on code in PR #1238:
URL: https://github.com/apache/iceberg-go/pull/1238#discussion_r3444362469
##########
table/arrow_utils.go:
##########
@@ -1190,8 +1178,33 @@ func (a *arrowProjectionVisitor) Primitive(_
iceberg.PrimitiveType, arr arrow.Ar
return arr
}
+// Variant reassembles a shredded variant back into the unshredded layout so
the
+// projected schema matches; without it the scanner fails to assemble the
table.
Review Comment:
I'm surprised that this is the case, this feels like a bug in
arrow-go/parquet rather than something we should have to do here. Can you take
a look at arrow-go/parquet and validate why it isn't already reassembling the
shredded column into a variant array properly and whether or not this is
something we should fix on that side instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
