orianED opened a new issue, #34101:
URL: https://github.com/apache/arrow/issues/34101
### Describe the bug, including details regarding any error messages,
version, and platform.
I'm using the arrow V11 package for reading parquet files.
Calling `pqarrow.NewFileReader` creates the schema manifest using
`pqarrow.NewSchemaManifest`.
When dealing with objects array fields, the record readers create an array
of primitives of the first field in the object. for example:
[[“a”:1, “b”:2},[“a”:3, “b”:4}] -> [1,3]
diving into the code I found this in `pqarrow/schema.go ->
listToSchemaField`:
```Go
listGroup := listNode.(*schema.GroupNode)
if listGroup.NumFields() == 1 && (listGroup.Name() == "array" ||
strings.HasSuffix(listGroup.Name(), "_tuple")) {
// list of primitive type
if err := groupToStructField(listGroup, currentLevels, ctx, out,
&out.Children[0]); err != nil {
return err
}
} else {
if err := nodeToSchemaField(listGroup.Field(0), currentLevels, ctx,
out, &out.Children[0]); err != nil {
return err
}
}
```
At this stage, the listGroup is the inner dictionary. Assuming the above
example listGroup.NumFields() returns 2 and the schema node is converted to a
primitive field taking the first field in the group (else clause).
It also seems like the `groupToStructField` function is more suitable for
this case.
changing the condition solves the problem and the field is being read as
expected.
```Go
listGroup := listNode.(*schema.GroupNode)
if listGroup.NumFields() != 1 && (listGroup.Name() == "array" ||
strings.HasSuffix(listGroup.Name(), "_tuple")) {
if err := groupToStructField(listGroup, currentLevels, ctx, out,
&out.Children[0]); err != nil {
return err
}
} else {
// list of primitive type
if err := nodeToSchemaField(listGroup.Field(0), currentLevels, ctx,
out, &out.Children[0]); err != nil {
return err
}
}
```
Can you take a look and fix it? the current state is misleading as the user
might think this is the real value
### Component(s)
Go, Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
