fpacanowski opened a new issue, #44742:
URL: https://github.com/apache/arrow/issues/44742
### Describe the bug, including details regarding any error messages,
version, and platform.
The issue occurs when there's a list of structs defined in schema. Here's a
minimal example:
```ruby
require 'arrow'
require 'parquet'
schema = Arrow::Schema.new(
[
Arrow::Field.new("structs", Arrow::ListDataType.new(
Arrow::StructDataType.new([
Arrow::Field.new("foo", :int64),
Arrow::Field.new("bar", :int64)
])
))
]
)
# This works.
table = Arrow::RecordBatchBuilder.build(schema, [
{ structs: [{foo: 1, bar: 2}, {foo: 3, bar: 4}] },
{ structs: [{foo: 5, bar: 6}] }
]).to_table
table.save('file.parquet')
# This errors out.
table = Arrow::RecordBatchBuilder.build(schema, [
{ structs: [] },
{ structs: [] },
]).to_table
table.save('file.parquet')
```
I expected the second invocation to produce a table with two rows with empty
lists in `structs` column. Instead I got the following error:
```
/home/filip/.rvm/gems/ruby-3.3.0/gems/gobject-introspection-4.2.4/lib/gobject-introspection/loader.rb:715:in
`invoke': [parquet][arrow][file-writer][write-table]: Invalid: Column 0: In
chunk 0: Invalid: List child array invalid: Invalid: Struct child array #0 has
length smaller than expected for struct array (0 < 2) (Arrow::Error::Invalid)
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/gobject-introspection-4.2.4/lib/gobject-introspection/loader.rb:715:in
`invoke'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/gobject-introspection-4.2.4/lib/gobject-introspection/loader.rb:583:in
`write_table'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-parquet-17.0.0/lib/parquet/arrow-table-savable.rb:41:in
`block (2 levels) in save_as_parquet'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-arrow-17.0.0/lib/arrow/block-closable.rb:25:in
`open'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-parquet-17.0.0/lib/parquet/arrow-table-savable.rb:38:in
`block in save_as_parquet'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-arrow-17.0.0/lib/arrow/block-closable.rb:25:in
`open'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-arrow-17.0.0/lib/arrow/table-saver.rb:115:in
`open_raw_output_stream'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-parquet-17.0.0/lib/parquet/arrow-table-savable.rb:37:in
`save_as_parquet'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-arrow-17.0.0/lib/arrow/table-saver.rb:77:in
`save_to_file'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-arrow-17.0.0/lib/arrow/table-saver.rb:53:in
`save'
from
/home/filip/.rvm/gems/ruby-3.3.0/gems/red-arrow-17.0.0/lib/arrow/table.rb:447:in
`save'
from repro.rb:27:in `<main>'
```
This is running on version 17.0.0 of `red-arrow` and `red-parquet`.
### Component(s)
Ruby
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
