alexandre-normand opened a new pull request, #736:
URL: https://github.com/apache/iceberg-go/pull/736
This test (not actually meant for merging) exercises a path that we found
triggered scanning/reading to panic. The test case is meant to make for the
simplest reproduction: it includes a schema with a single column called `tags`
which is a nullable map of string to a list of strings (i.e. `map<string,
list<string>>`).
The issue is that doing a scan on a table with such a schema (where it's
possible for some rows to have an empty `tags` field value) panics.
The runtime panic looked like this:
```
panic: arrow/array: map array child array should have two fields
goroutine 4357 [running]:
github.com/apache/arrow-go/v18/arrow/array.(*Map).validateData(0xfd25dd002f01?,
0x40035c8d20)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/map.go:68
+0x110
github.com/apache/arrow-go/v18/arrow/array.(*Map).setData(0x400089a5d0,
0x40035c8d20)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/map.go:77
+0x24
github.com/apache/arrow-go/v18/arrow/array.NewMapData({0x2811350,
0x40035c8d20})
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/map.go:41
+0xb8
github.com/apache/arrow-go/v18/arrow/array.init.0.func33({0x2811350?,
0x40035c8d20?})
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/array.go:170
+0x24
github.com/apache/arrow-go/v18/arrow/array.MakeFromData({0x2811350,
0x40035c8d20})
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/array.go:120
+0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*listReader).BuildArray(0x40065fb860,
0x20000)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/column_readers.go:469
+0x658
github.com/apache/arrow-go/v18/parquet/pqarrow.(*ColumnReader).NextBatch(0x40007c3c30,
0x20000)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:169
+0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func1(0x44,
0x4004f1ff38?)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:786
+0x54
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func2()
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:839
+0xe0
created by
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next in
goroutine 333
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:830
+0x2ac
```
Which is the same as this test:
```
=== RUN TestTableWriting/TestScanPanicOnMapStringKeyStringListValue
panic: arrow/array: map array child array should have two fields
goroutine 54 [running]:
github.com/apache/arrow-go/v18/arrow/array.(*Map).validateData(0x1083d0101?,
0x140009875e0)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/map.go:68
+0x110
github.com/apache/arrow-go/v18/arrow/array.(*Map).setData(0x1400085a3f0,
0x140009875e0)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/map.go:77
+0x24
github.com/apache/arrow-go/v18/arrow/array.NewMapData({0x1054c1e30,
0x140009875e0})
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/map.go:41
+0xb8
github.com/apache/arrow-go/v18/arrow/array.init.0.func33({0x1054c1e30?,
0x140009875e0?})
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/array.go:170
+0x24
github.com/apache/arrow-go/v18/arrow/array.MakeFromData({0x1054c1e30,
0x140009875e0})
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/arrow/array/array.go:120
+0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*listReader).BuildArray(0x14000e46b40,
0x20000)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/column_readers.go:469
+0x658
github.com/apache/arrow-go/v18/parquet/pqarrow.(*ColumnReader).NextBatch(0x14000e49290,
0x20000)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:169
+0x5c
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func1(0x0,
0x14000113738?)
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:786
+0x54
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next.func2()
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:839
+0xe0
created by
github.com/apache/arrow-go/v18/parquet/pqarrow.(*recordReader).next in
goroutine 16
/Users/alex.normand/go/pkg/mod/github.com/apache/arrow-go/[email protected]/parquet/pqarrow/file_reader.go:830
+0x2ac
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
