Sunyue opened a new issue, #464:
URL: https://github.com/apache/arrow-go/issues/464
### Describe the bug, including details regarding any error messages,
version, and platform.
Hi @zeroshade I find another bug when working with Decimal128, could you
help to have a look? I am performing filter on a arrow record with has column
with type Decimal128, it works fine when the number of rows <=32, but panics
when the number of row >32. As usual, please find my test case below:
```go
import (
"testing"
"github.com/apache/arrow-go/v18/arrow"
"github.com/apache/arrow-go/v18/arrow/array"
"github.com/apache/arrow-go/v18/arrow/compute"
"github.com/apache/arrow-go/v18/arrow/compute/exprs"
"github.com/apache/arrow-go/v18/arrow/decimal"
"github.com/apache/arrow-go/v18/arrow/memory"
"github.com/stretchr/testify/require"
"github.com/substrait-io/substrait-go/v4/expr"
"github.com/substrait-io/substrait-go/v4/types"
)
func Test_FilterOnDecmial128(t *testing.T) {
t.Parallel()
tt := []struct {
name string
n int
}{
{
name: "arrow.DECIMAL128 - number of records < 33 ok",
n: 32,
},
{
name: "arrow.DECIMAL128 - number of records >= 33
panic",
n: 33,
},
}
for _, tc := range tt {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
ctx := t.Context()
rq := require.New(t)
typ := &arrow.Decimal128Type{Precision: 3, Scale: 1}
field := arrow.Field{
Name: "col",
Type: typ,
Nullable: true,
}
schema := arrow.NewSchema([]arrow.Field{field}, nil)
db :=
array.NewDecimal128Builder(memory.DefaultAllocator, typ)
defer db.Release()
for i := range tc.n {
d, err :=
decimal.Decimal128FromFloat(float64(i), 3, 1)
rq.NoError(err, "Failed to create Decimal128
value")
db.Append(d)
}
rec := array.NewRecord(schema,
[]arrow.Array{db.NewArray()}, int64(tc.n))
extSet := exprs.GetExtensionIDSet(ctx)
builder := exprs.NewExprBuilder(extSet)
err := builder.SetInputSchema(schema)
rq.NoError(err, "Failed to set input schema")
v, p, s, err := expr.DecimalStringToBytes("10.0")
rq.NoError(err, "Failed to convert decimal string to
bytes")
lit, err := expr.NewLiteral(&types.Decimal{
Value: v[:16],
Precision: p,
Scale: s,
}, true)
rq.NoError(err, "Failed to create Decimal128 literal")
b, err := builder.CallScalar("less", nil,
builder.FieldRef("col"),
builder.Literal(lit),
)
rq.NoError(err, "Failed to call scalar")
e, err := b.BuildExpr()
rq.NoError(err, "Failed to build expression")
ctx = exprs.WithExtensionIDSet(ctx, extSet)
dr := compute.NewDatum(rec)
defer dr.Release()
_, err = exprs.ExecuteScalarExpression(ctx, schema, e,
dr)
rq.NoError(err, "Failed to execute scalar expression")
})
}
}
```
The panic comes from places like here:
https://github.com/apache/arrow-go/blame/4c2983c9e14ce4fb7ee9fc135815426b8f8c880f/arrow/compute/internal/kernels/scalar_comparisons.go#L316
(where `rt` contains all the decimal values which makes `i` can be much larger
than 32)
### Component(s)
Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
