andygrove opened a new issue, #6174:
URL: https://github.com/apache/datafusion-comet/issues/6174
`CometNativeUdfSuite` and the native adapter tests cover every supported
Spark type with nulls,
one- and two-argument calls, literal arguments in each position, and each
operator a UDF can be
placed in. What they do not cover yet:
- **Empty batches.** A zero-row batch reaches `invoke_with_args` with
`number_rows = 0`. Nothing
checks that the kernel sees zero-length arrays and that a zero-length
result passes the row count
check.
- **Sliced arrays.** An array with a non-zero offset (as produced by a
`LIMIT` or a filter over a
batch) crosses the C Data Interface with its offset, and the kernel has to
honor it. Nothing
exercises a non-zero offset in either direction.
- **Dictionary-encoded input.** A dictionary-encoded string column from a
Parquet scan reaches
the UDF as whatever Comet's scan hands the projection. The type the kernel
sees then depends on
whether the dictionary was unpacked first, and `return_field`
implementations that match on
`DataType::Utf8` would reject it.
Follow-up to #4459.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
