xiangfu0 commented on code in PR #18870:
URL: https://github.com/apache/pinot/pull/18870#discussion_r3599213874
##########
pinot-core/src/main/java/org/apache/pinot/core/util/SegmentProcessorAvroUtils.java:
##########
@@ -57,20 +61,50 @@ public static GenericData.Record
convertGenericRowToAvroRecord(GenericRow generi
*/
public static GenericData.Record convertGenericRowToAvroRecord(GenericRow
genericRow,
GenericData.Record reusableRecord, Set<String> fields) {
+ Schema avroSchema = reusableRecord.getSchema();
for (String field : fields) {
Object value = genericRow.getValue(field);
if (value instanceof Object[]) {
- reusableRecord.put(field, Arrays.asList((Object[]) value));
+ Schema.Field avroField = avroSchema.getField(field);
+ if (avroField != null && isUuidArrayLogicalType(avroField.schema())) {
+ // MV UUID columns are emitted with an Avro
array<string{logicalType:uuid}> schema; convert each
+ // 16-byte element to its canonical UUID string so
GenericDatumWriter accepts the record.
Review Comment:
Done in this PR — registered an Avro `Conversion` for the `uuid` logical
type (`UuidConversion` in `SegmentProcessorAvroUtils`) on a shared
`GenericData` model (`getAvroDataModel()`), wired into every segment-processing
writer: `FileBasedSegmentWriter`, `FlinkSegmentWriter`,
`PinotSegmentToAvroConverter`, and the Parquet converter (via
`AvroParquetWriter.withDataModel`). `convertGenericRowToAvroRecord` now passes
the raw 16-byte value straight through and the Conversion renders the canonical
string at write time, so the manual `UuidUtils.toString(...)` rendering is gone.
One implementation note: the conversion's converted type is `byte[]` (not
`ByteBuffer` or `java.util.UUID`), because `GenericDatumWriter` resolves
conversions by exact datum class — a `byte[]` always reports `byte[].class`, so
it resolves for both single values and MV/array elements, whereas a concrete
`HeapByteBuffer` would not match a `ByteBuffer`-typed conversion. Verified with
SV+MV round-trip tests through both the Avro `DataFileWriter` and the Parquet
writer.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
