amaechler opened a new pull request, #20485: URL: https://github.com/apache/druid/pull/20485
Fixes #18792. ### Description Protobuf ingestion turned proto3 fields that are not marked `optional` into `null` whenever they used their default value (`0`, `""`, `false`, or the first enum value). Users had to wrap such fields in `nvl()` in transforms to get the actual value back. `ProtobufConverter.convertMessage` built each row from `Message.getAllFields()`. That method only returns fields that are "set", and for proto3 fields without explicit presence a default value is indistinguishable from unset when serialized, so those fields were dropped from the row. `convertMessage` now also walks the message descriptor and adds every singular field where `FieldDescriptor.hasPresence()` is false and which `getAllFields()` did not already return, using `msg.getField()` to read the default. This mirrors `JsonFormat.printer().alwaysPrintFieldsWithNoPresence()`, with one deliberate difference: empty repeated and map fields are still omitted, as before, so they keep ingesting as `null` rather than an empty array. #### Release note Protobuf ingestion now keeps the default value of proto3 fields that are not marked `optional`. Previously an `int32` field holding `0` (or a `string` holding `""`, a `bool` holding `false`, an enum holding its first value) was ingested as `null`. Such fields now ingest as their actual value. <hr> This PR has: - [x] been self-reviewed. - [x] a release note entry in the PR description. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
