huan233usc opened a new pull request, #800:
URL: https://github.com/apache/iceberg-cpp/pull/800
## What
Part 3 of 4 of Iceberg v3 column default-value support (POC #731), built on
the
schema layer (#746) and the Parquet read path (#792).
When a column is present in the read (table) schema but absent from an Avro
data
file — because the column was added after those rows were written — fill it
with
the column's v3 `initial-default` instead of `null`.
## Changes
- **`arrow/literal_util`**: add `AppendDefaultToBuilder`, which appends a
`Literal` once to an Arrow `ArrayBuilder` (for the row-by-row Avro decode
paths). Like `MakeDefaultArray`, it targets the storage type for an
extension
builder (`arrow.uuid`), since `Scalar::CastTo` has no kernel for extension
types.
- **Avro projection** (`avro_schema_util.cc`): when a field is missing from
the
file and carries an `initial-default`, project it as
`FieldProjection::Kind::kDefault`, mirroring the generic / Parquet paths.
- **Avro decode** (`avro_data_util.cc`, `avro_direct_decoder.cc`):
materialize
the `kDefault` branch via `AppendDefaultToBuilder`.
## Tests
- `literal_util_test`: `AppendDefaultToBuilder` appends a value and casts to
the
builder type.
- `avro_data_test`: `AppendDatumToBuilder` fills missing required and
optional
default fields.
- `avro_test`: end-to-end — write an Avro file with an old schema, then read
it
through `ReaderFactoryRegistry` with an evolved schema carrying defaults
(`ReadMissingFieldsWithDefaults`).
## Stack
1. #746 — schema: represent / serialize / validate (merged)
2. #792 — read path: Parquet (merged)
3. **this PR** — read path: Avro
4. schema evolution: `addColumn` / `updateColumnDefault` (#793)
Draft while the earlier PRs in the stack settle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
