twmb opened a new pull request, #871: URL: https://github.com/apache/iceberg-go/pull/871
Replaces #870 ; this isn't critical but I figured to try after incorporating some improvements into my avro since Tyler opened 870. Obviously this has breaking changes due to the iceberg-go package exposing dependencies on hamba/avro in the API. This is very optional. Drop github.com/hamba/avro/v2 in favor of github.com/twmb/avro v1.4.0 for manifest reading, writing, and partition schema construction, replacing the archived hamba/avro. # Behavior differences visible to callers - DataFile.Partition() returns int32 for plain integer partition fields (was int). Correct per Avro spec — Avro int is 32-bit signed. Go code doing v.(int) on partition values needs to switch to v.(int32). - NewDataFileBuilder takes map[int]string instead of map[int]avro.LogicalType. The corresponding field on dataFile and the hasFieldToIDMap interface also now use string. Logical type constants live in the new github.com/twmb/avro/atype subpackage (atype.Date, atype.TimestampMicros, etc.), untyped string constants, drop-in wherever a string is expected. All other hamba/avro types used by the public surface are replaced: avro.Schema → *avro.Schema on the handful of exported helpers that referenced them; avro.SchemaCache (internal.AvroSchemaCache) → map[string]*avro.Schema (internal.AvroSchemaMap). # Dependency footprint Removes five transitive dependencies: json-iterator/go, modern-go/concurrent, modern-go/reflect2, ettle/strcase, and go-viper/mapstructure (hamba's chain). twmb/avro's only direct dependency is klauspost/compress, which iceberg-go already transitively requires. # Resolves the FixedType partition schema workaround FixedType partitions used BinarySchema (bytes) as a workaround for hamba/avro#571 — hamba couldn't match [N]byte arrays against fixed schemas in unions. twmb/avro does not have this bug, so FixedType now uses a proper Avro fixed schema. The TODO in schema_conversions.go is gone. # Tests New TestPartitionTypeToAvroSchemaDuplicateNamedTypes covers partition spec scenarios with multiple fields of the same named Avro type (two UUIDs, two same-size Fixed, two same-precision Decimal). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
