ShivamSharma43 opened a new issue, #2627:
URL: https://github.com/apache/iceberg-go/issues/2627
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
On a v3 table that has a struct, list or map column with a `write-default`,
every `UpdateSchema` fails, even one that only touches another column:
```
error applying schema changes: error encountered during schema visitor:
runtime error: comparing uncomparable type map[string]interface {}
```
The spec allows these defaults. A struct column's default "may be either
null or a non-null struct with no field values", stored as `{}`:
https://github.com/apache/iceberg/blob/52fc764127402ade8170b1b53ddd95fde13cb75c/format/spec.md?plain=1#L389-L391
List and map defaults are a JSON array and a JSON object. iceberg-go loads
such metadata fine (`validateComplexDefault` accepts exactly these shapes) and
keeps the defaults as `map[string]any` and `[]any`.
Reproduced on main (5803266):
```go
var sc iceberg.Schema
_ = json.Unmarshal([]byte(`{"type":"struct","schema-id":0,"fields":[
{"id":1,"name":"id","required":true,"type":"long"},
{"id":2,"name":"point","required":false,"initial-default":{},"write-default":{},
"type":{"type":"struct","fields":[{"id":3,"name":"x","required":false,"type":"int"}]}}]}`),
&sc)
md, _ := table.NewMetadata(&sc, iceberg.UnpartitionedSpec,
table.UnsortedSortOrder, "",
iceberg.Properties{"format-version": "3"})
txn := table.New([]string{"t"}, md, "", nil, nil).NewTransaction()
err := txn.UpdateSchema(true, false).
AddColumn([]string{"name"}, iceberg.PrimitiveTypes.String, "", false,
nil).
Commit()
fmt.Println(err)
```
Output:
```
error applying schema changes: error encountered during schema visitor:
runtime error: comparing uncomparable type map[string]interface {}
```
A list column with `"write-default": []` fails the same way with
`[]interface {}`.
Expected: the column is added.
**Cause:** `applyChanges.Struct` in `table/update_schema.go` checks whether
a field changed with `field.WriteDefault == writeDefault`. Both are `any`, and
`==` panics when they hold the same map or slice type. `iceberg.Visit` recovers
the panic and returns it as the error above. It runs for every field of every
struct in the schema, so one such column blocks all schema changes.
Java's `ApplyChanges.struct` compares the field with `equals`:
https://github.com/apache/iceberg/blob/52fc764127402ade8170b1b53ddd95fde13cb75c/core/src/main/java/org/apache/iceberg/SchemaUpdate.java#L623-L652
**Proposed fix:** compare with `reflect.DeepEqual`, as `NestedField.Equals`
and `unionUpdateColumn` already do.
I'd like to work on this. I have a fix and a unit test ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]