mzzz-zzm opened a new issue, #966:
URL: https://github.com/apache/iceberg-go/issues/966
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
`Schema.MarshalJSON()` in `schema.go` contains an unsynchronized write to a
field of the shared receiver:
```go
func (s *Schema) MarshalJSON() ([]byte, error) {
if s.IdentifierFieldIDs == nil {
s.IdentifierFieldIDs = []int{} // <-- write on shared *Schema
}
...
}
```
## How it is triggered
During manifest writing, `snapshotProducer.manifests()` launches multiple
goroutines via `errgroup` :
```go
g.Go(sp.manifestProducer(iceberg.ManifestContentData, ...)) //
goroutine A
g.Go(sp.manifestProducer(iceberg.ManifestContentDeletes, ...) // goroutine
B
// + goroutines for deleted-entry groups
```
Each goroutine calls `newManifestWriter` --> `sp.txn.meta.CurrentSchema()`
-- returning the same `*Schema` pointer -- then `ManifestWriter.meta()` -->
`json.Marshal(w.schema)` When `IdentifierFieldIDs` is nil on the first call,
all concurrent goroutines race to write `[]int{}` to it.
This shows up under `go test -race` on any concurrent-writer workload
## Fix
Work on a local copy instead of touching the receiver:
```go
func (s *Schema) MarshalJSON() ([]byte, error) {
ids := s.IdentifierFieldIDs
if ids == nil {
ids = []int{}
}
type Alias Schema
aliasCopy := *(*Alias)(s)
aliasCopy.IdentifierFieldIDs = ids
return json.Marshal(struct {
Type string `json:"type"`
Fields []NestedField `json:"fields"`
*Alias
}{Type: "struct", Fields: s.fields, Alias: &aliasCopy})
}
```
`aliasCopy` is stack-allocated per call -- never writes to `s` -- safe for
concurrent use.
## Note
The analogous race in `ExpressionEvaluator()` / `exprEvaluator.Eval` was
already fixed (the `Eval` method now constructs a fresh instance per call).
This is the remaining unfixed data race in the same concurrent
manifest-write code path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
