mzzz-zzm opened a new issue, #1028:
URL: https://github.com/apache/iceberg-go/issues/1028
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
**Component**: `literals.go`
**Affected version**: `main` (v0.5.0)
**Severity**: Silent wrong-precision type metadata for any decimal literal
read from a manifest
---
`DecimalLiteral.Type()` always returns `decimal(9, scale)` regardless of the
column's actual declared precision. Any code path that calls `.Type()` on a
manifest-read decimal literal — type bounds checks, cast decisions, predicate
projection — silently operates against the wrong precision.
---
## Affected code
`literals.go`:
```go
func (d DecimalLiteral) Type() Type { return DecimalTypeOf(9, d.Scale) }
```
`DecimalLiteral` is defined as:
```go
type DecimalLiteral Decimal
// Decimal carries Val (*big.Int) and Scale (int32), but NOT precision.
type Decimal struct {
Val *big.Int
Scale int32
}
```
---
## Root cause
`DecimalLiteral` (and `Decimal`) do not store the column's declared
precision — only the scale. When `Type()` needs to reconstruct the full
`DecimalType`, it has no precision to use, so it hardcodes `9`.
`9` is a valid precision for small decimal columns but is wrong for any
column with precision ≠ 9. Common production types like `decimal(10,2)`,
`decimal(18,4)`, or `decimal(38,10)` all silently get `decimal(9, scale)`
instead.
---
## Impact
All callers of `DecimalLiteral.Type()`:
| Call site | Effect of wrong precision |
|---|---|
| Literal bounds/range checks | Upper bound calculated against
`decimal(9,s)` range instead of actual column range |
| `Literal.To(targetType)` cast path | Cast validity check uses wrong source
type |
| Predicate projection (`BoundPredicate`) | `Inclusive`/`Exclusive` bounds
projected with wrong precision |
| `validateAddedDataFilesMatchingFilter` partition evaluation | Partition
equality compared against wrong type descriptor |
---
## Reproduction
```go
// Column is decimal(18, 4).
lit := iceberg.DecimalLiteral{Val: big.NewInt(12345678), Scale: 4}
fmt.Println(lit.Type()) // prints: decimal(9, 4) ← should be decimal(18, 4)
```
---
## Proposed fix
Two options:
**Option A — store precision in `DecimalLiteral`** (requires changing the
type):
```go
type DecimalLiteral struct {
Decimal
Precision int // declared column precision
}
func (d DecimalLiteral) Type() Type { return DecimalTypeOf(d.Precision,
d.Scale) }
```
This is a breaking change to the `DecimalLiteral` struct.
**Option B — thread precision through construction sites** (narrower change):
Ensure every constructor that produces a `DecimalLiteral` from a
manifest-read value (e.g. `convertAvroValueToIcebergType` in `manifest.go`)
also attaches the declared precision from the partition field's `DecimalType`.
The precision is available at the call site and just needs to be plumbed
through.
Option B is lower-risk and can be done without breaking the public API, at
the cost of requiring callers to supply precision explicitly.
---
## Related
- Bug: `convertDecimalValue` passes string length instead of declared
precision to `DecimalRequiredBytes` (companion bug in `manifest.go`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
