zhangfengcdt opened a new issue, #3326:
URL: https://github.com/apache/iceberg-rust/issues/3326
### Apache Iceberg Rust version
`main`. Verified at `1960386` (2026-10-01).
### Describe the bug
`PartitionSpec::partition_to_path` formats `float` / `double` partition
values differently from iceberg-java. Rust uses the float's `Display` output;
Java uses `Float.toString` / `Double.toString`. They disagree for whole
numbers, for values Java prints in scientific notation, and for infinities.
Paths for a `double` column `d` with an identity partition (Java column from
iceberg 1.11.0):
| Value | iceberg-rust | iceberg-java |
| --- | --- | --- |
| `1.0` | `d=1` | `d=1.0` |
| `100.0` | `d=100` | `d=100.0` |
| `0.0` | `d=0` | `d=0.0` |
| `-0.0` | `d=-0` | `d=-0.0` |
| `1.0e7` | `d=10000000` | `d=1.0E7` |
| `1.0e-5` | `d=0.00001` | `d=1.0E-5` |
| `1.23456789e20` | `d=123456789000000000000` | `d=1.23456789E20` |
| `+Infinity` | `d=inf` | `d=Infinity` |
| `-Infinity` | `d=-inf` | `d=-Infinity` |
`1.5`, `0.001` and `NaN` already match. `float` columns show the same
pattern.
The call path is `partition_to_path` → `Transform::to_human_string` →
`Datum::to_human_string` → `Display for Datum`, which writes the float with
`{val}`. In Java, `PartitionSpec.partitionToPath` →
`Transform.toHumanString(Type, T)` falls through to `value.toString()`.
This does not affect reads: readers take partition values from manifests,
and the spec does not define the path layout. The path is used in two places:
- Data file locations (`location_generator.rs`). A table written by both
implementations gets two directories for one partition value, for example
`d=1/` and `d=1.0/`.
- Keys of the per-partition snapshot summary, `partitions.<path>`
(`snapshot_summary.rs`). Java builds these keys from `partitionToPath` too, so
the same partition gets different keys depending on the writer.
### To Reproduce
```rust
use iceberg::spec::{Literal, PrimitiveType, Transform, Type};
let double = Type::Primitive(PrimitiveType::Double);
let human = |v: f64| Transform::Identity.to_human_string(&double,
Some(&Literal::double(v)));
// All pass today. iceberg-java gives "1.0", "-0.0", "1.0E7" and "Infinity".
assert_eq!(human(1.0), "1");
assert_eq!(human(-0.0), "-0");
assert_eq!(human(1.0e7), "10000000");
assert_eq!(human(f64::INFINITY), "inf");
```
### Expected behavior
`float` / `double` values are formatted as Java's `Float.toString` /
`Double.toString` do:
- at least one digit after the decimal point (`1.0`, `-0.0`);
- plain decimal when `1e-3 <= |v| < 1e7`, otherwise scientific notation such
as `1.0E7` and `1.0E-5`;
- `Infinity` / `-Infinity` for infinities.
Changing this changes the directory that new files are written to in tables
already written by iceberg-rust. Existing files stay valid, because manifests
store full paths.
### Willingness to contribute
None
### Additional context
Found while investigating #3325. It is independent of that issue: the cause
is value formatting, not partition equality.
#3022 aligned the human strings of the date and time transforms with Java in
the same function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
