leeyam24 opened a new issue, #16905:
URL: https://github.com/apache/iceberg/issues/16905
### Apache Iceberg version
1.11.0 (latest release)
### Query engine
Trino
### Please describe the bug 🐞
When using `rewrite_table_path` to migrate an Iceberg table between clusters
(e.g., different HDFS prefixes), the rewritten manifest list files contain
stale `manifest_length` values. Specifically,
`RewriteTablePathUtil.rewriteManifestList()` copies each `ManifestFile` entry
and only updates field `0` (the path), leaving field `1` (the `length`)
unchanged from the original:
```java
// core/src/main/java/org/apache/iceberg/RewriteTablePathUtil.java
for (ManifestFile file : manifestFiles) {
ManifestFile newFile = file.copy();
((StructLike) newFile).set(0, newPath(newFile.path(), sourcePrefix,
targetPrefix));
writer.add(newFile); // length (field 1) is NOT updated
...
}
```
The rewritten manifest `.avro` files have new path strings embedded inside
them (data file paths with the new prefix). If the target prefix is longer than
the source prefix, the rewritten manifest files are physically larger on disk
than what `manifest_length` records in the manifest list. The Iceberg spec
requires `manifest_length` (field `501`) to reflect the actual byte length of
the manifest file.
Trino's Iceberg connector enforces this field strictly and fails with:
```
Incorrect file size for file <manifest.avro> (end of stream not reached)
```
Spark does not validate `manifest_length` and succeeds.
**To Reproduce**
1. Have an Iceberg table on cluster A with prefix
`hdfs://cluster-a/warehouse/`.
2. Run `rewrite_table_path` with `source_prefix =
hdfs://cluster-a/warehouse/` and `target_prefix =
hdfs://cluster-b/longer-warehouse-path/` (target prefix is longer).
3. `distcp` the staged files to cluster B and register the table.
4. Query the table (any snapshot) with Trino → fails with `Incorrect file
size for file (end of stream not reached)`.
5. Query the same table with Spark → succeeds.
**Additional context: `rewrite_manifests` does not fix historical snapshots**
Running `rewrite_manifests` after the migration does not resolve the issue
for historical snapshots. `BaseRewriteManifests.apply()` only operates on
`base.currentSnapshot()`, so it creates a new snapshot with correct manifest
sizes, but all prior snapshots still point to manifest lists with stale
`manifest_length` values. Querying any old snapshot via Trino will continue to
fail.
**Environment**
- Apache Iceberg version: *1.11.0*
- Query engine: Trino *(fails)*, Spark *(succeeds)*
- Source/target filesystem: HDFS → HDFS (different clusters)
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
