morningman commented on code in PR #69031:
URL: https://github.com/apache/doris/pull/69031#discussion_r4269796941
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergManifestEntryKey.java:
##########
@@ -31,18 +33,25 @@
* cached payload, and a table-level invalidation (REFRESH TABLE)
intentionally does NOT drop it (legacy
* {@code testInvalidateTableKeepsManifestCache} parity). Runtime loader
context (the manifest/table instances)
* must not be stored here.
+ *
+ * <p>The key also carries the partition type the manifest is decoded with:
the reader converts partition values to
+ * the bound spec's types, and after a type promotion (e.g. INT to BIGINT) a
scan pinned to the old schema and a scan
+ * on the current one bind the same manifest to different types. Each binding
needs its own decoded copy.
*/
public class IcebergManifestEntryKey {
private final String manifestPath;
private final ManifestContent content;
+ private final Types.StructType partitionType;
Review Comment:
Fixed in 703a33ca3b1. `estimateManifestKey` now adds the key's bound
partition type through the existing `estimateIcebergType`, so an entry whose
value is empty (a delete manifest whose entries are all DELETED) is no longer
admitted below the size it retains. A partition type shared by several keys is
counted once per key; that overestimates, which is the safe direction for
admission. Covered by
`IcebergManifestCacheTest#manifestKeyWeightCountsTheBoundPartitionType` (an
empty delete-manifest value keyed by a 10-field vs a 1-field partition type),
which fails without the estimator change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
