pvary commented on code in PR #13340:
URL: https://github.com/apache/iceberg/pull/13340#discussion_r2165984691
##########
flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java:
##########
@@ -194,63 +208,62 @@ static class CacheItem {
private final boolean tableExists;
private final Set<String> branches;
- private final SchemaInfo schema;
+ private final Map<Integer, Schema> tableSchemas;
private final Map<Integer, PartitionSpec> specs;
+ private final LRUCache<Schema, ResolvedSchemaInfo> inputSchemas;
private CacheItem(
boolean tableExists,
Set<String> branches,
- SchemaInfo schema,
- Map<Integer, PartitionSpec> specs) {
+ Map<Integer, Schema> tableSchemas,
+ Map<Integer, PartitionSpec> specs,
+ int inputSchemaCacheMaximumSize) {
this.tableExists = tableExists;
this.branches = branches;
- this.schema = schema;
+ this.tableSchemas = tableSchemas;
this.specs = specs;
+ this.inputSchemas =
+ new LRUCache<>(inputSchemaCacheMaximumSize,
CacheItem::inputSchemaEvictionListener);
+ }
+
+ private static void inputSchemaEvictionListener(
+ Map.Entry<Schema, ResolvedSchemaInfo> evictedEntry) {
+ LOG.warn(
+ "Performance degraded as records with different schema is generated
for the same table. "
+ + "Likely the DynamicRecord.schema is not reused. "
+ + "Reuse the same instance if the record schema is the same to
improve performance");
}
@VisibleForTesting
- SchemaInfo getSchemaInfo() {
- return schema;
+ Map<Schema, ResolvedSchemaInfo> getInputSchemas() {
Review Comment:
Could we rename this method to `inputSchemas`. Sorry that I missed this:
https://iceberg.apache.org/contribute/#method-naming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
