kinolaev opened a new pull request, #18541: URL: https://github.com/apache/iceberg/pull/18541
With `immediateTableUpdate` disabled, `DynamicRecordProcessor`'s `TableMetadataCache` stores `NOT_FOUND` for an input schema that needs a table update and returns it on later lookups without checking the refresh interval. Nothing updates the processor's cache after `DynamicTableUpdateOperator` evolves the table, so every later record with that schema keeps going through the updater until the entry is evicted or an unrelated lookup refreshes the table. A cached `NOT_FOUND` is now re-resolved against a reloaded table once the refresh interval has elapsed. Within the interval it is still returned from the cache. Found while addressing review feedback on #18507. Tests: - `TestTableMetadataCache.refreshesUnresolvedSchemaAfterRefreshInterval`: a cached `NOT_FOUND` resolves to the evolved table schema after the refresh interval. Fails without the fix. - `TestDynamicRecordProcessor.routesRecordsToMainOutputAfterSchemaUpdateAndCacheRefresh`: after a schema change and the refresh interval, records go to the main output instead of the updater. Fails without the fix. - `TestDynamicRecordProcessor.routesRecordsToMainOutputAfterTableCreationAndCacheRefresh`: guards the same behavior after table creation, which already worked. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: fully reviewed - Prompt Summary: Fix the dynamic sink metadata cache never refreshing cached unresolved input schemas, with regression tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
