kevinjqliu opened a new pull request, #16882: URL: https://github.com/apache/iceberg/pull/16882
## Summary This PR refactors the input cache added in #16740 to make it easier to read and maintain. It keeps `inputCacheDir` as the class-scoped location for cached physical files and keeps `CACHED_DATA_FILES` as the in-memory cache. `CachedDataFilesKey` now uses structured fields: partition spec, table options, file count, row count, and partition count. `writeRecords` now handles cache reuse directly. On a hit, it appends cached files to the fresh table. On a miss, it writes inputs under `inputCacheDir`, plans files with column stats, caches them, and appends them to the target table. Verified locally with temporary instrumentation that the cache has both misses and later hits. --- **AI Disclosure** - Model: GPT-5 - Platform/Tool: Codex - Human Oversight: fully reviewed - Prompt Summary: Refactor Spark rewrite input caching while preserving test semantics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
