michalantkowicz commented on issue #13752: URL: https://github.com/apache/iceberg/issues/13752#issuecomment-3163892707
@nferrario I tested it carefeuly (hope it's last update 😓 ) and it looks that with 1 pod I don't observe data loss - while with two of them data gap remained after the restart now it's indeed reprocessed: ``` -- pod is unhealthy, "java.lang.IllegalStateException: Connection pool shut down" logs are appearing: bucket_start event_count_kafka event_count_iceberg diff diff_percent 2025-08-07 10:50:00 21265 21265 0 100.0 2025-08-07 10:52:00 18392 18392 0 100.0 2025-08-07 10:54:00 21523 21523 0 100.0 2025-08-07 10:56:00 19667 19667 0 100.0 2025-08-07 10:58:00 18558 18558 0 100.0 2025-08-07 11:00:00 23561 23561 0 100.0 2025-08-07 11:02:00 19303 19303 0 100.0 2025-08-07 11:04:00 20151 20151 0 100.0 2025-08-07 11:06:00 19417 18819 598 96 2025-08-07 11:08:00 18888 9613 9275 50 2025-08-07 11:10:00 20246 13564 6682 66 2025-08-07 11:12:00 19219 12904 6315 67 2025-08-07 11:14:00 29560 19855 9705 67 2025-08-07 11:16:00 18490 12406 6084 67 2025-08-07 11:18:00 18522 12427 6095 67 2025-08-07 11:20:00 21311 14295 7016 67 2025-08-07 11:22:00 18620 12493 6127 67 2025-08-07 11:24:00 18740 10300 8440 54 ``` then after restart: ``` -- pod is healthy, no error logs bucket_start event_count_kafka event_count_iceberg diff diff_percent 2025-08-07 10:50:00 21265 21265 0 100 2025-08-07 10:52:00 18392 18392 0 100 2025-08-07 10:54:00 21523 21523 0 100 2025-08-07 10:56:00 19667 19667 0 100 2025-08-07 10:58:00 18558 18558 0 100 2025-08-07 11:00:00 23561 23561 0 100 2025-08-07 11:02:00 19303 19303 0 100 2025-08-07 11:04:00 20151 20151 0 100 2025-08-07 11:06:00 19417 19417 0 100 2025-08-07 11:08:00 18888 18888 0 100 2025-08-07 11:10:00 20246 20246 0 100 2025-08-07 11:12:00 19219 19219 0 100 2025-08-07 11:14:00 29560 29560 0 100 2025-08-07 11:16:00 18490 18490 0 100 2025-08-07 11:18:00 18522 18522 0 100 2025-08-07 11:20:00 21311 21311 0 100 2025-08-07 11:22:00 18620 18620 0 100 2025-08-07 11:24:00 21105 21105 0 100 2025-08-07 11:26:00 19228 19228 0 100 2025-08-07 11:28:00 17477 17477 0 100 2025-08-07 11:30:00 22943 22943 0 100 ``` so as for now this issue looks much much closer to the issue you've linked 🙌 interesting why I was losing data with two pods but I may investigate is later do you know what is the release plan for Iceberg and when we can expect version 1.9.3? I saw that releases are usually happening ~once per month I wonder is there any other chance to have it in Aug... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
