akomisarek opened a new issue, #17634: URL: https://github.com/apache/iceberg/issues/17634
### Query engine Iceberg KafkaConnect writing to Unity Catalog Rest API ### Question I have a question about best practices/scaling/operating the Connector and why certain issues might be causing problems. For example I had a connector paused for a day, due to some operations we were running. I unpaused it now, for close to one hour, it hasn't produced as single record, despite being now 2M messages behind. In logs I only see constantly logs like: 2026-08-13 15:48:36,696 INFO [public.customer.delivery.v2-iceberg-connector|task-8] Handled event of type: START_COMMIT (org.apache.iceberg.connect.channel.Channel) [task-thread-public.customer.delivery.v2-iceberg-connector-8] 2026-08-13 15:48:36,700 INFO [public.customer.delivery.v2-unity-iceberg-connector|task-9] Handled event of type: DATA_COMPLETE (org.apache.iceberg.connect.channel.Channel) [iceberg-coord] 2026-08-13 15:48:36,700 WARN [public.customer.delivery.v2-unity-iceberg-connector|task-9] Received commit ready when no commit in progress, this can happen during recovery. Commit ID: d94b5d9e-07d6-4356-a9fb-509a754d1831 (org.apache.iceberg.connect.channel.CommitState) [iceberg-coord] However no data is written at all... we have got commit cadence set to 10 minutes so I would expect some records to appear every 10 minutes, but I guess there is something fundamental about Connector that I don't understand causing this. Any tips to improve this situations/monitor to know where it will successfully create something? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
