kumarpritam863 commented on issue #13457:
URL: https://github.com/apache/iceberg/issues/13457#issuecomment-3171281981
@rmoff two things I noticed is :
1. In case of dynamic commit, your workers are not able to put file metadata
in the control topic:
> ` [2025-07-03 16:41:01,953] INFO Sent workers commit trigger with
commit-id=605f92b4-f3b5-4a48-9263-6c7d88e2f1e9
(io.tabular.iceberg.connect.channel.Coordinator:104)
[2025-07-03 16:41:03,481] WARN [iceberg-sink-dynamic|task-0] Error aborting
producer transaction (io.tabular.iceberg.connect.channel.Channel:115)
org.apache.kafka.common.errors.TimeoutException: Timeout expired after
60000ms while awaiting EndTxn(false)
[2025-07-03 16:41:03,485] ERROR [iceberg-sink-dynamic|task-0]
WorkerSinkTask{id=iceberg-sink-dynamic-0} Task threw an uncaught and
unrecoverable exception. Task is being killed and will not recover until
manually restarted. Error: Timeout expired after 60000ms while awaiting
TxnOffsetCommitHandler (org.apache.kafka.connect.runtime.WorkerSinkTask:634)
org.apache.kafka.common.errors.TimeoutException: Timeout expired after
60000ms while awaiting TxnOffsetCommitHandler
[2025-07-03 16:41:03,485] INFO [iceberg-sink-dynamic|task-0] Channel
stopping (io.tabular.iceberg.connect.channel.Channel:163)`
2. When you are using without SMT, with route field as target, it always has
the value of "tmp.scratch".
For the point 1, you can set the producer max.block.ms as 5 minutes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
