HenryCaiHaiying opened a new pull request, #17025:
URL: https://github.com/apache/iceberg/pull/17025
Expose runtime visibility into the Iceberg Kafka Connect commit pipeline
using kafka-clients' built-in Metrics framework with a JmxReporter (no new
dependencies).
Two package-private classes own per-task and per-connector Metrics
registries:
WorkerMetrics — tags {connector, task}
CoordinatorMetrics — tag {connector}
Exposed MBeans under the iceberg-kafka-connect-metrics domain:
type=worker-metrics
save-time-{avg,max,total}
consume-available-time-{avg,max,total}
data-written-total
data-complete-total
type=coordinator-metrics
commit-time-{avg,max,total}
consume-available-time-{avg,max,total}
start-commit-total
commit-complete-total
commit-buffer-size (gauge, lazy)
ready-buffer-size (gauge, lazy)
Wiring:
- Channel.consumeAvailable wraps its body in a timer and calls a new
recordConsumeTime(long) hook overridden by Worker and Coordinator. The initial
poll inside Channel.start() is timed too.
- Worker times save() and counts DATA_WRITTEN/DATA_COMPLETE emissions;
closes its registry in stop().
- Coordinator times commit() (covering doCommit + endCurrentCommit on both
success and failure paths), counts START_COMMIT/ COMMIT_COMPLETE emissions, and
exposes lazy-supplier gauges backed by CommitState.commitBufferSize() /
readyBufferSize(); closes its registry in terminate().
- CommitState gains commitBufferSize() / readyBufferSize() accessors.
Construction is hardened against partial-init MBean leaks: the Metrics
registry is built into a local, sensors and gauges are registered inside a try
block, and on RuntimeException the local registry is closed before rethrowing.
Tests: TestWorkerMetrics (4 tests), TestCoordinatorMetrics (5 tests) verify
observable JMX behavior via the live MBeanServer. Registration assertions added
to TestWorker and TestCoordinator. ChannelTestBase gains default
config.connectorName()/taskId() stubs so existing Worker/Coordinator tests keep
constructing without NPE inside the JMX tag map. TestCoordinator's
coordinatorTest() helper now terminates and stops the Coordinator in a finally
block to release the CoordinatorMetrics registry between tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
