nastra commented on code in PR #16250: URL: https://github.com/apache/iceberg/pull/16250#discussion_r3460171061
########## docs/docs/metrics-reporting.md: ########## @@ -120,6 +120,227 @@ This is the default when using the [`RESTCatalog`](https://github.com/apache/ice Sending metrics via REST can be controlled with the `rest-metrics-reporting-enabled` (defaults to `true`) property. +### [`OtelMetricsReporter`](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/metrics/OtelMetricsReporter.java) + +Exports [`ScanReport`](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/metrics/ScanReport.java) and [`CommitReport`](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/metrics/CommitReport.java) through the [OpenTelemetry](https://opentelemetry.io/) API as `iceberg.scan.*` and `iceberg.commit.*` metrics. Any OTLP-compatible backend (Prometheus, CloudWatch, Datadog, Grafana Cloud, Honeycomb, etc.) can receive them through a host-owned OpenTelemetry SDK. + +#### Host responsibilities + +`OtelMetricsReporter` does not own the OpenTelemetry SDK. It calls `GlobalOpenTelemetry.get().getMeter("org.apache.iceberg")` in `initialize(...)` and reports through whatever SDK the host application has registered. If no SDK has been registered, OpenTelemetry returns the no-op implementation and metric calls are silently dropped — the standard OpenTelemetry API contract. + +The host application is therefore responsible for: + +1. Adding the OpenTelemetry **API**, **SDK**, and a **metric exporter** matching the target backend to the runtime classpath. Iceberg core compiles against `opentelemetry-api` only; it does not bundle the SDK or any exporter. +2. Building and registering an `OpenTelemetrySdk`, either via the [OpenTelemetry Java agent](https://opentelemetry.io/docs/zero-code/java/agent/) (which auto-instruments the SDK at JVM startup) or programmatically with `OpenTelemetrySdk.builder()...buildAndRegisterGlobal()`. +3. Configuring the exporter's endpoint, credentials, batching, retry, and resource attributes — typically via the [standard OpenTelemetry environment variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) (`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_SERVICE_NAME`, `OTEL_EXPORTER_OTLP_HEADERS`, ...). + +Because the host owns the SDK, Iceberg has no reporter-specific catalog properties for endpoint, protocol, headers, intervals, or resource attributes. The catalog only needs to know the reporter class: + +``` +metrics-reporter-impl=org.apache.iceberg.metrics.OtelMetricsReporter +``` + +#### Attribute set + +By default the reporter attaches the following attributes to every emitted metric point: + +| Metrics | Default attributes | +|---|---| +| Scan metrics (`iceberg.scan.*`) | `iceberg.table.name` | +| Commit metrics (`iceberg.commit.*`) | `iceberg.table.name`, `iceberg.operation` | + +The attribute set is configurable via the `iceberg.otel.metrics.attributes` catalog property, which takes a comma-separated allowlist of attribute short names. Recognized names: + +- `table-name` — emits `iceberg.table.name` +- `schema-id` — emits `iceberg.schema.id` (opt-in; useful for correlating scan performance with schema evolution) +- `operation` — emits `iceberg.operation` + +Attributes whose short names are not listed are omitted from emitted metric points. To additionally include the schema id: + +``` +iceberg.otel.metrics.attributes=table-name,schema-id,operation +``` + +To omit `iceberg.table.name` entirely in deployments with a very large number of tables: + +``` +iceberg.otel.metrics.attributes=operation +``` + +To emit metrics with no attributes at all (single aggregate time series per metric): + +``` +iceberg.otel.metrics.attributes= +``` + +When the property is not set, the default attribute set above is used. + +The snapshot id is deliberately not exposed as a metric attribute because snapshot ids are monotonically increasing and unique per commit; including them would create a new time series for every commit and risk unbounded cardinality in any time-series backend. + +#### Packaging the exporter + +Pick one OTLP exporter (or any other OpenTelemetry exporter for your backend) and add it to the host's runtime classpath alongside the API and SDK. The OTLP/HTTP path works against any OpenTelemetry Collector or backend that accepts OTLP/HTTP; OTLP/gRPC is functionally equivalent over gRPC. + +Gradle (for a Spark application or any plain JVM app): + +```groovy +dependencies { + // OpenTelemetry API + SDK + runtimeOnly "io.opentelemetry:opentelemetry-api:1.61.0" + runtimeOnly "io.opentelemetry:opentelemetry-sdk:1.61.0" + + // Pick one exporter for your backend + runtimeOnly "io.opentelemetry:opentelemetry-exporter-otlp:1.61.0" + // or "io.opentelemetry:opentelemetry-exporter-prometheus:1.61.0-alpha" +} +``` + +For Spark `spark-submit`, the same artifacts can be passed via `--packages`: + +```bash +spark-submit \ + --packages io.opentelemetry:opentelemetry-api:1.61.0,io.opentelemetry:opentelemetry-sdk:1.61.0,io.opentelemetry:opentelemetry-exporter-otlp:1.61.0 \ + ... +``` + +For Flink, add the same jars to the `lib/` directory of the Flink distribution. For Trino, OpenTelemetry is part of the platform classpath. + +#### Programmatic SDK registration + +A typical host bootstrap, executed once before the catalog is loaded: + +```java +SdkMeterProvider meterProvider = + SdkMeterProvider.builder() + .setResource( + Resource.getDefault().toBuilder() + .put(AttributeKey.stringKey("service.name"), "my-iceberg-app") + .build()) + .registerMetricReader( + PeriodicMetricReader.builder( + OtlpHttpMetricExporter.builder() + .setEndpoint("http://collector.example:4318/v1/metrics";) + .build()) + .setInterval(Duration.ofSeconds(30)) + .build()) + .build(); + +OpenTelemetrySdk.builder() + .setMeterProvider(meterProvider) + .buildAndRegisterGlobal(); +``` + +When using the [OpenTelemetry Java agent](https://opentelemetry.io/docs/zero-code/java/agent/), the agent registers the SDK automatically and no programmatic bootstrap is required. + +#### Examples: routing metrics to common backends Review Comment: this section seems to go beyond Iceberg's responsibility -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
