merlimat opened a new pull request, #25515:
URL: https://github.com/apache/pulsar/pull/25515
### Motivation
Pulsar already includes a `ConsoleJson` appender in `conf/log4j2.yaml` that
emits ECS-formatted JSON via `JsonTemplateLayout`, but enabling it requires
knowing internal Log4j2 system properties (e.g. setting
`PULSAR_ROUTING_APPENDER_DEFAULT=ConsoleJson`). This is undiscoverable for
users who want JSON logs in containerized environments (k8s, Docker) where log
aggregators like ELK, Loki, or Datadog parse JSON.
This PR adds a single, discoverable environment variable to flip both
console and file logging to JSON.
### Modifications
- **`bin/pulsar`**: Adds handling for a new `PULSAR_LOG_FORMAT` env var.
When set to `json`:
- `PULSAR_ROUTING_APPENDER_DEFAULT` switches to `ConsoleJson` (so the
routing appender's fallback writes JSON to stdout)
- When in daemon mode (`PULSAR_LOG_APPENDER=RollingFile`), it switches to
`RollingFileJson`
- **`conf/log4j2.yaml`**: Adds a new `RollingFileJson` appender — same
paths, rotation policies, and retention as the existing `RollingFile`, but uses
`JsonTemplateLayout` with the ECS template instead of `PatternLayout`. The
`RollingFile` section is restructured as a YAML list to host both entries
(matching how `Console` already does it).
- **`conf/pulsar_env.sh`**: Adds a documented commented-out example for
`PULSAR_LOG_FORMAT`.
- **slog 0.9.5 → 0.9.6**: The previous slog version triggered a NPE in
`JsonTemplateLayout` because `MutableLogEvent.clear()` left the context stack
null and Log4j2's `ThreadContextStackResolver.isResolvable()` does not
null-check it. slog 0.9.6 sets the context stack to `EMPTY_STACK` after
clearing the event. License files for both the server and shell distributions
are updated.
#### Usage
```bash
# Foreground (k8s, Docker stdout)
PULSAR_LOG_FORMAT=json bin/pulsar standalone
# Daemon mode (rolling JSON files)
PULSAR_LOG_FORMAT=json bin/pulsar-daemon start broker
```
Without `PULSAR_LOG_FORMAT`, behavior is unchanged.
### Verifying this change
- [x] Make sure that the change passes the CI checks.
This change is a trivial rework / code cleanup without any test coverage —
it only wires existing Log4j2 appenders behind an env var and bumps a
dependency.
Manually verified:
- `PULSAR_LOG_FORMAT=json bin/pulsar standalone` produces JSON on stdout
- Default behavior (no env var) is unchanged
- The slog 0.9.6 bump resolves the NPE that previously surfaced when JSON
logging was enabled
### Does this pull request potentially affect one of the following parts:
- [x] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [x] Anything that affects deployment
Bumps the `slog` dependency from 0.9.5 to 0.9.6 (only fix is the NPE in
`JsonTemplateLayout`). The new `PULSAR_LOG_FORMAT` env var is opt-in and does
not change any default behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
