ViliusS opened a new pull request, #3586: URL: https://github.com/apache/logging-log4j2/pull/3586
This is my first contribution to Log4J so I'm not 100% sure if I did everything right, but here it goes. This PR adjusts GcpLayout JSON formatting to match what is expected by GCP. First, it formats the log timestamp field to the correct format recognized by Fluent-Bit (component of Google Cloud Logging) and Google Ops Agent. I'm not sure how it worked before, probably it didn't, but Google Cloud Logging [expects timestampSecond + timestampNanos format](https://cloud.google.com/logging/docs/agent/logging/configuration#timestamp-processing). Ops Agent, which also can be used to feed the logs from external Log4J environment via Socket or RollingFile into GCP, [also expects the same format](https://cloud.google.com/logging/docs/agent/ops-agent/configuration#special-fields). More over, the same [timestamp format is always used in Spring GCP libraries](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/blob/cabb7a59fa6d49d280dd311a7f59ec4d922300e7/spring-cloud-gcp-logging/src/main/java/com/google/cloud/spring/logging/StackdriverJsonLayout.java#L270-L278). Secondly, severity field now must be prefixed with `logging.googleapis.com`. I'm not 100% sure regarding this, because Spring GCP libraries are still using just `severity` field name. However Ops Agent (link above) already uses `logging.googleapis.com/severity`, and simple `severity` name is only described in legacy logging agent which is deprecated. Third, counter cannot be used for `logging.googleapis.com/insertId` as it is duplicated on different threads. I've chosen to use time-based UUID, but if that increases garbage collection issues it safely can be removed. GCP will generate their own IDs anyway. And the last but not the least, I have renamed `exception`, `thread` and `logger` fields. I saw the PR discussion in https://github.com/apache/logging-log4j2/pull/543, however these fields are pretty standard when logging via [Logback's JSON layout](https://github.com/qos-ch/logback-contrib/blob/c68c2e5ddc0ed589b0f1e24bac7b9300a0ea5153/json/classic/src/main/java/ch/qos/logback/contrib/json/classic/JsonLayout.java#L105-L113) and then they are reused in Google's Spring GCP libraries. Sadly, I didn't find how to completely remove `exception` field from JSON output if it's empty. Looks like all empty fields are removed except for `pattern` type. I also found https://issues.apache.org/jira/browse/LOG4J2-3053 regarding that, but not sure about its state. This is not a big issue but is preferable for GCP. By the way, Log4J logs are a little bit incorrect. If `message` field exist, having separate `exception` does nothing to catch the error in Google Error Reporting. According to [Google docs](https://cloud.google.com/error-reporting/docs/formatting-error-messages#format-log-entry), if `message` field exist it must contain stack trace and only that stack trace is used in the Error Reporting UI. I tested this extensively. But I left separate `exception` field if one wants to filter it in the logs. All in all, this should provide much better JSON field mapping experience on GCP. If you think this PR is usable I then will go ahead and fix tests for timestamp processing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@logging.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
