I can see advantages in using Netty-http or similar to use async NIO, though those advantages will be a bit limited as long as we don't have a proper interface for async appenders (LOG4J2-1797 <https://issues.apache.org/jira/browse/LOG4J2-1797>).
But are there any significant advantages in using another synchronous HTTP client library? Given that an appender is a plugin by itself, I am a bit skeptical to adding another layer of pluggability unless there are are significant advantages with another library and we can share significant amount of code between different implementations. On Thu, May 4, 2017 at 4:42 PM, Matt Sicker <boa...@gmail.com> wrote: > Thanks for starting this! My idea in mind when I wrote the ticket was to > make the actual client pluggable so that users could use HttpURLConnection > (no dependencies), Apache HttpClient, Netty-HTTP, etc. > > On 4 May 2017 at 07:49, <mi...@apache.org> wrote: > > > Repository: logging-log4j2 > > Updated Branches: > > refs/heads/LOG4J2-1442 [created] 0af515f3b > > > > > > LOG4J2-1442 Generic HTTP appender > > > > > > Project: http://git-wip-us.apache.org/repos/asf/logging-log4j2/repo > > Commit: http://git-wip-us.apache.org/repos/asf/logging-log4j2/ > > commit/410f9d36 > > Tree: http://git-wip-us.apache.org/repos/asf/logging-log4j2/tree/ > 410f9d36 > > Diff: http://git-wip-us.apache.org/repos/asf/logging-log4j2/diff/ > 410f9d36 > > > > Branch: refs/heads/LOG4J2-1442 > > Commit: 410f9d360eabcdc6949c75973b786c9e2cec9c66 > > Parents: 8852cd1 > > Author: Mikael Ståldal <mikael.stal...@magine.com> > > Authored: Thu May 4 14:21:59 2017 +0200 > > Committer: Mikael Ståldal <mikael.stal...@magine.com> > > Committed: Thu May 4 14:45:04 2017 +0200 > > > > ---------------------------------------------------------------------- > > .../log4j/core/appender/HttpAppender.java | 161 > +++++++++++++++++++ > > .../log4j/core/appender/HttpManager.java | 82 ++++++++++ > > .../log4j/core/appender/HttpAppenderTest.java | 52 ++++++ > > .../src/test/resources/HttpAppenderTest.xml | 43 +++++ > > src/changes/changes.xml | 3 + > > src/site/site.xml | 1 + > > src/site/xdoc/manual/appenders.xml | 78 +++++++++ > > 7 files changed, 420 insertions(+) > > ---------------------------------------------------------------------- > > > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/log4j-core/src/main/java/org/apache/logging/log4j/ > > core/appender/HttpAppender.java > > ---------------------------------------------------------------------- > > diff --git a/log4j-core/src/main/java/org/apache/logging/log4j/core/ > appender/HttpAppender.java > > b/log4j-core/src/main/java/org/apache/logging/log4j/core/ > > appender/HttpAppender.java > > new file mode 100644 > > index 0000000..e0f1b27 > > --- /dev/null > > +++ b/log4j-core/src/main/java/org/apache/logging/log4j/core/ > > appender/HttpAppender.java > > @@ -0,0 +1,161 @@ > > +/* > > + * Licensed to the Apache Software Foundation (ASF) under one or more > > + * contributor license agreements. See the NOTICE file distributed with > > + * this work for additional information regarding copyright ownership. > > + * The ASF licenses this file to You under the Apache license, Version > 2.0 > > + * (the "License"); you may not use this file except in compliance with > > + * the License. You may obtain a copy of the License at > > + * > > + * http://www.apache.org/licenses/LICENSE-2.0 > > + * > > + * Unless required by applicable law or agreed to in writing, software > > + * distributed under the License is distributed on an "AS IS" BASIS, > > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > > implied. > > + * See the license for the specific language governing permissions and > > + * limitations under the license. > > + */ > > + > > +package org.apache.logging.log4j.core.appender; > > + > > +import java.io.IOException; > > +import java.io.Serializable; > > +import java.util.Objects; > > +import java.util.concurrent.TimeUnit; > > + > > +import org.apache.logging.log4j.core.Appender; > > +import org.apache.logging.log4j.core.Filter; > > +import org.apache.logging.log4j.core.Layout; > > +import org.apache.logging.log4j.core.LogEvent; > > +import org.apache.logging.log4j.core.config.Node; > > +import org.apache.logging.log4j.core.config.Property; > > +import org.apache.logging.log4j.core.config.plugins.Plugin; > > +import org.apache.logging.log4j.core.config.plugins. > > PluginBuilderAttribute; > > +import org.apache.logging.log4j.core.config.plugins. > PluginBuilderFactory; > > +import org.apache.logging.log4j.core.config.plugins.PluginElement; > > +import org.apache.logging.log4j.core.config.plugins.validation. > > constraints.Required; > > + > > +/** > > + * Sends log events over HTTP. > > + */ > > +@Plugin(name = "Http", category = Node.CATEGORY, elementType = > > Appender.ELEMENT_TYPE, printObject = true) > > +public final class HttpAppender extends AbstractAppender { > > + > > + /** > > + * Builds HttpAppender instances. > > + * @param <B> The type to build > > + */ > > + public static class Builder<B extends Builder<B>> extends > > AbstractAppender.Builder<B> > > + implements org.apache.logging.log4j.core. > util.Builder<HttpAppender> > > { > > + > > + @PluginBuilderAttribute > > + @Required(message = "No URL provided for HttpAppender") > > + private String url; > > + > > + @PluginBuilderAttribute > > + private String method = "POST"; > > + > > + @PluginBuilderAttribute > > + private int connectTimeoutMillis = 0; > > + > > + @PluginBuilderAttribute > > + private int readTimeoutMillis = 0; > > + > > + @PluginElement("Headers") > > + private Property[] headers; > > + > > + @Override > > + public HttpAppender build() { > > + final HttpManager httpManager = new > > HttpManager(getConfiguration(), getConfiguration().getLoggerContext(), > > + getName(), url, method, connectTimeoutMillis, > > readTimeoutMillis, headers); > > + return new HttpAppender(getName(), getLayout(), getFilter(), > > isIgnoreExceptions(), httpManager); > > + } > > + > > + public String getUrl() { > > + return url; > > + } > > + > > + public String getMethod() { > > + return method; > > + } > > + > > + public int getConnectTimeoutMillis() { > > + return connectTimeoutMillis; > > + } > > + > > + public int getReadTimeoutMillis() { > > + return readTimeoutMillis; > > + } > > + > > + public Property[] getHeaders() { > > + return headers; > > + } > > + > > + public B setUrl(final String url) { > > + this.url = url; > > + return asBuilder(); > > + } > > + > > + public B setMethod(final String method) { > > + this.method = method; > > + return asBuilder(); > > + } > > + > > + public B setConnectTimeoutMillis(int connectTimeoutMillis) { > > + this.connectTimeoutMillis = connectTimeoutMillis; > > + return asBuilder(); > > + } > > + > > + public B setReadTimeoutMillis(int readTimeoutMillis) { > > + this.readTimeoutMillis = readTimeoutMillis; > > + return asBuilder(); > > + } > > + > > + public B setHeaders(final Property[] headers) { > > + this.headers = headers; > > + return asBuilder(); > > + } > > + } > > + > > + /** > > + * @return a builder for a HttpAppender. > > + */ > > + @PluginBuilderFactory > > + public static <B extends Builder<B>> B newBuilder() { > > + return new Builder<B>().asBuilder(); > > + } > > + > > + private final HttpManager manager; > > + > > + private HttpAppender(final String name, final Layout<? extends > > Serializable> layout, final Filter filter, > > + final boolean ignoreExceptions, final > > HttpManager manager) { > > + super(name, filter, layout, ignoreExceptions); > > + Objects.requireNonNull(layout, "layout"); > > + this.manager = Objects.requireNonNull(manager, "manager"); > > + } > > + > > + @Override > > + public void append(final LogEvent event) { > > + try { > > + manager.send(getLayout(), event); > > + } catch (final Exception e) { > > + error("Unable to send HTTP in appender [" + getName() + "]", > > event, e); > > + } > > + } > > + > > + @Override > > + public boolean stop(final long timeout, final TimeUnit timeUnit) { > > + setStopping(); > > + boolean stopped = super.stop(timeout, timeUnit, false); > > + stopped &= manager.stop(timeout, timeUnit); > > + setStopped(); > > + return stopped; > > + } > > + > > + @Override > > + public String toString() { > > + return "HttpAppender{" + > > + "name=" + getName() + > > + ", state=" + getState() + > > + '}'; > > + } > > +} > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/log4j-core/src/main/java/org/apache/logging/log4j/ > > core/appender/HttpManager.java > > ---------------------------------------------------------------------- > > diff --git a/log4j-core/src/main/java/org/apache/logging/log4j/core/ > appender/HttpManager.java > > b/log4j-core/src/main/java/org/apache/logging/log4j/core/ > > appender/HttpManager.java > > new file mode 100644 > > index 0000000..8f69659 > > --- /dev/null > > +++ b/log4j-core/src/main/java/org/apache/logging/log4j/core/ > > appender/HttpManager.java > > @@ -0,0 +1,82 @@ > > +/* > > + * Licensed to the Apache Software Foundation (ASF) under one or more > > + * contributor license agreements. See the NOTICE file distributed with > > + * this work for additional information regarding copyright ownership. > > + * The ASF licenses this file to You under the Apache license, Version > 2.0 > > + * (the "License"); you may not use this file except in compliance with > > + * the License. You may obtain a copy of the License at > > + * > > + * http://www.apache.org/licenses/LICENSE-2.0 > > + * > > + * Unless required by applicable law or agreed to in writing, software > > + * distributed under the License is distributed on an "AS IS" BASIS, > > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > > implied. > > + * See the license for the specific language governing permissions and > > + * limitations under the license. > > + */ > > + > > +package org.apache.logging.log4j.core.appender; > > + > > +import java.io.IOException; > > +import java.io.OutputStream; > > +import java.net.HttpURLConnection; > > +import java.net.MalformedURLException; > > +import java.net.URL; > > +import java.util.Objects; > > + > > +import org.apache.logging.log4j.core.Layout; > > +import org.apache.logging.log4j.core.LogEvent; > > +import org.apache.logging.log4j.core.LoggerContext; > > +import org.apache.logging.log4j.core.config.Configuration; > > +import org.apache.logging.log4j.core.config.ConfigurationException; > > +import org.apache.logging.log4j.core.config.Property; > > + > > +public class HttpManager extends AbstractManager { > > + > > + private final Configuration configuration; > > + private final URL url; > > + private final String method; > > + private final int connectTimeoutMillis; > > + private final int readTimeoutMillis; > > + private final Property[] headers; > > + > > + public HttpManager(final Configuration configuration, LoggerContext > > loggerContext, final String name, > > + final String url, final String method, final int > > connectTimeoutMillis, final int readTimeoutMillis, > > + final Property[] headers) { > > + super(loggerContext, name); > > + this.configuration = Objects.requireNonNull(configuration); > > + try { > > + this.url = new URL(url); > > + } catch (MalformedURLException e) { > > + throw new ConfigurationException(e); > > + } > > + this.method = Objects.requireNonNull(method, "method"); > > + this.connectTimeoutMillis = connectTimeoutMillis; > > + this.readTimeoutMillis = readTimeoutMillis; > > + this.headers = headers != null ? headers : new Property[0]; > > + } > > + > > + public void send(final Layout<?> layout, final LogEvent event) > throws > > IOException { > > + HttpURLConnection urlConnection = (HttpURLConnection)url. > > openConnection(); > > + urlConnection.setAllowUserInteraction(false); > > + urlConnection.setDoOutput(true); > > + urlConnection.setDoInput(true); > > + urlConnection.setRequestMethod(method); > > + if (connectTimeoutMillis > 0) urlConnection.setConnectTimeout( > > connectTimeoutMillis); > > + if (readTimeoutMillis > 0) urlConnection.setReadTimeout( > > readTimeoutMillis); > > + if (layout.getContentType() != null) urlConnection. > > setRequestProperty("Content-Type", layout.getContentType()); > > + for (Property header : headers) { > > + urlConnection.setRequestProperty( > > + header.getName(), > > + header.isValueNeedsLookup() ? configuration. > > getStrSubstitutor().replace(event, header.getValue()) : > > header.getValue()); > > + } > > + byte[] msg = layout.toByteArray(event); > > + urlConnection.setFixedLengthStreamingMode(msg.length); > > + urlConnection.connect(); > > + try (OutputStream os = urlConnection.getOutputStream()) { > > + os.write(msg); > > + } > > + urlConnection.getInputStream().close(); > > + } > > + > > +} > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/log4j-core/src/test/java/org/apache/logging/log4j/ > core/appender/ > > HttpAppenderTest.java > > ---------------------------------------------------------------------- > > diff --git a/log4j-core/src/test/java/org/apache/logging/log4j/core/ > appender/HttpAppenderTest.java > > b/log4j-core/src/test/java/org/apache/logging/log4j/core/ > > appender/HttpAppenderTest.java > > new file mode 100644 > > index 0000000..98120a1 > > --- /dev/null > > +++ b/log4j-core/src/test/java/org/apache/logging/log4j/core/ > > appender/HttpAppenderTest.java > > @@ -0,0 +1,52 @@ > > +package org.apache.logging.log4j.core.appender; > > + > > +import org.apache.logging.log4j.Level; > > +import org.apache.logging.log4j.core.Appender; > > +import org.apache.logging.log4j.core.impl.Log4jLogEvent; > > +import org.apache.logging.log4j.junit.LoggerContextRule; > > +import org.apache.logging.log4j.message.SimpleMessage; > > +import org.junit.Rule; > > +import org.junit.Test; > > + > > +// TODO this test requires manual verification > > +public class HttpAppenderTest { > > + > > + private static final String LOG_MESSAGE = "Hello, world!"; > > + > > + private static Log4jLogEvent createLogEvent() { > > + return Log4jLogEvent.newBuilder() > > + .setLoggerName(HttpAppenderTest.class.getName()) > > + .setLoggerFqcn(HttpAppenderTest.class.getName()) > > + .setLevel(Level.INFO) > > + .setMessage(new SimpleMessage(LOG_MESSAGE)) > > + .build(); > > + } > > + > > + @Rule > > + public LoggerContextRule ctx = new LoggerContextRule(" > > HttpAppenderTest.xml"); > > + > > + @Test > > + public void testAppendSuccess() throws Exception { > > + final Appender appender = ctx.getRequiredAppender(" > HttpSuccess"); > > + appender.append(createLogEvent()); > > + } > > + > > + @Test > > + public void testAppendErrorIgnore() throws Exception { > > + final Appender appender = ctx.getRequiredAppender(" > > HttpErrorIgnore"); > > + appender.append(createLogEvent()); > > + } > > + > > + @Test(expected = AppenderLoggingException.class) > > + public void testAppendError() throws Exception { > > + final Appender appender = ctx.getRequiredAppender("HttpError"); > > + appender.append(createLogEvent()); > > + } > > + > > + @Test > > + public void testAppendSubst() throws Exception { > > + final Appender appender = ctx.getRequiredAppender("HttpSubst"); > > + appender.append(createLogEvent()); > > + } > > + > > +} > > \ No newline at end of file > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/log4j-core/src/test/resources/HttpAppenderTest.xml > > ---------------------------------------------------------------------- > > diff --git a/log4j-core/src/test/resources/HttpAppenderTest.xml > > b/log4j-core/src/test/resources/HttpAppenderTest.xml > > new file mode 100644 > > index 0000000..30edaa0 > > --- /dev/null > > +++ b/log4j-core/src/test/resources/HttpAppenderTest.xml > > @@ -0,0 +1,43 @@ > > +<?xml version="1.0" encoding="UTF-8"?> > > +<!-- > > + ~ Licensed to the Apache Software Foundation (ASF) under one or more > > + ~ contributor license agreements. See the NOTICE file distributed with > > + ~ this work for additional information regarding copyright ownership. > > + ~ The ASF licenses this file to You under the Apache license, Version > > 2.0 > > + ~ (the "License"); you may not use this file except in compliance with > > + ~ the License. You may obtain a copy of the License at > > + ~ > > + ~ http://www.apache.org/licenses/LICENSE-2.0 > > + ~ > > + ~ Unless required by applicable law or agreed to in writing, software > > + ~ distributed under the License is distributed on an "AS IS" BASIS, > > + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > > implied. > > + ~ See the license for the specific language governing permissions and > > + ~ limitations under the license. > > + --> > > +<Configuration name="HttpAppenderTest" status="WARN"> > > + <Appenders> > > + <Http name="HttpSuccess" url="http://localhost:9200/test/log4j/";> > > + <Property name="X-Test" value="header value" /> > > + <JsonLayout properties="true"/> > > + </Http> > > + <Http name="HttpErrorIgnore" url="http://localhost:9200/test/log4j/ > " > > method="PUT"> > > + <JsonLayout properties="true"/> > > + </Http> > > + <Http name="HttpError" url="http://localhost:9200/test/log4j/"; > > method="PUT" ignoreExceptions="false"> > > + <JsonLayout properties="true"/> > > + </Http> > > + <Http name="HttpSubst" url="http://localhost:9200/test/log4j/";> > > + <Property name="X-Test" value="$${java:runtime}" /> > > + <JsonLayout properties="true"/> > > + </Http> > > + </Appenders> > > + <Loggers> > > + <Root level="info"> > > + <AppenderRef ref="HttpSuccess"/> > > + <AppenderRef ref="HttpErrorIgnore"/> > > + <AppenderRef ref="HttpError"/> > > + <AppenderRef ref="HttpSubst"/> > > + </Root> > > + </Loggers> > > +</Configuration> > > \ No newline at end of file > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/src/changes/changes.xml > > ---------------------------------------------------------------------- > > diff --git a/src/changes/changes.xml b/src/changes/changes.xml > > index 379c21e..d0f8133 100644 > > --- a/src/changes/changes.xml > > +++ b/src/changes/changes.xml > > @@ -31,6 +31,9 @@ > > - "remove" - Removed > > --> > > <release version="2.9.0" date="2017-MM-DD" description="GA Release > > 2.9.0"> > > + <action issue="LOG4J2-1442" dev="mikes" type="add"> > > + Generic HTTP appender. > > + </action> > > <action issue="LOG4J2-1854" dev="mikes" type="add" due-to="Xavier > > Jodoin"> > > Support null byte delimiter in GelfLayout. > > </action> > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/src/site/site.xml > > ---------------------------------------------------------------------- > > diff --git a/src/site/site.xml b/src/site/site.xml > > index e380c82..aa161fe 100644 > > --- a/src/site/site.xml > > +++ b/src/site/site.xml > > @@ -134,6 +134,7 @@ > > <item name="JDBC" href="/manual/appenders.html#JDBCAppender"/> > > <item name="JMS" href="/manual/appenders.html#JMSAppender"/> > > <item name="JPA" href="/manual/appenders.html#JPAAppender"/> > > + <item name="HTTP" href="/manual/appenders.html#HttpAppender"/> > > <item name="Kafka" href="/manual/appenders.html# > KafkaAppender"/> > > <item name="Memory Mapped File" href="/manual/appenders.html# > > MemoryMappedFileAppender"/> > > <item name="NoSQL" href="/manual/appenders.html# > NoSQLAppender"/> > > > > http://git-wip-us.apache.org/repos/asf/logging-log4j2/blob/ > > 410f9d36/src/site/xdoc/manual/appenders.xml > > ---------------------------------------------------------------------- > > diff --git a/src/site/xdoc/manual/appenders.xml b/src/site/xdoc/manual/ > > appenders.xml > > index 28d9aa4..f7721df 100644 > > --- a/src/site/xdoc/manual/appenders.xml > > +++ b/src/site/xdoc/manual/appenders.xml > > @@ -1538,6 +1538,84 @@ public class JpaLogEntity extends > > AbstractLogEventWrapperEntity { > > ... > > }]]></pre> > > </subsection> > > + <a name="HttpAppender"/> > > + <subsection name="HttpAppender"> > > + <p> > > + The HttpAppender sends log events over HTTP. A Layout must > be > > provided to format the LogEvent. > > + </p> > > + <p> > > + Will set the <code>Content-Type</code> header according to > > the layout. Additional headers can be specified > > + with embedded Property elements. > > + </p> > > + <table> > > + <caption align="top">HttpAppender Parameters</caption> > > + <tr> > > + <th>Parameter Name</th> > > + <th>Type</th> > > + <th>Description</th> > > + </tr> > > + <tr> > > + <td>name</td> > > + <td>String</td> > > + <td>The name of the Appender.</td> > > + </tr> > > + <tr> > > + <td>filter</td> > > + <td>Filter</td> > > + <td>A Filter to determine if the event should be handled > by > > this Appender. More than one Filter > > + may be used by using a CompositeFilter.</td> > > + </tr> > > + <tr> > > + <td>layout</td> > > + <td>Layout</td> > > + <td>The Layout to use to format the LogEvent.</td> > > + </tr> > > + <tr> > > + <td>url</td> > > + <td>string</td> > > + <td>The URL to use. The URL scheme must be "http" or > > "https".</td> > > + </tr> > > + <tr> > > + <td>method</td> > > + <td>string</td> > > + <td>The HTTP method to use. Optional, default is > > "POST".</td> > > + </tr> > > + <tr> > > + <td>connectTimeoutMillis</td> > > + <td>integer</td> > > + <td>The connect timeout in milliseconds. Optional, default > > is 0 (infinite timeout).</td> > > + </tr> > > + <tr> > > + <td>readTimeoutMillis</td> > > + <td>integer</td> > > + <td>The socket read timeout in milliseconds. Optional, > > default is 0 (infinite timeout).</td> > > + </tr> > > + <tr> > > + <td>headers</td> > > + <td>Property[]</td> > > + <td>Additional HTTP headers to use. The values support <a > > href="lookups.html">lookups</a></td> > > + </tr> > > + <tr> > > + <td>ignoreExceptions</td> > > + <td>boolean</td> > > + <td>The default is <code>true</code>, causing exceptions > > encountered while appending events to be > > + internally logged and then ignored. When set to > > <code>false</code> exceptions will be propagated to the > > + caller, instead. You must set this to <code>false</code> > > when wrapping this Appender in a > > + <a href="#FailoverAppender">FailoverAppender</a>.</td> > > + </tr> > > + </table> > > + <p> > > + Here is a sample HttpAppender configuration snippet: > > + </p> > > + <pre class="prettyprint linenums"><![CDATA[<?xml version="1.0" > > encoding="UTF-8"?> > > + ... > > + <Appenders> > > + <Http name="Http" url="http://localhost:9200/test/log4j/";> > > + <Property name="X-Java-Runtime" value="$${java:runtime}" /> > > + <JsonLayout properties="true"/> > > + </Http> > > + </Appenders>]]></pre> > > + </subsection> > > <a name="KafkaAppender"/> > > <subsection name="KafkaAppender"> > > <p> > > > > > > > -- > Matt Sicker <boa...@gmail.com> > -- [image: MagineTV] *Mikael Ståldal* Senior software developer *Magine TV* mikael.stal...@magine.com Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email.
