[ https://issues.apache.org/jira/browse/LOG4J2-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834755#comment-16834755 ]
Ralph Goers commented on LOG4J2-1311: ------------------------------------- I ran into this when testing support for various methods of logging from a Docker container. This issue cannot be resolved. The problem is baked into how TCP works. The socket appender sends events that the OS tells us were accepted OK but they might not actually be delivered to the target endpoint right away - they might be buffered by the OS. The client only knows that the remote server is down when it gets an error from the operating system. Since no error is thrown there is no way for the SocketAppender to know it wasn't successfully sent. The only way to fix this is to use a protocol that sends acknowledgements, such as what Flume does. If you need guaranteed delivery of events you have to use something like Flume or Kafka. > SocketAppender will lose several events after re-connection to server > --------------------------------------------------------------------- > > Key: LOG4J2-1311 > URL: https://issues.apache.org/jira/browse/LOG4J2-1311 > Project: Log4j 2 > Issue Type: Bug > Components: Appenders > Affects Versions: 2.5 > Environment: OS: Ubuntu 14.04 LTS > Reporter: Xibing Liang > Priority: Major > Fix For: 2.9.0 > > > -------> Firstly, I'd like to describe some conditions of our environment: > 1) Our application uses SocketAppender(TCP) for audit logs and > SocketAppender(UDP) for system logs and the problem occurs for the TCP case. > 2) We have two logstash servers and there is one HAProxy > Server(172.16.64.236) for load balance usage between one application > server(10.111.131.28) and two logstash servers(172.16.64.33 and 172.16.64.3). > 3) Haproxy Server will break the connection if the connection is idle for > several minutes, e.g. 10 minutes as we set. > 4) The content of log4j2.properties(You can only focus on the TCP > SocketAppender part): > {code} > status=TRACE > name=TestOneLogConfig > > property.filename=/var/log/test-one-log/test-one-log.log > > #filters=threshold > > #filter.threshold.type=ThresholdFilter > #filter.threshold.level=debug > > appenders=console, rolling, syslog, auditlog > > appender.console.type=Console > appender.console.name=STDOUT > appender.console.layout.type=PatternLayout > appender.console.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %p %c:%L - %m%n > > appender.rolling.type=RollingFile > appender.rolling.name=RollingFile > appender.rolling.fileName=${filename} > appender.rolling.filePattern=/var/log/test-one-log/test-one-log-%d{MM-dd-yy-HH-mm-ss}-%i.log.gz > appender.rolling.layout.type=PatternLayout > appender.rolling.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %p %c:%L - %m%n > appender.rolling.policies.type=Policies > appender.rolling.policies.time.type=TimeBasedTriggeringPolicy > appender.rolling.policies.time.interval=2 > appender.rolling.policies.time.modulate=true > appender.rolling.policies.size.type=SizeBasedTriggeringPolicy > appender.rolling.policies.size.size=100MB > appender.rolling.strategy.type=DefaultRolloverStrategy > appender.rolling.strategy.max=5 > appender.syslog.type=Socket > appender.syslog.name=UDPSocket > appender.syslog.layout.type=JsonLayout > appender.syslog.layout.locationInfo=true > appender.syslog.layout.properties=true > appender.syslog.protocol=UDP > appender.syslog.host=172.16.64.3 > appender.syslog.port=4562 > appender.auditlog.type=Socket > appender.auditlog.name=TCPSocket > appender.auditlog.layout.type=JsonLayout > appender.auditlog.layout.locationInfo=true > appender.auditlog.layout.properties=true > appender.auditlog.protocol=TCP > appender.auditlog.host=172.16.64.236 > appender.auditlog.port=4561 > appender.auditlog.connectTimeoutMillis=3000 > appender.auditlog.reconnectionDelayMillis=3000 > appender.auditlog.immediateFail=true > appender.auditlog.immediateFlush=false > appender.auditlog.ignoreExceptions=false > #appender.asyncauditlog.type=Async > #appender.asyncauditlog.name=AsyncTCPSocket > #appender.asyncauditlog.blocking=false > #appender.asyncauditlog.bufferSize=2048 > #appender.asyncauditlog.ignoreExceptions=false > #appender.asyncauditlog.asyncaudit.type=AppenderRef > #appender.asyncauditlog.asyncaudit.ref=TCPSocket > loggers=auditlogger > > logger.auditlogger.name=auditlogger > logger.auditlogger.level=debug > #logger.auditlogger.additivity=false > logger.auditlogger.appenderRefs=auditlog > logger.auditlogger.appenderRef.auditlog.ref=TCPSocket > > rootLogger.level=debug > rootLogger.appenderRefs=console, rolling, syslog > rootLogger.appenderRef.console.ref=STDOUT > rootLogger.appenderRef.rolling.ref=RollingFile > rootLogger.appenderRef.syslog.ref=UDPSocket > {code} > -------> Secondly, the process/steps of meeting problem: > 1) Logstash servers, HAProxy servers and our application servers are up, and > the application servers send audit logs to haproxy and logstash successfully. > 2) As we didn't do any operation, there is no audit logs come into haproxy in > several minutes, and haproxy break the connection between it and application > server. I use "netstat -antp" to check tcp connection states and found that: > from the harpoxy server's pov, there is no connection to its 4561 tcp port > and from the application server's pov, the connection to haproxy's 4561 port > is CLOSE_WAIT > 3) We did a operation so that application server will write one audit log. > However nothing happen, SocketAppender didn't throw any exception. And I can > see that previous CLOSE_WAIT connection is also gone. This audit log didn't > come into haproxy and logstash at all. > 4) We did a second operation so that application server will write the second > audit log. However now SocketAppender throw a exception of and our > application catches it and write the system log about it. This audit log > didn't come into haproxy and logstash either. > 5) We did a third operation so that application server will write the third > audit log. This time SocketAppender works well and this audit log came into > haproxy and logstash. > -------> So our question is that: > 1) What's the reason of this situation, do you have any idea? > 2) Is there any solution to write the first and second audit log to log > server finally? The audit log is really importance for us. > P.S. > Here I paste the logs of console(from eclipse), you can see there are > serveral prints of "DEBUG auditlogger:57 - LXB 0.0.0.0 xClone roles > successfully" after re-connection, however logstash only receive the last one: > {code} > 三月 09, 2016 4:58:01 下午 org.apache.catalina.core.AprLifecycleListener init > 信息: The APR based Apache Tomcat Native library which allows optimal > performance in production environments was not found on the > java.library.path: > /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > 三月 09, 2016 4:58:01 下午 org.apache.tomcat.util.digester.SetPropertiesRule begin > 警告: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property > 'source' to 'org.eclipse.jst.j2ee.server:test-one-log' did not find a > matching property. > 三月 09, 2016 4:58:02 下午 org.apache.coyote.AbstractProtocol init > 信息: Initializing ProtocolHandler ["http-bio-8080"] > 三月 09, 2016 4:58:02 下午 org.apache.coyote.AbstractProtocol init > 信息: Initializing ProtocolHandler ["ajp-bio-8009"] > 三月 09, 2016 4:58:02 下午 org.apache.catalina.startup.Catalina load > 信息: Initialization processed in 1050 ms > 三月 09, 2016 4:58:02 下午 org.apache.catalina.core.StandardService startInternal > 信息: Starting service Catalina > 三月 09, 2016 4:58:02 下午 org.apache.catalina.core.StandardEngine startInternal > 信息: Starting Servlet Engine: Apache Tomcat/7.0.47 > 三月 09, 2016 4:58:03 下午 org.apache.catalina.loader.WebappClassLoader > validateJarFile > 信息: > validateJarFile(/home/xibing-liang/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp2/wtpwebapps/test-one-log/WEB-INF/lib/javax.servlet-api-3.1.0.jar) > - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: > javax/servlet/Servlet.class > 三月 09, 2016 4:58:06 下午 org.apache.catalina.core.ApplicationContext log > 信息: No Spring WebApplicationInitializer types detected on classpath > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". > SLF4J: Defaulting to no-operation (NOP) logger implementation > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further > details. > 三月 09, 2016 4:58:06 下午 org.apache.catalina.core.ApplicationContext log > 信息: Initializing Spring root WebApplicationContext > 三月 09, 2016 4:58:07 下午 org.apache.catalina.core.ApplicationContext log > 信息: Initializing Spring FrameworkServlet 'mvc-dispatcher' > 2016-03-09 16:58:07,694 localhost-startStop-1 DEBUG Initializing > configuration > org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@574c1328 > 2016-03-09 16:58:07,705 localhost-startStop-1 DEBUG Installed script engines > 2016-03-09 16:58:07,717 localhost-startStop-1 DEBUG Mozilla Rhino Version: > 1.7 release 3 PRERELEASE, Language: ECMAScript, Threading: MULTITHREADED, > Compile: true, Names: {js, rhino, JavaScript, javascript, ECMAScript, > ecmascript} > 2016-03-09 16:58:07,718 localhost-startStop-1 DEBUG PluginManager 'Core' > found 97 plugins > 2016-03-09 16:58:07,718 localhost-startStop-1 DEBUG PluginManager 'Level' > found 0 plugins > 2016-03-09 16:58:07,723 localhost-startStop-1 DEBUG No scheduled items > 2016-03-09 16:58:07,724 localhost-startStop-1 DEBUG Building > Plugin[name=property, class=org.apache.logging.log4j.core.config.Property]. > 2016-03-09 16:58:07,740 localhost-startStop-1 TRACE TypeConverterRegistry > initializing. > 2016-03-09 16:58:07,740 localhost-startStop-1 DEBUG PluginManager > 'TypeConverter' found 23 plugins > 2016-03-09 16:58:07,749 localhost-startStop-1 DEBUG > createProperty(name="filename", > value="/var/log/test-one-log/test-one-log.log") > 2016-03-09 16:58:07,750 localhost-startStop-1 DEBUG Building > Plugin[name=properties, > class=org.apache.logging.log4j.core.config.PropertiesPlugin]. > 2016-03-09 16:58:07,770 localhost-startStop-1 DEBUG > configureSubstitutor(={filename=/var/log/test-one-log/test-one-log.log}, > Configuration(TestOneLogConfig)) > 2016-03-09 16:58:07,771 localhost-startStop-1 DEBUG PluginManager 'Lookup' > found 13 plugins > 2016-03-09 16:58:07,771 localhost-startStop-1 DEBUG Building > Plugin[name=AppenderRef, > class=org.apache.logging.log4j.core.config.AppenderRef]. > 2016-03-09 16:58:07,772 localhost-startStop-1 DEBUG > createAppenderRef(ref="TCPSocket", level="null", Filter=null) > 2016-03-09 16:58:07,773 localhost-startStop-1 DEBUG Building > Plugin[name=logger, class=org.apache.logging.log4j.core.config.LoggerConfig]. > 2016-03-09 16:58:07,776 localhost-startStop-1 DEBUG > createLogger(additivity="null", level="DEBUG", name="auditlogger", > includeLocation="null", ={TCPSocket}, ={}, Configuration(TestOneLogConfig), > Filter=null) > 2016-03-09 16:58:07,779 localhost-startStop-1 DEBUG Building > Plugin[name=AppenderRef, > class=org.apache.logging.log4j.core.config.AppenderRef]. > 2016-03-09 16:58:07,780 localhost-startStop-1 DEBUG > createAppenderRef(ref="STDOUT", level="null", Filter=null) > 2016-03-09 16:58:07,780 localhost-startStop-1 DEBUG Building > Plugin[name=AppenderRef, > class=org.apache.logging.log4j.core.config.AppenderRef]. > 2016-03-09 16:58:07,783 localhost-startStop-1 DEBUG > createAppenderRef(ref="RollingFile", level="null", Filter=null) > 2016-03-09 16:58:07,783 localhost-startStop-1 DEBUG Building > Plugin[name=AppenderRef, > class=org.apache.logging.log4j.core.config.AppenderRef]. > 2016-03-09 16:58:07,784 localhost-startStop-1 DEBUG > createAppenderRef(ref="UDPSocket", level="null", Filter=null) > 2016-03-09 16:58:07,784 localhost-startStop-1 DEBUG Building > Plugin[name=root, > class=org.apache.logging.log4j.core.config.LoggerConfig$RootLogger]. > 2016-03-09 16:58:07,786 localhost-startStop-1 DEBUG > createLogger(additivity="null", level="DEBUG", includeLocation="null", > ={STDOUT, RollingFile, UDPSocket}, ={}, Configuration(TestOneLogConfig), > Filter=null) > 2016-03-09 16:58:07,787 localhost-startStop-1 DEBUG Building > Plugin[name=loggers, > class=org.apache.logging.log4j.core.config.LoggersPlugin]. > 2016-03-09 16:58:07,788 localhost-startStop-1 DEBUG > createLoggers(={auditlogger, root}) > 2016-03-09 16:58:07,788 localhost-startStop-1 DEBUG Building > Plugin[name=layout, class=org.apache.logging.log4j.core.layout.PatternLayout]. > 2016-03-09 16:58:07,821 localhost-startStop-1 DEBUG > PatternLayout$Builder(pattern="%d{yyyy-MM-dd HH:mm:ss} [%t] %p %c:%L - %m%n", > PatternSelector=null, Configuration(TestOneLogConfig), Replace=null, > charset="null", alwaysWriteExceptions="null", noConsoleNoAnsi="null", > header="null", footer="null") > 2016-03-09 16:58:07,822 localhost-startStop-1 DEBUG PluginManager 'Converter' > found 38 plugins > 2016-03-09 16:58:07,887 localhost-startStop-1 DEBUG Building > Plugin[name=appender, > class=org.apache.logging.log4j.core.appender.ConsoleAppender]. > 2016-03-09 16:58:07,895 localhost-startStop-1 INFO Log4j appears to be > running in a Servlet environment, but there's no log4j-web module available. > If you want better web container support, please add the log4j-web JAR to > your web archive or server lib directory. > 2016-03-09 16:58:07,896 localhost-startStop-1 DEBUG PluginManager 'Converter' > found 38 plugins > 2016-03-09 16:58:07,899 localhost-startStop-1 DEBUG Starting > OutputStreamManager SYSTEM_OUT.false-2 > 2016-03-09 16:58:07,930 localhost-startStop-1 DEBUG > ConsoleAppender$Builder(PatternLayout(%d{yyyy-MM-dd HH:mm:ss} [%t] %p %c:%L - > %m%n), Filter=null, target="null", name="STDOUT", follow="null", > ignoreExceptions="null") > 2016-03-09 16:58:07,938 localhost-startStop-1 DEBUG Starting > OutputStreamManager SYSTEM_OUT.false > 2016-03-09 16:58:07,940 localhost-startStop-1 DEBUG Building > Plugin[name=layout, class=org.apache.logging.log4j.core.layout.PatternLayout]. > 2016-03-09 16:58:07,942 localhost-startStop-1 DEBUG > PatternLayout$Builder(pattern="%d{yyyy-MM-dd HH:mm:ss} [%t] %p %c:%L - %m%n", > PatternSelector=null, Configuration(TestOneLogConfig), Replace=null, > charset="null", alwaysWriteExceptions="null", noConsoleNoAnsi="null", > header="null", footer="null") > 2016-03-09 16:58:07,943 localhost-startStop-1 DEBUG Building > Plugin[name=TimeBasedTriggeringPolicy, > class=org.apache.logging.log4j.core.appender.rolling.TimeBasedTriggeringPolicy]. > 2016-03-09 16:58:07,946 localhost-startStop-1 DEBUG > createPolicy(interval="2", modulate="true") > 2016-03-09 16:58:07,947 localhost-startStop-1 DEBUG Building > Plugin[name=SizeBasedTriggeringPolicy, > class=org.apache.logging.log4j.core.appender.rolling.SizeBasedTriggeringPolicy]. > 2016-03-09 16:58:07,951 localhost-startStop-1 DEBUG createPolicy(size="100MB") > 2016-03-09 16:58:07,955 localhost-startStop-1 DEBUG Building > Plugin[name=Policies, > class=org.apache.logging.log4j.core.appender.rolling.CompositeTriggeringPolicy]. > 2016-03-09 16:58:07,963 localhost-startStop-1 DEBUG > createPolicy(={TimeBasedTriggeringPolicy(nextRolloverMillis=0, interval=2, > modulate=true), SizeBasedTriggeringPolicy(size=104857600)}) > 2016-03-09 16:58:07,968 localhost-startStop-1 DEBUG Building > Plugin[name=DefaultRolloverStrategy, > class=org.apache.logging.log4j.core.appender.rolling.DefaultRolloverStrategy]. > 2016-03-09 16:58:07,973 localhost-startStop-1 DEBUG createStrategy(max="5", > min="null", fileIndex="null", compressionLevel="null", ={}, > stopCustomActionsOnError="true", Configuration(TestOneLogConfig)) > 2016-03-09 16:58:07,975 localhost-startStop-1 DEBUG Building > Plugin[name=appender, > class=org.apache.logging.log4j.core.appender.RollingFileAppender]. > 2016-03-09 16:58:07,980 localhost-startStop-1 DEBUG > createAppender(fileName="/var/log/test-one-log/test-one-log.log", > filePattern="/var/log/test-one-log/test-one-log-%d{MM-dd-yy-HH-mm-ss}-%i.log.gz", > append="null", name="RollingFile", bufferedIO="null", bufferSize="null", > immediateFlush="null", > Policies(CompositeTriggeringPolicy(policies=[TimeBasedTriggeringPolicy(nextRolloverMillis=0, > interval=2, modulate=true), SizeBasedTriggeringPolicy(size=104857600)])), > DefaultRolloverStrategy(DefaultRolloverStrategy(min=1, max=5)), > PatternLayout(%d{yyyy-MM-dd HH:mm:ss} [%t] %p %c:%L - %m%n), Filter=null, > ignoreExceptions="null", advertise="null", advertiseURI="null", > Configuration(TestOneLogConfig)) > 2016-03-09 16:58:07,990 localhost-startStop-1 DEBUG Starting > RollingFileManager /var/log/test-one-log/test-one-log.log > 2016-03-09 16:58:07,994 localhost-startStop-1 DEBUG PluginManager > 'FileConverter' found 2 plugins > 2016-03-09 16:58:07,997 localhost-startStop-1 TRACE > PatternProcessor.getNextTime returning 2016/03/09-16:45:52.000, > nextFileTime=2016/03/09-16:45:51.000, prevFileTime=1970/01/01-08:00:00.000, > current=2016/03/09-16:58:07.997, freq=EVERY_SECOND > 2016-03-09 16:58:08,003 localhost-startStop-1 TRACE > PatternProcessor.getNextTime returning 2016/03/09-16:45:52.000, > nextFileTime=2016/03/09-16:45:51.000, prevFileTime=2016/03/09-16:45:51.000, > current=2016/03/09-16:58:08.002, freq=EVERY_SECOND > 2016-03-09 16:58:08,003 localhost-startStop-1 DEBUG Building > Plugin[name=layout, class=org.apache.logging.log4j.core.layout.JsonLayout]. > 2016-03-09 16:58:08,012 localhost-startStop-1 DEBUG > createLayout(locationInfo="true", properties="true", complete="false", > compact="false", eventEol="false", charset="UTF-8") > 2016-03-09 16:58:08,301 localhost-startStop-1 DEBUG Building > Plugin[name=appender, > class=org.apache.logging.log4j.core.appender.SocketAppender]. > 2016-03-09 16:58:08,308 localhost-startStop-1 DEBUG > createAppender(host="172.16.64.3", port="4562", protocol="UDP", SSL=null, > connectTimeoutMillis="0", reconnectionDelayMillis="0", immediateFail="true", > name="UDPSocket", immediateFlush="true", ignoreExceptions="true", > JsonLayout(org.apache.logging.log4j.core.layout.JsonLayout@35e97fc3), > Filter=null, advertise="false", Configuration(TestOneLogConfig)) > 2016-03-09 16:58:08,311 localhost-startStop-1 DEBUG Starting > DatagramSocketManager UDP:172.16.64.3:4562 > 2016-03-09 16:58:08,312 localhost-startStop-1 DEBUG Building > Plugin[name=layout, class=org.apache.logging.log4j.core.layout.JsonLayout]. > 2016-03-09 16:58:08,313 localhost-startStop-1 DEBUG > createLayout(locationInfo="true", properties="true", complete="false", > compact="false", eventEol="false", charset="UTF-8") > 2016-03-09 16:58:08,313 localhost-startStop-1 DEBUG Building > Plugin[name=appender, > class=org.apache.logging.log4j.core.appender.SocketAppender]. > 2016-03-09 16:58:08,316 localhost-startStop-1 DEBUG > createAppender(host="172.16.64.236", port="4561", protocol="TCP", SSL=null, > connectTimeoutMillis="3000", reconnectionDelayMillis="3000", > immediateFail="true", name="TCPSocket", immediateFlush="false", > ignoreExceptions="false", > JsonLayout(org.apache.logging.log4j.core.layout.JsonLayout@6ab715ca), > Filter=null, advertise="false", Configuration(TestOneLogConfig)) > 2016-03-09 16:58:08,322 localhost-startStop-1 DEBUG Starting TcpSocketManager > TCP:172.16.64.236:4561 > 2016-03-09 16:58:08,323 localhost-startStop-1 DEBUG Building > Plugin[name=appenders, > class=org.apache.logging.log4j.core.config.AppendersPlugin]. > 2016-03-09 16:58:08,323 localhost-startStop-1 DEBUG createAppenders(={STDOUT, > RollingFile, UDPSocket, TCPSocket}) > 2016-03-09 16:58:08,324 localhost-startStop-1 DEBUG Configuration > org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@574c1328 > initialized > 2016-03-09 16:58:08,325 localhost-startStop-1 DEBUG Starting configuration > org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@574c1328 > 2016-03-09 16:58:08,326 localhost-startStop-1 DEBUG Started configuration > org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@574c1328 > OK. > 2016-03-09 16:58:08,326 localhost-startStop-1 TRACE Stopping > org.apache.logging.log4j.core.config.DefaultConfiguration@5332203... > 2016-03-09 16:58:08,327 localhost-startStop-1 TRACE DefaultConfiguration > notified 1 ReliabilityStrategies that config will be stopped. > 2016-03-09 16:58:08,327 localhost-startStop-1 TRACE DefaultConfiguration > stopping root LoggerConfig. > 2016-03-09 16:58:08,327 localhost-startStop-1 TRACE DefaultConfiguration > notifying ReliabilityStrategies that appenders will be stopped. > 2016-03-09 16:58:08,327 localhost-startStop-1 TRACE DefaultConfiguration > stopping remaining Appenders. > 2016-03-09 16:58:08,328 localhost-startStop-1 DEBUG Shutting down > OutputStreamManager SYSTEM_OUT.false-1 > 2016-03-09 16:58:08,328 localhost-startStop-1 TRACE DefaultConfiguration > stopped 1 remaining Appenders. > 2016-03-09 16:58:08,328 localhost-startStop-1 TRACE DefaultConfiguration > cleaning Appenders from 1 LoggerConfigs. > 2016-03-09 16:58:08,329 localhost-startStop-1 DEBUG Stopped > org.apache.logging.log4j.core.config.DefaultConfiguration@5332203 OK > 2016-03-09 16:58:08,339 localhost-startStop-1 TRACE Reregistering MBeans > after reconfigure. > Selector=org.apache.logging.log4j.core.selector.ClassLoaderContextSelector@769c48b > 2016-03-09 16:58:08,339 localhost-startStop-1 TRACE Reregistering context > (1/1): '602fc7b3' org.apache.logging.log4j.core.LoggerContext@30e6857a > 2016-03-09 16:58:08,340 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching 'org.apache.logging.log4j2:type=602fc7b3' > 2016-03-09 16:58:08,340 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=StatusLogger' > 2016-03-09 16:58:08,341 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=ContextSelector' > 2016-03-09 16:58:08,341 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=Loggers,name=*' > 2016-03-09 16:58:08,342 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=Appenders,name=*' > 2016-03-09 16:58:08,342 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=AsyncAppenders,name=*' > 2016-03-09 16:58:08,342 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=AsyncLoggerRingBuffer' > 2016-03-09 16:58:08,343 localhost-startStop-1 TRACE Unregistering but no > MBeans found matching > 'org.apache.logging.log4j2:type=602fc7b3,component=Loggers,name=*,subtype=RingBuffer' > 2016-03-09 16:58:08,344 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3 > 2016-03-09 16:58:08,350 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=StatusLogger > 2016-03-09 16:58:08,352 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=ContextSelector > 2016-03-09 16:58:08,354 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=Loggers,name= > 2016-03-09 16:58:08,355 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=Loggers,name=auditlogger > 2016-03-09 16:58:08,356 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=Appenders,name=UDPSocket > 2016-03-09 16:58:08,357 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=Appenders,name=RollingFile > 2016-03-09 16:58:08,357 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=Appenders,name=STDOUT > 2016-03-09 16:58:08,357 localhost-startStop-1 DEBUG Registering MBean > org.apache.logging.log4j2:type=602fc7b3,component=Appenders,name=TCPSocket > 2016-03-09 16:58:08,359 localhost-startStop-1 TRACE Using default SystemClock > for timestamps. > 2016-03-09 16:58:08,360 localhost-startStop-1 TRACE Using DummyNanoClock for > nanosecond timestamps. > 2016-03-09 16:58:08,370 localhost-startStop-1 DEBUG Reconfiguration complete > for context[name=602fc7b3] at URI NULL_SOURCE > (org.apache.logging.log4j.core.LoggerContext@30e6857a) with optional > ClassLoader: null > 2016-03-09 16:58:08,370 localhost-startStop-1 DEBUG Shutdown hook enabled. > Registering a new one. > 2016-03-09 16:58:08,371 localhost-startStop-1 DEBUG > LoggerContext[name=602fc7b3, > org.apache.logging.log4j.core.LoggerContext@30e6857a] started OK. > 三月 09, 2016 4:58:08 下午 org.hibernate.validator.internal.util.Version <clinit> > INFO: HV000001: Hibernate Validator 5.1.3.Final > 三月 09, 2016 4:58:09 下午 org.apache.coyote.AbstractProtocol start > 信息: Starting ProtocolHandler ["http-bio-8080"] > 三月 09, 2016 4:58:09 下午 org.apache.coyote.AbstractProtocol start > 信息: Starting ProtocolHandler ["ajp-bio-8009"] > 三月 09, 2016 4:58:09 下午 org.apache.catalina.startup.Catalina start > 信息: Server startup in 7725 ms > 2016-03-09 17:00:33 [http-bio-8080-exec-5] DEBUG > www.chinacloud.com.api.ProxyController:55 - roles xClone > 2016-03-09 17:00:33,022 http-bio-8080-exec-5 TRACE > PatternProcessor.getNextTime returning 2016/03/09-17:00:34.000, > nextFileTime=2016/03/09-17:00:33.000, prevFileTime=2016/03/09-16:45:51.000, > current=2016/03/09-17:00:33.022, freq=EVERY_SECOND > 2016-03-09 17:00:33,026 http-bio-8080-exec-5 TRACE > DefaultRolloverStrategy.purge() took 3.0 milliseconds > 2016-03-09 17:00:33,029 http-bio-8080-exec-5 DEBUG RollingFileManager > executing synchronous FileRenameAction[/var/log/test-one-log/test-one-log.log > to /var/log/test-one-log/test-one-log-03-09-16-16-45-51-1.log, > renameEmptyFiles=false] > 2016-03-09 17:00:33,029 http-bio-8080-exec-5 DEBUG RollingFileManager > executing async > GzCompressAction[/var/log/test-one-log/test-one-log-03-09-16-16-45-51-1.log > to /var/log/test-one-log/test-one-log-03-09-16-16-45-51-1.log.gz, > deleteSource=true] > 2016-03-09 17:00:33 [http-bio-8080-exec-5] DEBUG auditlogger:57 - LXB 0.0.0.0 > xClone roles successfully > PHPSESSID=kuk0kaclb0ogi2b9huei4hel02; expires=Wed, 09-Mar-2016 09:30:39 GMT; > Max-Age=1800; path=/; HttpOnly > 2016-03-09 17:14:03 [http-bio-8080-exec-7] DEBUG > www.chinacloud.com.api.ProxyController:55 - roles xClone > 2016-03-09 17:14:03,015 http-bio-8080-exec-7 TRACE > PatternProcessor.getNextTime returning 2016/03/09-17:14:04.000, > nextFileTime=2016/03/09-17:14:03.000, prevFileTime=2016/03/09-17:00:33.000, > current=2016/03/09-17:14:03.013, freq=EVERY_SECOND > 2016-03-09 17:14:03,017 http-bio-8080-exec-7 TRACE > DefaultRolloverStrategy.purge() took 1.0 milliseconds > 2016-03-09 17:14:03,017 http-bio-8080-exec-7 DEBUG RollingFileManager > executing synchronous FileRenameAction[/var/log/test-one-log/test-one-log.log > to /var/log/test-one-log/test-one-log-03-09-16-17-00-33-1.log, > renameEmptyFiles=false] > 2016-03-09 17:14:03,018 http-bio-8080-exec-7 DEBUG RollingFileManager > executing async > GzCompressAction[/var/log/test-one-log/test-one-log-03-09-16-17-00-33-1.log > to /var/log/test-one-log/test-one-log-03-09-16-17-00-33-1.log.gz, > deleteSource=true] > 2016-03-09 17:14:03 [http-bio-8080-exec-7] DEBUG auditlogger:57 - LXB 0.0.0.0 > xClone roles successfully > PHPSESSID=kuk0kaclb0ogi2b9huei4hel02; expires=Wed, 09-Mar-2016 09:44:08 GMT; > Max-Age=1800; path=/; HttpOnly > 2016-03-09 17:14:45 [http-bio-8080-exec-9] DEBUG > www.chinacloud.com.api.ProxyController:55 - roles xClone > 2016-03-09 17:14:45,034 http-bio-8080-exec-9 TRACE > PatternProcessor.getNextTime returning 2016/03/09-17:14:46.000, > nextFileTime=2016/03/09-17:14:45.000, prevFileTime=2016/03/09-17:14:03.000, > current=2016/03/09-17:14:45.034, freq=EVERY_SECOND > 2016-03-09 17:14:45,035 http-bio-8080-exec-9 TRACE > DefaultRolloverStrategy.purge() took 0.0 milliseconds > 2016-03-09 17:14:45,035 http-bio-8080-exec-9 DEBUG RollingFileManager > executing synchronous FileRenameAction[/var/log/test-one-log/test-one-log.log > to /var/log/test-one-log/test-one-log-03-09-16-17-14-03-1.log, > renameEmptyFiles=false] > 2016-03-09 17:14:45,035 http-bio-8080-exec-9 DEBUG RollingFileManager > executing async > GzCompressAction[/var/log/test-one-log/test-one-log-03-09-16-17-14-03-1.log > to /var/log/test-one-log/test-one-log-03-09-16-17-14-03-1.log.gz, > deleteSource=true] > 2016-03-09 17:14:45,048 http-bio-8080-exec-9 ERROR Unable to write to stream > TCP:172.16.64.236:4561 for appender TCPSocket > 2016-03-09 17:14:45,048 http-bio-8080-exec-9 ERROR An exception occurred > processing Appender TCPSocket > org.apache.logging.log4j.core.appender.AppenderLoggingException: Error > writing to TCP:172.16.64.236:4561 > at > org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:140) > at > org.apache.logging.log4j.core.appender.OutputStreamManager.write(OutputStreamManager.java:136) > at > org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:114) > at > org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:152) > at > org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:125) > at > org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:116) > at > org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) > at > org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:390) > at > org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:378) > at > org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:362) > at > org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:352) > at > org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) > at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:147) > at > org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:1022) > at > org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:980) > at > org.apache.logging.log4j.spi.AbstractLogger.debug(AbstractLogger.java:239) > at www.chinacloud.com.api.ProxyController.post(ProxyController.java:57) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) > at > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at www.chinacloud.com.filter.CORSFilter.doFilter(CORSFilter.java:28) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.net.SocketException: 断开的管道 > at java.net.SocketOutputStream.socketWrite0(Native Method) > at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) > at java.net.SocketOutputStream.write(SocketOutputStream.java:159) > at > org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:131) > ... 55 more > 2016-03-09 17:14:45 [http-bio-8080-exec-9] ERROR > www.chinacloud.com.api.ProxyController:61 - LXB Audit Error: Error writing to > TCP:172.16.64.236:4561 > PHPSESSID=kuk0kaclb0ogi2b9huei4hel02; expires=Wed, 09-Mar-2016 09:44:51 GMT; > Max-Age=1800; path=/; HttpOnly > 2016-03-09 17:14:48,095 Thread-3 DEBUG Connection to 172.16.64.236:4561 > reestablished. > 2016-03-09 17:15:39 [http-bio-8080-exec-1] DEBUG > www.chinacloud.com.api.ProxyController:55 - roles xClone > 2016-03-09 17:15:39,434 http-bio-8080-exec-1 TRACE > PatternProcessor.getNextTime returning 2016/03/09-17:15:40.000, > nextFileTime=2016/03/09-17:15:39.000, prevFileTime=2016/03/09-17:14:45.000, > current=2016/03/09-17:15:39.434, freq=EVERY_SECOND > 2016-03-09 17:15:39,435 http-bio-8080-exec-1 TRACE > DefaultRolloverStrategy.purge() took 0.0 milliseconds > 2016-03-09 17:15:39,436 http-bio-8080-exec-1 DEBUG RollingFileManager > executing synchronous FileRenameAction[/var/log/test-one-log/test-one-log.log > to /var/log/test-one-log/test-one-log-03-09-16-17-14-45-1.log, > renameEmptyFiles=false] > 2016-03-09 17:15:39,436 http-bio-8080-exec-1 DEBUG RollingFileManager > executing async > GzCompressAction[/var/log/test-one-log/test-one-log-03-09-16-17-14-45-1.log > to /var/log/test-one-log/test-one-log-03-09-16-17-14-45-1.log.gz, > deleteSource=true] > 2016-03-09 17:15:39 [http-bio-8080-exec-1] DEBUG auditlogger:57 - LXB 0.0.0.0 > xClone roles successfully > PHPSESSID=kuk0kaclb0ogi2b9huei4hel02; expires=Wed, 09-Mar-2016 09:45:45 GMT; > Max-Age=1800; path=/; HttpOnly > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)