carterkozak opened a new pull request #273: LOG4J2-2606: Substantially improve async logging performance under heavy load URL: https://github.com/apache/logging-log4j2/pull/273 This change applies synchronization to AsyncLoggerDisruptor blocking enqueue operations when the asynchronous logging queue is completely full. This new behavior may be disabled using the boolean property `AsyncLogger.SynchronizeEnqueueWhenQueueFull` (default true). Alternatives tested: * All available lmax disruptor `WaitStrategy` implementations. None of the available implementations provided a substantial difference in throughput or CPU utilization. In all cases my processor was stuck at or near 100% across all six cores. * Based on feedback from https://github.com/LMAX-Exchange/disruptor/issues/266 I attempted avoiding the blocking enqueue entirely in favor of `LockSupport.parkNanos` with values ranging from 1,000 through 1,000,000. 1,000,000 provided the most throughput among parkNanos values tested, but is too long for most scenarios. * Semaphore instead of synchronized. Testing with permits equivalent to the available processor count yielded higher CPU utilization and lower throughput than a semaphore with a single permit. Given that most work occurs on the (single) background thread, there's not much reason to allow multiple enqueues simultaneously. Updated benchmarks adding "ENQUEUE_UNSYNCHRONIZED" following the old unsynchronized path, where "ENQUEUE" shows an improvement: ``` Benchmark (queueFullPolicy) Mode Cnt Score Error Units ConcurrentAsyncLoggerToFileBenchmark.concurrentLoggingThreads ENQUEUE thrpt 3 1146649.538 ± 103899.750 ops/s ConcurrentAsyncLoggerToFileBenchmark.concurrentLoggingThreads ENQUEUE_UNSYNCHRONIZED thrpt 3 422244.124 ± 785429.301 ops/s ConcurrentAsyncLoggerToFileBenchmark.concurrentLoggingThreads SYNCHRONOUS thrpt 3 1084417.832 ± 354547.695 ops/s ConcurrentAsyncLoggerToFileBenchmark.singleLoggingThread ENQUEUE thrpt 3 1250748.284 ± 1202507.746 ops/s ConcurrentAsyncLoggerToFileBenchmark.singleLoggingThread ENQUEUE_UNSYNCHRONIZED thrpt 3 1280794.688 ± 832379.969 ops/s ConcurrentAsyncLoggerToFileBenchmark.singleLoggingThread SYNCHRONOUS thrpt 3 1227202.214 ± 1398451.960 ops/s ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
