All,
Summary
-------
While trying to align the documentation for maxConnections with the
actual implementation a couple of performance issue have been identified
with the BIO connector. All bar one of these have been fixed. A fix
needs to be agreed for the remaining issue, ideally before the next
7.0.x release.
Background
----------
New connections to the BIO connector are placed in a queue. When a
thread is available, it takes the next connection of the queue and
processes it. When the connection has been processed if the connection
is kept-alive it is placed back on the queue. If there is no keep-alive
the connection is closed.
Scenario
--------
This ended up being very long, so I moved it to the end. The exact
pattern of delays will vary depending on timeouts, request frequency
etc. but the scenario shows an example of how delays can occur. The
short version is that requests with data to process (particularly new
connections) tend to get delayed in the queue waiting for a thread to
process them when the threads are all tied up processing keep-alive
connections.
Root cause
----------
The underlying cause of all of the performance issues observed is when
the threads are tied up doing HTTP keep-alive when there is no data
process but there are other connections in the queue that do have data
that could be processed.
Solution A
----------
NIO is designed to handle this using a poller. That isn't available to
BIO so I attempted to simulate it. That generated excessive CPU load so
I do not think simulated polling is the tight solution.
Solution B
----------
Return to the Tomcat 6 implementation where maxConnections == maxThreads.
Additional clean-up
-------------------
maxConnections is unnecessary in APR since pollerSize performs the same
function.
Summary
-------
The proposed changes are:
a) restore disabling keep-alive when threads used >= 75% of maxThreads
b) remove maxConnections and associated code from the APR connector
c) remove the configuration options for maxConnections from the BIO
connector
d) use maxThreads instead of maxConnections for the BIO connector
e) update the docs
Mark
Scenario
--------
BIO connector, default configuration
- maxThreads=200
- maxConnections=10000
- keepAliveTimeout=connectionTimeout=20000 (20s)
When there are less than maxThreads current connections, no issues are
observed.
Consider the following scenario:
- Start from zero load.
- Clients make 1 request every 5 seconds and use HTTP keep-alive
- Clients ramp up from 0 to 1000 at the rate of 100 new clients every
second. I'll call each block of 100 clients group A, B, C, ..., J.
- Assume request processing takes 0s just to keep the numbers simple.
I've tried several ways to express this. This is verbose, but hopefully
clear. There may be some mistakes in the detail but the overall result
is consistent with observations of some JMeter test runs.
Time: 0s
Activity: None
Queue: None
Threads: None
Group A: Inactive
Group B: Inactive
Group C: Inactive
Group D: Inactive
Group E: Inactive
Group F: Inactive
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 1s
Activity: Group A clients connect and are added to the queue
100 threads are allocated to process the A connections
The A connections are processed and returned to the queue
100 threads are allocated to process the A connections
Queue: None
Threads: A
Group A: Allocated threads. In HTTP keep-alive for 0s
Group B: Inactive
Group C: Inactive
Group D: Inactive
Group E: Inactive
Group F: Inactive
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 2s
Activity: Group B clients connect and are added to the queue
100 threads are allocated to process the B connections
The B connections are processed and returned to the queue
100 threads are allocated to process the B connections
Queue: None
Threads: A, B
Group A: Allocated threads. In HTTP keep-alive for 1s
Group B: Allocated threads. In HTTP keep-alive for 0s
Group C: Inactive
Group D: Inactive
Group E: Inactive
Group F: Inactive
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 3s
Activity: Group C clients connect and are added to the queue
Queue: C
Threads: A, B
Group A: Allocated threads. In HTTP keep-alive for 2s
Group B: Allocated threads. In HTTP keep-alive for 1s
Group C: Queued for 0s (no threads available)
Group D: Inactive
Group E: Inactive
Group F: Inactive
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 4s
Activity: Group D clients connect and are added to the queue
Queue: C, D
Threads: A, B
Group A: Allocated threads. In HTTP keep-alive for 3s
Group B: Allocated threads. In HTTP keep-alive for 2s
Group C: Queued for 1s
Group D: Queued for 0s
Group E: Inactive
Group F: Inactive
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 5s
Activity: Group E clients connect and are added to the queue
Queue: C, D, E
Threads: A, B
Group A: Allocated threads. In HTTP keep-alive for 4s
Group B: Allocated threads. In HTTP keep-alive for 3s
Group C: Queued for 2s
Group D: Queued for 1s
Group E: Queued for 0s
Group F: Inactive
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 6s
Activity: Group F clients connect and are added to the queue
A connections send requests
A connections are processed and returned to the queue
C connections are processed and returned to the queue
D connections are processed and returned to the queue
E connections are processed and returned to the queue
F connections are processed and returned to the queue
A connections are allocated threads but are in keep-alive
Queue: C, D, E, F
Threads: A, B
Group A: Allocated threads. In HTTP keep-alive for 0s
Group B: Allocated threads. In HTTP keep-alive for 4s
Group C: Queued. In HTTP keep-alive for 0s
Group D: Queued. In HTTP keep-alive for 0s
Group E: Queued. In HTTP keep-alive for 0s
Group F: Queued. In HTTP keep-alive for 0s
Group G: Inactive
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 7s
Activity: Group G clients connect and are added to the queue
B connections send requests
B connections are processed and returned to the queue
C connections are allocated threads but are in keep-alive
Queue: D, E, F, G, B
Threads: C, A
Group A: Allocated threads. In HTTP keep-alive for 1s
Group B: Queued. In HTTP keep-alive for 0s
Group C: Allocated threads. In HTTP keep-alive for 1s
Group D: Queued. In HTTP keep-alive for 1s
Group E: Queued. In HTTP keep-alive for 1s
Group F: Queued. In HTTP keep-alive for 1s
Group G: Queued for 0s
Group H: Inactive
Group I: Inactive
Group J: Inactive
Time: 8s
Activity: Group H clients connect and are added to the queue
Queue: D, E, F, G, B, H
Threads: C, A
Group A: Allocated threads. In HTTP keep-alive for 2s
Group B: Queued. In HTTP keep-alive for 1s
Group C: Allocated threads. In HTTP keep-alive for 2s
Group D: Queued. In HTTP keep-alive for 2s
Group E: Queued. In HTTP keep-alive for 2s
Group F: Queued. In HTTP keep-alive for 2s
Group G: Queued for 1s
Group H: Queued for 0s
Group I: Inactive
Group J: Inactive
Time: 9s
Activity: Group I clients connect and are added to the queue
Queue: D, E, F, G, B, H, I
Threads: C, A
Group A: Allocated threads. In HTTP keep-alive for 3s
Group B: Queued. In HTTP keep-alive for for 2s
Group C: Allocated threads. In HTTP keep-alive for 3s
Group D: Queued. In HTTP keep-alive for 3s
Group E: Queued. In HTTP keep-alive for 3s
Group F: Queued. In HTTP keep-alive for 3s
Group G: Queued for 2s
Group H: Queued for 1s
Group I: Queued for 0s
Group J: Inactive
Time: 10s
Activity: Group J clients connect and are added to the queue
Queue: D, E, F, G, B, H, I, J
Threads: C, A
Group A: Allocated threads. In HTTP keep-alive for 4s
Group B: Queued. In HTTP keep-alive for for 3s
Group C: Allocated threads. In HTTP keep-alive for 4s
Group D: Queued. In HTTP keep-alive for 4s
Group E: Queued. In HTTP keep-alive for 4s
Group F: Queued. In HTTP keep-alive for 4s
Group G: Queued for 3s
Group H: Queued for 2s
Group I: Queued for 1s
Group J: Queued for 0s
Time: 11s
Activity: C connections send requests
C connections are processed and returned to the queue
A connections send requests
A connections are processed and returned to the queue
D connections send requests
D connections are processed and returned to the queue
E connections send requests
E connections are processed and returned to the queue
F connections send requests
F connections are processed and returned to the queue
G connections are processed and returned to the queue
B connections are allocated threads but are in keep-alive
H connections are processed and returned to the queue
I connections are processed and returned to the queue
J connections are processed and returned to the queue
C connections are allocated threads but are in keep-alive
Queue: A, D, E, F, G, H, I, J
Threads: B, C
Group A: Queued. In HTTP keep-alive for 0s
Group B: Allocated threads. In HTTP keep-alive for 4s
Group C: Allocated threads. In HTTP keep-alive for 0s
Group D: Queued. In HTTP keep-alive for 0s
Group E: Queued. In HTTP keep-alive for 0s
Group F: Queued. In HTTP keep-alive for 0s
Group G: Queued. In HTTP keep-alive for 0s
Group H: Queued. In HTTP keep-alive for 0s
Group I: Queued. In HTTP keep-alive for 0s
Group J: Queued. In HTTP keep-alive for 0s
The issues are:
At time 6s:
- 3s delay processing C first requests
- 2s delay processing D first requests
- 1s delay processing E first requests
At time 11s:
- 4s delay processing G first requests
- 3s delay processing H first requests
- 2s delay processing I first requests
- 1s delay processing J first requests
The delays are particularly bad for new connections which must wait
behind all the other requests in the queue.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
