Barry, Thanks for the detailed response. Very helpful.
On 12/3/20, 4:22 PM, "Barrett Oglesby" wrote:
Anil, you wrote:
- We need to be thinking about auto setting of configuration values
(dynamic) based on the load, resource availability and service agreements.
It would be cool to eventually remove this property altogether and
auto-configure it. Besides the things you mention, another thing that would
need to be considered is features being used. For example, wan requires
conserve-sockets=false. This discussion maybe should be moved to a different
thread so we don't distract from this one.
You also asked:
- Will there be dedicated channel for communication from the node where
conserve-socket is set to false to the remote nodes?
Since the server doing the op has conserve-sockets=false, and an unshared
p2p message reader is used on the remote member that means a dedicated
(thread-owned) connection is used.
ConnectionTable.get decides that. Here is a stack for creating a
thread-owned sender:
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1333)
at org.apache.geode.internal.tcp.Connection.(Connection.java:1224)
at
org.apache.geode.internal.tcp.Connection.createSender(Connection.java:1025)
at
org.apache.geode.internal.tcp.ConnectionTable.getThreadOwnedConnection(ConnectionTable.java:474)
at
org.apache.geode.internal.tcp.ConnectionTable.get(ConnectionTable.java:577)
at
org.apache.geode.internal.tcp.TCPConduit.getConnection(TCPConduit.java:800)
at
org.apache.geode.distributed.internal.direct.DirectChannel.getConnections(DirectChannel.java:452)
at
org.apache.geode.distributed.internal.direct.DirectChannel.sendToMany(DirectChannel.java:268)
at
org.apache.geode.distributed.internal.direct.DirectChannel.sendToOne(DirectChannel.java:182)
at
org.apache.geode.distributed.internal.direct.DirectChannel.send(DirectChannel.java:511)
Here are the same use cases with additional logging containing thread-owned
Connection creation or shared Connection usage:
Case 1:
The server (server1) that processes the put operation from the client is
primary and has conserve-sockets=false.
The server (server2) that handles the UpdateWithContextMessage has
conserve-sockets=true.
1. A ServerConnection thread in server1 sends the UpdateWithContextMessage:
ServerConnection on port 60539 Thread 3: TestDistributionMessageObserver
operation=beforeSendMessage; time=1607039519049;
message=UpdateOperation$UpdateWithContextMessage(...);
recipients=[192.168.1.8(server2:54360):41002]
2. The ServerConnection thread in server1 creates the thread-owned
Connection:
ServerConnection on port 60539 Thread 3: Connection.
sender=192.168.1.8(server2:54360):41002(uid=10);
socket=Socket[addr=/192.168.1.8,port=45823,localport=60595]; time=1607039519050
ServerConnection on port 60539 Thread 3: ConnectionTable.get using
threadOwnedConnection=192.168.1.8(server2:54360):41002(uid=10);
socket=Socket[addr=/192.168.1.8,port=45823,localport=60595]; time=1607039519051
3. A P2P Listener Thread in server2 creates the receiver Connection:
P2P Listener Thread /192.168.1.8:45823: Connection.
receiver=null(uid=0);
socket=Socket[addr=/192.168.1.8,port=60595,localport=45823]; time=1607039519050
4. The unshared P2P message reader in server2 reads the handshake from
server1's Connection:
P2P message reader for 192.168.1.8(server1:54333):41001 unshared
ordered uid=10 dom #1 local port=45823 remote port=60595:
Connection.readHandshakeForReceiver
receiver=192.168.1.8(server1:54333):41001(uid=10);
socket=Socket[addr=/192.168.1.8,port=60595,localport=45823]; time=1607039519050
5. The unshared P2P message reader in server2 handles the
UpdateWithContextMessage even though conserve-sockets=true:
P2P message reader for 192.168.1.8(server1:54333):41001 unshared
ordered uid=10 dom #1 local port=45823 remote port=60595:
TestDistributionMessageObserver operation=beforeProcessMessage;
time=1607039519051; message=UpdateOperation$UpdateWithContextMessage(...);
recipients=[null]
P2P message reader for 192.168.1.8(server1:54333):41001 unshared
ordered uid=10 dom #1 local port=45823 remote port=60595:
TestDistributionMessageObserver operation=afterProcessMessage;
time=1607039519052; message=UpdateOperation$UpdateWithContextMessage(...);
recipients=[null]
Case 2:
The server (server2) that processes the put operation from the client is
primary and has conserve-sockets=true.
The server (server1) that handles the UpdateWithContextMessage has
conserve-sockets=false.
1. A ServerConnection thread in server2 sends the UpdateWithContextMessage:
ServerConnection on port 60463 Thread 1: TestDistributionMessageObserver
operation=beforeSendMessage; time=1607039137587;
message=UpdateOperation$UpdateWithContextMessage(...);
recipients=[192.168.1.8(server1:53948):4
