I’m not familiar with the inner workings, but your writeup is excellent and makes a compelling case.
It sounds like you are saying that the original motivation for conserve-sockets=true was to improve performance, but in fact it makes performance worse. Do you have some numbers to quantify what magnitude of a difference this can make? It also looks bad that we actually document that Geode may deadlock if you use the default. To Udo’s concern about changing existing behavior, it sounds like you are saying there is no possible downside to conserve-sockets=false, but has that question been explored exhaustively? Is there ever any circumstance where we would still recommend a user to explicitly set conserve-sockets=true, once false is the default? From: Donal Evans <doev...@vmware.com> Date: Wednesday, November 18, 2020 at 5:04 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: [PROPOSAL] Change the default value of conserve-sockets to false Hi Geode dev, First, from the docs[1], a brief explanation of the purpose of the conserve-sockets property: "The conserve-sockets setting indicates whether application threads share sockets with other threads or use their own sockets for member communication. This setting has no effect on communication between a server and its clients, but it does control the server’s communication with its peers or a gateway sender’s communication with a gateway receiver." The current default value for the conserve-sockets property is true, which at first glance makes sense, since in an ideal world, existing sockets could be shared between threads and there would be no need to create and destroy new sockets for each process, which can be somewhat resource-intensive. However, in practice, there are several known issues with using the default setting of true. From the docs[1]: "For distributed regions, the put operation, and destroy and invalidate for regions and entries, can all be optimized with conserve-sockets set to false. For partitioned regions, setting conserve-sockets to false can improve general throughput. Note: When you have transactions operating on EMPTY, NORMAL or PARTITION regions, make sure that conserve-sockets is set to false to avoid distributed deadlocks." and[2]: "WAN deployments increase the messaging demands on a Geode system. To avoid hangs related to WAN messaging, always set `conserve-sockets=false` for Geode members that participate in a WAN deployment." Given that it is generally accepted as best practice to set conserve-sockets to false for almost all use cases of Geode beyond the most simple, it would make sense to also change the default value to false, to prevent people having to encounter a problem, search for the solution, then change the setting to what is almost always the "correct" value. I have done some experimenting to see what it would take to make this proposal a reality, and the changes required are very minimal, with only two existing DUnit tests that need to be modified to explicitly set the value of conserve-sockets that were previously relying on the default being true. Any feedback on this proposal would be very welcome, and if the response is positive, I can create a PR with the changes as soon as a decision is reached. Thanks, Donal [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F113%2Fmanaging%2Fmonitor_tune%2Fperformance_controls_controlling_socket_use.html&data=04%7C01%7Conichols%40vmware.com%7C73edfbbf4fe8435de38d08d88c271e40%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637413446927237420%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qGAh1BfdFovLpF11C0A1jtHUwRPpQDGVO48bksfzhOw%3D&reserved=0 [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F113%2Fmanaging%2Fmonitor_tune%2Fsockets_and_gateways.html&data=04%7C01%7Conichols%40vmware.com%7C73edfbbf4fe8435de38d08d88c271e40%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637413446927237420%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kRoPhR391aHl1iJLAyrs7JhzyGqG2fJ4dTFLxdb4Zb0%3D&reserved=0
