Hi I have a 3 node zk ensemble . I see a very high latency for zk responses and also a lot of outstanding requests (in the order of 30-40)
I also see that the requests are not going to all zookeeper nodes equally. One node has more requests/connections than the others. I see that CPU/Mem and disk usage limits are very normal (under 30% cpu, disk reads in the order of kb, jvm size is 2 Gb but it hasnt even reached 30% usage). The size of data in zk is around 50MB I also see a few zk timeout for solrcloud nodes causing them to be shown as "dead" in the cloud view. I have increased the connection timeout to around 3 minutes and still the same issue seems to be happening How do i make zk respond faster to requests and where does zk usually spend time while dealing with incoming requests? Any pointers on how to move forward will be great -- Best -- C
