On Wed, Feb 22, 2012 at 9:27 AM, prasenjit mukherjee <prasen....@gmail.com> wrote: > Is anybody aware of any effort regarding porting solr to a netty ( or > any other async-io based framework ) based framework. > > Even on medium load ( 10 parallel clients ) with 16 shards > performance seems to deteriorate quite sharply compared another > alternative ( async-io based ) solution as load increases.
By "16 shards" do you mean you have 16 nodes and each single client request causes a distributed search across all of them them? How many concurrent requests are your 10 clients making to each node? NIO works well when there are many clients, but when servicing those client requests only needs intermittent CPU. That's not the pattern we see for search. You *can* easily configure Solr's Jetty to use NIO when accepting client connections, but it won't do you any good, just as switching to Netty wouldn't do anything here. Where NIO could help a little is with the requests that Solr makes to other Solr instances. Solr is already architected for async request-response to other nodes, but the current underlying implementation uses HttpClient 3 (which doesn't have NIO). Anyway, it's unlikely that NIO vs BIO will make much of a difference with the numbers you're talking about (16 shards). Someone else reported that we have the number of connections per host set too low, and they saw big gains by increasing this. There's an issue open to make this configurable in 3x: https://issues.apache.org/jira/browse/SOLR-3079 We should probably up the max connections per host by default. -Yonik lucidimagination.com
