On Sat, Mar 17, 2018 at 1:06 AM, Shawn Heisey <apa...@elyograg.org> wrote:
> On 3/16/2018 7:38 AM, Deepak Goel wrote: > > I did a performance study of Solr a while back. And I found that it does > > not scale beyond a particular point on a single machine (could be due to > > the way its coded). Hence multiple instances might make sense. > > > > https://docs.google.com/document/d/1kUqEcZl3NhOo6SLklo5Icg3fMnn9O > tLY_lwnc6wbXus/edit?usp=sharing > > How did you *use* that code that you've shown? That is not apparent (at > least to me) from the document. > > If every usage of the SolrJ code went through ALL of the code you've > shown, then it's not done well. It appears that you're creating and > closing a client object with every query. This will be VERY inefficient. > > The client object should be created during an initialization step, and > then passed to the benchmark step to be used there. One client object > can be used by many threads. I wanted to test how many max connections can Solr handle concurrently. Also I would have to implement an 'connection pooling' of the client-object connections rather than a single connection thread However a single client object with thousands of queries coming in would surely become a bottleneck. I can test this scenario too. Very likely the ES client works the same, > but you'd need to ask them to be sure. > > That code seems to be doing an identical query on every run. If that's > what's happening, it's not a good indicator of performance. Running the > same query over and over will show better performance than you can > expect from a real-world query load What evidence do you see that Solr isn't scaling like you expect? > > The problem is the max throughput which I can get on the machine is around 28 tps, even though I increase the load further & only 65% CPU is utilised (there is still 35% which is not being used). This clearly indicates the software is a problem as there is enough hardware resources. Also very soon I would have a Linux environment with me, so I can conduct the test in the document on Linux too (for the users interested in Linux and not Windows) > Thanks, > Shawn > >
