Re: How to select the correct number of Shards in SolrCloud

2015-01-16 Thread Manohar Sripada
Thanks Daniel and Shawn for your valuable suggestions, Daniel, If you have a query and it needs to get results from 64 cores, if 63 return in 100ms but the last core is in GC pause and takes 500ms, your query will take just over 500ms. > There is only single JVM running per machine. I will get the

Re: How to select the correct number of Shards in SolrCloud

2015-01-16 Thread Shawn Heisey
On 1/15/2015 10:58 PM, Manohar Sripada wrote: > The reason I have created 64 Shards is there are 4 CPU cores on each VM; > while querying I can make use of all the CPU cores. On an average, Solr > QTime is around 500ms here. > > Last time to my other discussion, Erick suggested that I might be ove

Re: How to select the correct number of Shards in SolrCloud

2015-01-16 Thread Daniel Collins
Sharding a query lets you parallel the actual querying the index part of the search. But remember that as soon as you spread the query out more, you also need to bring all 64 results sets back together and consolidate them into a single result set for the end user. At some point, the gain of being

How to select the correct number of Shards in SolrCloud

2015-01-15 Thread Manohar Sripada
Hi All, My Setup is as follows. There are 16 nodes in my SolrCloud and 4 CPU cores on each Solr Node VM. Each having 64 GB of RAM, out of which I have allocated 32 GB to Solr. I have a collection which contains around 100 million Docs, which I created with 64 shards, replication factor 2, and 8 sh