Thanks Rafal for your reply, My scripts are running on other independent machines so they does not affect Solr, I did mention that the queries are not the same (that is why I removed the query cache from solrconfig.xml), and I only get 1 result from Solr (which is the top scored one so no sorting since it is by default ordred by score)
2013/12/26 Rafał Kuć <r....@solr.pl> > Hello! > > Could you tell us more about your scripts? What they do? If the > queries are the same? How many results you fetch with your scripts and > so on. > > -- > Regards, > Rafał Kuć > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > > Hi all, > > > I have multiple python scripts querying solr with the sunburnt module. > > > Solr was hosted on an Amazon ec2 m1.large (2 vCPU with 4 ECU, 7.5 GB > memory > > & 840 GB storage) and contained several cores for different usage. > > > When I manually executed a query through Solr Admin (a query containing > > 10~15 terms, with some of them having boosts over one field and limited > to > > one result without any sorting or faceting etc ....) it takes around 700 > > ms, and the Core contained 7 million documents. > > > When the scripts are executed things get slower, my query takes 7~10s. > > > Then what I did is to turn to SolrCloud expecting huge performance > increase. > > > I installed it on a cluster of 5 Amazon ec2 c3.2xlarge instances (8 vCPU > > with 28 ECU, 15 GB memory & 160 SSD storage), then I created one > collection > > to contain the core I was querying, I sharded it to 25 shards (each node > > containing 5 shards without replication), each shards took 54 MB of > storage. > > > Tested my query on the new SolrCloud, it takes 70 ms ! huge increase wich > > is very good ! > > > Tested my scripts again (I have 30 scripts running at the same time), and > > as a surprise, things run fast for 5 seconds then it turns realy slow > again > > (query time ). > > > I updated the solrconfig.xml to remove the query caches (I don't need > them > > since queries are very different and only 1 time queries) and changes the > > index memory to 1 GB, but only got a small increase (3~4s for each query > ?!) > > > Any ideas ? > > > PS: My index size will not stay with 7m documents, it will grow to +100m > > and that may get things worse > >
