Hi all, I have multiple python scripts querying solr with the sunburnt module.
Solr was hosted on an Amazon ec2 m1.large (2 vCPU with 4 ECU, 7.5 GB memory & 840 GB storage) and contained several cores for different usage. When I manually executed a query through Solr Admin (a query containing 10~15 terms, with some of them having boosts over one field and limited to one result without any sorting or faceting etc ....) it takes around 700 ms, and the Core contained 7 million documents. When the scripts are executed things get slower, my query takes 7~10s. Then what I did is to turn to SolrCloud expecting huge performance increase. I installed it on a cluster of 5 Amazon ec2 c3.2xlarge instances (8 vCPU with 28 ECU, 15 GB memory & 160 SSD storage), then I created one collection to contain the core I was querying, I sharded it to 25 shards (each node containing 5 shards without replication), each shards took 54 MB of storage. Tested my query on the new SolrCloud, it takes 70 ms ! huge increase wich is very good ! Tested my scripts again (I have 30 scripts running at the same time), and as a surprise, things run fast for 5 seconds then it turns realy slow again (query time ). I updated the solrconfig.xml to remove the query caches (I don't need them since queries are very different and only 1 time queries) and changes the index memory to 1 GB, but only got a small increase (3~4s for each query ?!) Any ideas ? PS: My index size will not stay with 7m documents, it will grow to +100m and that may get things worse
