1. Keep the number of collections down to the low hundreds max. Preferably no more than a few dozen or a hundred. 2. 8GB is too small to be useful. 16 GB min. 3. If you need large numbers of machines, organize them as separate clusters. 4. Figure 100 to 200 million documents on a Solr server. E.g., 1 billion documents would be 5 to 10 servers. Depends on document size and query latency requirements, practical limit could be higher or lower.
-- Jack Krupansky On Sat, Aug 15, 2015 at 6:22 AM, yura last <y_ura_2...@yahoo.com.invalid> wrote: > Hi All, I am testing a SolrCloud with many collections. The version is > 5.2.1 and I installed 3 machines – each one with 4 cores and 8 GB Ram.Then > I created collections with 3 shards and replication factor of 2. It gives > me 2 cores per collection on each machine.I reached almost 900 collections > and then the cluster was stuck and I couldn’t revive the cluster. As I > understand Solr have issues with many collections (thousands).If I will use > much more machines – does it will give me the ability to create tens of > thousands of collections or the limit is couple of thousands? I want to > build a cluster that will handle 10 billion of documents (currently I have > 1 billion) per day and to keep the data for 90 days.I want to support 2000 > customers so I would like to split them to collections and also to split it > by days. (180,000 collections)If I will create big collections I will have > performance issues with queries and also most of the queries are for a > specific customer. (I also have cross customers queries) How I can build an > appropriate design? Thanks a lot,Yuri
