When we have 49 shards per collection, there are more than 600 collections. Solr will have serious performance problems. I don't know how to deal with them. My advice to you is to minimize the number of collections. Our environment is 49 solr server nodes, each with 32cpu/128g, and the data volume is about 50 billion per day.
<https://mailtrack.io/> Sent with Mailtrack <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality&;> 2018-04-04 9:23 GMT+08:00 Yago Riveiro <yago.rive...@gmail.com>: > Hi, > > In my company we are running a 12 node cluster with 10 (american) Billion > documents 12 shards / 2 replicas. > > We do mainly faceting queries with a very reasonable performance. > > 36 million documents it's not an issue, you can handle that volume of > documents with 2 nodes with SSDs and 32G of ram > > Regards. > > -- > > Yago Riveiro > > On 4 Apr 2018 02:15 +0100, Abhi Basu <9000r...@gmail.com>, wrote: > > We have tested Solr 4.10 with 200 million docs with avg doc size of 250 > KB. > > No issues with performance when using 3 shards / 2 replicas. > > > > > > > > On Tue, Apr 3, 2018 at 8:12 PM, Steven White <swhite4...@gmail.com> > wrote: > > > > > Hi everyone, > > > > > > I'm about to start a project that requires indexing 36 million records > > > using Solr 7.2.1. Each record range from 500 KB to 0.25 MB where the > > > average is 0.1 MB. > > > > > > Has anyone indexed this number of records? What are the things I should > > > worry about? And out of curiosity, what is the largest number of > records > > > that Solr has indexed which is published out there? > > > > > > Thanks > > > > > > Steven > > > > > > > > > > > -- > > Abhi Basu > -- ============================== 联创科技 知行如一 ==============================
