For Solr 5 why don't we switch it to 64 bit ?? Bill Bell Sent from mobile
> On Dec 29, 2014, at 1:53 PM, Jack Krupansky <jack.krupan...@gmail.com> wrote: > > And that Lucene index document limit includes deleted and updated > documents, so even if your actual document count stays under 2^31-1, > deleting and updating documents can push the apparent document count over > the limit unless you very aggressively merge segments to expunge deleted > documents. > > -- Jack Krupansky > > -- Jack Krupansky > > On Mon, Dec 29, 2014 at 12:54 PM, Erick Erickson <erickerick...@gmail.com> > wrote: > >> When you say 2B docs on a single Solr instance, are you talking only one >> shard? >> Because if you are, you're very close to the absolute upper limit of a >> shard, internally >> the doc id is an int or 2^31. 2^31 + 1 will cause all sorts of problems. >> >> But yeah, your 100B documents are going to use up a lot of servers... >> >> Best, >> Erick >> >> On Mon, Dec 29, 2014 at 7:24 AM, Bram Van Dam <bram.van...@intix.eu> >> wrote: >>> Hi folks, >>> >>> I'm trying to get a feel of how large Solr can grow without slowing down >> too >>> much. We're looking into a use-case with up to 100 billion documents >>> (SolrCloud), and we're a little afraid that we'll end up requiring 100 >>> servers to pull it off. >>> >>> The largest index we currently have is ~2billion documents in a single >> Solr >>> instance. Documents are smallish (5k each) and we have ~50 fields in the >>> schema, with an index size of about 2TB. Performance is mostly OK. Cold >>> searchers take a while, but most queries are alright after warming up. I >>> wish I could provide more statistics, but I only have very limited >> access to >>> the data (...banks...). >>> >>> I'd very grateful to anyone sharing statistics, especially on the larger >> end >>> of the spectrum -- with or without SolrCloud. >>> >>> Thanks, >>> >>> - Bram >>
