Is there functionality for partitioning Solr indexes onto multiple machines? For this to work, I suppose that Solr would have to combine the results from the various machines. I think Nutch does this with the distributed searcher functionality.
-Nathan -----Original Message----- From: Mike Klaas [mailto:[EMAIL PROTECTED] Sent: Thursday, August 30, 2007 11:44 AM To: solr-user@lucene.apache.org Subject: Re: Multiple indexes On 29-Aug-07, at 10:21 PM, James liu wrote: > Does it affect with doc size？ > > for example 2 billion docs, 10k doc,,,,2 billion docs, but doc size > is 10m. There might be other places that have 2G limit (see lucene index format docs), but many things are vints and can grow larger. Of course you will hit physical limits of your machine long before you can achieve your hypothetical situation: that's 20,000 Tb, which is many, many times the size of a complete internet crawl. -Mike > > 2007/8/30, Mike Klaas <[EMAIL PROTECTED]>: >> >> 2 billion docs (signed int). >> >> On 29-Aug-07, at 6:24 PM, James liu wrote: >> >>> what is the limits for Lucene and Solr. >>> >>> 100m, 1000m, 5000m or other number docs? >>> >>> 2007/8/24, Walter Underwood <[EMAIL PROTECTED]>: >>>> >>>> It should work fine to index them and search them. 13 million >>>> docs is >>>> not even close to the limits for Lucene and Solr. Have you had >>>> problems? >>>> >>>> wunder >>>> >>>> On 8/23/07 7:30 AM, "Jae Joo" <[EMAIL PROTECTED]> wrote: >>>> >>>>> Is there any solution to handle 13 millions document shown as >>>>> below? >>>>> Each document is not big, but the number of ones is 13 million. >>>>> Any way to utilize the multiple indexes? >>>>> >>>>> Thanks, >>>>> Jae Joo >>>>> >>>>> >>>>> <doc><field name="trade2"></field> >>>>> <field name="company_name">Unlimi-Tech Software Inc</field> >>>>> <field name="phys_stabrv">ON</field> >>>>> <field name="trade4"></field> >>>>> <field name="status_id_descr">Single Location</field> >>>>> <field name="trade3"></field> >>>>> <field name="phys_country">Canada</field> >>>>> <field name="phys_zip">K1C 4R1</field> >>>>> <field name="phys_city">Ottawa</field> >>>>> <field name="phys_state">Ontario</field> >>>>> <field name="sic2">G2_Computer Software</field> >>>>> <field name="phys_address">1447a Youville Dr</field> >>>>> <field name="sic1">G_Technology & Communications</field> >>>>> <field name="duns_number">203439018</field> >>>>> <field name="trade1"></field> >>>>> <field name="phys_county">Carleton</field> >>>>> <field name="trade5"></field> >>>>> <field name="status_id_rank">30</field> >>>>> <field name="sic4">G2173_Computer Programming Services</field> >>>>> <field name="sic8">G217308D_Computer software development</field> >>>>> </doc> >>>> >>>> >>> >>> >>> -- >>> regards >>> jl >> >> > > > -- > regards > jl
