Re: Question on Solr Scalability

2010-02-11 Thread Yonik Seeley
On Thu, Feb 11, 2010 at 6:56 AM, abhishes wrote: > > Thanks really useful article. > > I am wondering about this statement in the article > > "Keep in mind that Solr does not calculate universal term/doc frequencies. > At a large scale, its not likely  to matter that tf/idf is calculated at the >

Re: Question on Solr Scalability

2010-02-11 Thread Erik Hatcher
There is already a patch available to address that short-coming in distributed search: http://issues.apache.org/jira/browse/SOLR-1632 On Feb 11, 2010, at 6:56 AM, abhishes wrote: Thanks really useful article. I am wondering about this statement in the article "Keep in mind that Solr

Re: Question on Solr Scalability

2010-02-11 Thread abhishes
Thanks really useful article. I am wondering about this statement in the article "Keep in mind that Solr does not calculate universal term/doc frequencies. At a large scale, its not likely to matter that tf/idf is calculated at the shard level - however, if your collection is heavily skewed in

Re: Question on Solr Scalability

2010-02-10 Thread David Stuart
Hi, I think your needs would meet better with Distributed Search http://wiki.apache.org/solr/DistributedSearch Which allows sharding to live on different servers and will search across all of those shard when a query comes in. There are a few patch which will hopefully be available in the S

Re: Question on Solr Scalability

2010-02-10 Thread Juan Pedro Danculovic
To scale solr, take a look to this article http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr Juan Pedro Danculovic CTO - www.linebee.com On Thu, Feb 11, 2010 at 4:12 AM, abhishes wrote: > > Suppose I am indexing very large data (5 billion rows