Re: Solr subset searching in 100-million document index

2013-10-25 Thread Aloke Ghoshal
Hi Sandeep, You are quite likely below capacity with this current set-up: http://wiki.apache.org/solr/SolrPerformanceProblems#OS_Disk_Cache Few things for you to confirm: 1. Which version of Solr are you using? 2. The size of your index. - Are fields stored? How much are these stored fields contr

Re: Solr subset searching in 100-million document index

2013-10-24 Thread Sandeep Gupta
Hi Joel, Thanks a lot for the information - I haven't worked with PostFilter's before but found an example at http://java.dzone.com/articles/custom-security-filtering-solr. Will try it over the next few days and come back if still have questions. Thanks again! Keep Walking, ~ Sandeep On Thu

Re: Solr subset searching in 100-million document index

2013-10-24 Thread Joel Bernstein
Sandeep, This type of operation can often be expressed as a PostFilter very efficiently. This is particularly true if the region id's are integer keys. Joel On Thu, Oct 24, 2013 at 7:46 AM, Sandeep Gupta wrote: > Hi, > > We have a Solr index of around 100 million documents with each document >

Solr subset searching in 100-million document index

2013-10-24 Thread Sandeep Gupta
Hi, We have a Solr index of around 100 million documents with each document being given a region id growing at a rate of about 10 million documents per month - the average document size being aronud 10KB of pure text. The total number of region ids are themselves in the range of 2.5 million. We w