Hm, hm, interesting. I was looking into something like this the other day (BIG indexed+stored text fields). After seeing enableLazyFieldLoading=true in solrconfig and after seeing "fl" didn't include those big fields, I though "hm, so Lucene/Solr will not be pulling those large fields from disk, OK".
You are saying that this may not be true based on your experiment? And what I'm calling your "experiment" means that you reindexed the same data, but without the 2 multi-valued text fields... .and that was the only change you made and got cca x10 search performance improvement? Sorry for repeating your words, just trying to confirm and understand. Thanks, Otis -- Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch ----- Original Message ---- > From: Raghuveer Kancherla <raghuveer.kanche...@aplopio.com> > To: solr-user@lucene.apache.org > Sent: Thu, December 3, 2009 8:43:16 AM > Subject: Re: Retrieving large num of docs > > Hi Hoss, > > I was experimenting with various queries to solve this problem and in one > such test I remember that requesting only the ID did not change the > retrieval time. To be sure, I tested it again using the curl command today > and it confirms my previous observation. > > Also, enableLazyFieldLoading setting is set to true in my solrconfig. > > Another general observation (off topic) is that having a moderately large > multi valued text field (~200 entries) in the index seems to slow down the > search significantly. I removed the 2 multi valued text fields from my index > and my search got ~10 time faster. :) > > - Raghu > > > On Thu, Dec 3, 2009 at 2:14 AM, Chris Hostetter wrote: > > > > > : I think I solved the problem of retrieving 300 docs per request for now. > > The > > : problem was that I was storing 2 moderately large multivalued text fields > > : though I was not retrieving them during search time. I reindexed all my > > : data without storing these fields. Now the response time (time for Solr > > to > > : return the http response) is very close to the QTime Solr is showing in > > the > > > > Hmmm.... > > > > two comments: > > > > 1) the example URL from your previous mail... > > > > : > > > > http://localhost:1212/solr/select/?rows=300&q=%28ResumeAllText%3A%28%28%28%22java+j2ee%22+%28java+j2ee%29%29%29%5E4%29%5E1.0%29&start=0&wt=python > > > > ...doesn't match your earlier statemnet that you are only returning hte id > > field (there is no "fl" param in that URL) ... are you certain you werent' > > returning those large stored fields in teh response? > > > > 2) assuming you were actually using an fl param to limit the fields, make > > sure you have this setting in your solrconfig.xml... > > > > true > > > > ..that should make it pretty fast to return only a few fields of each > > document, even if you do have some jumpto stored fields that aren't being > > returned. > > > > > > > > -Hoss > > > >
