Grant, Eric, Walter, and SOLR, Thank you so much for very prompt responses (with links!)
>From time to time I try to share... Happy Holidays!!!!!!! > -----Original Message----- > From: Walter Underwood [mailto:wun...@wunderwood.org] > Sent: December-24-09 1:51 PM > To: solr-user@lucene.apache.org > Subject: Re: SOLR Performance Tuning: Pagination > > Some bots will do that, too. Maybe badly written ones, but we saw that at > Netflix. It was causing search timeouts just before a peak traffic period, > so we set a page limit in the front end, something like 200 pages. > > It makes sense for that to be very slow, because a request for hit > 28838540 means that Solr has to calculate the relevance for 28838540 + 10 > documents. > > Fuad: Why are you benchmarking this? What user is looking at 20M > documents? > > wunder > > On Dec 24, 2009, at 10:44 AM, Erik Hatcher wrote: > > > > > On Dec 24, 2009, at 11:36 AM, Walter Underwood wrote: > >> When do users do a query like that? --wunder > > > > Well, SolrEntityProcessor "users" do :) > > > > http://issues.apache.org/jira/browse/SOLR-1499 > > (which by the way I plan on polishing and committing over the holidays) > > > > Erik > > > > > > > >> > >> On Dec 24, 2009, at 8:09 AM, Fuad Efendi wrote: > >> > >>> I used pagination for a while till found this... > >>> > >>> > >>> I have filtered query ID:[* TO *] returning 20 millions results (no > >>> faceting), and pagination always seemed to be fast. However, fast only > with > >>> low values for start=12345. Queries like start=28838540 take 40-60 > seconds, > >>> and even cause OutOfMemoryException. > >>> > >>> I use highlight, faceting on nontokenized "Country" field, standard > handler. > >>> > >>> > >>> It even seems to be a bug... > >>> > >>> > >>> Fuad Efendi > >>> +1 416-993-2060 > >>> http://www.linkedin.com/in/liferay > >>> > >>> Tokenizer Inc. > >>> http://www.tokenizer.ca/ > >>> Data Mining, Vertical Search > >>> > >> > >
