At the NOVA Apache Lucene/Solr Meetup last May, one of the speakers from Near Infinity (Aaron McCurry I think) mentioned that he had a patch for lucene that enabled unlimited depth memory-efficient paging. Is anyone in contact with him?
-Peter On Thu, Dec 24, 2009 at 11:27 AM, Grant Ingersoll <gsing...@apache.org> wrote: > > On Dec 24, 2009, at 11:09 AM, Fuad Efendi wrote: > >> I used pagination for a while till found this... >> >> >> I have filtered query ID:[* TO *] returning 20 millions results (no >> faceting), and pagination always seemed to be fast. However, fast only with >> low values for start=12345. Queries like start=28838540 take 40-60 seconds, >> and even cause OutOfMemoryException. > > Yeah, deep pagination in Lucene/Solr can be problematic due to the Priority > Queue management. See http://issues.apache.org/jira/browse/LUCENE-2127 and > the linked discussion on java-dev. > >> >> I use highlight, faceting on nontokenized "Country" field, standard handler. >> >> >> It even seems to be a bug... >> >> >> Fuad Efendi >> +1 416-993-2060 >> http://www.linkedin.com/in/liferay >> >> Tokenizer Inc. >> http://www.tokenizer.ca/ >> Data Mining, Vertical Search >> >> >> >> > > -------------------------- > Grant Ingersoll > http://www.lucidimagination.com/ > > Search the Lucene ecosystem using Solr/Lucene: > http://www.lucidimagination.com/search > > -- Peter M. Wolanin, Ph.D. Momentum Specialist, Acquia. Inc. peter.wola...@acquia.com
