Nagendra, Does RankingAlgorithm work with faceting which requires the use of cache? As new documents are added or updated, the cache will be constantly invalidated. So how would RankingAlgorithm work in this case?
________________________________ From: Nagendra Nagarajayya <nnagaraja...@transaxtions.com> To: solr-user@lucene.apache.org Sent: Tuesday, July 24, 2012 8:24 AM Subject: Re: [Announce] Solr 4.0-ALPHA with RankingAlgorithm 1.4.4 with Realtime NRT available for download Hi Yonik: Please see my comments below: On 7/23/2012 8:52 AM, Yonik Seeley wrote: > On Mon, Jul 23, 2012 at 11:37 AM, Nagendra Nagarajayya > <nnagaraja...@transaxtions.com> wrote: >> Realtime NRT algorithm enables NRT functionality in >> Solr by not closing the Searcher object and so is very fast. I am in the >> process of contributing the algorithm back to Apache Solr as a patch. > Since you're in the process of contributing this back, perhaps you > could explain your approach - it never made sense to me. > > Replacing the reader in an existing SolrIndexSearcher as you do means > that all the related caches will be invalid (meaning you can't use > solr's caches). You could just ensure that there is no auto-warming > set up for Solr's caches (which is now the default), or you could > disable caching altogether. It's not clear what you're comparing > against when you claim it's faster. Solr with RankingAlgorithm does not replace the reader in SolrIndexSearcher object. All it does is override the IndexSearcher.getIndexReader() method so as to supply a NRTReader if realtime is enabled. All direct references to the "reader" member has been replaced with a getIndexReader() method access. The performance is better as SolrIndexSearcher is not closed every 1 sec as in soft-commit. SolrIndexSearcher is a heavy object with caches, etc. and is reference counted. So every 1 sec this object needs to closed, re-allocated and the indexes need to be re-opened, caches invalidated, while waiting for existing searchers to complete, making this very expensive. realtime NRT does not close the SolrIndexSearcher object but makes available a new NRTReader with document updates ie. getIndexReader() returns a new NRTReader. > There are also consistency and concurrency issues with replacing the > reader in an existing SolrIndexSearcher, which is supposed to have a > static view of the index. If a reader replacement happens in the > middle of a request, it's bound to cause trouble, including returning > the wrong documents! The reader member is not replaced in the existing SolrIndexSearcher object. The IndexSearcher.getIndexReader() method has been overriden in SolrIndexSearcher and all direct reader member access has been replaced with a getIndexReader() method call allowing a NRT reader to be supplied when realtime is enabled. The concurrency is handled by the getNRTReader() method, with the static index view now increased to the granularity provided by the NRTIndexReader. Regards, Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org > -Yonik > http://lucidimagination.com > >
