ParallelReader is definitely out there on the Lucene landscape. See http://www.lucidimagination.com/search/page:2?q=ParallelReader
for some background discussion, including Doug's original post on it
and some others view of the use case. The key is that the small index
has to be rebuilt in exactly the same order as the large index, which
seems particularly onerous in high-update environments. I will add
that it is definitely one of those areas most people do not use, so
getting help on it may be difficult.
I've often thought about an AsynchronousParallelReader that maintained
a mapping between the two indexes such that you could let the indexes
get out of sync, but have never implemented it or gone far enough down
the path to know whether it would even work or not. The devil is
likely in the details what with Lucene's merging, etc.
Thinking out loud, you might also try a custom component (or some
changes to the QueryComponent) that uses the MultiSearcher or maybe
some lower level Solr changes. The MultiSearcher is also designed to
search across multiple indexes.
HTH,
Grant
On Feb 9, 2009, at 8:37 PM, Mark Kranz wrote:
Just an update on my own research:
I have discovered the 'ParallelReader' class (subclass of
IndexReader) in
lucene, which is designed for searching across multiple indexes.
This appears to suit our needs - and I do not expect will be too
difficult
to integrate into Solr.
--
View this message in context:
http://www.nabble.com/Vertical-Partitioning-advice-tp21906668p21926031.html
Sent from the Solr - User mailing list archive at Nabble.com.
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search
