On Mon, Oct 11, 2010 at 8:32 PM, Peter Keegan <peterlkee...@gmail.com> wrote: > I listened with great interest to Grant's presentation of the NoSQL > comparisons/alternatives to Solr/Lucene. It sounds like the jury is still > out on much of this. Here's a use case that might favor using a NoSQL > alternative for storing 'stored fields' outside of Lucene. > > When Solr does a distributed search across shards, it does this in 2 phases > (correct me if I'm wrong): > > 1. 1st query to get the docIds and facet counts > 2. 2nd query to retrieve the stored fields of the top hits > > The problem here is that the index could change between (1) and (2), so it's > not an atomic transaction.
Yep. As I discussed with Peter at Lucene Revolution, if this feature is important to people, I think the easiest way to solve it would be via leases. During a query, a client could request a lease for a certain amount of time on whatever index version is used to generate the response. Solr would then return the index version to the client along with the response, and keep the index open for that amount of time. The client could make consistent additional requests (such as the 2nd phase of a distributed request) by requesting the same version of the index. -Yonik