Somehow you'd have to create a custom collector, probably queue off the docs that made it to the collector and have some asynchronous thread consuming those docs and sending them in bits...
But this is so antithetical to how Solr operates that I suspect my hand-waving wouldn't really work out. The problem is, at heart, that Solr assumes that scoring matters. The scheme you've outlined has no way of knowing which documents are most important until after you've already queued *all* documents for return. Even the queueing will take forever, consider the query q=*:*. You're essentially queueing all the docs in the index to return. If you're talking about the other sort of distributed search (e.g. Shards), that's already built in to Solr, though admittedly the aggregator (whichever server distributes the requests in the first place) waits for a response from all the shards before assembling the response. It seems like you might leverage some of that. But I really don't understand the end goal very well here.... Best Erick On Wed, Mar 14, 2012 at 7:17 PM, Nicholas Ball <nicholas.b...@nodelay.com> wrote: > Hello all, > > I've been working on a plugin with a custom component and a few handlers for > a research project. It's aim is to do some interesting distributed work, > however I seem to have come to a road block when trying to respond to a > clients request in multiple steps. Not even sure if this is possible with > Solr but after no luck on the IRC channel, thought I'd ask here. > > What I'd like to achieve is to be able to have the requestHandler return > results to a user as soon as it has data available, then continue processing > or performing other distributed calls, and then return some more data, all on > the same single client request. > > Now my understanding is that solr does some kind of streaming. Not sure how > it's technically done over http in Solr so any information would be useful. I > believe something like this would work well but again not sure: > > http://en.m.wikipedia.org/wiki/Chunked_transfer_encoding > > I also came across this issue/feature request in JIRA but not completely sure > what the conclusion was or how someone might do/use this. Is it even relevant > to what I'm looking for? > > https://issues.apache.org/jira/browse/SOLR-578 > > Thank you very much for any help and time you can spare! > > Nicholas (incunix) > >
