Re: Distributed Search and the Stale Check

2013-02-27 Thread Ryan Zezeski
On Mon, Feb 25, 2013 at 8:26 PM, Mark Miller wrote: > Please file a JIRA issue and attach your patch. Great write up! (Saw it > pop up on twitter, so I read it a little earlier). > > Done. https://issues.apache.org/jira/browse/SOLR-4509

Re: Distributed Search and the Stale Check

2013-02-25 Thread Ryan Zezeski
On Mon, Feb 25, 2013 at 8:42 PM, Yonik Seeley wrote: > > > That's insane! > It is insane. Keep in mind this was a 5-node cluster on the same physical machine sharing the same resources. It consist of 5 smartos zones on the same global zone. On my MacBook Pro I saw ~1.5ms per stale check bu

Re: Distributed Search and the Stale Check

2013-02-25 Thread Yonik Seeley
> On my particular benchmark rig, each stale check call accounted for an > additional ~10ms. That's insane! It's still not even clear to me how the stale check works (reliably). Couldn't the server still close the connection between the stale check and the send of data by the client? -Yonik

Re: Distributed Search and the Stale Check

2013-02-25 Thread Mark Miller
On Feb 25, 2013, at 8:14 PM, Ryan Zezeski wrote: > I would like to see a > similar fix made upstream and that is why I am posting here. Please file a JIRA issue and attach your patch. Great write up! (Saw it pop up on twitter, so I read it a little earlier). - Mark

RE: Distributed Search and the Stale Check

2013-02-25 Thread Michael Ryan
I don't have anything to add besides saying "this is awesome". Great analysis. -Michael

Distributed Search and the Stale Check

2013-02-25 Thread Ryan Zezeski
Hello Solr Users, I just wrote up a piece about some work I did recently to improve the throughput of distributed search. http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html The short of it is that the stale check in Apache's HTTP Client used by SolrJ can add a lot of l