Thing is - I wouldn't expect any of the default options mentioned to change the 
behavior intermittently.

i.e. it's working for 95% of the health check requests, it's just the intermittent ones that seem to be cut off... I'm inquiring with haproxy devs since it appears that at least one other person on #haproxy is seeing the same behavior. Doesn't appear to be specific to solr.

-- Nathan

On 07/27/2014 10:44 PM, Shawn Heisey wrote:
On 7/27/2014 7:23 PM, Nathan Neulinger wrote:
Unfortunately, doesn't look like this clears the symptom.

The ping is responding almost instantly every time. I've tried setting a
15 second timeout on the check, with no change in occurences of the error.

Looking at a packet capture on the server side, there is a clear
distinction between working and failing/error-triggering connections.

It looks like in a "working" case, I see two packets immediately back to
back (one with header, and next a continuation with content) with no ack
in between, followed by ack, rst+ack, rst.

In the failing request, I see the GET request, acked, then the http/1.1
200 Ok response from Solr, a single ack, and then an almost
instantaneous reset sent by the client.


I'm only seeing this on traffic to/from haproxy checks. If I do a simple:

     while [ true ]; do curl -s http://host:8983/solr/admin/ping; done

from the same box, that flood runs with generally 10-20ms request times
and zero errors.

I won't claim to understand what's going on here, but it might be a
matter of the haproxy options.  Here are the options I'm using in the
"defaults" section of the config:

defaults
         log     global
         mode    http
         option  httplog
         option  dontlognull
         option  redispatch
         option  abortonclose
         option  http-server-close
         option  http-pretend-keepalive
         retries 1
         maxconn 1024
         timeout connect 1s
         timeout client  5s
         timeout server  30s

One bit of information I came across when I first started setting
haproxy up for Solr is that servlet containers like Jetty and Tomcat
require the "http-pretend-keepalive" option to work properly.  Are you
using this option?

Thanks,
Shawn


--
------------------------------------------------------------
Nathan Neulinger                       nn...@neulinger.org
Neulinger Consulting                   (573) 612-1412

Reply via email to