Author: schultz Date: Wed Jun 21 19:05:38 2017 New Revision: 1799498 URL: http://svn.apache.org/viewvc?rev=1799498&view=rev Log: Add LoadBalancerDrainingValve.
Added: tomcat/trunk/java/org/apache/catalina/valves/LoadBalancerDrainingValve.java (with props) tomcat/trunk/test/org/apache/catalina/valves/TestLoadBalancerDrainingValve.java (with props) Modified: tomcat/trunk/webapps/docs/changelog.xml tomcat/trunk/webapps/docs/config/valve.xml Added: tomcat/trunk/java/org/apache/catalina/valves/LoadBalancerDrainingValve.java URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/valves/LoadBalancerDrainingValve.java?rev=1799498&view=auto ============================================================================== --- tomcat/trunk/java/org/apache/catalina/valves/LoadBalancerDrainingValve.java (added) +++ tomcat/trunk/java/org/apache/catalina/valves/LoadBalancerDrainingValve.java Wed Jun 21 19:05:38 2017 @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.valves; + +import java.io.IOException; + +import javax.servlet.ServletException; +import javax.servlet.http.Cookie; +import javax.servlet.http.HttpServletResponse; + +import org.apache.catalina.LifecycleException; +import org.apache.catalina.connector.Request; +import org.apache.catalina.connector.Response; +import org.apache.catalina.util.SessionConfig; +import org.apache.juli.logging.Log; + +/** + * <p>A Valve to detect situations where a load-balanced node receiving a + * request has been deactivated by the load balancer (JK_LB_ACTIVATION=DIS) + * and the incoming request has no valid session.</p> + * + * <p>In these cases, the user's session cookie should be removed if it exists, + * any ";jsessionid" parameter should be removed from the request URI, + * and the client should be redirected to the same URI. This will cause the + * load-balanced to re-balance the client to another server.</p> + * + * <p>A request parameter is added to the redirect URI in order to avoid + * repeated redirects in the event of an error or misconfiguration.</p> + * + * <p>All this work is required because when the activation state of a node is + * DISABLED, the load-balancer will still send requests to the node if they + * appear to have a session on that node. Since mod_jk doesn't actually know + * whether the session id is valid, it will send the request blindly to + * the disabled node, which makes it take much longer to drain the node + * than strictly necessary.</p> + * + * <p>For testing purposes, a special cookie can be configured and used + * by a client to ignore the normal behavior of this Valve and allow + * a client to get a new session on a DISABLED node. See + * {@link #setIgnoreCookieName} and {@link #setIgnoreCookieValue} + * to configure those values.</p> + * + * <p>This Valve should be installed earlier in the Valve pipeline than any + * authentication valves, as the redirection should take place before an + * authentication valve would save a request to a protected resource.</p> + * + * @see http://tomcat.apache.org/connectors-doc/generic_howto/loadbalancers.html + */ +public class LoadBalancerDrainingValve + extends ValveBase +{ + /** + * The request attribute key where the load-balancer's activation state + * can be found. + */ + static final String ATTRIBUTE_KEY_JK_LB_ACTIVATION = "JK_LB_ACTIVATION"; + + /** + * The HTTP response code that will be used to redirect the request + * back to the load-balancer for re-balancing. Defaults to 307 + * (TEMPORARY_REDIRECT). + * + * HTTP status code 305 (USE_PROXY) might be an option, here. too. + */ + private int _redirectStatusCode = HttpServletResponse.SC_TEMPORARY_REDIRECT; + + /** + * The name of the cookie which can be set to ignore the "draining" action + * of this Filter. This will allow a client to contact the server without + * being re-balanced to another server. The expected cookie value can be set + * in the {@link #_ignoreCookieValue}. The cookie name and value must match + * to avoid being re-balanced. + */ + private String _ignoreCookieName; + + /** + * The value of the cookie which can be set to ignore the "draining" action + * of this Filter. This will allow a client to contact the server without + * being re-balanced to another server. The expected cookie name can be set + * in the {@link #_ignoreCookieValue}. The cookie name and value must match + * to avoid being re-balanced. + */ + private String _ignoreCookieValue; + + /** + * Local reference to the container log. + */ + protected Log containerLog = null; + + public LoadBalancerDrainingValve() + { + super(true); // Supports async + } + + // + // Configuration parameters + // + + /** + * Sets the HTTP response code that will be used to redirect the request + * back to the load-balancer for re-balancing. Defaults to 307 + * (TEMPORARY_REDIRECT). + */ + public void setRedirectStatusCode(int code) { + _redirectStatusCode = code; + } + + /** + * Gets the name of the cookie that can be used to override the + * re-balancing behavior of this Valve when the current node is + * in the DISABLED activation state. + * + * @return The cookie name used to ignore normal processing rules. + * + * @see #setIgnoreCookieValue + */ + public String getIgnoreCookieName() { + return _ignoreCookieName; + } + + /** + * Sets the name of the cookie that can be used to override the + * re-balancing behavior of this Valve when the current node is + * in the DISABLED activation state. + * + * There is no default value for this setting: the ability to override + * the re-balancing behavior of this Valve is <i>disabled</i> by default. + * + * @param cookieName The cookie name to use to ignore normal + * processing rules. + * + * @see #getIgnoreCookieValue + */ + public void setIgnoreCookieName(String cookieName) { + _ignoreCookieName = cookieName; + } + + /** + * Gets the expected value of the cookie that can be used to override the + * re-balancing behavior of this Valve when the current node is + * in the DISABLED activation state. + * + * @return The cookie value used to ignore normal processing rules. + * + * @see #setIgnoreCookieValue + */ + public String getIgnoreCookieValue() { + return _ignoreCookieValue; + } + + /** + * Sets the expected value of the cookie that can be used to override the + * re-balancing behavior of this Valve when the current node is + * in the DISABLED activation state. The "ignore" cookie's value + * <b>must</b> be exactly equal to this value in order to allow + * the client to override the re-balancing behavior. + * + * @param cookieValue The cookie value to use to ignore normal + * processing rules. + * + * @see #getIgnoreCookieValue + */ + public void setIgnoreCookieValue(String cookieValue) { + _ignoreCookieValue = cookieValue; + } + + @Override + public void initInternal() + throws LifecycleException + { + super.initInternal(); + + containerLog = getContainer().getLogger(); + } + + @Override + public void invoke(Request request, Response response) throws IOException, ServletException { + if("DIS".equals(request.getAttribute(ATTRIBUTE_KEY_JK_LB_ACTIVATION)) + && !request.isRequestedSessionIdValid()) { + + if(containerLog.isDebugEnabled()) + containerLog.debug("Load-balancer is in DISABLED state; draining this node"); + + boolean ignoreRebalance = false; // Allow certain clients + Cookie sessionCookie = null; + + // Kill any session cookie present + final Cookie[] cookies = request.getCookies(); + + final String sessionCookieName = request.getServletContext().getSessionCookieConfig().getName(); + + // Kill any session cookie present + if(null != cookies) { + for(Cookie cookie : cookies) { + final String cookieName = cookie.getName(); + if(containerLog.isTraceEnabled()) + containerLog.trace("Checking cookie " + cookieName + "=" + cookie.getValue()); + + if(sessionCookieName.equals(cookieName) + && request.getRequestedSessionId().equals(cookie.getValue())) { + sessionCookie = cookie; + } else + // Is the client presenting a valid ignore-cookie value? + if(null != _ignoreCookieName + && _ignoreCookieName.equals(cookieName) + && null != _ignoreCookieValue + && _ignoreCookieValue.equals(cookie.getValue())) { + ignoreRebalance = true; + } + } + } + + if(ignoreRebalance) { + if(containerLog.isDebugEnabled()) + containerLog.debug("Client is presenting a valid " + _ignoreCookieName + + " cookie, re-balancing is being skipped"); + + getNext().invoke(request, response); + + return; + } + + // Kill any session cookie that was found + // TODO: Consider implications of SSO cookies + if(null != sessionCookie) { + String cookiePath = request.getServletContext().getSessionCookieConfig().getPath(); + + if(request.getContext().getSessionCookiePathUsesTrailingSlash()) { + // Handle special case of ROOT context where cookies require a path of + // '/' but the servlet spec uses an empty string + // Also ensure the cookies for a context with a path of /foo don't get + // sent for requests with a path of /foobar + if (!cookiePath.endsWith("/")) + cookiePath = cookiePath + "/"; + + sessionCookie.setPath(cookiePath); + sessionCookie.setMaxAge(0); // Delete + sessionCookie.setValue(""); // Purge the cookie's value + response.addCookie(sessionCookie); + } + } + + // Re-write the URI if it contains a ;jsessionid parameter + String uri = request.getRequestURI(); + String sessionURIParamName = "jsessionid"; + SessionConfig.getSessionUriParamName(request.getContext()); + if(uri.contains(";" + sessionURIParamName + "=")) + uri = uri.replaceFirst(";" + sessionURIParamName + "=[^&?]*", ""); + + String queryString = request.getQueryString(); + + if(null != queryString) + uri = uri + "?" + queryString; + + // NOTE: Do not call response.encodeRedirectURL or the bad + // sessionid will be restored + response.setHeader("Location", uri); + response.setStatus(_redirectStatusCode); + } + else + getNext().invoke(request, response); + } +} Propchange: tomcat/trunk/java/org/apache/catalina/valves/LoadBalancerDrainingValve.java ------------------------------------------------------------------------------ svn:eol-style = native Added: tomcat/trunk/test/org/apache/catalina/valves/TestLoadBalancerDrainingValve.java URL: http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/catalina/valves/TestLoadBalancerDrainingValve.java?rev=1799498&view=auto ============================================================================== --- tomcat/trunk/test/org/apache/catalina/valves/TestLoadBalancerDrainingValve.java (added) +++ tomcat/trunk/test/org/apache/catalina/valves/TestLoadBalancerDrainingValve.java Wed Jun 21 19:05:38 2017 @@ -0,0 +1,257 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.valves; + +import java.util.ArrayList; +import java.util.List; + +import javax.servlet.ServletContext; +import javax.servlet.SessionCookieConfig; +import javax.servlet.http.Cookie; + +import org.junit.Test; + +import org.apache.catalina.Context; +import org.apache.catalina.Valve; +import org.apache.catalina.connector.Request; +import org.apache.catalina.connector.Response; +import org.apache.catalina.core.StandardPipeline; +import org.easymock.EasyMock; +import org.easymock.IMocksControl; + +public class TestLoadBalancerDrainingValve { + + static class MockResponse extends Response { + private List<Cookie> cookies; + @Override + public boolean isCommitted() { + return false; + } + @Override + public void addCookie(Cookie cookie) + { + if(null == cookies) + cookies = new ArrayList<Cookie>(1); + cookies.add(cookie); + } + public List<Cookie> getCookies() { + return cookies; + } + } + + static class CookieConfig implements SessionCookieConfig { + + private String name; + private String domain; + private String path; + private String comment; + private boolean httpOnly; + private boolean secure; + private int maxAge; + + @Override + public String getName() { + return name; + } + @Override + public void setName(String name) { + this.name = name; + } + @Override + public String getDomain() { + return domain; + } + @Override + public void setDomain(String domain) { + this.domain = domain; + } + @Override + public String getPath() { + return path; + } + @Override + public void setPath(String path) { + this.path = path; + } + @Override + public String getComment() { + return comment; + } + @Override + public void setComment(String comment) { + this.comment = comment; + } + @Override + public boolean isHttpOnly() { + return httpOnly; + } + @Override + public void setHttpOnly(boolean httpOnly) { + this.httpOnly = httpOnly; + } + @Override + public boolean isSecure() { + return secure; + } + @Override + public void setSecure(boolean secure) { + this.secure = secure; + } + @Override + public int getMaxAge() { + return maxAge; + } + @Override + public void setMaxAge(int maxAge) { + this.maxAge = maxAge; + } + } + + // A Cookie subclass that knows how to compare itself to other Cookie objects + static class MyCookie extends Cookie { + public MyCookie(String name, String value) { super(name, value); } + + @Override + public boolean equals(Object o) { + if(null == o) return false; + MyCookie mc = (MyCookie)o; + + return mc.getName().equals(this.getName()) + && mc.getPath().equals(this.getPath()) + && mc.getValue().equals(this.getValue()) + && mc.getMaxAge() == this.getMaxAge(); + } + + @Override + public String toString() { + return "Cookie { name=" + getName() + ", value=" + getValue() + ", path=" + getPath() + ", maxAge=" + getMaxAge() + " }"; + } + } + + @Test + public void testNormalRequest() throws Exception { + runValve("ACT", true, true, false, null); + } + + @Test + public void testDisabledValidSession() throws Exception { + runValve("DIS", true, true, false, null); + } + + @Test + public void testDisabledInvalidSession() throws Exception { + runValve("DIS", false, false, false, "foo=bar"); + } + + @Test + public void testDisabledInvalidSessionWithIgnore() throws Exception { + runValve("DIS", false, true, true, "foo=bar"); + } + + private void runValve(String jkActivation, + boolean validSessionId, + boolean expectInvokeNext, + boolean enableIgnore, + String queryString) throws Exception { + IMocksControl control = EasyMock.createControl(); + ServletContext servletContext = control.createMock(ServletContext.class); + Context ctx = control.createMock(Context.class); + Request request = control.createMock(Request.class); + Response response = control.createMock(Response.class); + + String sessionCookieName = "JSESSIONID"; + String sessionId = "cafebabe"; + String requestURI = "/test/path"; + SessionCookieConfig cookieConfig = new CookieConfig(); + cookieConfig.setDomain("example.com"); + cookieConfig.setName(sessionCookieName); + cookieConfig.setPath("/"); + + // Valve.init requires all of this stuff + EasyMock.expect(ctx.getMBeanKeyProperties()).andStubReturn(""); + EasyMock.expect(ctx.getName()).andStubReturn(""); + EasyMock.expect(ctx.getPipeline()).andStubReturn(new StandardPipeline()); + EasyMock.expect(ctx.getDomain()).andStubReturn("foo"); + EasyMock.expect(ctx.getLogger()).andStubReturn(org.apache.juli.logging.LogFactory.getLog(LoadBalancerDrainingValve.class)); + EasyMock.expect(ctx.getServletContext()).andStubReturn(servletContext); + + // Set up the actual test + EasyMock.expect(request.getAttribute(LoadBalancerDrainingValve.ATTRIBUTE_KEY_JK_LB_ACTIVATION)).andStubReturn(jkActivation); + EasyMock.expect(request.isRequestedSessionIdValid()).andStubReturn(validSessionId); + + ArrayList<Cookie> cookies = new ArrayList<Cookie>(); + if(enableIgnore) { + cookies.add(new Cookie("ignore", "true")); + } + + if(!validSessionId) { + MyCookie cookie = new MyCookie(cookieConfig.getName(), sessionId); + cookie.setPath(cookieConfig.getPath()); + cookie.setValue(sessionId); + + cookies.add(cookie); + + EasyMock.expect(request.getRequestedSessionId()).andStubReturn(sessionId); + EasyMock.expect(request.getRequestURI()).andStubReturn(requestURI); + EasyMock.expect(request.getCookies()).andStubReturn(cookies.toArray(new Cookie[cookies.size()])); + EasyMock.expect(servletContext.getSessionCookieConfig()).andStubReturn(cookieConfig); + EasyMock.expect(request.getServletContext()).andStubReturn(servletContext); + EasyMock.expect(request.getContext()).andStubReturn(ctx); + EasyMock.expect(ctx.getSessionCookiePathUsesTrailingSlash()).andStubReturn(true); + EasyMock.expect(servletContext.getSessionCookieConfig()).andStubReturn(cookieConfig); + EasyMock.expect(request.getQueryString()).andStubReturn(queryString); + + if(!enableIgnore) { + // Response will have cookie deleted + MyCookie expectedCookie = new MyCookie(cookieConfig.getName(), ""); + expectedCookie.setPath(cookieConfig.getPath()); + expectedCookie.setMaxAge(0); + + // These two lines just mean EasyMock.expect(response.addCookie) but for a void method + response.addCookie(expectedCookie); + EasyMock.expect(ctx.getSessionCookieName()).andReturn(sessionCookieName); // Indirect call + String expectedRequestURI = requestURI; + if(null != queryString) + expectedRequestURI = expectedRequestURI + '?' + queryString; + response.setHeader("Location", expectedRequestURI); + response.setStatus(307); + } + } + + Valve next = control.createMock(Valve.class); + + if(expectInvokeNext) { + // Expect the "next" Valve to fire + // Next 2 lines are basically EasyMock.expect(next.invoke(req,res)) but for a void method + next.invoke(request, response); + EasyMock.expectLastCall(); + } + + // Get set to actually test + control.replay(); + + LoadBalancerDrainingValve valve = new LoadBalancerDrainingValve(); + valve.setContainer(ctx); + valve.init(); + valve.setNext(next); + valve.setIgnoreCookieName("ignore"); + valve.setIgnoreCookieValue("true"); + + valve.invoke(request, response); + + control.verify(); + } +} Propchange: tomcat/trunk/test/org/apache/catalina/valves/TestLoadBalancerDrainingValve.java ------------------------------------------------------------------------------ svn:eol-style = native Modified: tomcat/trunk/webapps/docs/changelog.xml URL: http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/changelog.xml?rev=1799498&r1=1799497&r2=1799498&view=diff ============================================================================== --- tomcat/trunk/webapps/docs/changelog.xml (original) +++ tomcat/trunk/webapps/docs/changelog.xml Wed Jun 21 19:05:38 2017 @@ -138,6 +138,10 @@ variable for CGI executables is populated in a consistent way regardless of how the CGI servlet is mapped to a request. (markt) </fix> + <add> + Add LoadBalancerDrainingValve, a Valve designed to reduce the amount of + time required for a node to drain its authenticated users. (schultz) + </add> </changelog> </subsection> <subsection name="Coyote"> Modified: tomcat/trunk/webapps/docs/config/valve.xml URL: http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/config/valve.xml?rev=1799498&r1=1799497&r2=1799498&view=diff ============================================================================== --- tomcat/trunk/webapps/docs/config/valve.xml (original) +++ tomcat/trunk/webapps/docs/config/valve.xml Wed Jun 21 19:05:38 2017 @@ -700,6 +700,81 @@ <section name="Proxies Support"> + <subsection name="Load Balancer Draining Valve"> + <subsection name="Introduction"> + <p> + When using mod_jk or mod_proxy_ajp, the client's session id is used to + determine which back-end server will be used to serve the request. If the + target node is being "drained" (in mod_jk, this is the <i>DISABLED</i> + state; in mod_proxy_ajp, this is the <i>Drain (N)</i> state), requests + for expired sessions can actually cause the draining node to fail to + drain. + </p> + <p> + Unfortunately, AJP-based load-balancers cannot prove whether the + client-provided session id is valid or not and therefore will send any + requests for a session that appears to be targeted to that node to the + disabled (or "draining") node, causing the "draining" process to take + longer than necessary. + </p> + <p> + This Valve detects requests for invalid sessions, strips the session + information from the request, and redirects back to the same URL, where + the load-balancer should choose a different (active) node to handle the + request. This will accelerate the "draining" process for the disabled + node(s). + </p> + + <p> + The activation state of the node is sent by the load-balancer in the + request, so no state change on the node being disabled is necessary. Simply + configure this Valve in your valve pipeline and it will take action when + the activation state is set to "disabled". + </p> + + <p> + You should take care to register this Valve earlier in the Valve pipeline + than any authentication Valves, because this Valve should be able to + redirect a request before any authentication Valve saves a request to a + protected resource. If this happens, a new session will be created and + the draining process will stall because a new, valid session will be + established. + </p> + </subsection><!-- / Introduction --> + + <subsection name="Attributes"> + <p>The <strong>Load Balancer Draining Valve</strong> supports the + following configuration attributes:</p> + + <attributes> + <attribute name="className" required="true"> + <p>Java class name of the implementation to use. This MUST be set to + <strong>org.apache.catalina.valves.LoadBalancerDrainingValve</strong>. + </p> + </attribute> + + <attribute name="redirectStatusCode" required="false"> + <p>Allows setting a custom redirect code to be used when the client + is redirected to be re-balanced by the load-balancer. The default is + 307 TEMPORARY_REDIRECT.</p> + </attribute> + + <attribute name="ignoreCookieName" required="false"> + <p>When used with <code>ignoreCookieValue</code>, a client can present + this cookie (and accompanying value) that will cause this Valve to + do nothing. This will allow you to probe your <i>disabled</i> node + before re-enabling it to make sure that it is working as expected.</p> + </attribute> + + <attribute name="ignoreCookieValue" required="false"> + <p>When used with <code>ignoreCookieName</code>, a client can present + a cookie (and accompanying value) that will cause this Valve to + do nothing. This will allow you to probe your <i>disabled</i> node + before re-enabling it to make sure that it is working as expected.</p> + </attribute> + </attributes> + </subsection><!-- /Attributes --> + </subsection><!-- /Load Balancer Draining Valve --> <subsection name="Remote IP Valve"> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org