madrob commented on a change in pull request #1686: URL: https://github.com/apache/lucene-solr/pull/1686#discussion_r465122951
########## File path: solr/solr-ref-guide/src/rate-limiters.adoc ########## @@ -0,0 +1,116 @@ += Request Rate Limiters +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +Solr allows rate limiting per request type. Each request type can be allocated a maximum allowed number of concurrent requests +that can be active. The default rate limiting is implemented for updates and searches. + +If a request exceeds the request quota, further incoming requests are rejected with HTTP error code 429 (Too Many Requests). + +Note that rate limiting works at an instance (JVM) level, not at a core or collection level. Consider that when planning capacity. + +== When To Use Rate Limiters +Rate limiters should be used when the user wishes to allocate a guaranteed capacity of the request threadpool to a specific +request type. Indexing and search requests are mostly competing with each other for CPU resources. This becomes especially +pronounced under high stress in production workloads. + +== Rate Limiter Configurations +The default rate limiter is search rate limiter. Accordingly, it can be configured in web.xml under initParams for +SolrRequestFilter. + +[source,xml] +---- +<filter-name>SolrRequestFilter</filter-name> +---- + +==== Enable Query Rate Limiter +Controls enabling of query rate limiter. Default value is false. +[source,xml] +---- +<param-name>isQueryRateLimiterEnabled</param-name> +---- +[source,xml] +---- +<param-value>true</param-value> +---- + +==== Maximum Number Of Concurrent Requests +Allows setting maximum concurrent search requests at a given point in time. Default value is number of cores * 3. +[source,xml] +---- +<param-name>maxQueryRequests</param-name> +---- +[source,xml] +---- +<param-value>15</param-value> +---- + +==== Request Slot Allocation Wait Time +Wait time in ms for which a request will wait for a slot to be available when all slots are full, +before the request is put into the wait queue. This allows requests to have a chance to proceed if +the unavailability of the request slots for this rate limiter is a transient phenomenon. Default value +is -1, indicating no wait. 0 will represent the same -- no wait. +[source,xml] +---- +<param-name>queryWaitForSlotAllocationInMS</param-name> +---- +[source,xml] +---- +<param-value>100</param-value> +---- + +==== Slot Borrowing Enabled +If slot borrowing (described below) is enabled or not. Default value is false. +[source,xml] +---- +<param-name>queryAllowSlotBorrowing</param-name> +---- +[source,xml] +---- +<param-value>true</param-value> +---- + +==== Guaranteed Slots +The number of guaranteed slots that the query rate limiter will reserve irrespective +of the load of query requests. This is used only if slot borrowing is enabled and acts +as the threshold beyond which query rate limiter will not allow other request types to +borrow slots from its quota. Default value is allowed number of concurrent requests / 2. +[source,xml] +---- +<param-name>queryGuaranteedSlots</param-name> +---- +[source,xml] +---- +<param-value>200</param-value> +---- + +== Salient Points + +These are some of the things to keep in mind when using rate limiters + +=== Over Subscribing +It is possible to define a size of quota for a request type which exceeds the size +of the available threadpool. Solr does not enforce rules on the size of a quota that +can be define for a request type. This is intentionally done to allow users full +control on their quota allocation. However, if the quota exceeds the available threadpool's +size, the standard queuing policies of the threadpool will kick in. + +=== Slot Borrowing +If a quota does not have backlog but other quotas do, then the relatively less busier quota can +"borrow" slot from the busier quotas. This is done on a round robin basis today with a futuristic Review comment: Is there a JIRA for this? Would be good to link to it if it exists (or create it if it does not) ########## File path: solr/solr-ref-guide/src/rate-limiters.adoc ########## @@ -0,0 +1,116 @@ += Request Rate Limiters +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +Solr allows rate limiting per request type. Each request type can be allocated a maximum allowed number of concurrent requests +that can be active. The default rate limiting is implemented for updates and searches. + +If a request exceeds the request quota, further incoming requests are rejected with HTTP error code 429 (Too Many Requests). + +Note that rate limiting works at an instance (JVM) level, not at a core or collection level. Consider that when planning capacity. + +== When To Use Rate Limiters +Rate limiters should be used when the user wishes to allocate a guaranteed capacity of the request threadpool to a specific +request type. Indexing and search requests are mostly competing with each other for CPU resources. This becomes especially +pronounced under high stress in production workloads. + +== Rate Limiter Configurations +The default rate limiter is search rate limiter. Accordingly, it can be configured in web.xml under initParams for +SolrRequestFilter. + +[source,xml] +---- +<filter-name>SolrRequestFilter</filter-name> +---- + +==== Enable Query Rate Limiter +Controls enabling of query rate limiter. Default value is false. +[source,xml] +---- +<param-name>isQueryRateLimiterEnabled</param-name> +---- +[source,xml] +---- +<param-value>true</param-value> +---- + +==== Maximum Number Of Concurrent Requests +Allows setting maximum concurrent search requests at a given point in time. Default value is number of cores * 3. +[source,xml] +---- +<param-name>maxQueryRequests</param-name> +---- +[source,xml] +---- +<param-value>15</param-value> +---- + +==== Request Slot Allocation Wait Time +Wait time in ms for which a request will wait for a slot to be available when all slots are full, +before the request is put into the wait queue. This allows requests to have a chance to proceed if +the unavailability of the request slots for this rate limiter is a transient phenomenon. Default value +is -1, indicating no wait. 0 will represent the same -- no wait. Review comment: Add a indication here that setting higher slot allocation wait times may be reflected as higher QTime. It's probably apparent to us now, but somebody debugging why their queries are getting slow will likely appreciate the hint. ########## File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpSolrClient.java ########## @@ -387,7 +394,13 @@ protected HttpRequestBase createMethod(@SuppressWarnings({"rawtypes"})SolrReques throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "GET can't send streams!"); } - return new HttpGet(basePath + path + wparams.toQueryString()); + //return new HttpGet(basePath + path + wparams.toQueryString()); + + HttpGet result = new HttpGet(basePath + path + wparams.toQueryString()); + + result.addHeader(contextHeaders[0]); Review comment: let's extract this to a method, along with lines 445 and 366 ########## File path: solr/solr-ref-guide/src/rate-limiters.adoc ########## @@ -0,0 +1,116 @@ += Request Rate Limiters +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +Solr allows rate limiting per request type. Each request type can be allocated a maximum allowed number of concurrent requests +that can be active. The default rate limiting is implemented for updates and searches. + +If a request exceeds the request quota, further incoming requests are rejected with HTTP error code 429 (Too Many Requests). + +Note that rate limiting works at an instance (JVM) level, not at a core or collection level. Consider that when planning capacity. + +== When To Use Rate Limiters +Rate limiters should be used when the user wishes to allocate a guaranteed capacity of the request threadpool to a specific +request type. Indexing and search requests are mostly competing with each other for CPU resources. This becomes especially Review comment: This talks about indexing and searching requests, but I only see configuration for query rate limiters below. If update requests are not considered here, we should look at that as follow-on work, and the docs should reflect the state of things. ########## File path: solr/solrj/src/java/org/apache/solr/client/solrj/request/QueryRequest.java ########## @@ -76,5 +76,9 @@ public SolrParams getParams() { return query; } + @Override + public String getRequestType() { + return SolrRequestType.QUERY.toString(); Review comment: Something about this that I think bothers me is that we don't make a differentiation between distinct handlers, so we can't use this to build QoS levels. Like I can't say that RTG should always go through while `/select` might be acceptable to throttle and `/spellcheck` is even lower priority. ########## File path: solr/solrj/src/java/org/apache/solr/client/solrj/request/SolrPing.java ########## @@ -54,6 +54,11 @@ protected SolrPingResponse createResponse(SolrClient client) { public ModifiableSolrParams getParams() { return params; } + + @Override + public String getRequestType() { + return SolrRequestType.QUERY.toString(); Review comment: This is maybe more of an admin request? WDYT? ########## File path: solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpSolrClient.java ########## @@ -387,7 +394,13 @@ protected HttpRequestBase createMethod(@SuppressWarnings({"rawtypes"})SolrReques throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "GET can't send streams!"); } - return new HttpGet(basePath + path + wparams.toQueryString()); + //return new HttpGet(basePath + path + wparams.toQueryString()); Review comment: unneeded? ########## File path: solr/solr-ref-guide/src/rate-limiters.adoc ########## @@ -0,0 +1,116 @@ += Request Rate Limiters +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +Solr allows rate limiting per request type. Each request type can be allocated a maximum allowed number of concurrent requests +that can be active. The default rate limiting is implemented for updates and searches. + +If a request exceeds the request quota, further incoming requests are rejected with HTTP error code 429 (Too Many Requests). + +Note that rate limiting works at an instance (JVM) level, not at a core or collection level. Consider that when planning capacity. Review comment: Can we file a JIRA for finer grained request rate limiting? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
