noblepaul commented on a change in pull request #1906:
URL: https://github.com/apache/lucene-solr/pull/1906#discussion_r492668757
##########
File path: solr/core/src/java/org/apache/solr/handler/ClusterAPI.java
##########
@@ -171,10 +172,30 @@ public void setPlacementPlugin(PayloadObj<Map<String,
Object>> obj) {
// Need to reset to null first otherwise the mappings in
placementPluginConfig are added to existing ones
// in /clusterprops.json rather than replacing them. If removing the
config, that's all we do.
clusterProperties.setClusterProperties(
-
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+ if (!unset) {
+ clusterProperties.setClusterProperties(
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
placementPluginConfig));
+ }
+
+ } catch (Exception e) {
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error
in API", e);
+ }
+ }
+
+ @Command(name = "set-ratelimiters")
+ public void setRateLimiters(PayloadObj<Map<String, Object>> obj) {
+ Map<String, Object> rateLimiterConfig = obj.getDataMap();
+ final boolean unset = rateLimiterConfig.isEmpty();
Review comment:
wouldn't you do `set-ratelimters : null` instead of `set-ratelimters :
{}`
##########
File path: solr/core/src/java/org/apache/solr/handler/ClusterAPI.java
##########
@@ -171,10 +172,30 @@ public void setPlacementPlugin(PayloadObj<Map<String,
Object>> obj) {
// Need to reset to null first otherwise the mappings in
placementPluginConfig are added to existing ones
// in /clusterprops.json rather than replacing them. If removing the
config, that's all we do.
clusterProperties.setClusterProperties(
-
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+ if (!unset) {
+ clusterProperties.setClusterProperties(
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
placementPluginConfig));
+ }
+
+ } catch (Exception e) {
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error
in API", e);
+ }
+ }
+
+ @Command(name = "set-ratelimiters")
Review comment:
why plural? are there multiple rate limiters?
##########
File path: solr/core/src/java/org/apache/solr/handler/ClusterAPI.java
##########
@@ -171,10 +172,30 @@ public void setPlacementPlugin(PayloadObj<Map<String,
Object>> obj) {
// Need to reset to null first otherwise the mappings in
placementPluginConfig are added to existing ones
// in /clusterprops.json rather than replacing them. If removing the
config, that's all we do.
clusterProperties.setClusterProperties(
-
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+ if (!unset) {
+ clusterProperties.setClusterProperties(
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
placementPluginConfig));
+ }
+
+ } catch (Exception e) {
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error
in API", e);
+ }
+ }
+
+ @Command(name = "set-ratelimiters")
+ public void setRateLimiters(PayloadObj<Map<String, Object>> obj) {
Review comment:
Why not have a strongly typed java object? why are you using a loose
`Map` object
##########
File path: solr/core/src/java/org/apache/solr/servlet/RateLimiterConfig.java
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.servlet;
Review comment:
why is this in the `o.a.s.servlet` package? It's not a servlet/filter
##########
File path: solr/solr-ref-guide/src/rate-limiters.adoc
##########
@@ -31,67 +31,44 @@ pronounced under high stress in production workloads. The
current implementation
resources for indexing.
== Rate Limiter Configurations
-The default rate limiter is search rate limiter. Accordingly, it can be
configured in `web.xml` under `initParams` for
-`SolrRequestFilter`.
-
-[source,xml]
-----
-<filter-name>SolrRequestFilter</filter-name>
-----
+The default rate limiter is search rate limiter. Accordingly, it can be
configured using the following command:
+
+ curl -X POST -H 'Content-type:application/json' -d '{
+ "set-ratelimiters": {
+ "rlEnabled": "true",
Review comment:
why have the `rl` prefix everywhere? This is already used in the context
of a ratelimiter, why use attributes like `rlGuaranteedSlots` just use
`guaranteedSlots`
##########
File path: solr/core/src/java/org/apache/solr/servlet/QueryRateLimiter.java
##########
@@ -17,39 +17,82 @@
package org.apache.solr.servlet;
-import javax.servlet.FilterConfig;
+import java.util.Map;
import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.common.cloud.SolrZkClient;
+import org.apache.solr.common.cloud.ZkStateReader;
+import org.apache.solr.common.util.Utils;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.data.Stat;
-import static
org.apache.solr.servlet.RateLimitManager.DEFAULT_CONCURRENT_REQUESTS;
-import static
org.apache.solr.servlet.RateLimitManager.DEFAULT_SLOT_ACQUISITION_TIMEOUT_MS;
+import static org.apache.solr.servlet.RateLimiterConfig.RL_CONFIG_KEY;
/** Implementation of RequestRateLimiter specific to query request types. Most
of the actual work is delegated
* to the parent class but specific configurations and parsing are handled by
this class.
*/
public class QueryRateLimiter extends RequestRateLimiter {
- final static String IS_QUERY_RATE_LIMITER_ENABLED =
"isQueryRateLimiterEnabled";
- final static String MAX_QUERY_REQUESTS = "maxQueryRequests";
- final static String QUERY_WAIT_FOR_SLOT_ALLOCATION_INMS =
"queryWaitForSlotAllocationInMS";
- final static String QUERY_GUARANTEED_SLOTS = "queryGuaranteedSlots";
- final static String QUERY_ALLOW_SLOT_BORROWING = "queryAllowSlotBorrowing";
-
- public QueryRateLimiter(FilterConfig filterConfig) {
- super(constructQueryRateLimiterConfig(filterConfig));
+
+ public QueryRateLimiter(SolrZkClient solrZkClient) {
+ super(constructQueryRateLimiterConfig(solrZkClient));
+ }
+
+ @SuppressWarnings({"unchecked"})
+ public void processConfigChange(Map<String, Object> properties) {
+ RateLimiterConfig rateLimiterConfig = getRateLimiterConfig();
+ Map<String, Object> propertiesMap = (Map<String, Object>)
properties.get(RL_CONFIG_KEY);
+
+ constructQueryRateLimiterConfigInternal(propertiesMap, rateLimiterConfig);
+ }
+
+ // To be used in initialization
+ @SuppressWarnings({"unchecked"})
+ private static RateLimiterConfig
constructQueryRateLimiterConfig(SolrZkClient zkClient) {
+ try {
+
+ if (zkClient == null) {
+ return new RateLimiterConfig(SolrRequest.SolrRequestType.QUERY);
+ }
+
+ Map<String, Object> clusterPropsJson = (Map<String, Object>)
Utils.fromJSON(zkClient.getData(ZkStateReader.CLUSTER_PROPS, null, new Stat(),
true));
+ Map<String, Object> propertiesMap = (Map<String, Object>)
clusterPropsJson.get(RL_CONFIG_KEY);
+ RateLimiterConfig rateLimiterConfig = new
RateLimiterConfig(SolrRequest.SolrRequestType.QUERY);
+
+ constructQueryRateLimiterConfigInternal(propertiesMap,
rateLimiterConfig);
+
+ return rateLimiterConfig;
+ } catch (KeeperException.NoNodeException e) {
+ return new RateLimiterConfig(SolrRequest.SolrRequestType.QUERY);
+ } catch (KeeperException | InterruptedException e) {
+ throw new RuntimeException("Error reading cluster property",
SolrZkClient.checkInterrupted(e));
+ }
}
- protected static RequestRateLimiter.RateLimiterConfig
constructQueryRateLimiterConfig(FilterConfig filterConfig) {
- RequestRateLimiter.RateLimiterConfig queryRateLimiterConfig = new
RequestRateLimiter.RateLimiterConfig();
+ private static void constructQueryRateLimiterConfigInternal(Map<String,
Object> propertiesMap, RateLimiterConfig rateLimiterConfig) {
Review comment:
Why are you not using a POJO and jackson deserialization, like it's done
elsewhere? why all these boilerplate code
##########
File path: solr/solr-ref-guide/src/rate-limiters.adoc
##########
@@ -31,67 +31,44 @@ pronounced under high stress in production workloads. The
current implementation
resources for indexing.
== Rate Limiter Configurations
-The default rate limiter is search rate limiter. Accordingly, it can be
configured in `web.xml` under `initParams` for
-`SolrRequestFilter`.
-
-[source,xml]
-----
-<filter-name>SolrRequestFilter</filter-name>
-----
+The default rate limiter is search rate limiter. Accordingly, it can be
configured using the following command:
+
+ curl -X POST -H 'Content-type:application/json' -d '{
+ "set-ratelimiters": {
+ "rlEnabled": "true",
Review comment:
why string `"true"` why not `true` ?
##########
File path: solr/core/src/java/org/apache/solr/handler/ClusterAPI.java
##########
@@ -171,10 +172,30 @@ public void setPlacementPlugin(PayloadObj<Map<String,
Object>> obj) {
// Need to reset to null first otherwise the mappings in
placementPluginConfig are added to existing ones
// in /clusterprops.json rather than replacing them. If removing the
config, that's all we do.
clusterProperties.setClusterProperties(
-
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+
Collections.singletonMap(PlacementPluginConfigImpl.PLACEMENT_PLUGIN_CONFIG_KEY,
null));
+ if (!unset) {
Review comment:
why 2 write ops? Only one should be enough
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
