Re: Limiting the number of queries/updates to Solr

I tried using the Jetty's QoS filter for rate limiting the queries. It has a good option to apply different rates per URL pattern. However, the same is not being picked up by Solr and the details of the same are shared on https://stackoverflow.com/questions/45536986/why-is-this-qos-jetty-filter-no

Re: Limiting the number of queries/updates to Solr

timeAllowed parameter is a not a good choice for rate limiting and could crash the whole Solr cluster. In fact, timeAllowed parameter should increase the chances of crashing the whole cluster: When the timeAllowed for a query is over, it's client will get a failure but the server handling the quer

Re: Limiting the number of queries/updates to Solr

Hi Hrishikesh, I think SOLR-7344 is probably an important addition to Solr. It could help users isolate analytical queries ( streaming ) , search queries and indexing requests and throttle requests Let's continue the discussion on the Jira On Thu, Aug 3, 2017 at 2:03 AM, Rick Leir wrote: > > >

Re: Limiting the number of queries/updates to Solr

On 2017-08-02 11:33 PM, Shawn Heisey wrote: On 8/2/2017 8:41 PM, S G wrote: Problem is that peak load estimates are just estimates. It would be nice to enforce them from Solr side such that if a rate higher than that is seen at any core, the core will automatically begin to reject the reques

Re: Limiting the number of queries/updates to Solr

At one point I was working on SOLR-7344 (but it fell off the radar due to various reasons). Specifically I built a servlet request filter which implements a customizable queuing mechanism using asynchronous servlet API (Servlet 3 spec). This way you

Re: Limiting the number of queries/updates to Solr

> On Aug 2, 2017, at 8:33 PM, Shawn Heisey wrote: > > IMHO, intentionally causing connections to fail when a limit is exceeded > would not be a very good idea. When the rate gets too high, the first > thing that happens is all the requests slow down. The slowdown could be > dramatic. As the r

Re: Limiting the number of queries/updates to Solr

On 8/2/2017 8:41 PM, S G wrote: > Problem is that peak load estimates are just estimates. > It would be nice to enforce them from Solr side such that if a rate higher > than that is seen at any core, the core will automatically begin to reject > the requests. > Such a feature would contribute to

Limiting the number of queries/updates to Solr

Hi, My team provides Solr clusters to several other teams in my company. We get peak-requirements for query-rate and update-rate from our customers and load-test the cluster based on the same. This helps us arrive at a cluster suitable for a given peak load. Problem is that peak load estimates ar