Re: Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-08 Thread David Smiley
My comment, referring to "query tracking" was not a reference to your changes. It's a reference to Atri's changes for the query cancellation feature -- see CancellableCollector if you are curious On Wed, Feb 7, 2024 at 3:44 AM Andrzej Białecki wrote: > > This does reuse QueryTimeout infra and al

Re: Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-07 Thread Andrzej Białecki
This does reuse QueryTimeout infra and already existing collectors, it just extends / refactors it to allow other criteria than wall-time clock. > On 6 Feb 2024, at 23:01, David Smiley wrote: > > I will look at this! > > - Ticket for my refactoring > https://issues.apache.org/jira/browse/S

Re: Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-06 Thread Gus Heck
doh, cut/paste error https://issues.apache.org/jira/browse/SOLR-17140 On Tue, Feb 6, 2024 at 5:02 PM David Smiley wrote: > I will look at this! > >- Ticket for my refactoring >https://issues.apache.org/jira/browse/SOLR-1714 > > 1714, ehh? :-) > > Another thought I had that I want to shar

Re: Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-06 Thread David Smiley
I will look at this! - Ticket for my refactoring https://issues.apache.org/jira/browse/SOLR-1714 1714, ehh? :-) Another thought I had that I want to share is that I think the query tracking and cancellation could use the same QueryTimeout infrastructure. I don't like that it adds its own

Re: Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-06 Thread Gus Heck
To save folks some clicking around, and to provide a little more detail, here are a few links. - The first step was to refactor/replace QueryTimeoutImpl: PR https://github.com/apache/solr/pull/2237. - Building on that will be Andrzej's PR https://github.com/apache/solr/pull/2244/commi

Re: Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-06 Thread rajani m
I have updated the jira with my vote and totally agree that solr lacks this way to limit the usage and a single query bringing all the nodes in the cluster down. I'd love to review/contribute to PR . Thank you for bringing up this issue Andrzej. On Mon, Feb 5, 2024 at 4:16 PM Andrzej Białecki

Query Limits - extending timeAllowed to implement thread CPU/memory limits for a query

2024-02-05 Thread Andrzej Białecki
Hi all, I’d like to draw your attention to SOLR-17138. Jira description gives a more detailed background of the issue - but in short, Solr lacks a robust way to limit the resource usage (be it CPU time or memory consumption or other resource) on a per-query basis. CircuitBreakers help to an ext