Re: Backup, Restore, SplitShard should always internally be "async"

2023-06-29 Thread Ishan Chattopadhyaya
> I'm unsure Solr async responses should include documentation/help. I think it should return the URL to check status at. Not "documentation/help". On Thu, 29 Jun, 2023, 10:18 pm David Smiley, wrote: > To be extra clear, my proposal is about the *internal* operation of certain > commands. Thus

Re: Time for a 9.3 release?

2023-06-29 Thread Uwe Schindler
Please also make sure to enable Panama-Vectors if and only if Lucene 9.7+ is used with Java 20 or Java 21. This needs command line options added! Uwe Am 28.06.2023 um 00:42 schrieb Alessandro Benedetti: +1 and if you give me more or less 1-week we'll be able to push also Function Queries for v

Re: Backup, Restore, SplitShard should always internally be "async"

2023-06-29 Thread David Smiley
To be extra clear, my proposal is about the *internal* operation of certain commands. Thus a user/client issuing a backup command in the synchronous way will not be impacted at all; it'll wait and return the response. If a user/client issues an async command, presumably they have looked at the do

Re: Enable CodeQL scanning in Github Actions?

2023-06-29 Thread David Smiley
Thanks for raising awareness to this initiative. Given there's a PR, let's discuss there. ~ David On Mon, Jun 26, 2023 at 2:17 PM Jan Høydahl wrote: > Hi, > > We got a contribution from GitHub Security Lab in > https://github.com/apache/solr/pull/1721 to enable CodeQL < > https://codeql.githu

Re: Backup, Restore, SplitShard should always internally be "async"

2023-06-29 Thread Ishan Chattopadhyaya
+1. I think as an improvement, a helpful message on how to track the status of the async request should be returned as part of the response of async collection api calls. Even 1s poll for these commands is okay in real world. On Thu, 29 Jun, 2023, 8:05 pm David Smiley, wrote: > Some (most?) clu

Announce: SOLR-16837 introducing AffinityPlacementFactory.withCollectionSards

2023-06-29 Thread Mikhail Khludnev
Hello In scope of SOLR-16837 AffinityPlacementFactory obtained new property withCollectionSards. It allows to collocate shradN of one collection with shardN of another collection. The purpose is SOLR-16717 joining a sharded collection on both sides (will commit soon). Both should be released in 9.3

Backup, Restore, SplitShard should always internally be "async"

2023-06-29 Thread David Smiley
Some (most?) cluster admin commands can be executed in an async mode: https://solr.apache.org/guide/solr/latest/configuration-guide/collections-api.html#asynchronous-calls What I find really strange and unnecessary is that some of the commands *internally* operate differently based on whether the

Re: Limiting Backup IO

2023-06-29 Thread Pierre Salagnac
Jason, I haven't done much scalability testing, so it's hard to give accurate numbers on when we start having issues. For the environment I looked in detail we run a 16 nodes cluster, and the collection I wasn't able to backup has about 1500 shards, ~1.5 GB each. Core backups/restores are expensiv