Re: Change suggestion: more efficient replica state tracking

2023-09-28 Thread Mark Miller
Yeah, I think a jira issue or two was filed for it, but I didn't see anything user facing go in. You can do it for queries by asking the overseer to publish a DOWN state though. It won't drop indexing leadership until you close the core, but it will prevent the temporary slow/hotspot you get if you

Re: Change suggestion: more efficient replica state tracking

2023-09-28 Thread Houston Putman
Yeah ive mentioned it a number of times, but it’s absolutely something we should have. Give up leadership, dont accept new replicas, dont accept new requests. Maybe remove live_node! - Houstob On Thu, Sep 28, 2023 at 6:45 PM David Smiley wrote: > Somewhat related to this, we don't yet have a wa

Re: Build checks for missing javadocs

2023-09-28 Thread Eric Pugh
I did some searching, but didn’t find the specific reference I was hoping to share about how to encourage good practices on a large code set…. Maybe someone else can chime in. Basically, what you do is run additional checks on files that have been changed, so that as we modify a java file, if i

Re: Change suggestion: more efficient replica state tracking

2023-09-28 Thread David Smiley
Somewhat related to this, we don't yet have a way to signal to the cluster that a node will soon be shut down, but has not shut down yet. With such information, we'd prefer to not send queries there, and perhaps could even begin shard/overseer leadership changes. This was mentioned somewhere in c

Build checks for missing javadocs

2023-09-28 Thread David Smiley
When I see a class without any javadocs, especially a public top level class, it's really sad and disappointing. My colleagues have noted the same. I'd like to see a build mechanism to up our level of standards here. Of course we have tons of existing such classes so a solution would have to som

Re: [DISCUSS] 9.4 release

2023-09-28 Thread Alex Deparvu
Hi, It would be unfortunate to get this close and miss out on the Lucene release. I would want to give it one more day, if things do not work out, I will go ahead with the release. best, alex On Thu, Sep 28, 2023 at 2:25 PM Jan Høydahl wrote: > I see there is energy and traction into finaliz

Re: [DISCUSS] 9.4 release

2023-09-28 Thread Jan Høydahl
I see there is energy and traction into finalizing the Lucene 9.8 integration, so that's fine as long as we believe the risk to be low-ish. Jan > 28. sep. 2023 kl. 21:53 skrev Anshum Gupta : > > I don't like holding up releases, but it's nice to not have too many > releases from an operational

Re: Change suggestion: more efficient replica state tracking

2023-09-28 Thread Mark Miller
That did require some changes around live node handeling, which is why a different approach as you suggest would also be reasonable. You still do want to solve for the original motivation of DOWN - stopping search traffic to the node before things start closing.

Re: Change suggestion: more efficient replica state tracking

2023-09-28 Thread Mark Miller
Yeah, I took the DOWN state out all together in shutdown as its problematic and effectively sugar for the user view of the cluster state - as far as the system goes, if the ephemeral live node is gone, that node is down, regardless of the replica state. There is some value in being able to remove a

Re: [solr] branch main updated: More test cases for Coordinator node role (#1782)

2023-09-28 Thread David Smiley
It's a shame to see that we seem to maybe need to use the underlying Apache HttpClient in tests thanks to the presence of SSL randomization. Hossman, do you have ideas on how we could not have such a dependency? ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidws

Re: Re: [DISCUSS] 9.4 release

2023-09-28 Thread David Smiley
By default, Solr returns all metrics, of which there are a ton of them! That's great for looking at metrics if you are exploring them for the first time. But as you'll see in the ref guide, there are many filtering options which I would consider to be mandatory if you are going to use metrics "for

Re: [DISCUSS] 9.4 release

2023-09-28 Thread Anshum Gupta
I don't like holding up releases, but it's nice to not have too many releases from an operational standpoint. If fixing Solr for the Lucene upgrade will take time, I think it's worth the trade-off and moving forward with the older version of Lucene. On Thu, Sep 28, 2023 at 6:05 AM Jan Høydahl wro

Re: Change suggestion: more efficient replica state tracking

2023-09-28 Thread Justin Sweeney
You are right, sorry. We are also running a fork of Solr and in our fork we have made some optimizations to avoid processing DOWNNODE messages for nodes that only host PRS collections. Those optimizations have not made it upstream at this point. I can take a look at upstreaming those changes or som

Re: [DISCUSS] 9.4 release

2023-09-28 Thread Christine Poerschke (BLOOMBERG/ LONDON)
Hello, I have no strong views on Lucene 9.7 or Lucene 9.8 for Solr 9.4 but if it's not too late to ask, I'd love to see the (small) https://github.com/apache/solr/pull/1950 change for https://issues.apache.org/jira/browse/SOLR-16994 considered for being in the 9.4 release. Thanks, Christine

RE: Re: [DISCUSS] 9.4 release

2023-09-28 Thread Chris Schomaker
I'd like to raise an issue that I'm having with Solr 9.3 that might make a good case for getting another release (9.3.1 or 9.4) out soon. I've had a 9.3 cluster of 180 nodes up and running for about a week. After the weekend, I noticed that metrics from the cluster were failing to ship. Upon inves

Re: [DISCUSS] 9.4 release

2023-09-28 Thread Jan Høydahl
Can I propose we do NOT hold the 9.4 release to add the (too fresh) Lucene 9.8? We can do a rapid 9.5 with any benefits reaped from the new Lucene version. Jan > 28. sep. 2023 kl. 00:36 skrev Alex Deparvu : > > Hi, > > Some updates to share on the upgrade. > I have identified a cause for most