Re: [VOTE] Release Solr 9.8.1 RC1

[Kevin Risden]

+1

SUCCESS! [0:39:40.103601]

Kevin Risden


On Thu, Mar 6, 2025 at 4:28 PM Houston Putman  wrote:

> Please vote for release candidate 1 for Solr 9.8.1
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.8.1-RC1-rev-dab835e4a3249780230e5b1cefebffdf15ad5c96
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.8.1-RC1-rev-dab835e4a3249780230e5b1cefebffdf15ad5c96
>
> You can build a release-candidate of the official docker images (full &
> slim) using the following command:
>
> SOLR_DOWNLOAD_SERVER=
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.8.1-RC1-rev-dab835e4a3249780230e5b1cefebffdf15ad5c96/solr
> &&
> 
> \
>   docker build $SOLR_DOWNLOAD_SERVER/9.8.1/docker/Dockerfile.official-full
> \
> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> -t solr-rc:9.8.1-1 && \
>   docker build $SOLR_DOWNLOAD_SERVER/9.8.1/docker/Dockerfile.official-slim
> \
> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> -t solr-rc:9.8.1-1-slim
>
> The vote will be open for at least 72 hours i.e. until 2025-03-09 22:00
> UTC.
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
>

[VOTE] Release Solr 9.8.1 RC1

[Houston Putman]

Please vote for release candidate 1 for Solr 9.8.1

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/solr/solr-9.8.1-RC1-rev-dab835e4a3249780230e5b1cefebffdf15ad5c96

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/solr/solr-9.8.1-RC1-rev-dab835e4a3249780230e5b1cefebffdf15ad5c96

You can build a release-candidate of the official docker images (full &
slim) using the following command:

SOLR_DOWNLOAD_SERVER=
https://dist.apache.org/repos/dist/dev/solr/solr-9.8.1-RC1-rev-dab835e4a3249780230e5b1cefebffdf15ad5c96/solr
&& \
  docker build $SOLR_DOWNLOAD_SERVER/9.8.1/docker/Dockerfile.official-full \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.8.1-1 && \
  docker build $SOLR_DOWNLOAD_SERVER/9.8.1/docker/Dockerfile.official-slim \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.8.1-1-slim

The vote will be open for at least 72 hours i.e. until 2025-03-09 22:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Re: Environment Variables in the Solr CLI...

[Mike Drob]

I think the guidance from clig.dev is pretty clear - when the configuration
is generally stable from one invocation to the next, using environment
variables is nice. Imagine starting a container, you set ZK_HOST once and
then it's good for every invocation of the tools. Otherwise you have to
have logic for adding that flag to every command, which probably looks like
appending "--zk-host $ZK_HOST" every time, so you're setting the
environment variable in your deployment anyway.

On Thu, Mar 6, 2025 at 8:48 AM David Eric Pugh 
wrote:

> I've been thinking about the use of environment variables in the Solr CLI,
> and trying to figure out why it feels like a bad thing...I think that
> the reason I don't like them is that it feels like we are injecting "magic"
> into our CLI, and this makes them actually more brittle and difficult to
> comprehend.
>
> Some recent related tickets:
>   https://github.com/apache/solr-operator/issues/762
>   https://github.com/apache/solr/pull/3240
>
> In the recent revamp of the CLI a LOT of work went into makign the CLI
> more obvious.  We went through all the various arguments and cleaned them
> up so we didn't have different tools have overlapping parameters.   We
> standared the descriptive terminology we used for the various options.
> And we tried to make the CLI smarter about required and optional
> parameters.  Lastly, for situations where you could do A or B, like passing
> in Solr URL OR a ZK Host, we actually enforce that.
>
> We embraced Commons CLI, not because it's the best tool, but because it's
> the most widely used one in our code base, and pushed it to be used across
> all the CLI scripts (Prometheus Exporter, core CLI, the HDFS scripts.
>
> So...   Why this rant?
>
> It feels like using the environment variables to look up the connection
> information pattern that we are using goes against making our CLI clearer,
> and is why we are seeing various bugs in solr-operator.   The contract
> between the CLI and the caller is the set of options that you pass to the
> CLI.   Using environment variables avoids the checking that Commmons CLI
> does for us.   We should be explicit when we call a script and pass in
> everything it needs.
>
> I was very much hoping the guide lines we were using to steer us in
> revamping the CLI would support my contention that the use of the
> environment variables to specify connectivity options is a bad practice,
> https://clig.dev/#configuration, but unfortunantly it doesn't clearly
> state that;-)
>
> I mostly wanted to share this to get it off my chest, versus any specific
> "we must change this now" request
> Thanks for reading!
> Eric
>

Environment Variables in the Solr CLI...

[David Eric Pugh]

I've been thinking about the use of environment variables in the Solr CLI, and 
trying to figure out why it feels like a bad thing...    I think that the 
reason I don't like them is that it feels like we are injecting "magic" into 
our CLI, and this makes them actually more brittle and difficult to comprehend. 
  

Some recent related tickets:
  https://github.com/apache/solr-operator/issues/762
  https://github.com/apache/solr/pull/3240

In the recent revamp of the CLI a LOT of work went into makign the CLI more 
obvious.  We went through all the various arguments and cleaned them up so we 
didn't have different tools have overlapping parameters.   We standared the 
descriptive terminology we used for the various options.   And we tried to make 
the CLI smarter about required and optional parameters.  Lastly, for situations 
where you could do A or B, like passing in Solr URL OR a ZK Host, we actually 
enforce that.

We embraced Commons CLI, not because it's the best tool, but because it's the 
most widely used one in our code base, and pushed it to be used across all the 
CLI scripts (Prometheus Exporter, core CLI, the HDFS scripts.

So...   Why this rant?

It feels like using the environment variables to look up the connection 
information pattern that we are using goes against making our CLI clearer, and 
is why we are seeing various bugs in solr-operator.   The contract between the 
CLI and the caller is the set of options that you pass to the CLI.   Using 
environment variables avoids the checking that Commmons CLI does for us.   We 
should be explicit when we call a script and pass in everything it needs.

I was very much hoping the guide lines we were using to steer us in revamping 
the CLI would support my contention that the use of the environment variables 
to specify connectivity options is a bad practice, 
https://clig.dev/#configuration, but unfortunantly it doesn't clearly state 
that;-) 

I mostly wanted to share this to get it off my chest, versus any specific "we 
must change this now" request
Thanks for reading!
Eric

Re: Bugfix release Solr 9.8.1

[Houston Putman]

Sounds good, starting to build the candidate.

I also have a draft of the release notes up:
https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote9_8_1

- Houston

On Wed, Mar 5, 2025 at 4:40 PM Jan Høydahl  wrote:

> From an operator point of view, upgrading to 0.9.1 which then works for
> all 9.x versions is a good path.
>
> For the solr CLI bug of not reading ZK_HOST, it is annoying but not
> serious enough to block the release.
>
> Jan
>
> > 5. mars 2025 kl. 22:37 skrev Houston Putman :
> >
> > Thanks for mentioning that Jan. I think Jason's idea of just doing a
> 0.9.1
> > release of the operator is good. That way we can unblock this release and
> > take the time we need to make the CLI behave the right way.
> >
> > Thoughts?
> >
> > - Houston
> >
> > On Tue, Mar 4, 2025 at 7:25 PM Jan Høydahl 
> wrote:
> >
> >> I'd like to get https://issues.apache.org/jira/browse/SOLR-17690 into
> >> 9.8.1. It's a blocker for bootstrapping custom security.json with
> >> solr-operator 0.9 and Solr 9.8.
> >>
> >> PR in progress.
> >>
> >> Jan
> >>
> >>> 28. feb. 2025 kl. 18:59 skrev Christine Poerschke (BLOOMBERG/ LONDON) <
> >> cpoersc...@bloomberg.net>:
> >>>
> >>> Hi Houston,
> >>>
> >>> Yes, I can aim to merge this on Monday London time but would appreciate
> >> at least one extra pair of eyes for the change to the default logic:
> >> https://github.com/apache/solr/pull/3224/files
> >>>
> >>> Thanks,
> >>> Christine
> >>>
> >>> From: dev@solr.apache.org At: 02/28/25 16:27:34 UTCTo:
> >> dev@solr.apache.org
> >>> Subject: Re: Bugfix release Solr 9.8.1
> >>>
> >>> Thanks to you both!
> >>>
> >>> Christine was trying to fix
> >>> https://issues.apache.org/jira/browse/SOLR-17686 and get it in for
> >> 8.9.1.
> >>>
> >>> Christine, do you think you can get this is by tuesday next week? If so
> >> I’m
> >>> happy to wait. Otherwise, I’ll likely create an RC today.
> >>>
> >>> - Houston
> >>>
> >>> On Fri, Feb 28, 2025 at 9:45 AM David Smiley 
> wrote:
> >>>
>  Sanjay and I teamed up and fixed
>  https://issues.apache.org/jira/browse/SOLR-17644 which has been
>  backported.  I suppose the release is unblocked!
> 
>  On Tue, Feb 25, 2025 at 9:12 AM David Smiley 
> >> wrote:
> 
> > https://issues.apache.org/jira/browse/SOLR-17644 is a significant
> bug
> > that should get into 9.8.1.  We've got a PR... probably a bit
> drafty; a
> > complex problem.
> >
> > On Tue, Feb 25, 2025 at 8:35 AM Jason Gerlowski <
> gerlowsk...@gmail.com
> >>>
> > wrote:
> >
> >> I just finished backporting SOLR-17677 this morning; so "all clear"
> on
>  my
> >> end!
> >>
> >> On Mon, Feb 24, 2025 at 3:45 PM Houston Putman 
> >> wrote:
> >>>
> >>> How are we looking on the backports for all of these?
> >>>
> >>> - Houston
> >>>
> >>> On Thu, Feb 20, 2025 at 3:32 PM Jason Gerlowski <
>  gerlowsk...@gmail.com>
> >>> wrote:
> >>>
>  Alright - I've put together a PR for SOLR-17677 that can at least
>  block the problem-queries:
> https://github.com/apache/solr/pull/3203
>  .
>  Hoping I can merge this tomorrow or Monday, if that's alright with
>  folks?
> 
>  Longer term we probably want to remove the reliance on
>  SolrIndexSearcher that causes the problem here, but that's
> probably
>  not quick enough to tackle on the 9.8.1 timeframe
> 
>  Sorry for my part in holding things up!
> 
>  Best,
> 
>  Jason
> 
>  On Wed, Feb 19, 2025 at 11:47 PM David Smiley  >
> >> wrote:
> >
> > https://github.com/apache/solr/pull/3199
> > Hoping to get this tiny compatibility fix into 9.8.1.
> >
> > On Wed, Feb 19, 2025 at 1:25 PM Houston Putman <
>  hous...@apache.org>
>  wrote:
> >
> >> Sounds good, let me know how to proceed.
> >>
> >> - Houston
> >>
> >> On Wed, Feb 19, 2025 at 10:44 AM Jason Gerlowski <
>  gerlowsk...@gmail.com>
> >> wrote:
> >>
> >>> I ran into a nasty little bug yesterday that I'd like to fix
>  in
>  9.8.1,
> >>> if no one minds.
> >>>
> >>> https://issues.apache.org/jira/browse/SOLR-17677 - DBQ's
> >> containing
>  a
> >>> "join" will trigger a "tragic" Lucene exception that closes
>  the
> >>> IndexWriters and puts the surrounding SolrCore into a bad
>  state.
>  Will
> >>> investigate and scope today; if a fix would take too long I
> >> don't
>  want
> >>> to hold the release up.
> >>>
> >>> Best,
> >>>
> >>> Jason
> >>>
> >>> On Tue, Feb 18, 2025 at 11:07 PM David Smiley <
> >> dsmi...@apache.org>
> >> wrote:
> 
>  I'm done.  I cherry-picked 3 bugs into 9.8.1.  Tests and

Re: Environment Variables in the Solr CLI...

[Jan Høydahl]

I love environment variables, since it is the common denominator across all 
runtimes you can imagine, including containers.
So having a broad ENV strategy for the solr server itself, as we are 
approaching with EnvUtils and SOLR_ prefix etc, is AGoodThing™.

Now, when it comes to the CLI Tools, that do not start solr but rather help us 
manage solr, like create a collection, request info, add auth,
all those tools have always been piggy-backing on whatever ENVs are set in 
solr.in.sh or exported in your shell. This was mostly handled
in the shell scripts, but are increasingly being handled in Java. In this 
latest incident there was a regression where Java Zk tool did not pick up 
ZK_HOST as before and users needed to pass the cli flag.

I just tested Solr 9.2, and it respects ZK_HOST for "solr zk ls /"

I then testet Solr 9.8 and it does not, but falls back to a hardcoded default 
of http://localhost:8983 to lookup zk_host, even if solr runs on a different 
port.


So rather than disregarding ENV for tools, let's make their use consistent with 
CLI options so we don't need so much glue code.

Jan


> 6. mars 2025 kl. 16:33 skrev Mike Drob :
> 
> I think the guidance from clig.dev is pretty clear - when the configuration
> is generally stable from one invocation to the next, using environment
> variables is nice. Imagine starting a container, you set ZK_HOST once and
> then it's good for every invocation of the tools. Otherwise you have to
> have logic for adding that flag to every command, which probably looks like
> appending "--zk-host $ZK_HOST" every time, so you're setting the
> environment variable in your deployment anyway.
> 
> On Thu, Mar 6, 2025 at 8:48 AM David Eric Pugh 
> wrote:
> 
>> I've been thinking about the use of environment variables in the Solr CLI,
>> and trying to figure out why it feels like a bad thing...I think that
>> the reason I don't like them is that it feels like we are injecting "magic"
>> into our CLI, and this makes them actually more brittle and difficult to
>> comprehend.
>> 
>> Some recent related tickets:
>>  https://github.com/apache/solr-operator/issues/762
>>  https://github.com/apache/solr/pull/3240
>> 
>> In the recent revamp of the CLI a LOT of work went into makign the CLI
>> more obvious.  We went through all the various arguments and cleaned them
>> up so we didn't have different tools have overlapping parameters.   We
>> standared the descriptive terminology we used for the various options.
>> And we tried to make the CLI smarter about required and optional
>> parameters.  Lastly, for situations where you could do A or B, like passing
>> in Solr URL OR a ZK Host, we actually enforce that.
>> 
>> We embraced Commons CLI, not because it's the best tool, but because it's
>> the most widely used one in our code base, and pushed it to be used across
>> all the CLI scripts (Prometheus Exporter, core CLI, the HDFS scripts.
>> 
>> So...   Why this rant?
>> 
>> It feels like using the environment variables to look up the connection
>> information pattern that we are using goes against making our CLI clearer,
>> and is why we are seeing various bugs in solr-operator.   The contract
>> between the CLI and the caller is the set of options that you pass to the
>> CLI.   Using environment variables avoids the checking that Commmons CLI
>> does for us.   We should be explicit when we call a script and pass in
>> everything it needs.
>> 
>> I was very much hoping the guide lines we were using to steer us in
>> revamping the CLI would support my contention that the use of the
>> environment variables to specify connectivity options is a bad practice,
>> https://clig.dev/#configuration, but unfortunantly it doesn't clearly
>> state that;-)
>> 
>> I mostly wanted to share this to get it off my chest, versus any specific
>> "we must change this now" request
>> Thanks for reading!
>> Eric
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org