Re: First class support for node roles

[Mike Drob]

t; it's same as roles=coordinator:on)
> >>>>> roles=coordinator:on,data:off (role as coordinator, disallow data)
> >>>>>
> >>>>>
> >>>>> On Sun, Dec 5, 2021 at 11:01 AM Ilan Ginzburg 
> wrote:
> >>>>>>
> >>>>>> If we go with no negative node roles and overseer node role is not
> strict (i.e. it’s a "preferred overseer"), then one would need to define a
> second node role "no_overseer" to explicitly exclude a node from ever
> becoming overseer (which I think is a useful feature until we switch the
> cluster default to not using the overseer), plus the implementation of
> these two node roles will obviously be coupled (and what if a node has both
> defined?).
> >>>>>>
> >>>>>> I prefer strict node roles.
> >>>>>> Maybe we could have node roles with [optional] parameters to let
> the node role implementation decide ?
> >>>>>> The overseer node role for example could have one of 3 values
> defined for each node: “preferred” (default, equivalent to the existing
> overseer role), "accepted" (equivalent to currently not defining the
> overseer role) and "no_way" (does not exist today).
> >>>>>>
> >>>>>> This could be useful in other contexts. A node role “data” could be
> “fast” or “slow” depending on type of local persistent storage for example…
> >>>>>>
> >>>>>> Ilan
> >>>>>>
> >>>>>> On Fri 3 Dec 2021 at 16:10, Gus Heck  wrote:
> >>>>>>>
> >>>>>>> I really don't think we should have types of roles. Not
> negative/positive and not strict/non-strict. You have a role or you don't.
> What that means is up to the code implementing the role.
> >>>>>>>
> >>>>>>> Roles should be free to configure a preference order (binary, or
> n-ary or whatever, strict or loose), prohibit behavior, or enable behavior.
> In this SIP I feel we should focus on How to identify what node has what
> role, How to designate what roles a node has via config/params, and the
> API's for interacting with roles.
> >>>>>>>
> >>>>>>> We should for example be able to support roles such as
> >>>>>>>
> >>>>>>> PREFERRED_OVERSEER
> >>>>>>> DATA
> >>>>>>> NO_ROUTED_ALIAS  (just an example, not something I mean to suggest)
> >>>>>>>
> >>>>>>> Details about role implementation should probably be discussed in
> a thread about that role.  Obviously we should think about the name
> carefully to leave options open should we want to enhance things later so
> maybe
> >>>>>>>
> >>>>>>> OVERSEER_PREF  or just  OVERSEER
> >>>>>>>
> >>>>>>> would be better since it merely reades that the node implements
> some sort of preference or config regarding overseer... but all this can be
> decided on a per role basis
> >>>>>>>
> >>>>>>> On Thu, Dec 2, 2021 at 11:44 PM Noble Paul 
> wrote:
> >>>>>>>>
> >>>>>>>> Negative roles have a place
> >>>>>>>>
> >>>>>>>> Example is overseer
> >>>>>>>>
> >>>>>>>> There are 3 possible choices for that role
> >>>>>>>>
> >>>>>>>> a) preferred: always be in front of the election queue
> >>>>>>>> b) on: not preferred, but can be an overseer if no preferred
> overseer nodes are available
> >>>>>>>> c) off: never become an overseer
> >>>>>>>>
> >>>>>>>> Today we only have options 'a' and 'b' . In a future ticket, we
> may implement C
> >>>>>>>>
> >>>>>>>> On Fri, Dec 3, 2021, 11:59 AM Mike Drob  wrote:
> >>>>>>>>>
> >>>>>>>>> Negative roles add a lot of complexity, I would really want to
> stay away from them. That’s why I want strict roles up front. It’s maybe ok
> to push this decision out, but it also seems like the sort of thing we
> should consider at the start.
> >>>>>>>>>
> >>>>>>>>> On Thu, Dec 2, 2021 at 5:52 PM Noble Paul 

Re: Strange error in Solr with Lucene 9.0.0

[Mike Drob]

I was just able to reproduce this on my local macOS machine with

gradlew test --tests TestRandomDVFaceting.testRandomFaceting
-Dtests.seed=3B93BA61C91F26D4 -Dtests.slow=true -Dtests.locale=uz-Latn
-Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true
-Dtests.file.encoding=UTF-8

on commit 0287458f836

with openjdk version "11.0.13"

On Wed, Dec 8, 2021 at 12:30 PM Robert Muir  wrote:

> I also generally tried to "beast" this
> SimpleFacetsTest.testRangeFacetFilterVsDocValuesRandom without any
> seed at all (JDK 17/linux), but I didn't manage to provoke any
> failures.
>
> On Wed, Dec 8, 2021 at 1:08 PM Robert Muir  wrote:
> >
> > I checked out the commit hash and re-ran the test with the same seed,
> > it doesn't fail. I only ran the test twice, once with
> > TieredStopAtLevel=1 and once without that.
> >
> > Given that the stacktrace looks "impossible", i suspect this might be
> > an openjdk issue? I assume this test usually reproduces failures for
> > given random seeds?
> >
> > On Wed, Dec 8, 2021 at 12:44 PM Robert Muir  wrote:
> > >
> > > Uwe, it looks a little crazy:
> > >
> > > we've got asserts here that "index" is in bounds and certainly not -1
> > > right before the method call!
> > >
> https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java#L1123-L1125
> > >
> > > On Wed, Dec 8, 2021 at 12:33 PM Uwe Schindler  wrote:
> > > >
> > > > I updated the dependencies of Solr's main branch to Lucene 9.0
> > > >
> > > > On Mac and Linux, the following error occurs sometimes:
> > > >
> https://jenkins.thetaphi.de/job/Solr-main-Linux/2070/testReport/junit/org.ap
> > > >
> ache.solr.request/SimpleFacetsTest/testRangeFacetFilterVsDocValuesRandom/
> > > >
> https://jenkins.thetaphi.de/job/Solr-main-MacOSX/517/testReport/junit/org.ap
> > > > ache.solr/TestRandomDVFaceting/testRandomFaceting/
> > > >
> > > > "ArrayIndexOutOfBoundsException: Index -1 out of bounds for length
> 1" at
> > > >
> org.apache.lucene.util.packed.DirectMonotonicReader.get(DirectMonotonicReade
> > > > r.java:161)
> > > >
> > > > This looks like a bug (or is Solr not using something with docvalues
> > > > correctly, but an AIOOBE should not happen)!
> > > >
> > > > When searching through the failure mails, it looks like this was
> already
> > > > there with the Lucene preview from summer 2021.
> > > >
> > > > Uwe
> > > >
> > > > -
> > > > Uwe Schindler
> > > > Achterdiek 19, D-28357 Bremen
> > > > https://www.thetaphi.de
> > > > eMail: u...@thetaphi.de
> > > >
> > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Strange error in Solr with Lucene 9.0.0

[Mike Drob]

Sorry, meant to add that the reproduction is before the update to lucene
9.0.0 release. But I'm also able to reproduce it after the upgrade, like
Uwe noted.

On Wed, Dec 8, 2021 at 12:53 PM Mike Drob  wrote:

> I was just able to reproduce this on my local macOS machine with
>
> gradlew test --tests TestRandomDVFaceting.testRandomFaceting
> -Dtests.seed=3B93BA61C91F26D4 -Dtests.slow=true -Dtests.locale=uz-Latn
> -Dtests.timezone=America/Santa_Isabel -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
>
> on commit 0287458f836
>
> with openjdk version "11.0.13"
>
> On Wed, Dec 8, 2021 at 12:30 PM Robert Muir  wrote:
>
>> I also generally tried to "beast" this
>> SimpleFacetsTest.testRangeFacetFilterVsDocValuesRandom without any
>> seed at all (JDK 17/linux), but I didn't manage to provoke any
>> failures.
>>
>> On Wed, Dec 8, 2021 at 1:08 PM Robert Muir  wrote:
>> >
>> > I checked out the commit hash and re-ran the test with the same seed,
>> > it doesn't fail. I only ran the test twice, once with
>> > TieredStopAtLevel=1 and once without that.
>> >
>> > Given that the stacktrace looks "impossible", i suspect this might be
>> > an openjdk issue? I assume this test usually reproduces failures for
>> > given random seeds?
>> >
>> > On Wed, Dec 8, 2021 at 12:44 PM Robert Muir  wrote:
>> > >
>> > > Uwe, it looks a little crazy:
>> > >
>> > > we've got asserts here that "index" is in bounds and certainly not -1
>> > > right before the method call!
>> > >
>> https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java#L1123-L1125
>> > >
>> > > On Wed, Dec 8, 2021 at 12:33 PM Uwe Schindler 
>> wrote:
>> > > >
>> > > > I updated the dependencies of Solr's main branch to Lucene 9.0
>> > > >
>> > > > On Mac and Linux, the following error occurs sometimes:
>> > > >
>> https://jenkins.thetaphi.de/job/Solr-main-Linux/2070/testReport/junit/org.ap
>> > > >
>> ache.solr.request/SimpleFacetsTest/testRangeFacetFilterVsDocValuesRandom/
>> > > >
>> https://jenkins.thetaphi.de/job/Solr-main-MacOSX/517/testReport/junit/org.ap
>> > > > ache.solr/TestRandomDVFaceting/testRandomFaceting/
>> > > >
>> > > > "ArrayIndexOutOfBoundsException: Index -1 out of bounds for length
>> 1" at
>> > > >
>> org.apache.lucene.util.packed.DirectMonotonicReader.get(DirectMonotonicReade
>> > > > r.java:161)
>> > > >
>> > > > This looks like a bug (or is Solr not using something with docvalues
>> > > > correctly, but an AIOOBE should not happen)!
>> > > >
>> > > > When searching through the failure mails, it looks like this was
>> already
>> > > > there with the Lucene preview from summer 2021.
>> > > >
>> > > > Uwe
>> > > >
>> > > > -
>> > > > Uwe Schindler
>> > > > Achterdiek 19, D-28357 Bremen
>> > > > https://www.thetaphi.de
>> > > > eMail: u...@thetaphi.de
>> > > >
>> > > >
>> > > >
>> > > >
>> -
>> > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > > > For additional commands, e-mail: dev-h...@lucene.apache.org
>> > > >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: Log4J RCE vulnerability

[Mike Drob]

I’ve already posted an item to our security page. If you have improvements,
please add them, my copy was certainly rushed.

 Informing our user list is a good idea too!

On Fri, Dec 10, 2021 at 10:13 AM Cassandra Targett 
wrote:

> Uwe, I understand Log4J 2.15.0 is going to address the vulnerability, but
> do you think we should add the system property
> 'log4j2.formatMsgNoLookups=true' anyway, just as a general good practice? I
> got that impression from reading your earlier message and wanted to confirm
> if I was correct.
>
> Even though we don’t need a CVE, I think we should proactively a) add a
> news item to the security.html page with the simple mitigation; and b) post
> the same to the solr-user list. I could tackle these if there are no
> objections.
>
>
> Cassandra
> On Dec 10, 2021, 7:44 AM -0600, Uwe Schindler , wrote:
>
> Hi,
>
>
>
> there was a message by the ASF security team to the members list: All
> projects should upgrade, but a CVE for each project is NOT needed:
>
>
>
> “Note: any updates of ASF projects needed to address this should
> reference CVE-2021-44228 and do not require a project-specific CVE.”
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
> 
>
> https://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Gus Heck 
> *Sent:* Friday, December 10, 2021 1:32 PM
> *To:* dev@solr.apache.org
> *Subject:* Re: Log4J RCE vulnerability
>
>
>
> In progress already it seems
> https://issues.apache.org/jira/browse/SOLR-15843
>
>
>
> On Fri, Dec 10, 2021 at 7:29 AM Gus Heck  wrote:
>
> i think upgrade is a preferable solution lest we field repeated
> emails/jiras about vulnerability scanners detecting it.
>
>
>
> On Fri, Dec 10, 2021 at 6:24 AM Uwe Schindler  wrote:
>
> With log4j 2.15.0 this should be fixed and by default all expansions on
> log messages were disabled:
> https://issues.apache.org/jira/browse/LOG4J2-3198
>
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> 
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>
> > -Original Message-
> > From: Uwe Schindler 
> > Sent: Friday, December 10, 2021 11:10 AM
> > To: dev@solr.apache.org
> > Subject: RE: Log4J RCE vulnerability
> >
> > In general the sysprop "log4j2.formatMsgNoLookups=true" fix is the only
> > correct fix (maybe add it to the bootstrap class of solr). Updating
> log4j is not
> > really needed. This prevents any of those shit. There's no reason ever
> to parse
> > ${} escapes in log messages. The only place where this can be used is the
> > format pattern in the config file, but WTF was the idea behind that to
> pass ALL
> > log messages through the expansion?
> >
> > Man man, SNEAKY log4j!!! 😊
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> 
> > https://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> > > -Original Message-
> > > From: Uwe Schindler 
> > > Sent: Friday, December 10, 2021 10:35 AM
> > > To: dev@solr.apache.org
> > > Subject: RE: Log4J RCE vulnerability
> > >
> > > Hi,
> > >
> > > I did some checks:
> > > - The problem also exists with logging parameters, so it is also
> executed if you
> > > call (which is IMHO a design failure in log4j, the reason for this is
> that the
> > > expansion is happending on printing the complete formatted log string
> to the
> > > output file): logger.info("Foobar: {}", "${badPayload}")
> > > - It also triggers if the message of an exception has a malicous
> payload! So
> > > happens easily if some input is validated and there's an
> > > IllegalArgumentException logged on validation errors
> > >
> > > To try out, and see it live do the following (can be done on any
> server, I tested
> > > it on my own servers, worked always):
> > >
> > > Start a local netcat:
> > > root@pangaea-mw1:~# nc -lp 1234
> > >
> > > Go to any user interface of you application, e.g. solr or send a query
> > containing
> > > this payload, e.g. as part of a query string that is logged:
> > > ${jndi:ldap://127.0.0.1:1234/abc}
> > >
> > > You will see cryptic text with emojis in the above netcat output. This
> shows
> > > that it definitely made an external request.
> > >
> > > We should fix this in 8.11 by doing the following:
> > > a) add "-Dlog4j2.formatMsgNoLookups=true" to Solr's start scripts
> (easy fix, I
> > > did the same on all my servers). Add this to the *main shell script*,
> not to the
> > > solr.sh.in files, as those are modified by users.
> > > b) possibly update log4j, but with above fix it's not urgent and
> should not be
> > > done in 10.0.
> > >
> > > Uwe
> > >
> > > -
> > > Uwe Schindler
> > > Achterdiek 19, D-28357 Bremen
> 
> > > https:

Re: Log4j < 2.15.0 may still be vulnerable even if -Dlog4j2.formatMsgNoLookups=true is set

[Mike Drob]

The MDC Patterns used by solr are for the collection, shard, replica, core
and node names, and a potential trace id. All of those are restricted to
alphanumeric, no special characters like $ or { needed for the injection.
And trying to access a collection that didn’t exist Returns 404 without
logging.

Upgrading is always going to be more complete, but I think we’re still ok
for now, at least until the next iteration of this attack surfaces.



On Tue, Dec 14, 2021 at 3:37 PM solr  wrote:

> Only setting -Dlog4j2.formatMsgNoLookups=true might not be enough to
> mitigate the log4j vulnerability.
>
> See https://github.com/kmindi/log4shell-vulnerable-app
> “So even with LOG4J_FORMAT_MSG_NO_LOOKUPS true version 2.14.1 of log4j is
> vulnerable when using ThreadContextMap in PatternLayout.”
>
> ThreadContext.put(key, value) is used under the hood by MDC.  I’m not sure
> wether any user-input is actually stored in MDC in SOLR.
>
>
> Probably this should be updated:
> https://solr.apache.org/security.html#apache-solr-affected-by-apache-log4j-cve-2021-44228
>
> And maybe consider releasing patch releases for other versions than 8.11
> as well which includes log4j 2.16.0?
>
>
>
> Regards,
>
>
> Fredrik
>
>
> --
> Fredrik Rødland   Cell:+47 99 21 98 17
> Maisen Pedersens vei 1Twitter: @fredrikr
> NO-1363 Høvik, NORWAY flickr:  http://www.flickr.com/fmmr/
> http://rodland.no about.me http://about.me/fmr
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Quarterly Committer Meetings

[Mike Drob]

David,

Thank you for taking the initiative! Happy New Year!

In your text you propose noon, but the time link is for 11am Boston Time.
Which did you mean?

Mike

On Mon, Jan 10, 2022 at 8:36 PM David Smiley  wrote:

> Hello everyone,
>
> I would like to propose that we have Solr committer online meetings, as we
> did sporadically previously, but henceforth quarterly & scheduled in
> advance.  I enjoyed seeing all your faces, complementing each other on our
> fine work, and getting down to business of discussing the evolution of
> Solr.  Mike D's proposal to share complements was awesome, from last time!
> I think seeing each other is really helpful for the group of us on multiple
> levels.  If we schedule them in advance with predetermined organizers
> and maybe even the time, they will happen and not be forgotten.  It should
> also be less work to plan if it's automatic.  Credit on this idea is shared
> with Eric Pugh.
>
> I volunteer to do the next 4 of them, and to use the Google Meet
> platform.  I can record them for the benefit of anyone who can't make it.
>
> Let's start next week on Thursday, January 20th; okay?  I routinely meet
> with colleagues between Europe and US West coast, and I think
> noon probably is the balance between the two.  In India, this is at
> 10:30pm.  If you would prefer I do the doodle.com thing to find the best
> coordinated time; I can do that.
>
>
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=Solr+Committer+Meeting&iso=20220120T11&p1=43&ah=1
>
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>

[DISCUSS] Mark Rate Limiting as Experimental for Solr 9.0

[Mike Drob]

Howdy folks,

After finding two quick bugs in the rate limiting code[1][2], I suspect
that this feature has largely gone untested in our code up until now. I
also looked at TestRequestRateLimiter and at least locally I'm getting a
lot of EofException in the logs, which I think is unintended. This might be
easy to fix as well, but I don't know what the next bug will be if we keep
going down this path.

I think Atri has mentioned that he was using a version of this feature with
8.x, but there's probably been some divergence between the two
implementations. Maybe we can get interested folks to try it with 9.0 and
then I have high confidence we can get it stable in time for 9.1? I do not
think reverting the feature is necessary, and it should stay in the code.

Thoughts?
Mike

[1]: https://github.com/apache/solr/pull/516
[2]: https://github.com/apache/solr/pull/505

Re: Discuss SIP-14 Embedded Zookeeper

[Mike Drob]

I put together a draft PR for the first stage of the SIP which is to cut
over unit tests to using the embedded ZK. There's a few things missing,
like setting limiters and our injected use of ZKDatabase, but we've also
had discussions about how those are not particularly useful as is.

The biggest change is the removal of SolrZooKeeper class, which was mainly
a reflection scaffolding around getting internals. We can use zookeeper
testing constructs a bit more cleanly for this, but is does affect the
public API for clients.

https://github.com/apache/solr/pull/522

On Wed, Sep 15, 2021 at 9:17 AM Mike Drob  wrote:

> > If people are running 100 or 1000 node clusters and use each node as a
> ZK server, by default, what kind of impact would that have?
>
> Bad. Very very bad. The largest ZK quorum I've personally seen is 9, and
> I've heard rumors of somebody running 15. I think the recommended approach
> for distributing load is to use Observers[1], which may provide some
> tiering benefits or may be redundant with traditional ZK clients. Maybe an
> Observer per failure zone makes sense for the Solr Operator?
>
> [1]: https://zookeeper.apache.org/doc/current/zookeeperObservers.html
>
>
>
> On Wed, Sep 15, 2021 at 8:33 AM Houston Putman  wrote:
>
>> If we were to make this work, and support productionized embedded
>> Zookeeper, then it would absolutely be something that we want to support by
>> default in the Solr Operator.
>>
>> I don't think we'd be able to cut the Zookeeper Operator dependency
>> really quickly, because this is going in at the earliest in Solr 9 and more
>> likely Solr 10 (probably). The Solr Operator still needs to support older
>> versions, especially Solr 8 for a fair amount of time. So once the minimum
>> supported Solr version is one that has this feature, then we can get rid of
>> the Zookeeper Operator for good. This is probably my favorite thing about
>> the SIP. The Zookeeper Operator is fine, but removing that dependency would
>> lift a huge burden off of the Solr Operator's shoulders.
>>
>> I also think it's a good idea to be able to start solr in a ZK-Only mode.
>>
>> Also you should be able to tell Solr whether you want it to start as a ZK
>> member or observer, or not run ZK on that node at all. I'm not extremely in
>> touch with the ZK community at this point, but what cluster sizes are
>> people scaling up to nowadays? If people are running 100 or 1000 node
>> clusters and use each node as a ZK server, by default, what kind of impact
>> would that have?
>>
>> On Tue, Sep 14, 2021 at 8:20 PM Mike Drob  wrote:
>>
>>> I like the idea of starting nodes in a ZK-only mode, probably we would
>>> call it something like coordination mode. It ties in to ideas that I've had
>>> while discussing with other folks about other Solr node specializations,
>>> like "edge" nodes that are part of a cluster but do not host collections
>>> and exist solely for routing http queries to the appropriate places.
>>>
>>> I think it could be useful in a k8s deployment as well, but I'd have to
>>> think about how we want to do all the port magic there. I know that I've
>>> had conversations with Houston about wanting to move away from
>>> ZookeeperOperator, but those haven't quite taken hold yet.
>>>
>>> On Tue, Sep 14, 2021 at 6:02 PM Jan Høydahl 
>>> wrote:
>>>
>>>> Thanks for kicking this off Mike. I added a few "rejected alternatives"
>>>> and put a few questions for thought in a comment. You may want to keep all
>>>> discussion in this email thread, so here are the questions copied:
>>>>
>>>>
>>>> *This is promising! Question: Would this mode be valuable also for
>>>> Kubernetes deployments, i.e. we could get rid of the ZookeeperOperator and
>>>> instead let the SolrOperator keep track of which Solr pods that also act as
>>>> ZK nodes?Would we allow a Solr node to start in a ZK-only mode, i.e. not
>>>> eligible for collections/cores/overseer? This would also support those huge
>>>> clusters where you want dedicated ZKs.*
>>>>
>>>> This also ties in with SIP-6 Solr should own the bootstrap process
>>>> <https://cwiki.apache.org/confluence/display/SOLR/SIP-6+Solr+should+own+the+bootstrap+process>,
>>>> as we'd want to control startup/shutdown behavior wrt the zk, e.g. start
>>>> embedded zk before solr and stop solr before stopping zk. Perhaps also
>>>> gracefully exiting the quorum on planned shutdown?
>>>>
>>>> Jan
>>>>
>>>> 14. sep. 2021 kl. 22:09 skrev Mike Drob :
>>>>
>>>> Devs,
>>>>
>>>> We've previously discussed maintaining ZK as being an operational
>>>> hurdle for some groups getting started or migrating to SolrCloud from
>>>> non-ZK cloud mode. I'd like to discuss the idea of embedding ZK in our own
>>>> process control.
>>>>
>>>> Please see the SIP at
>>>> https://cwiki.apache.org/confluence/display/SOLR/SIP-14+Embedded+Zookeeper
>>>>
>>>> Thank you,
>>>> Mike
>>>>
>>>>
>>>>

Re: New branch and feature freeze for Solr 9.0.0

[Mike Drob]

I'd like to consider getting SOLR-15919
 into 9.0 because I'm not
sure if it would be allowed in 9.1. I just realized we would want this
while looking at the code tonight, don't even have a patch to present yet.

It would be a fairly substantial change to the public API of SolrZkClient,
which I'm not sure if we consider client facing or not. However, I feel
like the change would be low risk given that we are already often
converting between File and Path in the implementations.

Would I need to start a separate thread for it to discuss?

On Tue, Jan 11, 2022 at 4:24 PM Joel Bernstein  wrote:

> I think the key is to get 9.0 out the door so we can start regular
> releases again. I'm excited about vector search as well but it will be
> impactful in 9.0 or 9.1.
>
> I'll be adding more documentation also for the new the Temporal Graph
> features in Solr 9.0 (SOLR-15197). With the new DAY and WEEKDAY time
> widows this is a Fintech event correlation tool.
>
> Existing documentation for temporal graph queries are here:
>
>
> https://github.com/apache/solr/blob/main/solr/solr-ref-guide/src/graph.adoc#temporal-graph-expressions
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Mon, Jan 10, 2022 at 5:21 PM Jan Høydahl  wrote:
>
>> I don't disagree with you that it is cool with announcements of killer
>> features. But we have known for almost 2 years that 9.0 is eventually
>> coming, but still chose to backport every single cool feature to 8.x,
>> leaving less for the 9.0 release. And I cannot blame us, since we all
>> thought of 9.0 as something in the far future. Still, the best interest of
>> the project and our users is getting what we have out there, not stalling
>> releases for 4, 5, 6 months. So hopefully first RC in 3-4 weeks from now..
>> Blocker JIRAs that have not seen active work towards completion will be
>> candidates for demotion, except blockers that are about security or compat
>> break. If we are almost ready with "Killer feature Y" when time for RC1
>> comes, then let's talk, maybe it is ready to be included in Beta state? But
>> if it is still a month or two away, it has to jump on the next train.
>>
>> There is still time to propose JIRAs to be added as blockers. I hear
>> people complain about lack of features but I don't see anyone saying
>> "Please can I add SOLR-XYZ, as blocker, I'm going to work hard and polish
>> it and make sure it is ready by DD.MM".
>>
>> If we, after 4-5 weeks of stabilization still doubt the stability /
>> reliability of the entire release, then we should consider a 9.0.0-BETA. It
>> would however be very useful if you outline exactly what functionality you
>> currently feel is beta quality, and why.
>>
>> I have earlier proposed that we can choose to label only a certain
>> Feature-X as BETA, by tagging it's documentation as such and mentioning it
>> in the release notes, e.g. "Solr 9.0.0 features a BETA version of vector
>> search..." I.e. we could have a non-beta Solr release with a few features
>> in beta, promoting them to GA in a later point release. If I remember
>> right, Noble claimed during the release of the package manager that we
>> could not tag a single feature as BETA, but I disagree. I think it is a
>> good practice, to pre-release a feature as beta if we cannot guarantee it's
>> quality in the first release.
>>
>> Finally, I'd also like to remind us that we are now 100% free as a
>> project to plan for a 10.0.0 (Solr X) release even before Lucene 10 is out,
>> with bells and whistles and cool features from here to the moon, Java 17
>> and what we like. Just a though. We are free.
>>
>> Jan
>>
>> 10. jan. 2022 kl. 21:58 skrev David Smiley :
>>
>> Major releases are special, not like minor releases.  More people will
>> look at what's neat in a major release than a minor one.  It's a de facto
>> PR event for the project to *potentially* shine.  If Lucene or whatever
>> project didn't use them to its advantage years ago then it was a missed
>> opportunity.  Also, just as importantly, it's a rare event to make
>> backwards-incompatible changes.  So I don't think we should release a major
>> version simply because someone is willing to go through the procedures to
>> do so.  Of course this is motivated by 8x being in feature freeze, which we
>> are somewhat beholden to because of our unfortunate wedlock with Lucene
>> that is not yet fully severed.  Yes, we all knew this time was coming and
>> we were all busy and didn't get to some of the things we all wanted to do
>> -- (heavy sigh).  Personally, a metaphorical fire is under my butt to try
>> to make the changes in Solr I want, and I hope you all are mustering the
>> time to do likewise.  I'm not sure if they will all make Jan's deadlines or
>> not.  Scant few are marked release blockers because in my mind it's more of
>> a best-effort.
>>
>> A beta version would help here -- a "real" release that may not have all
>> of the changes we want 

Re: Quarterly Committer Meetings

[Mike Drob]

+1

Do we have a place to gather agenda items ahead of time?

On Thu, Jan 13, 2022 at 8:12 AM Gus Heck  wrote:

> +1, please send a link to me too.
>
> On Thu, Jan 13, 2022 at 3:52 AM Bruno Roustant 
> wrote:
>
>> +1
>> I'll try to make it, probably with some lateness.
>>
>> Bruno
>>
>> Le jeu. 13 janv. 2022 à 03:34, Shawn Heisey  a
>> écrit :
>>
>>> On 1/12/2022 3:46 AM, Alessandro Benedetti wrote:
>>> > +1, the time works for me and I am in!
>>> >
>>> > Do we have a calendar event already?
>>>
>>> Looking forward to seeing a Google Meet link!  I blocked out some time
>>> on my calendar for this.
>>>
>>> Thanks,
>>> Shawn
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>>> For additional commands, e-mail: dev-h...@solr.apache.org
>>>
>>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>

Re: [VOTE] Stop releasing Solr binary ZIP artifact

[Mike Drob]

+0

I have no issue with this, but would maybe like to see a discussion on the
user@ list first and to hear the thoughts of our Windows users (Uwe? Steve?
Shawn?)

On Sun, Jan 16, 2022 at 1:08 PM Jan Høydahl  wrote:

> Hi,
>
> I know that ZIP vs TGZ has been discussed before, but I'd like to propose
> that we ditch the Solr ZIP artifacts for the 9.0 release.
> Below is a plot of downloads
> 
>  from
> DLCDN last 3 months, where we see that the zip artifact now constitutes
> only 9% of the downloads.
>
> As the tar utility ships with Windows since several years, and users are
> well aware of the format, I see no reason to keep publishing the ZIP.
> We can simplify the release, speed up the smoketester and save the
> environment, all at the same time :-)
>
> Here is my +1
>
>

Re: [DISCUSS] Mark Rate Limiting as Experimental for Solr 9.0

[Mike Drob]

I meant to bring this up at the meeting, but forgot. How _do_ we mark it?
Probably red guide, javadocs, and sample solrconfig.xml need to be updated?
Maybe a log message if it’s turned on?

Mike

On Thu, Jan 20, 2022 at 12:45 PM Jan Høydahl  wrote:

> +1 to mark as Beta or experimental. Is this a doc-only change?
>
> Jan
>
> 12. jan. 2022 kl. 16:59 skrev Mike Drob :
>
> Howdy folks,
>
> After finding two quick bugs in the rate limiting code[1][2], I suspect
> that this feature has largely gone untested in our code up until now. I
> also looked at TestRequestRateLimiter and at least locally I'm getting a
> lot of EofException in the logs, which I think is unintended. This might be
> easy to fix as well, but I don't know what the next bug will be if we keep
> going down this path.
>
> I think Atri has mentioned that he was using a version of this feature
> with 8.x, but there's probably been some divergence between the two
> implementations. Maybe we can get interested folks to try it with 9.0 and
> then I have high confidence we can get it stable in time for 9.1? I do not
> think reverting the feature is necessary, and it should stay in the code.
>
> Thoughts?
> Mike
>
> [1]: https://github.com/apache/solr/pull/516
> [2]: https://github.com/apache/solr/pull/505
>
>
>

Re: Solr partial shutdown problem

[Mike Drob]

Is this a 9.0 blocker? Seems like any APIs we change would be internal only?

On Mon, Jan 24, 2022 at 9:26 AM Jan Høydahl  wrote:

> Hi,
>
> Solr runs in a servlet container (Jetty), and normally you want one webapp
> to not be able to cause issues for another one by e.g. running
> System.exit().
> But Solr is different, it is the only servlet in the container, and in
> certain cases (such as syntax error in config files during startup), there
> are exceptions in the logs (such as ClassNotFound), Solr's servlet was
> never initialized, so returns a cryptic HTML page, while Jetty remains
> running.
>
> Is there a way we could allow some classes responsible for bringing up
> Solr to call Jetty's life-cycle command to shut down the entire Jetty JVM?
> I don't think we should allow all plugins, 3rd party tools etc to do this,
> but our own bootstrapping and config parsing code.
>
> Jan
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: Warning - Ref Guide has been migrated to Antora

[Mike Drob]

I just went through the process of merging up an older PR and the ref guide
link checker helpfully alerted me to my old broken links! Took me a few
minutes to find the right page, but overall the speedbump was very very
minor.

Thanks for taking this across the finish line, Houston!

On Thu, Feb 10, 2022 at 2:31 PM Houston Putman  wrote:

> Ok, created a different folder in the nightlies for the new nightly
> ref-guide build:
>
>
> https://nightlies.apache.org/solr/draft-guides/solr-reference-guide-nightly/solr/9_0/index.html
>
> Note: The nightly builds will include all branches (including main and
> branch_*x), whereas the release build will only include release branches
> (branch_*_*).
> So when you go the the nightlies site linked above you can check out 10.0,
> 9.1 and 9.0, however when 9.0.0 is released, it will only show 9.0
>
> - Houston
>
> On Thu, Feb 10, 2022 at 1:58 PM Houston Putman  wrote:
>
>> Right?? By far makes this whole thing worth it, besides the other
>> wonderful goodies we get (like changing solr versions).
>>
>> Basically it works by adding this line: (along with gradle build stuff of
>> downloading that extension and putting stuff in the right place)
>>
>> https://github.com/apache/solr/blob/main/solr/solr-ref-guide/playbook.template.yml#L40
>>
>> Antora handles the rest.
>> You can find the extension here:
>> https://gitlab.com/antora/antora-lunr-extension
>>
>> Sadly it doesn't use Solr, but it is a small price to pay so that we, a
>> search engine, finally have search enabled for our docs...
>>
>> - Houston
>>
>> On Thu, Feb 10, 2022 at 1:42 PM Gus Heck  wrote:
>>
>>> :) nice. highlighting of search terms too!
>>>
>>> On Thu, Feb 10, 2022 at 1:38 PM David Smiley  wrote:
>>>
 OMG I see we have search now -- good search with snippets too!  How
 does it work?

 ~ David Smiley
 Apache Lucene/Solr Search Developer
 http://www.linkedin.com/in/davidwsmiley


 On Thu, Feb 10, 2022 at 1:09 PM David Smiley 
 wrote:

> This is a nice fresh look; thanks Cassandra, Houston, and Mike!
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Thu, Feb 10, 2022 at 12:47 PM Houston Putman 
> wrote:
>
>> Hey everyone. The new ref guide is officially merged into all
>> relevant branches (down to branches_9_0). Luckily, there shouldn't be 
>> much
>> change to your workflow!
>>
>> *Building*
>> In order to build a local site, use: "gradlew buildLocalSite", or
>> just "gradlew assemble".
>> The output will be in "solr/solr-ref-guide/build/site/index.html",
>> but this is also output when the task is run.
>>
>> *Page source*
>> The source for the pages is now found under the
>> "solr/solr-ref-guide/modules" directory. At first it might be hard to 
>> find
>> files, but they are pretty logically separated out.
>>
>> The syntax is still asciidoctor, but there is a large change in how
>> you link between pages in the ref guide. You can find lots of examples
>> throughout the existing pages, but it is documented here:
>> https://github.com/apache/solr/blob/main/dev-docs/ref-guide/asciidoc-syntax.adoc#link-to-other-pagessections-of-the-guide
>>
>> Please make sure that the merge goes cleanly for the PRs you have
>> already created, before the new ref-guide was committed. The only real
>> issue you should see is the new link syntax, mentioned above, but there 
>> is
>> a possibility there will be worse problems. I'm happy to help with any
>> merge issues you run into so please reach out.
>>
>> *Check it out*
>> You can check out the local build here:
>> https://nightlies.apache.org/solr/draft-guides/solr-reference-guide-antora/solr/10_0/
>>
>> That link is up-to-date as of yesterday. Soon we will have it up to
>> date with all current versions (9.0, 9.1 and 10.0), hopefully tomorrow at
>> some point. (I will also go through and make sure we didn't backport 
>> things
>> from 10 and 9.1 to 9.0 that shouldn't have been included...)
>>
>>
>> Thanks to Cassandra for doing the heavy lifting here (and Mike as
>> well)! This is a major improvement for our docs, and I'm really excited 
>> to
>> have it out there soon!
>>
>> - Houston
>>
>
>>>
>>> --
>>> http://www.needhamsoftware.com (work)
>>> http://www.the111shift.com (play)
>>>
>>

Re: Warning - Ref Guide has been migrated to Antora

[Mike Drob]

The build should download and install node/npm/npx for you under .gradle,
so no new prerequisites aside from gradle pulling in new dependency trees.

Do you have more detail on the error message so we can investigate further?

On Mon, Feb 14, 2022 at 10:30 PM David Smiley  wrote:

> When I do "gw check", I see an error about a a "npx" program not being
> found:
>
> > Process 'command
>> '/Users/dsmiley/SFDCDev/solr_main/.gradle/node/nodejs/node-v16.13.2-darwin-x64/bin/npx''
>> finished with non-zero exit value 1
>>
>
> Um... are there new prerequisites to do a build?  Is this documented?
>

Re: Define what requires a JIRA and an entry in CHANGES.txt?

[Mike Drob]

If this is about my PR https://github.com/apache/solr/pull/641 then I'm
happy to add a JIRA for it. I initially said that I wasn't sure if it was
big enough to warrant an issue because it was almost entirely test changes,
and even then not really functional changes. But it's 200 lines of
difference, so that's probably well past the "minor change" territory.

I don't think there is a written consensus on this, but it's come up a few
times on the mailing list before. I don't think we ever reached
consensus there either.

Mike

On Wed, Feb 16, 2022 at 2:09 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> IMO, any code change should have a JIRA and CHANGES.txt. Substantial ref
> guide changes should have a JIRA.
> Refactoring should definitely get a JIRA, I'll be surprised if any
> refactoring has happened without a JIRA.
>
> On Wed, Feb 16, 2022 at 12:52 PM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
>
>> Hi all!Do we have a written definition of what requires a JIRA and/or
>> an entry in CHANGES.txt?   Something that could go in our developer docs?
>>
>> Is there a consensus on this?   For example, changes to documentation
>> typically have NOT gone into the CHANGES.txt file.   It appears that some
>> refactoring don’t need a JIRA issue as well.
>>
>> Thoughts on this?
>>
>>
>> Eric
>>
>> ___
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> 
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> 
>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless
>> of whether attachments are marked as such.
>>
>>

Re: [Operator] 1.0 Release Questions

[Mike Drob]

On Fri, Mar 4, 2022 at 9:43 AM Timothy Potter  wrote:

> Also, we should draw a hard line in the sand on which versions of Solr
> the 1.0 operator supports! It'll likely be one of the later 8.x
> versions.
>

In addition to this, I think we will need to be aware of what versions of
Kubernetes each version of the operator will support. I know that k8s 1.22
removed many deprecated APIs, but I don't recall if that required changes
to our operator. Would similarly scoped changes in the future require us to
go from 1.0 to 2.0 to 3.0?

Re: CloudSolrClient; do we deprecate or not?

[Mike Drob]

I feel like CloudSolrClient doesn't imply anything about HTTP 1 or 2,
anything about Apache or Jetty (or java.net.http). If we have exposed those
internal details in some ways, then that is unfortunate and should be
addressed.

I personally never use CloudHttp2SolrClient because I kind of assumed that
it was an implementation detail and the various builders would give me the
http2 client when I needed it. Maybe that's not the case. I've never
thought about it too much. CloudSolrClient looks like the "simpler" one to
use so that's what people gravitate towards.

A quick look in my editor suggests that we have 100 uses of
CloudSolrClient, including some in the ref guide. If we want to deprecate
this, then we should update our documentation to guide people away from it
as well. I suspect that if we try to examine which uses of CloudSolrClient
in our code could just be SolrClient, we wouldn't make much progress on
this though.

I know this isn't offering much in the way of solutions, but I'm mostly
trying to say that I agree it is a problem.


Mike

On Wed, Mar 16, 2022 at 12:05 AM David Smiley  wrote:

> On Tue, Mar 15, 2022 at 8:47 AM Jan Høydahl  wrote:
>
>> I re-opened SOLR-15223 to highlight that we are still blocked by this
>> decision.
>>
>> I don't clearly see the full effects of your suggestion right now. Does
>> your proposal also involve deprecating CloudHttp2SolrClient as a separate
>> class?
>>
>
> No; it would stay.  Perhaps ideally it would have a name reflecting it
> uses the Jetty client but no big deal; it can stay as-is.  Its name already
> isn't necessarily true; you can use this class (and thus the Jetty client)
> and tell it not to use Http2 :-). I'm reminded that HdfsDirectory doesn't
> require HDFS :-). (It requires the HDFS client libs but not necessarily an
> HDFS backend, if you're curious).
>
>
>> I would imagine users with existing SolrJ code would after upgrading get
>> an instance of BaseCloudSolrClient (with a new name) using Jetty client
>> under the hood? What if that application code assumes org.apache.http as
>> client and tries to obtain HttpSolrClient and even org.apache.http objects
>> based on CloudSolrClient? The code would fail since the contract is broken.
>>
>
> If the client/user truly assumes org.apache.http well clearly they will be
> disrupted by this change.  You want to call that a "contract" -- shrug; I
> call it an implementation detail that can change :-).  They may be calling
> getLbClient and may be using the LBHttpSolrClient subclass of LBSolrClient,
> perhaps.  Or similarly specifying builder options relating to this advanced
> option.  It's possible and it's undeniable _some_ clients will be
> impacted.  We can only hypothesize _why_ a client is dependent in the first
> place (vs. perhaps an accidental dependency/assumption e.g. in dependency
> management).  Perhaps to tweak/tune advanced options, timeouts.  Perhaps to
> instrument mTLS details; although I know from experience it can be done
> without calling special methods on builders; it can be done via setting
> special system properties referring to one's own classes that are called in
> certain ways.  If you do that (and we have), the way to do it for the
> Apache based client differs from Jetty; we've done it for both because Solr
> uses both internally.  Anyway, this is off the beaten path of most users.
>
>
>>
>> With the current pure deprecation and switch to CloudHttp2SolrClient,
>> existing users' code would continue to work..
>>
>
> Hey, this is a major release; let's not hold ourselves to a standard that
> is too onerous for us to maintain.  We can make our intentions clear in
> upgrade notes.
>
> ~ David
>
>
>> Jan
>>
>>
>> 14. mar. 2022 kl. 15:40 skrev David Smiley :
>>
>> I want to bring an important SolrJ decision to the dev list.
>>
>> There's a JIRA issue https://issues.apache.org/jira/browse/SOLR-15223
>> "Deprecate HttpSolrClient and friends in 9.0"
>>
>> Sounds great by the title -- we want to transition over time to the Jetty
>> client instead.  Jan submitted a PR to deprecate CloudSolrClient and some
>> others, and I approved it because these classes intimately assume the
>> Apache HttpClient.  It's merged.
>>
>> But I have serious doubts now and wish to discuss it with the dev list.
>> Copying my last message on the issue:
>>
>> Now that I'm "seeing" the results of this in my IDE, seeing the
>>> cross-through of deprecated usage on innocent looking classes like
>>> CloudSolrClient in particular, I have doubts on the approach.
>>> "CloudSolrClient" is an intuitive/obvious name to a user that wants to talk
>>> to SolrCloud. The particulars of which HTTP protocol or wether the client
>>> is using whatever HTTP library is all an implementation detail. Ideally
>>> such decisions would be done in the builder, either a common builder or if
>>> not then a builder specific to those libraries if needed (less nice but
>>> acceptable IMO).
>>>
>>> The easiest way to get there is to rename 

Re: Welcome Houston Putman as Solr's new PMC chair

[Mike Drob]

Thank you both for the effort you’ve put in and will put in!

On Thu, Mar 17, 2022 at 12:11 PM Anshum Gupta 
wrote:

> Thank you, Jan for all the effort and for serving as the chair.
>
> Congratulations and good luck, Houston!
>
>
>
> On Thu, Mar 17, 2022 at 6:14 AM Jan Høydahl  wrote:
>
>> Hi,
>>
>> It's been an honour to serve as chair for the Apache Solr PMC (Project
>> Management Committee) for the last year. Quite a busy year for the project!
>>
>> We have a tradition to rotate the role among PMC members, and the PMC has
>> elected Houston Putman as the next Solr Chair.
>>
>> Congrats Houston, and good luck!
>>
>> Jan
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>
>>
>
> --
> Anshum Gupta
>

Re: [VOTE] Release Solr 9.0.0 RC1

[Mike Drob]

-1:

Java 11 Only - SUCCESS! [0:35:46.949352]

Java 11 + 17 via python3 -u dev-tools/scripts/smokeTestRelease.py
--test-java17 /Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home
https://dist.apache.org/...

run tests w/ Java 17 and testArgs='-Dtests.nightly=false
-Dtests.badapples=false '...

command "export
JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home"
PATH="/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home/bin:$PATH"
JAVACMD="/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home/bin/java";
./gradlew --no-daemon clean test -Dtests.nightly=false
-Dtests.badapples=false " failed:
sh: ./gradlew: No such file or directory

Attempted to follow the tutorial exercises in the ref guide, got this
error right at the start:

mdrob-imp:/tmp/smoke_solr_9.0.0_4ea6ecc35c0f6585c76e0713acfe917ccfe3b8cf_2/unpack/solr-9.0.0/solr
$ bin/solr start -e cloud

*** [WARN] *** Your open file limit is currently 256.
 It should be set to 65000 to avoid operational disruption.
 If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to
false in your profile or solr.in.sh
*** [WARN] ***  Your Max Processes Limit is currently 11136.
 It should be set to 65000 to avoid operational disruption.
 If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to
false in your profile or solr.in.sh
Error: Could not find or load main class org.apache.solr.util.SolrCLI
Caused by: java.lang.ClassNotFoundException: org.apache.solr.util.SolrCLI


On Tue, Mar 29, 2022 at 1:18 PM Jan Høydahl  wrote:

> Please vote for release candidate 1 for Solr 9.0.0
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC1-rev-4ea6ecc35c0f6585c76e0713acfe917ccfe3b8cf
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC1-rev-4ea6ecc35c0f6585c76e0713acfe917ccfe3b8cf
>
> NB: Make sure to do a 'git pull' on branch_9_0 to get the latest
> smoketester version.
>
> Since this is a major release I encourage a more thorough manual
> inspection and test of the artifacts than the smoke tester alone.
>
> A Docker image is available at docker hub as "cominvent/solr:9.0.0-rc1"
>
> The vote will be open for at least 72 hours i.e. until 2022-04-01 19:00
> UTC.
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
>
> SUCCESS! [0:51:42.777256]
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: [VOTE] Release Solr 9.0.0 RC1

[Mike Drob]

Never mind, I think that last one is because I ran from a src unpack and
not a bin unpack like I thought I was running. Probably could have a better
error message instead of that CNFE.

On Tue, Mar 29, 2022 at 3:41 PM Mike Drob  wrote:

> -1:
>
> Java 11 Only - SUCCESS! [0:35:46.949352]
>
> Java 11 + 17 via python3 -u dev-tools/scripts/smokeTestRelease.py
> --test-java17 /Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home
> https://dist.apache.org/...
>
> run tests w/ Java 17 and testArgs='-Dtests.nightly=false 
> -Dtests.badapples=false '...
>
> command "export 
> JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home" 
> PATH="/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home/bin:$PATH" 
> JAVACMD="/Library/Java/JavaVirtualMachines/jdk-17.jdk/Contents/Home/bin/java";
>  ./gradlew --no-daemon clean test -Dtests.nightly=false 
> -Dtests.badapples=false " failed:
> sh: ./gradlew: No such file or directory
>
> Attempted to follow the tutorial exercises in the ref guide, got this error 
> right at the start:
>
> mdrob-imp:/tmp/smoke_solr_9.0.0_4ea6ecc35c0f6585c76e0713acfe917ccfe3b8cf_2/unpack/solr-9.0.0/solr
>  $ bin/solr start -e cloud
>
> *** [WARN] *** Your open file limit is currently 256.
>  It should be set to 65000 to avoid operational disruption.
>  If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false 
> in your profile or solr.in.sh
> *** [WARN] ***  Your Max Processes Limit is currently 11136.
>  It should be set to 65000 to avoid operational disruption.
>  If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false 
> in your profile or solr.in.sh
> Error: Could not find or load main class org.apache.solr.util.SolrCLI
> Caused by: java.lang.ClassNotFoundException: org.apache.solr.util.SolrCLI
>
>
> On Tue, Mar 29, 2022 at 1:18 PM Jan Høydahl  wrote:
>
>> Please vote for release candidate 1 for Solr 9.0.0
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC1-rev-4ea6ecc35c0f6585c76e0713acfe917ccfe3b8cf
>>
>> You can run the smoke tester directly with this command:
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC1-rev-4ea6ecc35c0f6585c76e0713acfe917ccfe3b8cf
>>
>> NB: Make sure to do a 'git pull' on branch_9_0 to get the latest
>> smoketester version.
>>
>> Since this is a major release I encourage a more thorough manual
>> inspection and test of the artifacts than the smoke tester alone.
>>
>> A Docker image is available at docker hub as "cominvent/solr:9.0.0-rc1"
>>
>> The vote will be open for at least 72 hours i.e. until 2022-04-01 19:00
>> UTC.
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>> Here is my +1
>>
>> SUCCESS! [0:51:42.777256]
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>
>>

Re: [VOTE] Release Solr 9.0.0 RC2

[Mike Drob]

Anshum and I are both seeing the smoke tester leak GPG agents, filed
https://issues.apache.org/jira/browse/SOLR-16132 for it.

On Thu, Mar 31, 2022 at 9:06 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Sure, I'll take a stab at it. But, no need to wait for me to get to it, I
> might be able to do this in 1-2 days.
> Even if it doesn't make it to Solr 9.0, we can use Lucene 9.1 for a Solr
> 9.1 for sure :-)
>
> On Thu, Mar 31, 2022 at 7:13 PM Jan Høydahl  wrote:
>
>> You make it seem dead smiple and risk free, but there are a ton of
>> changes that may trigger Solr changes, with another round of testing and
>> stabilization. But hey, won't stop you from whipping up a JIRA and a PR,
>> and then the PR review along with passing tests can shed more light on the
>> benefit, cost, risk and eta :)
>>
>> Jan
>>
>> 31. mar. 2022 kl. 15:23 skrev Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>>
>>
>> https://lucene.apache.org/core/9_1_0/changes/Changes.html#v9.1.0.optimizations
>>
>> On Thu, Mar 31, 2022 at 6:52 PM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> > Can you tell the benefit, cost, risk and ETA for a Lucene 9.1 upgrade?
>>> I'm not able to judge before I know.
>>>
>>> I can see lots of performance optimizations in Lucene 9.1, and can't see
>>> any significant risk (judging by changelog).
>>>
>>> On Thu, Mar 31, 2022 at 6:34 PM Jan Høydahl 
>>> wrote:
>>>
 Can you tell the benefit, cost, risk and ETA for a Lucene 9.1 upgrade?
 I'm not able to judge before I know.

 Yea, RC2 will likely fail due to SQL. But it is valuable for people to
 continue testing RC2 and potentially find other bugs, so we avoid too many
 RCs... So I'll leave the vote open a bit longer, to collect more -1's :)

 Jan

 31. mar. 2022 kl. 14:35 skrev Ishan Chattopadhyaya <
 ichattopadhy...@gmail.com>:

 If this vote is going to fail, can we re-spin based on Lucene 9.1?

 On Thu, Mar 31, 2022 at 5:55 PM Joel Bernstein 
 wrote:

> Here is the jira https://issues.apache.org/jira/browse/SOLR-16131.
>
> My official vote for the RC:
>
> -1
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Thu, Mar 31, 2022 at 4:15 AM Jan Høydahl 
> wrote:
>
>> Joel,
>>
>> I could reproduce the error
>> > Failed to load JDBC driver for
>> 'org.apache.solr.handler.sql.CalciteSolrDriver'
>>
>> And I see that JDBCStream uses a plain
>> Class.forName(driverClassName); to fail early if the driver is wrong.
>>
>> The sql module is loaded:
>> > o.a.s.c.NodeConfig Added module sql.
>> libPath=...solr-9.0.0/solr/packaging/build/solr-9.0.0/modules/sql/lib 
>> with
>> 9 libs
>>
>> I wonder if we need to use SolrResourceLoader to load the class in
>> order to find classes in sharedLib. But JDBCStream is in solrj while
>> ResourceLoader is in core... Hmm
>>
>> Please file an issue for this, and cast your formal vote.
>>
>> Jan
>>
>> 31. mar. 2022 kl. 02:57 skrev Joel Bernstein :
>>
>> I tried adding the module in the solr.in.sh and it can't find the
>> SQLHandler after starting up. So the sql module appears to not be 
>> working.
>>
>> "error":{
>> "metadata":[
>>   "error-class","org.apache.solr.common.SolrException",
>>   "root-error-class","java.lang.ClassNotFoundException"],
>> "msg":" Error loading class 'solr.SQLHandler'",
>>
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Wed, Mar 30, 2022 at 8:39 PM Joel Bernstein 
>> wrote:
>>
>>> I tried loading the sql module but ran into this error when running
>>> a sql query:
>>>
>>> Failed to load JDBC driver for
>>> 'org.apache.solr.handler.sql.CalciteSolrDriver'
>>>
>>> I loaded the module using system prop from startup command line:
>>>
>>> bin/solr start -c -Dsolr.modules=sql
>>>
>>>
>>>
>>>
>>>
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>>
>>>
>>> On Wed, Mar 30, 2022 at 8:16 PM Jan Høydahl 
>>> wrote:
>>>
 I found a minor glitch.

 In solr-exporter, the default config solr-exporter-config.xml still
 uses solr_metrics_core_replication_master as key for the
 REPLICATION./replication.isLeader metric.

 In 8.x the metrics was named REPLICATION./replication.isMaster and
 REPLICATION./replication.isSlave, but renamed to isLeader / isFollower 
 in
 9.0.
 The prometheus key for follower is correctly renamed as
 solr_metrics_core_replication_follower already.
 I think the master -> leader should also have been done.

 However I cannot see it being referenced in Grafana dashboard or
 elsewhere, so probably not worth a respi

Re: Solr github permissions

[Mike Drob]

Reach out to ASF Infra for help troubleshooting either on slack #asfinfra
or through INFRA jira.

Make sure your profile on id.apache.org is linked to your GitHub profile

On Fri, Apr 1, 2022 at 8:32 AM Joel Bernstein  wrote:

> My github permissions appear to be incorrect or missing for the Solr
> repository. I'm able to push to the apache gitbox repo, where my commits
> are mirrored to github. But my permissions on github seem to be
> non-existent for the project. My github account is joel-bernstein. Is there
> someone on the project that manages github permissions or is this done
> through a centralized apache process?
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>

Re: [VOTE] Release Solr 9.0.0 RC3

[Mike Drob]

If we're not doing anything with the release artifacts anyway (since you'll
be on holiday, Jan), would it be fine to leave it open over the weekend? I
didn't get nearly the amount of testing done on this that I wanted to,
mainly trying to figure out if SOLR-16143 was a real problem (yes) or just
a test issue (no) or worthy of blocking a release (probably not).

On Fri, Apr 8, 2022 at 4:40 PM Anshum Gupta  wrote:

> and just around the deadline, I got the smoke-tester to pass. Thanks to
> everyone who helped :)
>
> I changed a ton of things but most likely, it was an init.gradle file with
> some random stuff that was causing the issue. I remember having deleted
> that file a few months ago, but not sure what job regenerates that and adds
> that to the ~/.gradle folder.
>
> Here's my +1 (binding)
> SUCCESS! [0:46:31.241458]
>
> Also, tested SolrJ and some basic search/indexing using some sample app.
>
>
> On Fri, Apr 8, 2022 at 1:06 PM Jan Høydahl  wrote:
>
>> This vote ends in an hour or so.
>> While there are currently five +1's and a few +0, I feel it is
>> inconclusive due to the SolrCell issue?
>> Kevin, guess your vote will decide :)
>>
>> Jan
>>
>> 8. apr. 2022 kl. 18:00 skrev Kevin Risden :
>>
>> The smoke tester passed for me: SUCCESS! [0:54:21.639508]
>>
>> However, I'm running into issues checking Tika integration / Solr Cell.
>> Following
>> https://nightlies.apache.org/solr/draft-guides/solr-reference-guide-nightly/solr/9_0/indexing-guide/indexing-with-tika.html
>>
>> 2022-04-08 14:52:18.768 ERROR (qtp201274566-26) []
>>> o.a.s.s.SolrRequestParsers Couldn't get multipart parts in order to delete
>>> them => java.lang.IllegalStateException: No multipart config for servlet
>>> at org.eclipse.jetty.server.Request.getParts(Request.java:2420)
>>> java.lang.IllegalStateException: No multipart config for servlet
>>
>>
>> I tried curl and the bin/post examples and they didn't work. I also tried
>> running Solr with "./bin/solr -e schemaless -Dsolr.module=extraction" but
>> the behavior was the same.
>>
>> This happened to be the first module I started trying to make sure I
>> didn't break anything with tons of the module changes in 9.0. I know SQL
>> issues were fixed.
>>
>> Anshum - I can try to help debug the security manager stuff on the ticket
>> you filed https://issues.apache.org/jira/browse/SOLR-16135
>>
>> Kevin Risden
>>
>> Kevin Risden
>>
>>
>> On Thu, Apr 7, 2022 at 5:28 PM Anshum Gupta 
>> wrote:
>>
>>> I can't seem to get the smoketest to pass on this RC, like the previous
>>> two.
>>>
>>> At this point, I've tried:
>>> - Fresh clone of the repository
>>> - Multiple JDK versions from JDK11, 14, 17
>>> - Cleanup gradle cache
>>> - Multiple machines (Macbook and iMac)
>>>
>>> At this point I have the HDFS module tests and the adminUI test failing
>>> with SecurityManager related exceptions e.g.
>>>
>>>
>>>
>>> *org.apache.solr.hdfs.store.HdfsDirectoryTest > classMethod FAILED
>>> java.security.AccessControlException: access denied
>>> ("java.io.FilePermission"
>>> "/Users/anshumg/workspace/apache/solr/solr/modules/hdfs/build/tmp" "write")*
>>>
>>> I don't have a custom security policy and even running with 
>>> *-Djava.security.debug=access,failure,policy
>>> *didn't get any useful information.
>>>
>>> I'll continue looking into it, but if I don't get anywhere, I'll vote
>>> based on my custom tests and manual testing.
>>>
>>> -Anshum
>>>
>>>
>>> On Tue, Apr 5, 2022 at 2:58 PM Jan Høydahl 
>>> wrote:
>>>
 Please vote for release candidate 3 for Solr 9.0.0

 The artifacts can be downloaded from:

 https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC3-rev-e9e64f83a8c972b5a3f3460899c81ee9ccde2d1e

 You can run the smoke tester directly with this command:

 python3 -u dev-tools/scripts/smokeTestRelease.py \

 https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC3-rev-e9e64f83a8c972b5a3f3460899c81ee9ccde2d1e

 You are encouraged to do an extra thorough test and manual inspection
 beyond
 running the smoketester, since this is a major release.

 You can build a release-candidate of the official docker image using
 the following command:

 DIST_BASE=https://dist.apache.org/repos/dist/dev/solr && \
   RC_FOLDER=solr-9.0.0-RC3-rev-e9e64f83a8c972b5a3f3460899c81ee9ccde2d1e
 && \
   docker build $DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official \
   --build-arg
 SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-9.0.0.tgz \
   -t solr-rc:9.0.0-3

 The vote will be open for at least 72 hours i.e. until 2022-04-08 22:00
 UTC.

 [ ] +1  approve
 [ ] +0  no opinion
 [ ] -1  disapprove (and reason why)

 Here is my +1

 SUCCESS! [0:46:42.638796]
 -
 To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
 For additional commands, e-mail: dev-h...@solr.apache.org


>

Re: [RESULT][VOTE] Release Solr 9.0.0 RC3

[Mike Drob]

Did we already tag something as 9.0.0 in our Git repo?

I understand the unpleasantness of having our first release be 9.0.1 but
that might be the easiest way forward.

On Wed, Apr 13, 2022 at 4:59 PM Anshum Gupta  wrote:

> Thanks for the update, Houston.
>
> I think this calls for an RC4.
>
> I'll be happy to review the PRs as they come in.
>
> On Wed, Apr 13, 2022 at 2:32 PM Houston Putman  wrote:
>
>> Hello everyone,
>>
>> Unfortunately, while the build passed I noticed some issues when building
>> the remaining infrastructure to finish the release process (A fair amount
>> of stuff left to do since the docker and gradle migration).
>>
>> The biggest issue I found was that the solr-core maven artifact cannot be
>> used, since it still relies on the solr-server artifact, which doesn't
>> exist. There are also modules that don't have maven artifacts created. (And
>> module artifacts rely on solr-core, so they also cannot be used)
>>
>> I think this warrants an RC4 (and while we do have the RC3 release
>> artifacts in https://dist.apache.org/repos/dist/release/solr/solr/), we
>> can work with Apache infra to get that removed.
>>
>> If y'all think we should continue with the release anyways, I can do that
>> as well. Will wait for some consensus while continuing to build in the
>> necessary release steps. (for RC3 or RC4)
>>
>> Will have a fair number of PRs coming up to fix these things, so help
>> reviewing them would be much appreciated.
>>
>> - Houston
>>
>> On Fri, Apr 8, 2022 at 6:55 PM Jan Høydahl  wrote:
>>
>>> It's been >72h since the vote was initiated and the result is:
>>>
>>> +1  7  (7 binding)
>>>  0  1
>>> -1  0
>>>
>>> This vote has PASSED
>>>
>>> Congratulations everyone for reaching this milestone!
>>>
>>> There are still a few loose ends before the release can be published and
>>> announced.
>>> I'm away the next week, and Houston Putman has agreed to take over the
>>> RM job from here.
>>>
>>> Thanks for stepping up Houston!
>>>
>>> Jan
>>>
>>> 8. apr. 2022 kl. 23:43 skrev Mike Drob :
>>>
>>> If we're not doing anything with the release artifacts anyway (since
>>> you'll be on holiday, Jan), would it be fine to leave it open over the
>>> weekend? I didn't get nearly the amount of testing done on this that I
>>> wanted to, mainly trying to figure out if SOLR-16143 was a real problem
>>> (yes) or just a test issue (no) or worthy of blocking a release (probably
>>> not).
>>>
>>> On Fri, Apr 8, 2022 at 4:40 PM Anshum Gupta 
>>> wrote:
>>>
>>>> and just around the deadline, I got the smoke-tester to pass. Thanks to
>>>> everyone who helped :)
>>>>
>>>> I changed a ton of things but most likely, it was an init.gradle file
>>>> with some random stuff that was causing the issue. I remember having
>>>> deleted that file a few months ago, but not sure what job regenerates that
>>>> and adds that to the ~/.gradle folder.
>>>>
>>>> Here's my +1 (binding)
>>>> SUCCESS! [0:46:31.241458]
>>>>
>>>> Also, tested SolrJ and some basic search/indexing using some sample app.
>>>>
>>>>
>>>> On Fri, Apr 8, 2022 at 1:06 PM Jan Høydahl 
>>>> wrote:
>>>>
>>>>> This vote ends in an hour or so.
>>>>> While there are currently five +1's and a few +0, I feel it is
>>>>> inconclusive due to the SolrCell issue?
>>>>> Kevin, guess your vote will decide :)
>>>>>
>>>>> Jan
>>>>>
>>>>> 8. apr. 2022 kl. 18:00 skrev Kevin Risden :
>>>>>
>>>>> The smoke tester passed for me: SUCCESS! [0:54:21.639508]
>>>>>
>>>>> However, I'm running into issues checking Tika integration / Solr
>>>>> Cell. Following
>>>>> https://nightlies.apache.org/solr/draft-guides/solr-reference-guide-nightly/solr/9_0/indexing-guide/indexing-with-tika.html
>>>>>
>>>>> 2022-04-08 14:52:18.768 ERROR (qtp201274566-26) []
>>>>>> o.a.s.s.SolrRequestParsers Couldn't get multipart parts in order to 
>>>>>> delete
>>>>>> them => java.lang.IllegalStateException: No multipart config for servlet
>>>>>> at org.eclipse.jetty

Re: [VOTE] Release Solr 9.0.0 RC4

[Mike Drob]

+1 (binding)


Smoketester succeeded with Java 11 and Java 17 -- SUCCESS! [2:04:03.678208]

Unlike for some others, this succeeded on my first try. I guess I'm just
lucky :)


Tested building an application that uses EmbeddedSolrServer and depends on
our maven artifacts - validated SOLR-16157, SOLR-16117

Tested a mixed state cluster with 8.11.1 and 9.0.0 nodes (no security).
Queries worked as expected.
Tested a rolling upgrade from 8.11.1 to 9.0.0 with basic authentication
enabled. Queries failed (as expected) at first, but then succeeded when
setting solr.pki.sendVersion=v1 and solr.pki.acceptVersions=v1,v2on the
Solr 9 node, as described in our ref guide and upgrade notes.

Manually checked for license headers in source release, missing headers on
the following:
* SYSTEM_REQUIREMENTS.md
* solr/core/src/resources/SystemCollection*.xml
* lots and lots of stuff under solr-ref-guide, unclear which pieces of this
are our code and which are copied from external sources.
* modules/*/README.md

Will file a follow up JIRA for the license issues.

On Wed, Apr 27, 2022 at 8:15 AM Jan Høydahl  wrote:

> Please vote for release candidate 4 for Solr 9.0.0
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>
> You are encouraged to do an extra thorough test and manual inspection
> beyond
> running the smoketester, since this is a major release.
>
> You can build a release-candidate of the official docker image using the
> following command:
>
> DIST_BASE=https://dist.apache.org/repos/dist/dev/solr && \
>   RC_FOLDER=solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc &&
> \
>   docker build $DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official \
>   --build-arg SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-9.0.0.tgz \
>   -t solr-rc:9.0.0-4
>
> The vote will be open for at least 72 hours i.e. until 2022-04-30 13:00
> UTC.
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
>
> SUCCESS! [0:56:56.134141]
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: [VOTE] Release Solr 9.0.0 RC4

[Mike Drob]

I'll change my vote to a -1.

Two reasons, #1 is based on the license header issue that I mentioned, and
will work on getting that sorted out this week.
#2 is that while I personally don't see the severity of the latest
identified issues as blocking, if we're going to immediately turn around
and start on 9.0.1 then we might as well fix them now and save ourselves a
vote cycle.

Mike

On Mon, May 2, 2022 at 4:42 PM Anshum Gupta  wrote:

> I went through the vote thread again, and it seems like there are issues
> that folks aren't comfortable releasing with.
>
> Jan, as you've been driving this, I'll leave it to you but I think if we
> are certain about doing another follow up release with the fixes, we might
> as well just do that one release with things fixed. I'd be happy to help.
>
> Either way, thanks a lot for all of the effort and patience!
>
> On Mon, May 2, 2022 at 2:39 PM Anshum Gupta 
> wrote:
>
>> Even though the votes are not decisive, here's my report and vote for
>> Solr 9.0 RC4.
>>
>> 1. Smoke Tester passed in a single run! SUCCESS! [0:58:24.657506]
>> 2. Manual testing of basic index/search capability using sample code.
>> 3. Tested out some issues that were fixed in 8.11 and 9.0 to work
>> correctly in this release.
>>
>> Here's my +1 based on what I saw.
>>
>>
>>
>>
>> On Wed, Apr 27, 2022 at 6:15 AM Jan Høydahl 
>> wrote:
>>
>>> Please vote for release candidate 4 for Solr 9.0.0
>>>
>>> The artifacts can be downloaded from:
>>>
>>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>>>
>>> You can run the smoke tester directly with this command:
>>>
>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>
>>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>>>
>>> You are encouraged to do an extra thorough test and manual inspection
>>> beyond
>>> running the smoketester, since this is a major release.
>>>
>>> You can build a release-candidate of the official docker image using the
>>> following command:
>>>
>>> DIST_BASE=https://dist.apache.org/repos/dist/dev/solr && \
>>>   RC_FOLDER=solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>>> && \
>>>   docker build $DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official \
>>>   --build-arg
>>> SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-9.0.0.tgz \
>>>   -t solr-rc:9.0.0-4
>>>
>>> The vote will be open for at least 72 hours i.e. until 2022-04-30 13:00
>>> UTC.
>>>
>>> [ ] +1  approve
>>> [ ] +0  no opinion
>>> [ ] -1  disapprove (and reason why)
>>>
>>> Here is my +1
>>>
>>> SUCCESS! [0:56:56.134141]
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>>> For additional commands, e-mail: dev-h...@solr.apache.org
>>>
>>>
>>
>> --
>> Anshum Gupta
>>
>
>
> --
> Anshum Gupta
>

Re: [RESULT] [VOTE] Release Solr 9.0.0 RC4

[Mike Drob]

I created SOLR-16177  for
the license header issue.

I am backporting SOLR-16133 as a minor smoke tester improvement.

Mike

On Tue, May 3, 2022 at 8:35 AM Kevin Risden  wrote:

> I'm backporting https://issues.apache.org/jira/browse/SOLR-16164 -
> ConfigSet API returns error if untrusted user creates from _default
> configset
>
> I am NOT backporting SOLR-16174 TestBulkSchemaConcurrent - its still
> baking on master/9x. Saw a new Jenkins failure in last ~24hrs about thread
> leak I need to look into.
>
>
> Kevin Risden
>
>
> On Tue, May 3, 2022 at 4:53 AM Jan Høydahl  wrote:
>
>> It's been >72h since the vote was initiated and the result is:
>>
>> +1  5  (5 binding)
>>  0  0
>> -1  3
>>
>> The conclusion is that the vote has *FAILED* due to lack of clear
>> consensus. We'll prepare for RC5.
>>
>> I added these as blockers for 9.0 RC5:
>>
>>
>>- https://issues.apache.org/jira/browse/SOLR-16164 - ConfigSet API
>>returns error if untrusted user creates from _default configset
>>- https://issues.apache.org/jira/browse/SOLR-16176 - Reproducing
>>failure in TestCloudJSONFacetSKGEquiv.testRandom
>>
>>
>> There's also Mike's fix for missing License headers, please add it as
>> blocker yourself.
>>
>> We can now also backport other important bug fixes from branch9_x. Just
>> drop a note here and then you are free to backport your fix - bug fixes
>> only, no features :)
>> Test-improvements and stabilizations are always welcome too, such as
>> SOLR-16174 TestBulkSchemaConcurrent
>>
>> Jan
>>
>>
>> 27. apr. 2022 kl. 15:15 skrev Jan Høydahl :
>>
>> Please vote for release candidate 4 for Solr 9.0.0
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>>
>> You can run the smoke tester directly with this command:
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc
>>
>> You are encouraged to do an extra thorough test and manual inspection
>> beyond
>> running the smoketester, since this is a major release.
>>
>> You can build a release-candidate of the official docker image using the
>> following command:
>>
>> DIST_BASE=https://dist.apache.org/repos/dist/dev/solr && \
>>  RC_FOLDER=solr-9.0.0-RC4-rev-d6e36d590896755ca962c6d2ddedf78ca4f463cc &&
>> \
>>  docker build $DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official \
>>  --build-arg SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-9.0.0.tgz \
>>  -t solr-rc:9.0.0-4
>>
>> The vote will be open for at least 72 hours i.e. until 2022-04-30 13:00
>> UTC.
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>> Here is my +1
>>
>> SUCCESS! [0:56:56.134141]
>>
>>
>>

Re: [VOTE] Release Solr 9.0.0 RC5

[Mike Drob]

[INFO] There was an issue with SOLR-16133 that caused the smoke tester to
fail with gpg errors on macOS. That change has been reverted from
branch_9_0 and if you run into it, please try fetching the latest changes
and running the smoke tester command again.

On Thu, May 5, 2022 at 2:48 AM Jan Høydahl  wrote:

> Please vote for release candidate 5 for Solr 9.0.0
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC5-rev-a4eb7aa123dc53f8dac74d80b66a490f2d6b4a26
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC5-rev-a4eb7aa123dc53f8dac74d80b66a490f2d6b4a26
>
> You can build a release-candidate of the official docker image using the
> following command:
>
> DIST_BASE=https://dist.apache.org/repos/dist/dev/solr && \
>   RC_FOLDER=solr-9.0.0-RC5-rev-a4eb7aa123dc53f8dac74d80b66a490f2d6b4a26 &&
> \
>   docker build $DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official \
>   --build-arg SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-9.0.0.tgz \
>   -t solr-rc:9.0.0-5
>
> The vote will be open for at least 72 hours (plus weekend) i.e. until
> 2022-05-10 08:00 UTC.
>
> [x] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
>
> SUCCESS! [0:59:56.100439]
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: [VOTE] Release Solr 9.0.0 RC5

[Mike Drob]

Michael - can you file a JIRA issue with details of the environment and
failure that you saw? Happy to take a look at it once we know what the
problem is.

On Mon, May 9, 2022 at 3:55 PM Michael Gibney 
wrote:

> +1 (non-binding)
>
> SUCCESS! [0:49:30.542503]
>
> I ran the smoketester, but did not do any manual testing.
>
> It may be worth noting: I struggled a bit to get the integration tests to
> run on Alpine Linux (with the BusyBox version of `ps`) -- solved by `apk
> add procps`. It's possible that this would not be newly an issue with 9.0;
> but I did find that fewer integration tests failed when I reverted
> SOLR-15558, so there may be some cause for caution. This would be unlikely
> to matter for most users, but the essence of the issue is that the `solr
> stop` script doesn't detect running PIDs properly. Pre-SOLR-15558 this was
> still a problem, it just manifested differently I think.
>
> Michael
>
> On Mon, May 9, 2022 at 1:27 PM Houston Putman  wrote:
>
>> +1 (binding)
>>
>> SUCCESS! [0:38:46.406539]
>>
>> Tested:
>>
>>- Built the official docker image
>>- Local and S3 backups
>>- Solr Operator integration - smoke tester
>>- Integration with SolrJ 8.10
>>
>> The smoketester failed once, but just due to threadleak issues.
>>
>> - Houston
>>
>>
>> On Mon, May 9, 2022 at 8:59 AM David Smiley  wrote:
>>
>>> +1 (binding)
>>>
>>> SUCCESS! [0:45:47.169811]
>>>
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Mon, May 9, 2022 at 8:58 AM Jason Gerlowski 
>>> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>> SUCCESS! [1:02:41.827601]
>>>>
>>>> Ran the smoketester and did some manual tests for incremental backups.
>>>>
>>>> On Sat, May 7, 2022 at 12:56 PM Joel Bernstein 
>>>> wrote:
>>>>
>>>>> +1 (binding)
>>>>>
>>>>> SUCCESS! [0:50:47.511601]
>>>>>
>>>>>
>>>>>
>>>>> Joel Bernstein
>>>>> http://joelsolr.blogspot.com/
>>>>>
>>>>>
>>>>> On Fri, May 6, 2022 at 11:26 AM Eric Pugh <
>>>>> ep...@opensourceconnections.com> wrote:
>>>>>
>>>>>> +1 (binding)
>>>>>>
>>>>>> SUCCESS! [1:05:27.496955]
>>>>>>
>>>>>> On May 6, 2022, at 11:16 AM, Timothy Potter 
>>>>>> wrote:
>>>>>>
>>>>>> +1 (binding)
>>>>>>
>>>>>> SUCCESS! [0:47:39.581876]
>>>>>>
>>>>>> Just ran the smoke tester for this one, didn't have time to do any
>>>>>> other manual tests.
>>>>>>
>>>>>> On Thu, May 5, 2022 at 1:47 PM Mike Drob  wrote:
>>>>>>
>>>>>>
>>>>>> [INFO] There was an issue with SOLR-16133 that caused the smoke
>>>>>> tester to fail with gpg errors on macOS. That change has been reverted 
>>>>>> from
>>>>>> branch_9_0 and if you run into it, please try fetching the latest changes
>>>>>> and running the smoke tester command again.
>>>>>>
>>>>>> On Thu, May 5, 2022 at 2:48 AM Jan Høydahl 
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> Please vote for release candidate 5 for Solr 9.0.0
>>>>>>
>>>>>> The artifacts can be downloaded from:
>>>>>>
>>>>>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC5-rev-a4eb7aa123dc53f8dac74d80b66a490f2d6b4a26
>>>>>>
>>>>>> You can run the smoke tester directly with this command:
>>>>>>
>>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>>>>
>>>>>> https://dist.apache.org/repos/dist/dev/solr/solr-9.0.0-RC5-rev-a4eb7aa123dc53f8dac74d80b66a490f2d6b4a26
>>>>>>
>>>>>> You can build a release-candidate of the official docker image using
>>>>>> the following command:
>>>>>>
>>>>>> DIST_BASE=https://dist.apache.org/repos/dist/dev/solr && \
>>>>>>  RC_FOLDER=solr-9.0.0-RC5-rev-a4eb7aa123dc53f8dac74d80b66a490f2d6b4a26
>>>>>> &

Bugfix release Lucene/Solr 8.11.2

[Mike Drob]

To: dev@lucene, dev@solr

NOTICE:

I am planning on preparing a bugfix release from branch branch_8_11 (likely
mid next week)

Please observe the normal rules for committing to this branch:

* Before committing to the branch, reply to this thread and argue
  why the fix needs backporting and how long it will take.
** If you're backporting stuff this week still or over the weekend, then
skip
the bit about how long it will take.
* All issues accepted for backporting should be marked with 8.11.2
  in JIRA, and issues that should delay the release must be marked as
Blocker
* All patches that are intended for the branch should first be committed
  to the unstable branch, merged into the stable branch, and then into
  the current release branch.
* Only Jira issues with Fix version 8.11.2 and priority "Blocker" will delay
  a release candidate build.

Also, please observe that since 9.0 already exists, there cannot be any
index format breaking changes. It really should only be bug fixes that have
already been verified on the 9x branch.

Thanks,
Mike

Re: Update Lucene dependency version

[Mike Drob]

Some folks have mentioned being interested in working on this, but I don't
know that anybody has picked it up yet. Most were likely waiting to do that
work post 9.0 release which is still just wrapping up.

The big change that worries me is the module system, I wonder if we will be
affected or if it will end up being no concern at all.

Also, notably, Lucene 9.2 seems to be just around the corner, so maybe we
consider skipping 9.1 and going straight to the latest release when it
arrives?

In any case, if you're interested in helping out, then we would be happy to
review and accept any contributions you can provide!

Thanks,
Mike

On Mon, May 16, 2022 at 3:52 AM Elia Porciani  wrote:

> Hi All,
> Is anyone currently working on updating the Lucene dependency version to
> 9.1?
>
> Cheers
> -
> Elia Porciani
> Search Consultant, R&D Software Engineer
> www.sease.io
>
>

Re: Cleaning up IntelliJ warnings in code base

[Mike Drob]

I would try to handle them in several small PRs either grouped by module or
by warning type.

On Fri, May 27, 2022 at 9:24 AM Eric Pugh 
wrote:

> Hey all, was poking around at a unit test while watching TV and noticed
> lots of warnings from IntelliJ, little stuff like exceptions being thrown
> that don’t need to be thrown, unused variables, or typos.
>
> I was thinking about going through and fixing those, just to get the long
> list of problems down….  Is this best handled with a single PR with a JIRA
> issue?   Say a JIRA issue like “Clean up IntelliJ warnings for Test Code”?
>
>
> Eric
>
>
>
> ___
> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com | My Free/Busy
> 
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> 
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
>

Re: Cleaning up IntelliJ warnings in code base

[Mike Drob]

Declaring an unused thrown exception in tests isn't the most critical
change, but cleaning this up might help us discover accidental API
signature changes in the future. If a test throws an exception then JUnit
will figure it out and fail the test anyway, which is probably what we want
to do regardless.

I'm +0 on this, like I am not going to go out of my way to refactor that,
but now that you've done it I don't want to just throw away your work so
it's probably fine to commit. I hope this was some automated fix you could
apply instead of doing manually.
But I'm also not going to review it, so I hope you trust the automated
tooling and are willing to volunteer watching Jenkins for a few days after.
:)

Unused exceptions anywhere under src/main I would be _very_ interested in,
on the other hand.

Mike

On Fri, May 27, 2022 at 7:33 PM Eric Pugh 
wrote:

> So, going through and cleaning up unused throwing of exceptions, I’ve
> touched all these files listed below.   I was thinking I would do ONE
> commit for all of the “remove unused Exception”….   Before I keep going,
> wanted to make sure that makes sense…..
>
> modified:
> solr/core/src/test/org/apache/solr/analysis/CommonGramsPhraseQueryTest.java
> modified:
> solr/core/src/test/org/apache/solr/analysis/PathHierarchyTokenizerFactoryTest.java
> modified:
> solr/core/src/test/org/apache/solr/analysis/TestLuceneMatchVersion.java
> modified:
> solr/core/src/test/org/apache/solr/analysis/TestReversedWildcardFilterFactory.java
> modified:
> solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/ActionThrottleTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/AssignBackwardCompatibilityTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/ChaosMonkeyShardSplitTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/CollectionPropsTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/CollectionsAPISolrJTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/ConcurrentCreateRoutedAliasTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/ConfigSetApiLockingTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/CreateRoutedAliasTest.java
> modified:   solr/core/src/test/org/apache/solr/cloud/DeleteShardTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/DistribJoinFromCollectionTest.java
> modified:   solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java
> modified:   solr/core/src/test/org/apache/solr/cloud/HttpPartitionTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/LeaderElectionTest.java
> modified:   solr/core/src/test/org/apache/solr/cloud/OverseerTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/ReindexCollectionTest.java
> modified:   solr/core/src/test/org/apache/solr/cloud/SSLMigrationTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/SolrCLIZkUtilsTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestAuthenticationFramework.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestBaseStatsCacheCloud.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestCloudDeleteByQuery.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestCloudInspectUtil.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestCloudPivotFacet.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestHashPartitioner.java
> modified:   solr/core/src/test/org/apache/solr/cloud/TestPrepRecovery.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestRebalanceLeaders.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestSSLRandomization.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/TestStressCloudBlindAtomicUpdates.java
> modified:   solr/core/src/test/org/apache/solr/cloud/TestTlogReplica.java
> modified:   solr/core/src/test/org/apache/solr/cloud/ZkCLITest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/AsyncCallRequestStatusResponseTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/BackupRestoreApiErrorConditionsTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/CollectionApiLockingTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/CollectionTooManyReplicasTest.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/ReplicaPropertiesBase.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/TestLocalFSCloudBackupRestore.java
> modified:
> solr/core/src/test/org/apache/solr/cloud/api/collections/TestReplicaProperties.java
> modified:
> solr/core/src/test/org/apache/solr/cluster/events/impl/CollectionsRepairEventListenerTest.java
> modified:
> solr/core/src/test/org/apache/solr/core/AlternateDirectoryTest.java
> modified:
> solr/core/src/test/org/apache/solr/core/ConfigureRecoveryStrategyTest.java
> modified:
> solr/core/src/test/org/apache/

Re: Bugfix release Lucene/Solr 8.11.2

[Mike Drob]

Howdy folks, now that Lucene 9.2 has wrapped and we're past the holiday
weekend in the United States, I'd like to take a look at getting this
rolling by the end of the week. I see an open PR for
backporting LUCENE-10236 but it doesn't look like anything else would
really be waiting at the moment.

I will plan to build a release candidate on Thursday (sooner if
LUCENE-10236 is committed, later if somebody else shouts that they have
other issues).

Thanks!

On Tue, May 24, 2022 at 3:48 PM Jan Høydahl  wrote:

> I bumped Jackson in https://issues.apache.org/jira/browse/SOLR-16213 and
> also backported to 8_11. Wdyt?
>
> Jan
>
> 18. mai 2022 kl. 15:22 skrev Gus Heck :
>
> SOLR-16194 is in and ported to 8.11,.2
>
> On Wed, May 18, 2022 at 7:12 AM Jan Høydahl  wrote:
>
>> I was pinged on https://issues.apache.org/jira/browse/SOLR-16019 because
>> I have an in-flight PR with a backport. I'll complete and merge that PR.
>>
>> Jan
>>
>>
>> 13. mai 2022 kl. 01:03 skrev Mike Drob :
>>
>> To: dev@lucene, dev@solr
>>
>> NOTICE:
>>
>> I am planning on preparing a bugfix release from branch branch_8_11
>> (likely mid next week)
>>
>> Please observe the normal rules for committing to this branch:
>>
>> * Before committing to the branch, reply to this thread and argue
>>   why the fix needs backporting and how long it will take.
>> ** If you're backporting stuff this week still or over the weekend, then
>> skip
>> the bit about how long it will take.
>> * All issues accepted for backporting should be marked with 8.11.2
>>   in JIRA, and issues that should delay the release must be marked as
>> Blocker
>> * All patches that are intended for the branch should first be committed
>>   to the unstable branch, merged into the stable branch, and then into
>>   the current release branch.
>> * Only Jira issues with Fix version 8.11.2 and priority "Blocker" will
>> delay
>>   a release candidate build.
>>
>> Also, please observe that since 9.0 already exists, there cannot be any
>> index format breaking changes. It really should only be bug fixes that have
>> already been verified on the 9x branch.
>>
>> Thanks,
>> Mike
>>
>>
>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
>
>

Re: Bugfix release Lucene/Solr 8.11.2

[Mike Drob]

Yes, please fix and backport SOLR-16227, it looks almost ready from the
conversation on the PR. I will plan to do the first RC on Monday if we can
get the backport completed today.

On Thu, Jun 2, 2022 at 9:18 AM kiran chitturi 
wrote:

> Hi Mike,
>
> I found a new issue in Solr SQL (SOLR-16227) and have a fix for it (
> https://github.com/apache/solr/pull/887). Can you wait on the release for
> 8.11.2 till I can backport this?
>
> Thank you,
> Kiran.
>
> On Tue, May 31, 2022 at 9:21 AM Mike Drob  wrote:
>
>> Howdy folks, now that Lucene 9.2 has wrapped and we're past the holiday
>> weekend in the United States, I'd like to take a look at getting this
>> rolling by the end of the week. I see an open PR for
>> backporting LUCENE-10236 but it doesn't look like anything else would
>> really be waiting at the moment.
>>
>> I will plan to build a release candidate on Thursday (sooner if
>> LUCENE-10236 is committed, later if somebody else shouts that they have
>> other issues).
>>
>> Thanks!
>>
>> On Tue, May 24, 2022 at 3:48 PM Jan Høydahl 
>> wrote:
>>
>>> I bumped Jackson in https://issues.apache.org/jira/browse/SOLR-16213 and
>>> also backported to 8_11. Wdyt?
>>>
>>> Jan
>>>
>>> 18. mai 2022 kl. 15:22 skrev Gus Heck :
>>>
>>> SOLR-16194 is in and ported to 8.11,.2
>>>
>>> On Wed, May 18, 2022 at 7:12 AM Jan Høydahl 
>>> wrote:
>>>
>>>> I was pinged on https://issues.apache.org/jira/browse/SOLR-16019 because
>>>> I have an in-flight PR with a backport. I'll complete and merge that PR.
>>>>
>>>> Jan
>>>>
>>>>
>>>> 13. mai 2022 kl. 01:03 skrev Mike Drob :
>>>>
>>>> To: dev@lucene, dev@solr
>>>>
>>>> NOTICE:
>>>>
>>>> I am planning on preparing a bugfix release from branch branch_8_11
>>>> (likely mid next week)
>>>>
>>>> Please observe the normal rules for committing to this branch:
>>>>
>>>> * Before committing to the branch, reply to this thread and argue
>>>>   why the fix needs backporting and how long it will take.
>>>> ** If you're backporting stuff this week still or over the weekend,
>>>> then skip
>>>> the bit about how long it will take.
>>>> * All issues accepted for backporting should be marked with 8.11.2
>>>>   in JIRA, and issues that should delay the release must be marked as
>>>> Blocker
>>>> * All patches that are intended for the branch should first be committed
>>>>   to the unstable branch, merged into the stable branch, and then into
>>>>   the current release branch.
>>>> * Only Jira issues with Fix version 8.11.2 and priority "Blocker" will
>>>> delay
>>>>   a release candidate build.
>>>>
>>>> Also, please observe that since 9.0 already exists, there cannot be any
>>>> index format breaking changes. It really should only be bug fixes that have
>>>> already been verified on the 9x branch.
>>>>
>>>> Thanks,
>>>> Mike
>>>>
>>>>
>>>>
>>>
>>> --
>>> http://www.needhamsoftware.com (work)
>>> http://www.the111shift.com (play)
>>>
>>>
>>>

[VOTE] Release Lucene/Solr 8.11.2 RC1

[Mike Drob]

to: dev@lucene, dev@solr

Please vote for release candidate 1 for Lucene/Solr 8.11.2

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC1-reva9ed1e5fccbd1a84c78194a1329a7e1a3032ffc6

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC1-reva9ed1e5fccbd1a84c78194a1329a7e1a3032ffc6

Please see draft release notes (and edit as appropriate) at
https://cwiki.apache.org/confluence/display/LUCENE/ReleaseNote8_11_2
https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote8_11_2

The vote will be open for at least 72 hours not including weekend i.e.
until 2022-06-14 01:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Re: [VOTE] Release Lucene/Solr 8.11.2 RC1

[Mike Drob]

Thanks for finding that, Houston! It was an issue during backporting that
I've corrected. I'll respin and put up a new RC with the fix.

On Sat, Jun 11, 2022 at 11:21 AM Houston Putman  wrote:

> +0
>
> SUCCESS! [1:02:38.547629]
>
> I saw this in the example logs during the smoketester:
>
>> ps: Invalid process id: i��\r\001
>> Waiting up to 180 seconds to see Solr running on port 8983 [/]
>> Started Solr server on port 8983 (pid=16758). Happy searching!
>>
> This seems related to SOLR-16191
> <https://issues.apache.org/jira/browse/SOLR-16191>, which is included in
> this release.
> Not exactly sure what went wrong, but the example still passed?
>
> - Houston
>
> On Wed, Jun 8, 2022 at 8:50 PM Mike Drob  wrote:
>
>> to: dev@lucene, dev@solr
>>
>> Please vote for release candidate 1 for Lucene/Solr 8.11.2
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC1-reva9ed1e5fccbd1a84c78194a1329a7e1a3032ffc6
>>
>> You can run the smoke tester directly with this command:
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC1-reva9ed1e5fccbd1a84c78194a1329a7e1a3032ffc6
>>
>> Please see draft release notes (and edit as appropriate) at
>> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseNote8_11_2
>> https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote8_11_2
>>
>> The vote will be open for at least 72 hours not including weekend i.e.
>> until 2022-06-14 01:00 UTC.
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>> Here is my +1
>>
>

Re: [VOTE] Release Lucene/Solr 8.11.2 RC1

[Mike Drob]

This RC did not receive enough votes to pass, I've fixed the bug pointed
out by Houston and will be moving on to RC2. Thanks!

On Sun, Jun 12, 2022 at 2:57 PM Mike Drob  wrote:

> Thanks for finding that, Houston! It was an issue during backporting that
> I've corrected. I'll respin and put up a new RC with the fix.
>
> On Sat, Jun 11, 2022 at 11:21 AM Houston Putman 
> wrote:
>
>> +0
>>
>> SUCCESS! [1:02:38.547629]
>>
>> I saw this in the example logs during the smoketester:
>>
>>> ps: Invalid process id: i��\r\001
>>> Waiting up to 180 seconds to see Solr running on port 8983 [/]
>>> Started Solr server on port 8983 (pid=16758). Happy searching!
>>>
>> This seems related to SOLR-16191
>> <https://issues.apache.org/jira/browse/SOLR-16191>, which is included in
>> this release.
>> Not exactly sure what went wrong, but the example still passed?
>>
>> - Houston
>>
>> On Wed, Jun 8, 2022 at 8:50 PM Mike Drob  wrote:
>>
>>> to: dev@lucene, dev@solr
>>>
>>> Please vote for release candidate 1 for Lucene/Solr 8.11.2
>>>
>>> The artifacts can be downloaded from:
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC1-reva9ed1e5fccbd1a84c78194a1329a7e1a3032ffc6
>>>
>>> You can run the smoke tester directly with this command:
>>>
>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC1-reva9ed1e5fccbd1a84c78194a1329a7e1a3032ffc6
>>>
>>> Please see draft release notes (and edit as appropriate) at
>>> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseNote8_11_2
>>> https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote8_11_2
>>>
>>> The vote will be open for at least 72 hours not including weekend i.e.
>>> until 2022-06-14 01:00 UTC.
>>>
>>> [ ] +1  approve
>>> [ ] +0  no opinion
>>> [ ] -1  disapprove (and reason why)
>>>
>>> Here is my +1
>>>
>>

[VOTE] Release Lucene/Solr 8.11.2 RC2

[Mike Drob]

Please vote for release candidate 2 for Lucene/Solr 8.11.2

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC2-rev17dee71932c683e345508113523e764c3e4c80fa

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC2-rev17dee71932c683e345508113523e764c3e4c80fa

The vote will be open for at least 72 hours i.e. until 2022-06-16 20:00 UTC

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Older Docker Images

[Mike Drob]

I noticed that with the 9.0 release we made the decision to yank all of the
older docker images. I wanted to get some clarity on the practical
implications.

On our Official Images page I see that we list 8.11.1 and 9.0 - is the plan
going forward to continue to list only the latest in each line? So 8.11.1
will be replaced by 8.11.2 (soon, hopefully), etc...

Just before writing this email, I was able to verify that I can still
docker pull solr:7.7 - do we know how long this will continue to exist?
Does dockerhub have an age-off policy?
https://www.docker.com/blog/expanded-support-for-open-source-software-projects/
suggests that we are exempt from the data pull rate throttle, but I am
unclear regarding retention times.

I'm all for encouraging our users to move on to the latest and greatest
version, but I'd prefer that we do it via carrots of new features, not the
stick of their downloads disappearing.

Thanks,
Mike

Re: v2 API "Broader Changes" and Next Steps

[Mike Drob]

I suspect that #2 will inspire #1, once we start trying to use it
everywhere in SolrJ and other areas we’ll see patterns and antipatterns
emerge and guide us on what the API should be.

On Thu, Jun 16, 2022 at 6:05 AM Jason Gerlowski 
wrote:

> Hi all,
>
> I've been working over the last few months to get our "v2" API into
> shape to eventually replace (and allow us to sunset) v1.  At a high
> level the two biggest next-steps are to (1) make whatever changes we'd
> like to the v2 endpoints while they're still "experimental", and then
> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>
> The big question in my mind is: what all should change in the v2
> endpoints as part of item #1 above. Do people like the current v2
> design enough to move forward as-is (or with some minor tweaking)?  Or
> is there interest in seeing a larger re-alignment around other design
> principles (e.g. REST, OpenAPI, etc.) while v2 is still
> "experimental"?
>
> My impression from past discussions is that there's interest in seeing
> broader changes to v2 (see [1] and [2] below).  I tend to agree and
> I'm willing to head up that effort.  But I wanted to double-check
> first that there was consensus on the overall idea of "broader
> changes" before I start filing JIRAs and working out specifics.
>
> Assuming consensus, my next step will be to write up the current v2
> endpoints and some proposed changes in a spreadsheet that we can
> iterate on.  (Feedback on specific APIs, changes, etc. definitely
> welcome at that point once I get there.)
>
> Best,
>
> Jason
>
> [1] "Should v2 API be experimental" -
> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
> [2] "Solr 9.0 Release Blockers"" -
> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

[RESULT] [VOTE] Release Lucene/Solr 8.11.2 RC2

[Mike Drob]

It's been >72h since the vote was initiated and the result is:

+1  6  (6 binding)
 0  1
-1  0

This vote has PASSED

On Mon, Jun 13, 2022 at 12:05 PM Mike Drob  wrote:

> Please vote for release candidate 2 for Lucene/Solr 8.11.2
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC2-rev17dee71932c683e345508113523e764c3e4c80fa
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.2-RC2-rev17dee71932c683e345508113523e764c3e4c80fa
>
> The vote will be open for at least 72 hours i.e. until 2022-06-16 20:00 UTC
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
> 
>

Re: [RESULT] [VOTE] Release Lucene/Solr 8.11.2 RC2

[Mike Drob]

We're currently having issues with updating the project websites, which we
need before we can officially announce the release on the mailing list.
It's being tracked under INFRA-23404

On Tue, Jun 21, 2022 at 9:47 AM Colvin Cowie 
wrote:

> Thanks!
>
> On Tue, 21 Jun 2022 at 16:31, Shawn Heisey  wrote:
>
>> On 6/21/2022 8:22 AM, Colvin Cowie wrote:
>> > Hello, I see 8.11.2 is on the mirrors and in maven central, but
>> > there's not yet been a release announcement on
>> > https://solr.apache.org/news.html etc. Is that coming?
>>
>> Yes.  You can expect an announcement soon.
>>
>> When a new release is built and the VOTE passes, the person who has
>> volunteered as Release Manager will wait for a while before publishing
>> the announcement, so that the new release has had time to propagate to
>> most of the mirror network.
>>
>> Thanks,
>> Shawn
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>
>>

[ANNOUNCE] Apache Solr 8.11.2 released

[Mike Drob]

The Lucene and Solr PMCs are pleased to announce the release of Apache Solr 
8.11.2.

Solr is the popular, blazing fast, open source NoSQL search platform from the 
Apache Lucene project. Its major features include powerful full-text search, 
hit highlighting, faceted search, dynamic clustering, database integration, 
rich document handling, and geospatial search. Solr is highly scalable, 
providing fault tolerant distributed search and indexing, and powers the search 
and navigation features of many of the world's largest internet sites.

Solr 8.11.2 is available for immediate download at:

  

### Solr 8.11.2 Release Highlights:

Security

* SOLR-15871: Update Log4J to 2.17.1
* SOLR-15961: Fix bug in PKIAuthenticationPlugin that can cause a request to 
fail with 401 Unauthorized instead of re-fetching expired remote keys from 
other nodes.
* SOLR-14569: Configuring a shardHandlerFactory on the /select requestHandler 
results in HTTP 401 when searching on alias in secured Solr.
* SOLR-16022: Enforce special character requirements on passwords with length 
less than 15
* SOLR-16075: ShowFileHandler path parameter is now validated to be relative to 
instance conf dir in standalone mode

Bugfixes

* SOLR-15849: Fix the connection reset problem caused by the incorrect use of 
4LW with \n when monitoring zooKeeper status
* SOLR-16199: Improve query syntax construction for SQL LIKE clause with 
phrases and wildcards
* SOLR-16143: SolrConfig can miss updates from ZooKeeper when deleting and 
recreating file items

Please refer to the Upgrade Notes in the Solr Ref Guide for information on 
upgrading from previous Solr versions:

  

Please read CHANGES.txt for a full list of bugfixes:

  

Solr 8.11.2 also includes bugfixes in the corresponding Apache Lucene release:

  

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Missing tags for recent 8.x Solr releases

[Mike Drob]

+dev@solr
-dev@lucene to bcc

Hi Nemo,

The canonical location of our tags for the 8.x release line is
https://gitbox.apache.org/repos/asf?p=lucene-solr.git;a=tags

The tags for 9.x (and future) will all be at
https://gitbox.apache.org/repos/asf?p=solr.git;a=tags as you have found.
This is the unfortunate result of some continued split development.

We may be able to copy the 8.x tags to the solr repo as well, but I'm not
certain what our options are.

Mike

On Mon, Jul 11, 2022 at 12:10 AM Nemo  wrote:

>
> Hi,
>
> It appears that recent 8.x Solr releases (post 8.8.1) haven't been
> tagged on the Git repository[1]
>
> Is this an intentional release process change? Seeing as how the 9.0.0
> release was correctly tagged, could the missing releases be backtagged
> with the correct dates? The list of untagged releases is quite small:
>
> 8.8.2, 8.9.0, 8.10.0, 8.10.1
> 8.11.0, 8.11.1, 8.11.2
>
> Context: Writing this on behalf of the endoflife.date project, where we
> recently[2] added a endoflife.date/solr page. To track new releases
> automatically, we track tags on upstream repositories. In this case,
> tracking the repo would get us 8.8.1 as the latest release instead of
> 8.11.2. We could also track Docker images, but they seem to lag behind
> by a few days, and wouldn't give us good enough coverage.
>
> Please keep me in cc for replies.
>
> Thanks,
> Nemo
>
> [1]: https://gitbox.apache.org/repos/asf?p=solr.git;a=tags.
> [2]: https://github.com/endoflife-date/endoflife.date/pull/1389
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

On tests labelled @Slow...

[Mike Drob]

Howdy devs,

I stumbled onto https://issues.apache.org/jira/browse/SOLR-16304 while
trying to upgrade our Lucene dependency and it's motivated me to take a
little bit of a look at our tests. I know that there are dragons here and
I'm under no illusions that I can fix everything, but I feel like a
thorough audit might be useful.

The short of it is that @Slow is going away. We have choices on what to do.
We currently have 112 tests annotated as such.

Let's start with some definitions? What is our threshold for how slow
is @Slow? Obviously this will vary from machine to machine, but maybe let's
say that anything under 10s on my 2017 iMac Pro is fast and anything longer
is slow? Arbitrary, and I reserve the right to move this later if I feel
there's a better cut off.

So maybe some tests get a new breath on life by being unlabelled. Maybe
some other ones get fixed (reducing data size is one idea...)

Some tests are slow because we have distributed systems and
propagation delay and lots of gross sleeps and waits, and I don't want to
touch those. Maybe those become Nightlies.

Are there other approaches? What do folks want to do to move us forward?

Mike

Re: On tests labelled @Slow...

[Mike Drob]

While I would agree with you in principle, I don't think the Slow tests are
currently running anywhere right now. I tried running them locally and
immediately got three reproducible failures.

Uwe's jenkins doesn't run the slow tests and I don't see any jobs on ASF
Jenkins that seem to do that either.

On Thu, Jul 21, 2022 at 3:42 PM David Smiley  wrote:

> Thanks for spearheading this!
>
> Your definition of "slow" seems fine.  We can change it later.  As long as
> the build publishes tests with a runtime exceeding this threshold, we can
> maintain this easily.
>
> I think keeping @Slow makes sense so that we can identify these tests
> as-such to avoid running them at the CLI during normal development to keep
> us productive.  Obviously, slow tests need to run _sometimes_, which I
> think should be at least CI & probably PR validation too.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Thu, Jul 21, 2022 at 4:00 PM Mike Drob  wrote:
>
> > Howdy devs,
> >
> > I stumbled onto https://issues.apache.org/jira/browse/SOLR-16304 while
> > trying to upgrade our Lucene dependency and it's motivated me to take a
> > little bit of a look at our tests. I know that there are dragons here and
> > I'm under no illusions that I can fix everything, but I feel like a
> > thorough audit might be useful.
> >
> > The short of it is that @Slow is going away. We have choices on what to
> do.
> > We currently have 112 tests annotated as such.
> >
> > Let's start with some definitions? What is our threshold for how slow
> > is @Slow? Obviously this will vary from machine to machine, but maybe
> let's
> > say that anything under 10s on my 2017 iMac Pro is fast and anything
> longer
> > is slow? Arbitrary, and I reserve the right to move this later if I feel
> > there's a better cut off.
> >
> > So maybe some tests get a new breath on life by being unlabelled. Maybe
> > some other ones get fixed (reducing data size is one idea...)
> >
> > Some tests are slow because we have distributed systems and
> > propagation delay and lots of gross sleeps and waits, and I don't want to
> > touch those. Maybe those become Nightlies.
> >
> > Are there other approaches? What do folks want to do to move us forward?
> >
> > Mike
> >
>

Re: On tests labelled @Slow...

[Mike Drob]

Hmm... correction here - the failing Slow tests also happen to be AwaitsFix
tests so they were broken anyway. I wonder why my gradle command decided to
include them.

On Thu, Jul 21, 2022 at 8:14 PM Mike Drob  wrote:

> While I would agree with you in principle, I don't think the Slow tests
> are currently running anywhere right now. I tried running them locally and
> immediately got three reproducible failures.
>
> Uwe's jenkins doesn't run the slow tests and I don't see any jobs on ASF
> Jenkins that seem to do that either.
>
> On Thu, Jul 21, 2022 at 3:42 PM David Smiley  wrote:
>
>> Thanks for spearheading this!
>>
>> Your definition of "slow" seems fine.  We can change it later.  As long as
>> the build publishes tests with a runtime exceeding this threshold, we can
>> maintain this easily.
>>
>> I think keeping @Slow makes sense so that we can identify these tests
>> as-such to avoid running them at the CLI during normal development to keep
>> us productive.  Obviously, slow tests need to run _sometimes_, which I
>> think should be at least CI & probably PR validation too.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Thu, Jul 21, 2022 at 4:00 PM Mike Drob  wrote:
>>
>> > Howdy devs,
>> >
>> > I stumbled onto https://issues.apache.org/jira/browse/SOLR-16304 while
>> > trying to upgrade our Lucene dependency and it's motivated me to take a
>> > little bit of a look at our tests. I know that there are dragons here
>> and
>> > I'm under no illusions that I can fix everything, but I feel like a
>> > thorough audit might be useful.
>> >
>> > The short of it is that @Slow is going away. We have choices on what to
>> do.
>> > We currently have 112 tests annotated as such.
>> >
>> > Let's start with some definitions? What is our threshold for how slow
>> > is @Slow? Obviously this will vary from machine to machine, but maybe
>> let's
>> > say that anything under 10s on my 2017 iMac Pro is fast and anything
>> longer
>> > is slow? Arbitrary, and I reserve the right to move this later if I feel
>> > there's a better cut off.
>> >
>> > So maybe some tests get a new breath on life by being unlabelled. Maybe
>> > some other ones get fixed (reducing data size is one idea...)
>> >
>> > Some tests are slow because we have distributed systems and
>> > propagation delay and lots of gross sleeps and waits, and I don't want
>> to
>> > touch those. Maybe those become Nightlies.
>> >
>> > Are there other approaches? What do folks want to do to move us forward?
>> >
>> > Mike
>> >
>>
>

Re: On tests labelled @Slow...

[Mike Drob]

Ok, another correction, tests.slow is enabled by default, so if they're
already running most of the time then it's pretty "safe" to just axe the
annotations.

On Thu, Jul 21, 2022 at 8:19 PM Mike Drob  wrote:

> Hmm... correction here - the failing Slow tests also happen to be
> AwaitsFix tests so they were broken anyway. I wonder why my gradle command
> decided to include them.
>
> On Thu, Jul 21, 2022 at 8:14 PM Mike Drob  wrote:
>
>> While I would agree with you in principle, I don't think the Slow tests
>> are currently running anywhere right now. I tried running them locally and
>> immediately got three reproducible failures.
>>
>> Uwe's jenkins doesn't run the slow tests and I don't see any jobs on ASF
>> Jenkins that seem to do that either.
>>
>> On Thu, Jul 21, 2022 at 3:42 PM David Smiley  wrote:
>>
>>> Thanks for spearheading this!
>>>
>>> Your definition of "slow" seems fine.  We can change it later.  As long
>>> as
>>> the build publishes tests with a runtime exceeding this threshold, we can
>>> maintain this easily.
>>>
>>> I think keeping @Slow makes sense so that we can identify these tests
>>> as-such to avoid running them at the CLI during normal development to
>>> keep
>>> us productive.  Obviously, slow tests need to run _sometimes_, which I
>>> think should be at least CI & probably PR validation too.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Thu, Jul 21, 2022 at 4:00 PM Mike Drob  wrote:
>>>
>>> > Howdy devs,
>>> >
>>> > I stumbled onto https://issues.apache.org/jira/browse/SOLR-16304 while
>>> > trying to upgrade our Lucene dependency and it's motivated me to take a
>>> > little bit of a look at our tests. I know that there are dragons here
>>> and
>>> > I'm under no illusions that I can fix everything, but I feel like a
>>> > thorough audit might be useful.
>>> >
>>> > The short of it is that @Slow is going away. We have choices on what
>>> to do.
>>> > We currently have 112 tests annotated as such.
>>> >
>>> > Let's start with some definitions? What is our threshold for how slow
>>> > is @Slow? Obviously this will vary from machine to machine, but maybe
>>> let's
>>> > say that anything under 10s on my 2017 iMac Pro is fast and anything
>>> longer
>>> > is slow? Arbitrary, and I reserve the right to move this later if I
>>> feel
>>> > there's a better cut off.
>>> >
>>> > So maybe some tests get a new breath on life by being unlabelled. Maybe
>>> > some other ones get fixed (reducing data size is one idea...)
>>> >
>>> > Some tests are slow because we have distributed systems and
>>> > propagation delay and lots of gross sleeps and waits, and I don't want
>>> to
>>> > touch those. Maybe those become Nightlies.
>>> >
>>> > Are there other approaches? What do folks want to do to move us
>>> forward?
>>> >
>>> > Mike
>>> >
>>>
>>

Re: 9.1 RC1 build errors (GPG)

[Mike Drob]

>From `gradlew helpPublishing`

If you the following command fails with your GPG configuration, you can
not use an external GPG process with gradle:
echo foo | gpg --batch --no-tty --armor --detach-sign --use-agent
--local-user YOUR_KEY_NAME

Can you verify that command works for you? You might need to do some steps
to configure your pinentry settings

On Tue, Oct 25, 2022 at 12:46 PM Houston Putman  wrote:

> Ok, I have a fix for this to start, it does require a commit:
> https://github.com/apache/solr/pull/1125
>
> For a more complete fix can you share your gpg agent conf
> file: ~/.gnupg/gpg-agent.conf?
>
> This will help me setup the release wizard for default setups.
>
> On Tue, Oct 25, 2022 at 1:40 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
> > I tried putting the keyid and passphrase in the gradle.properties. But
> the
> > signing.secretKeyRingFile didn't work as, AFAICT, Fedora doesn't have a
> > secret key file lying around in the filesystem, I think it uses some
> secure
> > storage for it.
> >
> > On Tue, 25 Oct, 2022, 10:32 pm Houston Putman, 
> wrote:
> >
> > > The GPG stuff has been tested and works, though with certain inputs. I
> > can
> > > get it to work with `-PuseGPG` and providing the following properties
> > > `-Psigning.gnupg.keyName` (with the full fingerprint) and
> > > `-Psigning.gnupg.passphrase`.
> > >
> > > I think the problem is getting the passphrase to the GPG stuff through
> > the
> > > python script isn't working.
> > >
> > > Doing some local testing to see how we can fix it.
> > >
> > > In the meantime, if you put "signing.gnupg.passphrase=" in your
> > > ~/.gradle/gradle.properties then I bet the "-PuseGPG" option will
> work...
> > >
> > > - Houston
> > >
> > > On Tue, Oct 25, 2022 at 12:54 PM Ishan Chattopadhyaya <
> > > ichattopadhy...@gmail.com> wrote:
> > >
> > > > Has the GPG signing ever been tested with Linux (or Fedora in
> > > particular)?
> > > > Any ideas on how to proceed?
> > > >
> > > >
> > > > On Tue, Oct 25, 2022 at 10:22 PM Ishan Chattopadhyaya <
> > > > ichattopadhy...@gmail.com> wrote:
> > > >
> > > > > Here are the logs with external GPG.
> > > > >
> > > > > ishan@x1extreme ~/code/solr (branch_9_1) $ python3 -u
> > > > > dev-tools/scripts/buildAndPushRelease.py \
> > > > > --logfile ~/.solr-releases/9.1.0/RC1/logs/buildAndPushRelease.log \
> > > > > --push-local "~/.solr-releases/9.1.0/RC1/dist" \
> > > > > --rc-num 1 \
> > > > > --sign 2085660D9C1FCCACC4A479A3BF160FF14992A24C \
> > > > > --gpg-pass-noprompt
> > > > > Logfile is:
> > > > > /home/ishan/.solr-releases/9.1.0/RC1/logs/buildAndPushRelease.log
> > > > > Building version: 9.1.0
> > > > > Verify your gpg key is in the main KEYS file
> > > > > Using online KEYS file https://archive.apache.org/dist/solr/KEYS
> > > > > Found key 2085660D9C1FCCACC4A479A3BF160FF14992A24C in KEYS file at
> > > > > https://archive.apache.org/dist/solr/KEYS
> > > > > Will not prompt for gpg password. Make sure your signing setup
> > supports
> > > > > this.
> > > > >
> > > > > Prepare release...
> > > > > git pull...
> > > > > git clone is clean
> > > > > git rev: 36d8f3977d65525dab0b5e25a35e137aac4e1580
> > > > > Check DOAP files
> > > > > ./gradlew --no-daemon -Dtests.badapples=false clean check
> > > > > prepare-release
> > > > > Signing method is gpg tool
> > > > > Running: ./gradlew --no-daemon assembleRelease
> > -Dversion.release=9.1.0
> > > > > -Psign --max-workers 2 -PuseGpg -Psigning.gnupg.keyName=
> > > > > "2085660D9C1FCCACC4A479A3BF160FF14992A24C"
> > > > > FAILED: ./gradlew --no-daemon assembleRelease
> -Dversion.release=9.1.0
> > > > > -Psign --max-workers 2 -PuseGpg -Psigning.gnupg.keyName=
> > > > > "2085660D9C1FCCACC4A479A3BF160FF14992A24C" [see log
> > > > > /home/ishan/.solr-releases/9.1.0/RC1/logs/buildAndPushRelease.log]
> > > > > Traceback (most recent call last):
> > > > > File
> > "/home/ishan/code/solr/dev-tools/scripts/buildAndPushRelease.py",
> > > > > line 419, in 
> > > > > main()
> > > > > File
> > "/home/ishan/code/solr/dev-tools/scripts/buildAndPushRelease.py",
> > > > > line 397, in main
> > > > > prepare(c.root, c.version, c.key_id, c.key_password,
> > gpg_home=gpg_home,
> > > > > sign_gradle=c.sign_method_gradle)
> > > > > File
> > "/home/ishan/code/solr/dev-tools/scripts/buildAndPushRelease.py",
> > > > > line 147, in prepare
> > > > > run(cmd)
> > > > > File
> > "/home/ishan/code/solr/dev-tools/scripts/buildAndPushRelease.py",
> > > > > line 45, in run
> > > > > raise RuntimeError(msg)
> > > > > RuntimeError: FAILED: ./gradlew --no-daemon assembleRelease
> > > > > -Dversion.release=9.1.0 -Psign --max-workers 2 -PuseGpg
> > > > > -Psigning.gnupg.keyName="2085660D9C1FCCACC4A479A3BF160FF14992A24C"
> > [see
> > > > > log
> > /home/ishan/.solr-releases/9.1.0/RC1/logs/buildAndPushRelease.log]
> > > > >
> > > > >
> > > > >
> > > > > FAILURE: Build failed with an exception.
> > > > >
> > > > > * What went wrong:
> > > > > Execution failed for task
> > > 

Re: Publishing dependency vulnerability information

[Mike Drob]

Hi Arnout,

Thanks for starting this conversation, I have had similar thoughts recently
but hadn’t put them to action yet.

>From my understanding, SBOM are meaningful in the context of a release, not
necessarily an arbitrary code point. VEX on the other hand could be updated
between releases as information comes in about new CVEs and such. I think
that’s an important duality to consider when it comes to storage of these.

I’ll play with the VEX file you generated a little bit more later this
week, definitely looking forward to it! Can you also put up a PR for how
you generated the SBOM? I’m not sure we want to commit to that yet, but at
least a draft state would be best to collaborate on.

Mike

On Wed, Nov 30, 2022 at 3:15 AM Arnout Engelen  wrote:

> Hi,
>
> We regularly get questions asking whether Solr is affected by
> vulnerabilities that were disclosed for a dependency. With all the
> recent enthusiasm around vulnerability scanning and SBOM's, I think we
> can expect the number of such questions to rise.
>
> Solr already does a great job of collecting known false positives at
>
> https://cwiki.apache.org/confluence/display/SOLR/SolrSecurity#SolrSecurity-SolrandVulnerabilityScanningTools
> . I think it would be interesting to experiment with sharing this
> information in a machine-readable way. I've been reading up and
> experimenting a bit, and it's clear that it is early days, and a lot
> of work still needs to be done in the wider ecosystem: there are
> various SBOM/VEX file formats, and even within a format most tools
> rely on their own dialect.
>
> That said, I've had some success generating a VEX file from the table
> on the Solr wiki, attached. When I create an SBOM for solr 9.0.0 with
> the gradle org.cyclonedx.bom plugin, load it into the OWASP
> dependencytrack tool, and when I apply the VEX indeed it filters out
> some of the reported CVE's and marks the Calcite problem
> (CVE-2022-39135) as 'exploitable' - so that's at least a start. I
> would be interested in pointing people with questions about
> vulnerability scanner results to that, and working with them to gain
> experience on what we can do to make this useful.
>
> I would be happy to maintain a VEX file for Solr, be the contact point
> for feedback and questions on how to use it, etc. It doesn't look like
> the wiki allows file uploads, perhaps we could include it in the
> solr-site repo? We could also expand the "Solr and Vulnerability
> Scanning Tools" section on the wiki, explaining in more detail what to
> do when their CVE scanning tool flags a problem in Solr. I'd also be
> happy to propose a first draft of such a paragraph.
>
> Curious to hear your thoughts!
>
>
> Kind regards,
>
> Arnout
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org

Re: Welcome Michael Gibney to the Solr PMC

[Mike Drob]

Welcome and well deserved!

On Wed, Dec 21, 2022 at 3:41 PM Michael Gibney 
wrote:

> Thank you for the invitation, and the warm welcome! It's an honor to
> be a part of this community.
>
> Michael
>
> On Wed, Dec 21, 2022 at 4:06 PM Jan Høydahl  wrote:
> >
> > Congrats and welcome Michael!
> >
> > Jan Høydahl
> >
> > > 21. des. 2022 kl. 18:11 skrev Houston Putman :
> > >
> > > I am pleased to announce that Michael Gibney has accepted the PMC's
> > > invitation to join.
> > >
> > > Congratulations and welcome, Michael!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: Running all tests via Crave.io

[Mike Drob]

Having massive infrastructure to run PRs is pretty cool.

I'm worried about letting arbitrary people run code on these
machines though - a single 'crave run -- mine_bitcoin.exe' would ruin the
system for everybody, or it's not hard to imagine a slightly more indirect
case where an attacker adds a test that launches an undesirable process and
runs crave. What safeguards exist to protect against this? At least with
GHA we have to approve non-committers tests to run, but opening it up to
local command line access sounds very broad.

Mike

On Wed, Jan 25, 2023 at 10:34 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> This is very cool. Thanks for working on this, David. Can multiple
> developers execute their tests at the same time?
>
> On Thu, 26 Jan, 2023, 5:07 am Noble Paul,  wrote:
>
> > This is interesting.
> >
> > So, if the PR is merged , we will have the full test running on crave.io
> > for every PR raised?
> >
> > On Thu, Jan 26, 2023 at 9:22 AM David Smiley  wrote:
> >
> > > We haven't been running all our tests in GitHub Actions (i.e. PR
> > > validation) because it was too time consuming to do so.  I don't recall
> > how
> > > slow it was when someone last tried; it's probably better now but still
> > > slow.  To make up for this, there is a GHA only for SolrJ if a PR
> touches
> > > SolrJ.
> > >
> > > There's now a PR here to introduce a new GHA that builds on Crave.io
> on a
> > > beefy machine: https://github.com/apache/solr/pull/1303  The PR
> > validation
> > > took 11 minutes which is similar to the amount of time it took a GHA to
> > > just do precommit checks -- 10 minutes :-)
> > > I think we can remove the SolrJ specific GHA as it'll be redundant.
> > >
> > > Furthermore, anyone can use this to run tests from the convenience of
> > your
> > > laptop at the CLI while you are in the middle of any change (doesn't
> > matter
> > > what you have committed or not, pushed or not).  To do so, run: crave
> run
> > > -- './gradlew localSettings && ./gradlew --max-workers=`nproc`
> > > -Ptests.jvms=48 test'
> > >
> > > Yeah that's long.  There is a discussion in JIRA underway that may lead
> > to
> > > eliminating the "localSettings" step if, for example, it's moved to a
> > bash
> > > script executed by the gradle wrapper (my proposal).  I should also be
> > able
> > > to configure crave with a default run configuration with this baked in.
> > > I'll post an update when I'm able to do that.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> >
> >
> > --
> > -
> > Noble Paul
> >
>

Re: Running all tests via Crave.io

[Mike Drob]

@Ishan - not necessarily, a committer has to explicitly approve running the
GHA checks for first time contributors. So a random person with a drive-by
PR won't be automatically handed the keys to our infra. I believe we can
set that to needing approval for each run by a non-committer, or maybe
that's already set, somebody would need to dig through GH configs and
verify (or experiment).

I think that would still run into issues with my second case. Even if we
allow 'gradlew test' in the list of allowed commands (which I think we
would have to), what's stopping a user from creating a new test (or
modifying an existing one) to do malicious things and then execute it via
the test runner. It doesn't matter if they're launching bitcoin miners on
their own machine - using our tests to do that would be a very indirect and
inefficient way to do it. But giving people access to unfettered compute
resources suddenly makes for an enticing target.

I'm not sure if I'm being clear enough with my description, so let me try
an example. `gradlew test` will run all tests. The attacker creates a new
test "DoBadThingsTest" that will connect out to the internet and do the
previously described bad things. They launch crave run from their machine
with the changed code, and... does this upload their current code state to
crave to run the tests?

This is probably possible to do safely with sufficient isolation - gradle
needs access to a specific set of known hosts to download dependencies, and
the tests need network connectivity to talk to the multiple Solr servers
that we launch during testing, but otherwise I don't think there's a need
for external traffic. This is hard to do correctly and completely, but
would essentially limits the threat model to a DoS.

Anyway, it's not my place to tell Crave how to run their business, and I
don't mean to turn down their generous contribution with a heavy-handed
security audit. It's a very nice thing that they are offering and I think
we should take advantage of it while it is available.

Mike

On Thu, Jan 26, 2023 at 9:18 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Though, if a malicious user creates a PR that executes harmful code, that
> PR will also get executed (via "gradlew test"), right?
>
> On Thu, 26 Jan, 2023, 8:36 pm Yuvraaj Kelkar,  wrote:
>
> > Hm. I see your point. The first solution I thought of is a bit blunt, but
> > it will work: We disable command line arguments when using ZeroConf. Only
> > preset commands are allowed.
> > Also, to allow the Solr GHA to run whatever commands needed not just for
> > now, but also for any future changes, we set up regular (not ZeroConf)
> > access for the GHA runner.
> > This way GHA always has the ability to run any commands - but it is
> > strictly controlled by review requests.
> > Regular developers only get to use the commands that are approved by the
> > Solr community leaders.
> >
> > Would that work?
> > Thanks,
> > -Uv
> >
> > On Jan 26 2023, at 8:09 am, Mike Drob  wrote:
> > > Having massive infrastructure to run PRs is pretty cool.
> > >
> > > I'm worried about letting arbitrary people run code on these
> > > machines though - a single 'crave run -- mine_bitcoin.exe' would ruin
> the
> > > system for everybody, or it's not hard to imagine a slightly more
> > indirect
> > > case where an attacker adds a test that launches an undesirable process
> > and
> > > runs crave. What safeguards exist to protect against this? At least
> with
> > > GHA we have to approve non-committers tests to run, but opening it up
> to
> > > local command line access sounds very broad.
> > >
> > > Mike
> > > On Wed, Jan 25, 2023 at 10:34 PM Ishan Chattopadhyaya <
> > > ichattopadhy...@gmail.com> wrote:
> > >
> > > > This is very cool. Thanks for working on this, David. Can multiple
> > > > developers execute their tests at the same time?
> > > >
> > > > On Thu, 26 Jan, 2023, 5:07 am Noble Paul, 
> > wrote:
> > > >
> > > > > This is interesting.
> > > > >
> > > > > So, if the PR is merged , we will have the full test running on
> > crave.io
> > > > > for every PR raised?
> > > > >
> > > > > On Thu, Jan 26, 2023 at 9:22 AM David Smiley 
> > wrote:
> > > > >
> > > > > > We haven't been running all our tests in GitHub Actions (i.e. PR
> > > > > > validation) because it was too time consuming to do so. I don't
> > recall
> > > &

Re: Welcome David Smiley as Solr's new PMC chair

[Mike Drob]

Thank you for taking this mantle, David.

On Fri, Mar 31, 2023 at 3:39 PM Gus Heck  wrote:

> Congratulations!
>
> On Fri, Mar 31, 2023 at 3:51 PM Andy Webb  wrote:
>
> > Congratulations David!
> >
> > Andy
> >
> > On Fri, 31 Mar 2023 at 20:47, Alessandro Benedetti  >
> > wrote:
> >
> > > Congrats David, very well deserved!
> > >
> > > On Fri, 31 Mar 2023, 20:29 Alex Deparvu,  wrote:
> > >
> > > > Congratulations David!
> > > >
> > > > On Fri, Mar 31, 2023 at 10:04 AM Houston Putman 
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Solr has had quite a year, with a major release and many cool new
> > > > > initiatives!
> > > > > It's been an honor to serve as the PMC Chair over that time.
> > > > >
> > > > > Our PMC has traditionally rotated the position every year, and this
> > > year
> > > > > the PMC has chosen David Smiley to be the next Solr PMC Chair.
> > > > >
> > > > > Congrats David, and thanks in advance!
> > > > >
> > > > > - Houston
> > > > >
> > > >
> > >
> >
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>

Re: Rethinking dependency upgrades

[Mike Drob]

I think for a lot of us the rapid release cycle of other projects is
surprising coming from our ASF context. Here a release takes three votes
and at least three days and comparatively quite a bit of process. For some
other project, a release involves as little as clicking a button to make a
github tag and then everything else flows from there. So for those
projects, a single PR or patch can be enough to precipitate a release,
while we batch hundreds of PRs before our next release.

This is not to say one way or the other is better, but to highlight the
difference in mental models and processes between our project and others.
So even though we release roughly quarterly, it is not fair (although
highly tempting) to expect that our dependencies also update on a similar
cadence.

Mike

On Mon, Apr 3, 2023 at 3:10 PM David Smiley  wrote:

> gradle/validation/versions-props-sorted.gradle
> -- will fail if you add a comment to this file.  It insists that the file
> is sorted.  It also generates the file sorted if it isn't.  So if we want
> to support comments, we'll need to remove generating the file and remove
> the comments in checking for sorted order.
>
> Another point to note is that we'll probably have fewer dependency bot PRs
> after the longer backlog of dated dependencies is addressed.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, Apr 3, 2023 at 4:01 PM Jan Høydahl  wrote:
>
> > +1 to keeping dependencies up to date.
> >
> > But we don't necessarily need to do it every week, if people feel it is
> > noisy. We could disable the bot until we start thinking about a new
> > release, and then get a ton of upgrades to merge for the new release.
> > Personally I prefer smaller bulks each week. Less to manage for RM and
> more
> > time to test upgrades.
> >
> > > So an outcome here is: groom versions.properties to not have needless
> > references to transitive dependencies.
> >
> > Absolutely, we have done so several times already, and there may be even
> > more left.
> >
> > > It's too bad the format of this file doesn't support comments so that
> we
> >
> > The versions.props file actually supports #comments on separate lines, so
> > it could be a good idea to do that. Perhaps maintain a separate section
> at
> > the bottom for temporary transitive overrides?
> >
> > Jan
> >
> > > 3. apr. 2023 kl. 18:30 skrev David Smiley :
> > >
> > > So an outcome here is:  groom versions.properties to not have needless
> > > references to transitive dependencies.
> > > It's too bad the format of this file doesn't support comments so that
> we
> > > can explain our justification for listing a transitive dependency.
> Maybe
> > > that would be easy for us to add.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > >
> > > On Mon, Apr 3, 2023 at 12:26 PM Kevin Risden 
> wrote:
> > >
> > >> solrbot only updates things in versions.props. The PRs have found a
> few
> > >> cases where we could remove a few old transitive dependency pins in
> > >> versions.props. versions.props only has direct dependencies so not
> going
> > >> out of the way to upgrade transitive dependencies. That being said,
> the
> > >> direct dependency upgrades do in many cases upgrade transitive
> > dependencies
> > >> (as seen in versions.lock)  but the PRs are not specific for that.
> > >>
> > >> Kevin Risden
> > >>
> > >>
> > >> On Mon, Apr 3, 2023 at 11:30 AM Gus Heck  wrote:
> > >>
> > >>> The only thing I think I would add is that perhaps we should think of
> > >>> things in terms of upgrading our direct dependencies. That ensures
> the
> > >>> proper testing at the preceding levels. Updates of transitive deps
> are
> > >>> somewhat more risky, though justifiable if there is a valid security
> > >>> concern such as log4shell or similar of course.
> > >>>
> > >>> On Mon, Apr 3, 2023 at 10:47 AM Houston Putman 
> > >> wrote:
> > >>>
> >  I agree with Jason and Kevin that it's better to err on the side of
> >  updating dependencies faster than updating them slower.
> > 
> >  We have (hopefully) comprehensive testing for a lot of the features
> > >> that
> >  these dependencies are used for, and as Jason said we have ultimate
> >  discretion in merging.
> > 
> >  In general I'm surprised these libraries have so many updates, I was
> > >> not
> >  imagining that we'd get a dozen updates a week.
> > 
> >  - Houston
> > 
> >  On Mon, Apr 3, 2023 at 9:01 AM Jason Gerlowski <
> gerlowsk...@gmail.com
> > >
> >  wrote:
> > 
> > > Hi all,
> > >
> > > New releases of dependencies can introduce new bugs for sure.  But
> I
> > > think the rationale is generally that on the whole, a new release
> of
> > > dependency Foo is going to fix more than it breaks (otherwise why
> > > would the Foo project have done the release).
> > >
> > > Particu

Re: [Discuss] Solr 9.2.1 BugFix Release

[Mike Drob]

Eid Mubarak to all who celebrate and happy Friday to everybody else. And
happy weekend to those who want to rest and not check emails constantly.
And hey, wouldn't you know it, the next workday isn't until 72 hours away!
Let's try to be accommodating to all of our friends, yes?

On Thu, Apr 20, 2023 at 6:15 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> 24 hours are enough to cover all timezones on earth. Only those on other
> planets need more time.
>
> On Fri, 21 Apr, 2023, 4:42 am Jan Høydahl,  wrote:
>
> > Bureaucracy sure, but needless? Don't think so. How else can we make
> > decisions across time zones and national holidays that ensures every PMC
> > member has a chance to vote? Of course, in this matter it would probably
> be
> > possible to persuade INFRA to let Solr "inherit" Lucene's policy
> > immediately, and then change it back, only if a PMC vote demands it... :)
> >
> > Jan
> >
> > > 21. apr. 2023 kl. 01:01 skrev Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com>:
> > >
> > > Vote and 3 days of wait!? This whole Apache process feels like a
> needless
> > > bureaucracy, stuck in the twentieth century. If such a vote is a strict
> > > necessity, I think we should also get a vote on whether to stick with
> > > Apache.
> > >
> > > On Fri, 21 Apr, 2023, 3:52 am Justin Sweeney, <
> > justin.sweene...@gmail.com>
> > > wrote:
> > >
> > >> I believe I can continue the release for most of the steps before this
> > will
> > >> be a blocking issue. If as a result of the vote, it is decided not to
> > grant
> > >> write permissions to non-PMC committers, I'd just need additional
> > support
> > >> from a PMC member to complete the 9.2.1 release.
> > >>
> > >> On Thu, Apr 20, 2023 at 6:08 PM Jan Høydahl 
> > wrote:
> > >>
> > >>> Ah, I never knew this was the default. I see no reason to change our
> > >>> release practice, so guess we can do a proper VOTE and wait 3 days
> for
> > it
> > >>> to complete before the release can happen. I suppose several RM steps
> > can
> > >>> happen before this is a blocker, or?
> > >>>
> > >>> Jan
> > >>>
> >  20. apr. 2023 kl. 22:24 skrev Houston Putman :
> > 
> >  Ahhh I think I know the issue. We haven't had a non-pmc committer
> do a
> >  release since the Lucene/Solr split. I guess Lucene was setup to
> > >> support
> >  this, but by default Apache doesn't allow it, so Solr doesn't have
> it
> >  enabled now.
> > 
> >  As per the JIRA infra can do this. Should we get a PMC vote on this?
> > 
> >  - Houston
> > 
> >  On Thu, Apr 20, 2023 at 4:06 PM Houston Putman 
> > >>> wrote:
> > 
> > > I commented.
> > >
> > > On Thu, Apr 20, 2023 at 3:59 PM Justin Sweeney <
> > >>> justin.sweene...@gmail.com>
> > > wrote:
> > >
> > >> I've created an Infra ticket here:
> > >> https://issues.apache.org/jira/browse/INFRA-24485. It would
> > probably
> > >>> be
> > >> useful if a few PMC members can comment support for it, I am not
> > sure
> > >> what an approval process looks like for this.
> > >>
> > >> On Thu, Apr 20, 2023 at 3:44 PM Jan Høydahl <
> jan@cominvent.com>
> > >> wrote:
> > >>
> > >>> +1 to give svn release permission to all committers!
> > >>>
> > >>> Jan Høydahl
> > >>>
> >  20. apr. 2023 kl. 18:01 skrev Houston Putman <
> hous...@apache.org
> > >:
> > 
> >  Yeah it definitely seems like a change in policy.
> > 
> >  Justin already can't add his GPG key to the KEYS file, so we
> know
> > >> this is
> >  going to be a blocker now and again when the release vote
> passes.
> > 
> >  The infra page does say "*By default*, only PMC/PPMC members
> have
> > >> write
> >  access to the dist/release directories." So it seems like we can
> > >> change
> >  that for solr if we want to. I have no problem reverting to the
> > old
> >  permission scheme, since committers will not be able to do
> > releases
> >  otherwise.
> > 
> >  - Houston
> > 
> > > On Thu, Apr 20, 2023 at 11:38 AM Jan Høydahl <
> > >> jan@cominvent.com
> > 
> > >>> wrote:
> > >
> > > It has not been a blocker before, so I’d just proceed with the
> > >> release
> > >>> and
> > > ask again if it becomes a problem. Non-pmc committers have
> > >> certainly
> > >>> been
> > > RM before.
> > >
> > > Jan Høydahl
> > >
> > >> 20. apr. 2023 kl. 17:19 skrev Justin Sweeney <
> > >>> justin.sweene...@gmail.com
> > >> :
> > >>
> > >> While working on the 9.2.1 release process, I've hit a bit
> of a
> > > blocker. It
> > >> looks like you must be a PMC member to push to
> > >> https://dist.apache.org/repos/dist/release/solr (based on
> > >> https://infra.apache.org/release-publishing.html#normal

Re: [Proposal] Security Working Group

[Mike Drob]

Howdy folks. I'd be happy to step into this working group.

On Mon, May 1, 2023 at 12:34 PM Gus Heck  wrote:

> Awesome, glad to have you Jason, I in the end feel the same way about my
> spot. Mostly I qualify as "concerned citizen", possibly with "who thought
> about it some and has ideas" added. If we get more than 5 volunteers we can
> start comparing credentials.
>
> On Mon, May 1, 2023 at 1:17 PM Jason Gerlowski 
> wrote:
>
> > Hi Gus,
> >
> > I think this is a great idea.
> >
> > I don't have much security background that'd make me a particularly
> > good fit, but absent someone with that background stepping up, I'm
> > willing to volunteer for one of the spots.  (I'd be more than happy to
> > bow out if better qualified folks come along.)
> >
> > Best,
> >
> > Jason
> >
> > On Sun, Apr 30, 2023 at 7:14 PM David Smiley  wrote:
> > >
> > > Pretty sleepy thread so far; apparently nobody else is interested in
> > > talking about Solr security -- LOL ;-)
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > >
> > > On Wed, Apr 26, 2023 at 8:25 AM Gus Heck  wrote:
> > >
> > > > Thanks David. It would be great to have you if you can find time for
> > it. As
> > > > far as time commitment goes, I think it should become minimal after a
> > while
> > > > unless we have a flood of security reports to respond to. For a
> little
> > > > while after initial organization, I think the members will want to
> put
> > a
> > > > bit of effort into hitting some of the goals I mentioned.
> > > >
> > > > On Tue, Apr 25, 2023 at 12:28 AM David Smiley 
> > wrote:
> > > >
> > > > > This is a thoughtful organization attempt and needed, I think.
> > Thanks
> > > > Gus!
> > > > >
> > > > > I want to see if I could get a security specialist/engineer where I
> > work
> > > > to
> > > > > help us with this.  I'm tempted to say I'm joining this thing but
> I'm
> > > > weary
> > > > > of dedicating time per week.
> > > > >
> > > > > ~ David Smiley
> > > > > Apache Lucene/Solr Search Developer
> > > > > http://www.linkedin.com/in/davidwsmiley
> > > > >
> > > > >
> > > > > On Mon, Apr 24, 2023 at 1:33 PM Gus Heck 
> wrote:
> > > > >
> > > > > > *Rationale*
> > > > > >
> > > > > > Over the course of the last decade the way software security is
> > viewed
> > > > > has
> > > > > > changed. Solr has changed significantly over this time too and we
> > have
> > > > > > gained some important security features and fixed a variety of
> > > > > > vulnerabilities. However, I think as a project we have not really
> > > > > developed
> > > > > > a clear vision of what our security goals and use cases are. I
> have
> > > > > > witnessed a fair bit of variability in the responses to security
> > > > related
> > > > > > queries, and I think much of the variability comes from
> conflation
> > > > among
> > > > > > "good practical advice", "somewhat dated advice" and "varying
> > notions
> > > > of
> > > > > > supported use cases". We also regularly receive reports to the
> > > > > > secur...@solr.apache.org address that involve investigations
> into
> > > > > systems
> > > > > > that are not properly secured to begin with or configured to
> > explicitly
> > > > > > allow the dangerous behavior and it's a shame to see security
> > > > researchers
> > > > > > waste their time on that. Finally, the PMC and set of people
> > subscribed
> > > > > to
> > > > > > secur...@solr.apache.org is a large enough group that incoming
> > mails
> > > > > often
> > > > > > seem to languish in a classic example of nobody having actual
> > specific
> > > > > > responsibility for responding.
> > > > > >
> > > > > > *Proposal*
> > > > > > The Solr PMC should appoint from among its members either 3 to 5
> > > > > > individuals to serve as a "security working group" Membership in
> > the
> > > > > > "Security Working Group" requires subscribing to
> > > > > secur...@solr.apache.org,
> > > > > > and a 30 minute conference call once or twice a month. This
> working
> > > > group
> > > > > > would have the following goals.
> > > > > >
> > > > > >1. Establish a relationship with someone who's core job
> > function is
> > > > > >computer security, rather than providing search (I'm hoping
> the
> > ASF
> > > > > has
> > > > > >some people who secure their systems that could be a
> resource).
> > This
> > > > > > person
> > > > > >should be willing to offer a systems security perspective on
> our
> > > > goals
> > > > > > and
> > > > > >the security functionality we provide.
> > > > > >2. Develop a clear statement of the security use cases we
> would
> > like
> > > > > to
> > > > > >support, and exposition of some scenarios that are clearly out
> > of
> > > > > scope.
> > > > > >This results in a proposal to be discussed on the dev list and
> > users
> > > > > > list
> > > > > >and eventually voted on.
> > > > > >3. Identification of use cases we would like to support tha

Re: [Proposal] Security Working Group

[Mike Drob]

Just a quick update here - it sounds like the project may opt to allow
committers (non-PMC members) to join the security list. Discussion here:
https://lists.apache.org/thread/k9rt56y3j4vd2gczbn257qf4x272vz1o

I expect the same logic would apply to this WG.

Mike

On Tue, May 2, 2023 at 7:40 PM Gus Heck  wrote:

> @Kevin, Cool, I think with 4-5 people volunteering this is a go, and
> perhaps the working group can do a quick kick off (30 min) online call
> somewhere around the 15th?
>
> @Marcus Please don't hesitate to suggest improvements (or implement them!)
> Also feel 100% free to suggest improvements to my list of goals or
> brainstorm ideas to flesh them out. Happy to have community involvement at
> all levels. The core idea of the working group is to get a few people
> invested in this particular aspect of solr and improve the timeliness and
> quality of our responses to reports. The more help we get the better. One
> of the best possible results would be if this got people thinking and we
> got more participation out of it.
>
> -Gus
>
> On Tue, May 2, 2023 at 7:19 PM Marcus Eagan  wrote:
>
> > Also happy to contribute from the outside, or one foot in rather :-)
> >
> > Security is my motivation for most of the work that I have done in the
> > project to date.
> >
> >
> > On Tue, May 2, 2023 at 3:51 PM Kevin Risden  wrote:
> >
> > > I'm happy to contribute.
> > >
> > > Kevin Risden
> > >
> > >
> > > On Tue, May 2, 2023 at 3:47 PM Arrieta, Alejandro <
> > > aarri...@perrinsoftware.com> wrote:
> > >
> > > > Hi Gus,
> > > >
> > > > thx 4 clarification.
> > > > Well I need to work on those 2 requirements then :-)
> > > >
> > > > Thanks
> > > > Alejandro Arrieta
> > > >
> > > >
> > > > On Tue, May 2, 2023 at 3:40 PM Gus Heck  wrote:
> > > >
> > > > > Unfortunately, since part of the duties will be responding to the
> > > queries
> > > > > sent to secur...@solr.apache.org, one must be both a committer
> and a
> > > PMC
> > > > > member. However, I expect that this group will make suggestions
> about
> > > > > anything unrelated to un-announced security issues to the wider
> list
> > > for
> > > > a
> > > > > typical discussion/proposal/vote cycle.
> > > > >
> > > > > On Tue, May 2, 2023 at 3:28 PM Arrieta, Alejandro <
> > > > > aarri...@perrinsoftware.com> wrote:
> > > > >
> > > > > >  Hello Team,
> > > > > >
> > > > > > Do you need to be a committer to join the group?
> > > > > >
> > > > > > Kind Regards,
> > > > > > Alejandro Arrieta
> > > > > >
> > > > > > On Tue, May 2, 2023 at 3:23 PM Gus Heck 
> > wrote:
> > > > > >
> > > > > > > Cool that means so far we have:
> > > > > > >
> > > > > > >1. Me (Gus Heck)
> > > > > > >2. Jason Gerlowski
> > > > > > >3. Mike Drob
> > > > > > >4. (maybe?) David Smiley
> > > > > > >
> > > > > > >
> > > > > > > On Tue, May 2, 2023 at 3:02 PM Mike Drob 
> > wrote:
> > > > > > >
> > > > > > > > Howdy folks. I'd be happy to step into this working group.
> > > > > > > >
> > > > > > > > On Mon, May 1, 2023 at 12:34 PM Gus Heck  >
> > > > wrote:
> > > > > > > >
> > > > > > > > > Awesome, glad to have you Jason, I in the end feel the same
> > way
> > > > > about
> > > > > > > my
> > > > > > > > > spot. Mostly I qualify as "concerned citizen", possibly
> with
> > > "who
> > > > > > > thought
> > > > > > > > > about it some and has ideas" added. If we get more than 5
> > > > > volunteers
> > > > > > we
> > > > > > > > can
> > > > > > > > > start comparing credentials.
> > > > > > > > >
> > > > > > > > > On Mon, May 1, 2023 at 1:17 PM Jason Gerlowski <
> > > > > > gerlowsk...@gmail.com>
> > > > > > > > > wrote:
> &

Re: Welcome Alex Deparvu as Solr committer

[Mike Drob]

Welcome and congratulations!

On Wed, Aug 2, 2023 at 8:47 AM David Smiley 
wrote:

> Congratulations and well deserved Alex!
>
> ~ David
>
>
> On Wed, Aug 2, 2023 at 4:10 AM Anshum Gupta  wrote:
>
> > Hello everyone.
> >
> > On behalf of the Apache Solr PMC, I'm pleased to announce that Alex
> Deparvu
> > has accepted the invitation to become a Solr committer.
> >
> > Alex - it's a tradition that you introduce yourself with a brief bio.
> >
> > Congratulations, welcome, and happy committing!
> >
> > -Anshum
> >
>

Re: PR reviewed

[Mike Drob]

Hi Isabelle,

It looks like there is a precommit check issue with the code still, can you
take a look and address that? You should be able to click on "details" for
the failing check, or run it locally using ./gradlew check -x test

Thanks for your contribution!

On Wed, Aug 23, 2023 at 9:52 AM Isabelle Giguere
 wrote:

> Hello Solr committers!
>
> This PR was reviewed by Christine Poerschke
> and Alex D.  Thank you both 🙂
>
> IMHO, the code is as clean as it's going to get.  I hope it can be merged
> some day.
>
> https://github.com/apache/solr/pull/1632
>
> Regards;
>
> Isabelle Giguère
>
>

Re: PR reviewed

[Mike Drob]

Resending with corrected email address

On Wed, Aug 23, 2023 at 11:13 AM Mike Drob  wrote:

> Hi Isabelle,
>
> It looks like there is a precommit check issue with the code still, can
> you take a look and address that? You should be able to click on
> "details" for the failing check, or run it locally using ./gradlew check
> -x test
>
> Thanks for your contribution!
>
> On Wed, Aug 23, 2023 at 9:52 AM Isabelle Giguere
>  wrote:
>
>> Hello Solr committers!
>>
>> This PR was reviewed by Christine Poerschke<https://github.com/cpoerschke>
>> and Alex D<https://github.com/stillalex>.  Thank you both 🙂
>>
>> IMHO, the code is as clean as it's going to get.  I hope it can be merged
>> some day.
>>
>> https://github.com/apache/solr/pull/1632
>>
>> Regards;
>>
>> Isabelle Giguère
>>
>>

Re: [PR] SOLR-16835: Add approximate-/select to OAS [solr]

[Mike Drob]

I think the goal is incremental migration from hand written to generated?
So over time these endpoints converge rather than fragment. Maybe this
isn’t the right way though, what do you think an appropriate direction
would be?

On Thu, Nov 30, 2023 at 8:33 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Having a /select and a similar (approximate) endpoint that is not exactly
> like /select but similar will create fragmentation. Having an automatically
> generated client is not a goal worthy of sacrificing on API consistency (by
> creating a separate /select like endpoint).
>
> On Wed, 29 Nov, 2023, 10:23 pm gerlowskija (via GitHub), 
> wrote:
>
> >
> > gerlowskija merged PR #2079:
> > URL: https://github.com/apache/solr/pull/2079
> >
> >
> > --
> > This is an automated message from the Apache Git Service.
> > To respond to the message, please log on to GitHub and use the
> > URL above to go to the specific comment.
> >
> > To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
> >
> > For queries about this service, please contact Infrastructure at:
> > us...@infra.apache.org
> >
> >
> > -
> > To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
> > For additional commands, e-mail: issues-h...@solr.apache.org
> >
> >
>

Re: Is SolrBot too noisy / being ignored...?

[Mike Drob]

That’s probably a question for asf legal

On Thu, Apr 18, 2024 at 5:36 PM Eric Pugh 
wrote:

> Thanks for the work that has been done on some of these.
>
> I actually just ran through the process of updating commons-cli based on
> what SolrBot provided.   I *did* have to update a Java class, and I did
> regenerate the licenses, and that was about it…
>
> Which made me wonder..   If SolrBot opens a dependency upgrade, and
> recommit and the tests pass, could we have it just commit automatically the
> update?
>
> I looked at one that I constantly see, the update to the awssdk:
> https://github.com/apache/solr/pull/2056.   The tests all pass, and it
> appears all I need to do to make precommit happy is drop in some new
> licenses.   Other than that, I believe that I could merge that PR, and I
> wouldn’t need to do any other steps….  So, if there were no new license and
> precommit had passed, couldn’t SolrBot merge it for us?
>
> Basically, do we actually need a human in the loop on this when at least
> this human, me, wouldn’t really be doing anything else if all the checks
> passed….
>
> > On Apr 9, 2024, at 8:01 AM, Eric Pugh 
> wrote:
> >
> > The update that I see a lot is for the software.amazon.awssdk and
> com.google.cloud packages….  I checked renovate.json and they should only
> happen once a month.
> >
> > I just checked and there has been an update today, yesterday, and the
> day before for the software.amazon.awssdk package.
> >
> > Looks like they all go to https://github.com/apache/solr/pull/2056
> however.  Is this because once it opens the PR, it is just updating the PR
> as needed?
> >
> > How can we get a smoother workflow?   The constant updates are noisy,
> and now I think they are just ignored…!   I saw that Kevin approved this
> back in November 2023.   Do we want to be more on top of these and merge as
> they go?
> >
> > And maybe for these frequently changing ones, maybe move to a quarterly
> schedule?   Or, do we add it to the release manager process, though I know
> that approach was discussed and then viewed as too burdensome for the RM.
> >
> >
> >
> > Eric
> >
> >
> >
> >
> >
> > ___
> > Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
> ___
> Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>

Re: SolrCloud architecture overview doc?

[Mike Drob]

I think Mark has given this as a presentation a few times, maybe
https://www.youtube.com/watch?v=eVK0wLkLw9w

On Thu, May 30, 2024 at 3:03 AM Ilan Ginzburg  wrote:

> Hi,
>
> I'm looking for a blog post or documentation giving a good overview of the
> architecture of SolrCloud (not the user experience and use of Solr). The
> audience would be engineers knowledgeable about distributed systems that
> know nothing about SolrCloud (side note: I looked for something similar for
> Elasticsearch and didn't find either).
>
> Documentation that will touch on the index structure
> (collection/shard/replica), the way they're materialized as cores, the
> coordination done by ZooKeeper (shard leader élections). How these replicas
> interact with each other and the update log in steady state and in recovery
> scenarios.
> The Overseer role (I wrote the detailed overseer doc
> https://github.com/apache/solr/blob/main/dev-docs/overseer/overseer.adoc)
> and how it interacts with ZooKeeper, the storage of all metadata in
> ZooKeeper, the way nodes start, the way cluster state is propagated and
> managed (watches for collections with replicas on a node vs all other
> collection) etc.
>
> If anybody knows about a doc that gives a complete overview, I'm very
> interested.
> Otherwise might end up writing it 🤓
>
> Thanks,
> Ilan
>

Welcome Bruno to the Apache Solr PMC

[Mike Drob]

I am pleased to announce that Bruno has accepted an invitation to join the
Solr PMC!

Congratulations, and welcome aboard!

Re: [solr] branch branch_8_8 created (now 90671d8)

[Mike Drob]

Yep, this was a mistake of pushing to the wrong remote. I've already
deleted it, sorry about the noise. :(

On Wed, Mar 24, 2021 at 1:31 PM Jan Høydahl 
wrote:

> We should not create the branch_8_8 in this repo. Use repo lucene-solr.git
> for backports.
>
> Jan Høydahl
>
> > 24. mar. 2021 kl. 18:01 skrev md...@apache.org:
> >
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > mdrob pushed a change to branch branch_8_8
> > in repository https://gitbox.apache.org/repos/asf/solr.git.
> >
> >
> >  at 90671d8  SOLR-15249 Properly set ZK ACLs on /security.json
> >
> > This branch includes the following new commits:
> >
> > new 90671d8  SOLR-15249 Properly set ZK ACLs on /security.json
> >
> > The 1 revisions listed above as "new" are entirely new to this
> > repository and will be described in separate emails.  The revisions
> > listed as "add" were already present in the repository and have only
> > been added to this reference.
> >
>

Bugfix release Lucene/Solr 8.8.2

[Mike Drob]

I am now preparing for a bugfix release from branch branch_8_8

I plan to have the RC built and vote started on Tuesday, Mar 30. If you
have small, low risk bug fixes to backport before then, please do so using
your best judgement.

Please observe the normal rules for committing to this branch:

* Before committing to the branch, reply to this thread and argue
  why the fix needs backporting and how long it will take.
* All issues accepted for backporting should be marked with 8.8.2
  in JIRA, and issues that should delay the release must be marked as
Blocker
* All patches that are intended for the branch should first be committed
  to the unstable branch, merged into the stable branch, and then into
  the current release branch.
* Only Jira issues with Fix version 8.8.2 and priority "Blocker" will delay
  a release candidate build.

Thanks,
Mike

Re: Bugfix release Lucene/Solr 8.8.2

[Mike Drob]

Ishan,

Thank you for bringing this up. I’m comfortable delaying an extra week to
accommodate the multitude of holidays (Holi, Passover, others) coming up.

I will adjust my schedule to start the vote Tuesday, Apr 6.

Please make sure that all back ports are appropriately marked with
fixVersion in Jira and have corresponding CHANGES entries.

Mike

On Fri, Mar 26, 2021 at 11:11 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hi Mike,
>
> I wish to get https://issues.apache.org/jira/browse/SOLR-15288 in, but
> will likely be able to wrap up by 2 April or so (on vacation right now due
> to the festival of Holi)
>
> Regards,
> Ishan
>
> On Sat, 27 Mar, 2021, 7:41 am Mike Drob,  wrote:
>
>> I am now preparing for a bugfix release from branch branch_8_8
>>
>> I plan to have the RC built and vote started on Tuesday, Mar 30. If you
>> have small, low risk bug fixes to backport before then, please do so using
>> your best judgement.
>>
>> Please observe the normal rules for committing to this branch:
>>
>> * Before committing to the branch, reply to this thread and argue
>>   why the fix needs backporting and how long it will take.
>> * All issues accepted for backporting should be marked with 8.8.2
>>   in JIRA, and issues that should delay the release must be marked as
>> Blocker
>> * All patches that are intended for the branch should first be committed
>>   to the unstable branch, merged into the stable branch, and then into
>>   the current release branch.
>> * Only Jira issues with Fix version 8.8.2 and priority "Blocker" will
>> delay
>>   a release candidate build.
>>
>> Thanks,
>> Mike
>>
>

Re: Bugfix release Lucene/Solr 8.8.2

[Mike Drob]

Ignacio, Alan - I have looked at the patches and these should be safe to
backport and useful in a bugfix. Please go ahead and commit, and update
CHANGES entry as well.

Cassandra - yes, I plan to republish the ref guide, your expertise is
absolutely appreciated. It looks like there will also be some gradle->ant
translation. Was this mostly find-and-replace or was there more order to it?

On Mon, Mar 29, 2021 at 12:32 PM Alan Woodward  wrote:

> Hi Mike,
>
> I’d like to back port LUCENE-9744 (which fixes an NPE in intervals
> queries) and LUCENE-9762 (which fixes a bug in QueryValuesSource).
>
> Thanks, Alan
>
> On 29 Mar 2021, at 09:47, Ignacio Vera  wrote:
>
> I'd like to backport Lucene-9870 which is a bug in distance queries that
> causes no matches along indexed lines and polygon edges. This fix
> only touches one class tho so very low risk.
> lucene
>
> On Sat, Mar 27, 2021 at 3:24 PM Mike Drob  wrote:
>
>> Ishan,
>>
>> Thank you for bringing this up. I’m comfortable delaying an extra week to
>> accommodate the multitude of holidays (Holi, Passover, others) coming up.
>>
>> I will adjust my schedule to start the vote Tuesday, Apr 6.
>>
>> Please make sure that all back ports are appropriately marked with
>> fixVersion in Jira and have corresponding CHANGES entries.
>>
>> Mike
>>
>> On Fri, Mar 26, 2021 at 11:11 PM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> Hi Mike,
>>>
>>> I wish to get https://issues.apache.org/jira/browse/SOLR-15288 in, but
>>> will likely be able to wrap up by 2 April or so (on vacation right now due
>>> to the festival of Holi)
>>>
>>> Regards,
>>> Ishan
>>>
>>> On Sat, 27 Mar, 2021, 7:41 am Mike Drob,  wrote:
>>>
>>>> I am now preparing for a bugfix release from branch branch_8_8
>>>>
>>>> I plan to have the RC built and vote started on Tuesday, Mar 30. If you
>>>> have small, low risk bug fixes to backport before then, please do so using
>>>> your best judgement.
>>>>
>>>> Please observe the normal rules for committing to this branch:
>>>>
>>>> * Before committing to the branch, reply to this thread and argue
>>>>   why the fix needs backporting and how long it will take.
>>>> * All issues accepted for backporting should be marked with 8.8.2
>>>>   in JIRA, and issues that should delay the release must be marked as
>>>> Blocker
>>>> * All patches that are intended for the branch should first be committed
>>>>   to the unstable branch, merged into the stable branch, and then into
>>>>   the current release branch.
>>>> * Only Jira issues with Fix version 8.8.2 and priority "Blocker" will
>>>> delay
>>>>   a release candidate build.
>>>>
>>>> Thanks,
>>>> Mike
>>>>
>>>
>

Re: Bugfix release Lucene/Solr 8.8.2

[Mike Drob]

I feel kind of weird about this one, is there a slightly less minimal patch
that is still reasonably safe and would let us continue to avoid
problematic terminology?

On Mon, Mar 29, 2021 at 1:18 PM Tomás Fernández Löbbe 
wrote:

> Mike,
> I'd like to backport SOLR-15216 (UI Fix)
>
> On Mon, Mar 29, 2021 at 10:55 AM Mike Drob  wrote:
>
>> Ignacio, Alan - I have looked at the patches and these should be safe to
>> backport and useful in a bugfix. Please go ahead and commit, and update
>> CHANGES entry as well.
>>
>> Cassandra - yes, I plan to republish the ref guide, your expertise is
>> absolutely appreciated. It looks like there will also be some gradle->ant
>> translation. Was this mostly find-and-replace or was there more order to it?
>>
>> On Mon, Mar 29, 2021 at 12:32 PM Alan Woodward 
>> wrote:
>>
>>> Hi Mike,
>>>
>>> I’d like to back port LUCENE-9744 (which fixes an NPE in intervals
>>> queries) and LUCENE-9762 (which fixes a bug in QueryValuesSource).
>>>
>>> Thanks, Alan
>>>
>>> On 29 Mar 2021, at 09:47, Ignacio Vera  wrote:
>>>
>>> I'd like to backport Lucene-9870 which is a bug in distance queries that
>>> causes no matches along indexed lines and polygon edges. This fix
>>> only touches one class tho so very low risk.
>>> lucene
>>>
>>> On Sat, Mar 27, 2021 at 3:24 PM Mike Drob  wrote:
>>>
>>>> Ishan,
>>>>
>>>> Thank you for bringing this up. I’m comfortable delaying an extra week
>>>> to accommodate the multitude of holidays (Holi, Passover, others) coming 
>>>> up.
>>>>
>>>> I will adjust my schedule to start the vote Tuesday, Apr 6.
>>>>
>>>> Please make sure that all back ports are appropriately marked with
>>>> fixVersion in Jira and have corresponding CHANGES entries.
>>>>
>>>> Mike
>>>>
>>>> On Fri, Mar 26, 2021 at 11:11 PM Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> wrote:
>>>>
>>>>> Hi Mike,
>>>>>
>>>>> I wish to get https://issues.apache.org/jira/browse/SOLR-15288 in,
>>>>> but will likely be able to wrap up by 2 April or so (on vacation right now
>>>>> due to the festival of Holi)
>>>>>
>>>>> Regards,
>>>>> Ishan
>>>>>
>>>>> On Sat, 27 Mar, 2021, 7:41 am Mike Drob,  wrote:
>>>>>
>>>>>> I am now preparing for a bugfix release from branch branch_8_8
>>>>>>
>>>>>> I plan to have the RC built and vote started on Tuesday, Mar 30. If
>>>>>> you have small, low risk bug fixes to backport before then, please do so
>>>>>> using your best judgement.
>>>>>>
>>>>>> Please observe the normal rules for committing to this branch:
>>>>>>
>>>>>> * Before committing to the branch, reply to this thread and argue
>>>>>>   why the fix needs backporting and how long it will take.
>>>>>> * All issues accepted for backporting should be marked with 8.8.2
>>>>>>   in JIRA, and issues that should delay the release must be marked as
>>>>>> Blocker
>>>>>> * All patches that are intended for the branch should first be
>>>>>> committed
>>>>>>   to the unstable branch, merged into the stable branch, and then into
>>>>>>   the current release branch.
>>>>>> * Only Jira issues with Fix version 8.8.2 and priority "Blocker" will
>>>>>> delay
>>>>>>   a release candidate build.
>>>>>>
>>>>>> Thanks,
>>>>>> Mike
>>>>>>
>>>>>
>>>

Re: Bugfix release Lucene/Solr 8.8.2

[Mike Drob]

Thanks, Cassandra. When I skimmed the initial changeset, I saw that there
were changes to build.gradle - I assumed that there would have to be some
changes to build.xml in the 8x branch, but I didn't investigate further on
how they would differ. Was just trying to be helpful and give you a heads
up about it, but looks like I caused more confusion than help!

On Mon, Mar 29, 2021 at 1:31 PM Cassandra Targett 
wrote:

> I pushed up the URL changes I mentioned.
>
> Not sure what you mean, Mike, by “gradle->ant translation”? The build
> process is different in branch_8x but it’s documented in the
> how-to-contribute page in the ref guide that talks about contributing and
> building, etc.
> On Mar 29, 2021, 12:55 PM -0500, Mike Drob , wrote:
>
>
> Cassandra - yes, I plan to republish the ref guide, your expertise is
> absolutely appreciated. It looks like there will also be some gradle->ant
> translation. Was this mostly find-and-replace or was there more order to it?
>
> On Mon, Mar 29, 2021 at 12:32 PM Alan Woodward  <https://mailto:romseyg...@gmail.com>> wrote:
>
> Hi Mike,
>
> I’d like to back port LUCENE-9744 (which fixes an NPE in intervals
> queries) and LUCENE-9762 (which fixes a bug in QueryValuesSource).
>
> Thanks, Alan
>
> On 29 Mar 2021, at 09:47, Ignacio Vera  <https://mailto:iver...@gmail.com>> wrote:
>
> I'd like to backport Lucene-9870 which is a bug in distance queries that
> causes no matches along indexed lines and polygon edges. This fix
> only touches one class tho so very low risk.
>
> lucene
>
> On Sat, Mar 27, 2021 at 3:24 PM Mike Drob  <https://mailto:md...@mdrob.com>> wrote:
>
> Ishan,
>
> Thank you for bringing this up. I’m comfortable delaying an extra week to
> accommodate the multitude of holidays (Holi, Passover, others) coming up.
>
> I will adjust my schedule to start the vote Tuesday, Apr 6.
>
> Please make sure that all back ports are appropriately marked with
> fixVersion in Jira and have corresponding CHANGES entries.
>
> Mike
>
> On Fri, Mar 26, 2021 at 11:11 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com <https://mailto:ichattopadhy...@gmail.com>>
> wrote:
>
> Hi Mike,
>
> I wish to get https://issues.apache.org/jira/browse/SOLR-15288 in, but
> will likely be able to wrap up by 2 April or so (on vacation right now due
> to the festival of Holi)
>
> Regards,
> Ishan
>
> On Sat, 27 Mar, 2021, 7:41 am Mike Drob,  <https://mailto:md...@mdrob.com>> wrote:
>
> I am now preparing for a bugfix release from branch branch_8_8
>
> I plan to have the RC built and vote started on Tuesday, Mar 30. If you
> have small, low risk bug fixes to backport before then, please do so using
> your best judgement.
>
> Please observe the normal rules for committing to this branch:
>
> * Before committing to the branch, reply to this thread and argue
>   why the fix needs backporting and how long it will take.
> * All issues accepted for backporting should be marked with 8.8.2
>   in JIRA, and issues that should delay the release must be marked as
> Blocker
> * All patches that are intended for the branch should first be committed
>   to the unstable branch, merged into the stable branch, and then into
>   the current release branch.
> * Only Jira issues with Fix version 8.8.2 and priority "Blocker" will delay
>   a release candidate build.
>
> Thanks,
> Mike
>
>
>

Re: Bugfix release Lucene/Solr 8.8.2

[Mike Drob]

I am going to start building the 8.8.2 RCs and running some tests locally
before the vote. Please hold on further commits to branch_8_8, or let me
know that I need to restart the builds if we find something critical in the
next few hours.

On Tue, Mar 30, 2021 at 6:26 AM Alan Woodward  wrote:

> LUCENE-9744 and LUCENE-9762 are both backported.
>
> On 30 Mar 2021, at 09:03, Ignacio Vera  wrote:
>
> Hi Mike,
>
> LUCENE-9870 has been backported, Thanks!
>
> On Mon, Mar 29, 2021 at 10:32 PM Cassandra Targett 
> wrote:
>
>> Oh, I see what you meant! I actually just redid the changes from scratch
>> instead of trying to backport the 9.0 changes as a patch, so no worries on
>> that front. Thanks for thinking of me there though!
>>
>> I just committed some changes for morelikethis.adoc that I decided I’d
>> like to backport into branch_8_8 since we’re re-publishing…it’s docs-only,
>> though.
>> On Mar 29, 2021, 2:25 PM -0500, Mike Drob , wrote:
>>
>> Thanks, Cassandra. When I skimmed the initial changeset, I saw that there
>> were changes to build.gradle - I assumed that there would have to be some
>> changes to build.xml in the 8x branch, but I didn't investigate further on
>> how they would differ. Was just trying to be helpful and give you a heads
>> up about it, but looks like I caused more confusion than help!
>>
>> On Mon, Mar 29, 2021 at 1:31 PM Cassandra Targett 
>> wrote:
>>
>>> I pushed up the URL changes I mentioned.
>>>
>>> Not sure what you mean, Mike, by “gradle->ant translation”? The build
>>> process is different in branch_8x but it’s documented in the
>>> how-to-contribute page in the ref guide that talks about contributing and
>>> building, etc.
>>> On Mar 29, 2021, 12:55 PM -0500, Mike Drob , wrote:
>>>
>>>
>>> Cassandra - yes, I plan to republish the ref guide, your expertise is
>>> absolutely appreciated. It looks like there will also be some gradle->ant
>>> translation. Was this mostly find-and-replace or was there more order to it?
>>>
>>> On Mon, Mar 29, 2021 at 12:32 PM Alan Woodward >> <https://mailto:romseyg...@gmail.com/>> wrote:
>>>
>>> Hi Mike,
>>>
>>> I’d like to back port LUCENE-9744 (which fixes an NPE in intervals
>>> queries) and LUCENE-9762 (which fixes a bug in QueryValuesSource).
>>>
>>> Thanks, Alan
>>>
>>> On 29 Mar 2021, at 09:47, Ignacio Vera >> <https://mailto:iver...@gmail.com/>> wrote:
>>>
>>> I'd like to backport Lucene-9870 which is a bug in distance queries that
>>> causes no matches along indexed lines and polygon edges. This fix
>>> only touches one class tho so very low risk.
>>>
>>> lucene
>>>
>>> On Sat, Mar 27, 2021 at 3:24 PM Mike Drob >> <https://mailto:md...@mdrob.com/>> wrote:
>>>
>>> Ishan,
>>>
>>> Thank you for bringing this up. I’m comfortable delaying an extra week
>>> to accommodate the multitude of holidays (Holi, Passover, others) coming up.
>>>
>>> I will adjust my schedule to start the vote Tuesday, Apr 6.
>>>
>>> Please make sure that all back ports are appropriately marked with
>>> fixVersion in Jira and have corresponding CHANGES entries.
>>>
>>> Mike
>>>
>>> On Fri, Mar 26, 2021 at 11:11 PM Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com <https://mailto:ichattopadhy...@gmail.com/>>
>>> wrote:
>>>
>>> Hi Mike,
>>>
>>> I wish to get https://issues.apache.org/jira/browse/SOLR-15288 in, but
>>> will likely be able to wrap up by 2 April or so (on vacation right now due
>>> to the festival of Holi)
>>>
>>> Regards,
>>> Ishan
>>>
>>> On Sat, 27 Mar, 2021, 7:41 am Mike Drob, >> <https://mailto:md...@mdrob.com/>> wrote:
>>>
>>> I am now preparing for a bugfix release from branch branch_8_8
>>>
>>> I plan to have the RC built and vote started on Tuesday, Mar 30. If you
>>> have small, low risk bug fixes to backport before then, please do so using
>>> your best judgement.
>>>
>>> Please observe the normal rules for committing to this branch:
>>>
>>> * Before committing to the branch, reply to this thread and argue
>>>   why the fix needs backporting and how long it will take.
>>> * All issues accepted for backporting should be marked with 8.8.2
>>>   in JIRA, and issues that should delay the release must be marked as
>>> Blocker
>>> * All patches that are intended for the branch should first be committed
>>>   to the unstable branch, merged into the stable branch, and then into
>>>   the current release branch.
>>> * Only Jira issues with Fix version 8.8.2 and priority "Blocker" will
>>> delay
>>>   a release candidate build.
>>>
>>> Thanks,
>>> Mike
>>>
>>>
>>>
>

[DISCUSS] Zookeeper 3.7 for Solr 9.0

[Mike Drob]

Like it says on the box - what do folks think about this? Is anybody
already trying it?

Updating the clients? I think there's new security options available.

What does the back-compat story look like?

Mike

[VOTE] Release Lucene/Solr 8.8.2 RC1

[Mike Drob]

Please vote for release candidate 1 for Lucene/Solr 8.8.2

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.2-RC1-reva92a05e195b775b30ca410bc0a26e8e79e7b3bfb

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.2-RC1-reva92a05e195b775b30ca410bc0a26e8e79e7b3bfb

The vote will be open until 2021-04-12 00:00 UTC. I will tally votes on
Monday morning.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

[RESULT] [VOTE] Release Lucene/Solr 8.8.2 RC1

[Mike Drob]

It's been >72h since the vote was initiated and the result is:

+1  5 (Mike Drob, Tim Potter, Anshum Gupta, Bruno Roustant, Ignacio Vera)
 0  2 (Uwe Schindler, Jan Høydahl)
-1  0

This vote has PASSED

Re: NOTICE.txt requirements for new/updated dependencies

[Mike Drob]

If the dependency you are using has a NOTICE file, then we need to preserve
that content in our own NOTICE file (see Apache Licence section 4d)

I imagine the intent was to make sure we are using compatible licenses
according to the ASF release policies which defines category A/B/X and also
to be aware of the transitive dependencies that we might be bringing in.

On Mon, Apr 12, 2021 at 12:18 PM Jason Gerlowski 
wrote:

> Hey all,
>
> I have a PR open that needs access to a new JAR library, so I've been
> trying to understand the full set of steps involved in adding a
> dependency to Solr.  I followed the steps mentioned in
> `help/dependencies.txt` (i.e. 'gradlew helpDependencies') without too
> much trouble: the JAR is visible on the classpath and the
> 'solr/licenses/' directory has the appropriate checksum, license and
> NOTICE.txt files.  Everything looks good.
>
> Just when I thought I was done though, I noticed one more step in
> 'solr/licenses/README.committers.txt':
>
> > Under no circumstances should any new files be added to this directory
> > without careful consideration of how LICENSE.txt and NOTICE.txt in the
> > parent directory should be updated to reflect the addition.
>
> Does anyone remember the context around this step, and whether it is
> still valid today?  It seems like it might be outdated, but maybe not.
>
> If appending to the top-level NOTICE.txt is still required: under what
> conditions?  'solr/NOTICE.txt' only has a NOTICE.txt file for ~40
> dependencies: conspicuously few compared to the ~150 non empty
> NOTICE.txt files in the 'solr/licenses/' directory.
>
> Appreciate any context people can offer here: just looking to make
> sure I get the process right.
>
> Best,
>
> Jason
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: NOTICE.txt requirements for new/updated dependencies

[Mike Drob]

Many of the notices that we have there are zero bytes.

The top level aggregate notice is part of asf release policy iirc.

On Mon, Apr 12, 2021 at 12:52 PM Jason Gerlowski 
wrote:

> > Why do we need a jar dependency?
> We don't.  Just the "normal" dependency on a java library that's
> distributed through maven central, etc.  Sorry if my wording suggested
> otherwise.
>
> > If the dependency you are using has a NOTICE file, then we need to
> preserve that content in our own NOTICE file
>
> AFAICT, ASL section 4d doesn't say _where_ the NOTICE content needs to
> live, does it?  And we're already including a NOTICE.txt entry for
> each dependency in the 'solr/licenses/' directory. [1]  Isn't the
> content of 'solr/licenses/' sufficient to meet 4d?  If not, have the
> majority of our deps been added incorrectly? (40ish in
> solr/NOTICE.txt, ~150 in 'solr/licenses/')
>
> [1] https://github.com/apache/solr/tree/main/solr/licenses
>
> On Mon, Apr 12, 2021 at 1:42 PM Ishan Chattopadhyaya
>  wrote:
> >
> > Why do we need a jar dependency? Is the artifact not available through
> maven central? IIRC, @Uwe Schindler once mentioned that we should avoid
> adding jar files to the project.
> >
> > On Mon, 12 Apr, 2021, 11:03 pm Mike Drob,  wrote:
> >>
> >> If the dependency you are using has a NOTICE file, then we need to
> preserve that content in our own NOTICE file (see Apache Licence section 4d)
> >>
> >> I imagine the intent was to make sure we are using compatible licenses
> according to the ASF release policies which defines category A/B/X and also
> to be aware of the transitive dependencies that we might be bringing in.
> >>
> >> On Mon, Apr 12, 2021 at 12:18 PM Jason Gerlowski 
> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> I have a PR open that needs access to a new JAR library, so I've been
> >>> trying to understand the full set of steps involved in adding a
> >>> dependency to Solr.  I followed the steps mentioned in
> >>> `help/dependencies.txt` (i.e. 'gradlew helpDependencies') without too
> >>> much trouble: the JAR is visible on the classpath and the
> >>> 'solr/licenses/' directory has the appropriate checksum, license and
> >>> NOTICE.txt files.  Everything looks good.
> >>>
> >>> Just when I thought I was done though, I noticed one more step in
> >>> 'solr/licenses/README.committers.txt':
> >>>
> >>> > Under no circumstances should any new files be added to this
> directory
> >>> > without careful consideration of how LICENSE.txt and NOTICE.txt in
> the
> >>> > parent directory should be updated to reflect the addition.
> >>>
> >>> Does anyone remember the context around this step, and whether it is
> >>> still valid today?  It seems like it might be outdated, but maybe not.
> >>>
> >>> If appending to the top-level NOTICE.txt is still required: under what
> >>> conditions?  'solr/NOTICE.txt' only has a NOTICE.txt file for ~40
> >>> dependencies: conspicuously few compared to the ~150 non empty
> >>> NOTICE.txt files in the 'solr/licenses/' directory.
> >>>
> >>> Appreciate any context people can offer here: just looking to make
> >>> sure I get the process right.
> >>>
> >>> Best,
> >>>
> >>> Jason
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >>> For additional commands, e-mail: dev-h...@solr.apache.org
> >>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

[ANNOUNCE] Apache Solr 8.8.2 released

[Mike Drob]

The Solr PMC is pleased to announce the release of Apache Solr 8.8.2

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search and analytics, rich document
parsing, geospatial search, extensive REST APIs as well as parallel SQL.
Solr is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.

This release contains several bug fixes. The release is available for
immediate download at:

https://solr.apache.org/downloads.html
Solr 8.8.2 Release Highlights

   - SOLR-15249: Properly set ZK ACLs on /security.json
   - SOLR-15233: Set doAs param in ConfigurableInternodeAuthHadoopPlugin
   - SOLR-15217: Use shardsWhitelist in ReplicationHandler
   - SOLR-15288: Hardening NODEDOWN event in collections using
   PerReplicaStates

Please report any feedback to the mailing lists (
https://solr.apache.org/community.html#mailing-lists-irc)


A summary of important changes is published in the Solr Reference Guide at
https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html.

For the most exhaustive list, see the full release notes at
https://lucene.apache.org/solr/8_8_2/changes/Changes.html or by viewing the
CHANGES.txt file accompanying the distribution.

Solr's release notes usually don't include Lucene layer changes.  Lucene's
release notes are at
https://lucene.apache.org/core/8_8_2/changes/Changes.html
Note: The Apache Software Foundation uses an extensive mirroring network for
distributing releases. It is possible that the mirror you are using may not
have
replicated the release yet. If that is the case, please try another mirror.
This also applies to Maven access.

Re: [Operator] [VOTE] Release the Solr Operator v0.3.0 RC1

[Mike Drob]

-1

[OK] Verified signatures
[OK] Verified checksums

[BAD] helm-carts/index.yaml missing licence header
[BAD] most yaml files in source tarball are missing license headers
[BAD] Makefile missing license header

[UNKNOWN] what is helm-charts/solr-operator-0.3.0.tgz.prov? It looks
like a gpg signed yaml file, maybe this is part of the helm standard
but the filename looks confusing to me.
[UNKNOWN] why are we releasing zookeeperclusters.yaml with our crds?
It's not our code, I don't want us to be forking it. Do we have to
include it?'

Will continue testing and reviewing, but those were the first things that I saw.

On Thu, Apr 15, 2021 at 4:20 PM Houston Putman  wrote:
>
> Updated the release wizard, but not my draft email
>
> You need to clone the solr-operator repository and checkout the release-0.3 
> branch before running the smoke test.
> And you should run the smoke test from the base directory of the 
> solr-operator repo.
>
> Hope it goes smoothly for everyone!
>
> - Houston
>
> On Thu, Apr 15, 2021 at 3:53 PM Houston Putman  wrote:
>>
>> Please vote for release candidate 1 for the Solr Operator v0.3.0
>>
>> The artifacts can be downloaded from:
>> https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.3.0-RC1-rev8b14ce2567599561358eaa7d75210cd29fd21a41
>>
>> You can run the full smoke tester, with instructions below.
>> However, it is also encouraged to go and use the artifacts yourself in a 
>> test Kubernetes cluster.
>> The smoke tester does not require you to download or install the RC 
>> artifacts before running.
>> If you plan on just running the smoke tests, then ignore all other 
>> instructions.
>>
>> The artifacts are layed out in the following way:
>>   * solr-operator-v0.3.0.tgz - Contains the source release
>>   * crds/ - Contains the CRD files
>>   * helm/ - Contains the Helm release packages
>>
>> The RC Docker image can be found at:
>>   apache/solr-operator:v0.3.0-rc1
>>
>> The RC Helm repo can be added with:
>>   helm repo add solr-operator-rc 
>> https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.3.0-RC1-rev8b14ce2567599561358eaa7d75210cd29fd21a41/helm-charts
>>
>> You can install the RC Solr Operator and CRDs with:
>>   kubectl create -f 
>> https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.3.0-RC1-rev8b14ce2567599561358eaa7d75210cd29fd21a41/crds/all-with-dependencies.yaml
>>   helm install --verify solr-operator solr-operator-rc/solr-operator --set 
>> image.tag=v0.3.0-rc1
>>
>> You can run the full smoke tester directly with this command:
>>
>> ./hack/release/smoke_test/smoke_test.sh -v "v0.3.0" -s "8b14ce2" -i 
>> "apache/solr-operator:v0.3.0-rc1" \
>> -l 
>> 'https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.3.0-RC1-rev8b14ce2567599561358eaa7d75210cd29fd21a41'
>>
>> Make sure you have the following installed before running the smoke test:
>>   - Docker (Give it enough memory and CPU to run ~12 containers, 3 of which 
>> are Solr nodes)
>>   - Go 1.16
>>   - Kubectl
>>   - GnuPG
>>   - Helm v3.4.0+
>>   - Kustomize (v4.0.0+) This will be installed for you, but NOT upgraded if 
>> a lower version is already installed.
>>   - yq
>>   - jq
>>
>> The vote will be open for at least 72 hours i.e. until 2021-04-20 21:00 UTC.
>> (I have extended this to Tuesday, since the vote occurs over a weekend)
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>> Here is my +1

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Jenkins Nightly Tests Failing to launch

[Mike Drob]

https://ci-builds.apache.org/job/Solr/job/Solr-NightlyTests-main/38/console
looks like it is failing because something is wrong with either the
checkout or the connection to the jenkins workers or agents. I don't
really know enough about Jenkins to troubleshoot this, but bringing it
up in case somebody else can take a look.

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Solr Roadmap (and 9.0 release planning)

[Mike Drob]

Absent anything else, I want us to stop breaking whenever Lucene makes
an incompatible change reflected in a snapshot. Pinning to a 9.0
release of theirs would give us that. Everything else is gravy.

9.0 will be built with Gradle, built after the split, there's already
a lot going into it. The way that we get more "selling points" into it
(if we need them), is to stop releasing 8.x releases.

On Wed, Apr 28, 2021 at 1:39 PM Alexandre Rafalovitch
 wrote:
>
> On v2 API,
>
> I did some review a while ago, though the goal was to compare its
> coverage with solrconfig.xml, rather than with V1.
>
> My discoveries (umbrella and children) are in SOLR-14795 .
>
> Regards,
>Alex.
>
> On Wed, 28 Apr 2021 at 12:04, Jan Høydahl  wrote:
> >
> > Hi all,
> >
> > We've come a long way since I last looked at the Roadmap WIKI page at 
> > https://cwiki.apache.org/confluence/display/SOLR/Roadmap
> >
> > We have setup the Solr TLP, released a few new versions and landed some of 
> > the major features for 9.0
> >
> > Now is probably a good time to do some forward-thinking, preparing what 
> > features need to land in 8.9 to make a smooth upgrade path for 9.0, and 
> > what the 9.0 blockers are.
> > I've updated the Roadmap page by removing outdated content. Feel free to 
> > edit it, perhaps remove stuff that is irrelevant or will not be done.
> >
> > Let's start 8.9 and 9.0 visionary release planning in this mail thread.
> >
> > Some of my questions are
> > * Will 9.0 have a working, tested alternative to Autoscaling API? Or are we 
> > missing some pieces that should be 9.0 blockers?
> > * Will we deprecate v1 APIs? When? I see some v2 work done lately...
> > * Will 9.0 ship with some 1st party plugins or will we still ship all 
> > contribs?
> > * What will be the main selling point for the Solr 9.0 release? Other than 
> > "runs Lucene 9"?
> > * etc
> >
> > Jan
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Splitting the collection API into separate per-command class files

[Mike Drob]

Would the individual commands register via classpath scanning or static
initializers or is this something that @Endpoint already does for us? In
general, I would be in favor of splitting out the commands, although some
of them might still make sense to group together.

On Thu, Apr 29, 2021 at 4:05 PM David Smiley  wrote:

> For a long time, I have not been fond of how the code for our collection
> APIs are organized.  I would prefer to see more separation between the
> commands (thus no long class files implementing many commands), and fewer
> places to touch generally.  If more of a command's logic is in its own
> source file, I think this would be easier to understand (less indirection
> across many classes), easier to code review, and easier to observe from a
> source control perspective.
>
> I just did a review of "MODIFYCOLLECTION" to illustrate the situation and
> also because Jason was looking at this one in SOLR-15351:
>
> Solr-Core:
> * collections.collection.Commands.modify.json (43 lines of metadata)
> * CollectionsHandler: CollectionOperation enum entry (~20 lines)
> * Overseer: processMessage switch entry (2 lines, trivial code)
> * DistributedClusterStateUpdateer: MutatingCommand enum entry (6 lines,
> trivial code)
> * CollectionMutator: (55 lines)
> SolrJ:
> * CollectionParams: CollectionAction enum entry (one line, trivial code)
> * CollectionApiMapping: Meta enum (~4 lines, trivial code)
> * CollectionAdminRequest: convenience method & class for users/clients
> (~80 lines)
>
> Furthermore, in SOLR-15351, Jason has a proposal to add another class with
> per-command code (~30 lines for this command) that would replace the
> modify.json one above.  I *really* like the essence of SOLR-15351 but I
> want to discuss on the dev list how we organize our code for these admin
> API commands.
>
> Imagine a "ModifyCollectionApi" class file.  It would have a method with
> an @EndPoint annotation that Jason shows in SOLR-15351.  Unlike the current
> state of that PR, it could also absorb more than that.  Let's have it
> absorb the logic in CollectionsHandler for this command.  Assuming we fully
> implement this approach, it would mean CollectionsHandler would be
> massively gutted and wouldn't be on the list above.
>
> That separation would be nice IMO but there are still many places to touch
> to understand how this one command works.  After this stage, it's worth
> seeing if it's best to have ModifyCollectionApi also contain the
> CollectionMutator logic for this command.  Probably?  Some of the APIs have
> differing backends to their implementation.
>
> The Core admin APIs could be improved to benefit from "@EndPoint" but the
> situation there isn't quite so bad ever since "CoreAdminOp" came into being
> with the separation of each implementation into separate source files that
> occurred in SOLR-9523 in 2016.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>

Re: Proposal to pin the Lucene snapshot version on main

[Mike Drob]

Note that this happened again last night, and Jason was able to
quickly fix it. But it makes things like 'git bisect' impossible to
chase a bug because none of the older versions will compile.

I don't think we can pin to a SNAPSHOT version because there are no
guarantees about how long those versions stick around. Maybe we can
ask Lucene to release a 9.0.0-alpha just so that we have some kind of
tag that we can stick to, with the understanding that this alpha tag
has no guarantees about future compatibility.

On Fri, Apr 30, 2021 at 2:36 AM Dawid Weiss  wrote:
>
> But how do you pin to an intermediate version without a snapshot maven
> repository that would keep those pinned artifacts?
>
> Dawid
>
> On Thu, Apr 29, 2021 at 9:45 PM David Smiley  wrote:
> >
> > +1 Jan, that sounds complementary to what I propose.  We'd get 
> > notifications via Jenkins that there are some compatibility issues.  But 
> > we'd still pin a version and upgrade at a time of our choosing.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Thu, Apr 29, 2021 at 11:35 AM Jan Høydahl  
> > wrote:
> >>
> >> It could be feasible to let Jenkins do a periodic run (weekly?) of main 
> >> branch with lucene SNAPSHOT. We can perhaps define a gradle property to 
> >> override lucene version? -Dlucene.version=10.0.0-SNAPSHOT which Jenkins 
> >> could easily trigger. I think a similar thought has been discussed before.
> >>
> >> Jan
> >>
> >> 29. apr. 2021 kl. 17:20 skrev Gus Heck :
> >>
> >> We should still have some way of detecting these breakages early rather 
> >> than later (or worse yet after lucene has released something). The easiest 
> >> time to fix a problem is before someone else built on top of it.
> >>
> >> On Thu, Apr 29, 2021 at 11:15 AM Jason Gerlowski  
> >> wrote:
> >>>
> >>> +1
> >>>
> >>> On Thu, Apr 29, 2021 at 11:12 AM Jan Høydahl
> >>>  wrote:
> >>> >
> >>> > +1 to pin. There will probably be a few more months until 9.0 given 
> >>> > that 8.9 must be released first etc.
> >>> >
> >>> > Jan
> >>> >
> >>> > 29. apr. 2021 kl. 17:08 skrev David Smiley :
> >>> >
> >>> > There have been some discussions previously about whether to pin the 
> >>> > Lucene snapshot version until 9.0 is out, so that we update it manually 
> >>> > instead of it being ~daily.  Most recently in Slack but also this 
> >>> > thread "Solr fails with current lucene-9.0.0-SNAPSHOT (LUCENE-9387)".  
> >>> > I think the rate of spontaneous breaking has increased beyond my 
> >>> > comfort level from being ambivalent on the matter to preferring more 
> >>> > control of when we update.  I know that may be as late as possible :-) 
> >>> > but it minimizes surprises/disruptions.  If there are no vetos on the 
> >>> > matter in this thread, I'll throw up a PR.
> >>> >
> >>> > ~ David Smiley
> >>> > Apache Lucene/Solr Search Developer
> >>> > http://www.linkedin.com/in/davidwsmiley
> >>> >
> >>> >
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >>> For additional commands, e-mail: dev-h...@solr.apache.org
> >>>
> >>
> >>
> >> --
> >> http://www.needhamsoftware.com (work)
> >> http://www.the111shift.com (play)
> >>
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Proposal to pin the Lucene snapshot version on main

[Mike Drob]

I think adding git submodule means that we have to add back in all the
build code for it. At which point, I'd rather just copy and commit the
code they have so we don't have to learn another git system. I've
heard submodules don't play nice with Jenkins, but don't have any
direct experience with them.

On Fri, Apr 30, 2021 at 2:08 PM David Smiley  wrote:
>
> Other than literally adding the git submodule, would we do anything else to 
> modify the gradle build so that or do we (and Jenkinsfile) have to manually
> do a step to install Lucene before proceeding?  Manually wouldn't necessarily 
> be bad, even if a first step, considering this is likely to only last until 
> 9.0.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Apr 30, 2021 at 3:03 PM Ishan Chattopadhyaya 
>  wrote:
>>
>> Can we give submodules a try for few weeks and then take an informed 
>> decision?
>>
>> On Sat, 1 May, 2021, 12:32 am Ishan Chattopadhyaya, 
>>  wrote:
>>>
>>> I fully support git submodules. +1 dawid.
>>> Most of the problems with it that people complain about, are perceived 
>>> problems, not real ones.
>>>
>>> On Fri, 30 Apr, 2021, 10:50 pm Dawid Weiss,  wrote:
>>>>
>>>> We've had that discussion before -- this is where git submodules are
>>>> excellent to work with - you know exactly which version you had for
>>>> each commit... this said, there are other pitfalls to using submodules
>>>> so it's not all rosy either.
>>>>
>>>> On Fri, Apr 30, 2021 at 4:41 PM Mike Drob  wrote:
>>>> >
>>>> > Note that this happened again last night, and Jason was able to
>>>> > quickly fix it. But it makes things like 'git bisect' impossible to
>>>> > chase a bug because none of the older versions will compile.
>>>> >
>>>> > I don't think we can pin to a SNAPSHOT version because there are no
>>>> > guarantees about how long those versions stick around. Maybe we can
>>>> > ask Lucene to release a 9.0.0-alpha just so that we have some kind of
>>>> > tag that we can stick to, with the understanding that this alpha tag
>>>> > has no guarantees about future compatibility.
>>>> >
>>>> > On Fri, Apr 30, 2021 at 2:36 AM Dawid Weiss  
>>>> > wrote:
>>>> > >
>>>> > > But how do you pin to an intermediate version without a snapshot maven
>>>> > > repository that would keep those pinned artifacts?
>>>> > >
>>>> > > Dawid
>>>> > >
>>>> > > On Thu, Apr 29, 2021 at 9:45 PM David Smiley  
>>>> > > wrote:
>>>> > > >
>>>> > > > +1 Jan, that sounds complementary to what I propose.  We'd get 
>>>> > > > notifications via Jenkins that there are some compatibility issues.  
>>>> > > > But we'd still pin a version and upgrade at a time of our choosing.
>>>> > > >
>>>> > > > ~ David Smiley
>>>> > > > Apache Lucene/Solr Search Developer
>>>> > > > http://www.linkedin.com/in/davidwsmiley
>>>> > > >
>>>> > > >
>>>> > > > On Thu, Apr 29, 2021 at 11:35 AM Jan Høydahl 
>>>> > > >  wrote:
>>>> > > >>
>>>> > > >> It could be feasible to let Jenkins do a periodic run (weekly?) of 
>>>> > > >> main branch with lucene SNAPSHOT. We can perhaps define a gradle 
>>>> > > >> property to override lucene version? 
>>>> > > >> -Dlucene.version=10.0.0-SNAPSHOT which Jenkins could easily 
>>>> > > >> trigger. I think a similar thought has been discussed before.
>>>> > > >>
>>>> > > >> Jan
>>>> > > >>
>>>> > > >> 29. apr. 2021 kl. 17:20 skrev Gus Heck :
>>>> > > >>
>>>> > > >> We should still have some way of detecting these breakages early 
>>>> > > >> rather than later (or worse yet after lucene has released 
>>>> > > >> something). The easiest time to fix a problem is before someone 
>>>> > > >> else built on top of it.
>>>> > > >>
>>>> > > >> On Thu, Apr 29, 202

Re: Proposal to pin the Lucene snapshot version on main

[Mike Drob]

Is this allowed by ASF policy? Are the Lucene “releases” that we are making
something that needs to be voted on and approved by the Lucene PMC?
https://apache.org/legal/release-policy.html#what

> In our case, that means any publication outside the group of people on
the product dev list.

Solr dev list is not the Lucene dev list.

On Fri, May 7, 2021 at 12:55 AM Uwe Schindler  wrote:

> Hi Cassandra. Sure, We can create a job on ASF Jenkins that is only
> manually triggered.
> It builds the maven artifacts into a local file system repo. As post build
> job we publish the output folder on nighties repo like the ref guide.
>
> The version number is passed as -DreleaseVersion system property (e.g.
> populated by Jenkins from the build number), like we do on artifacts builds
> on jenkins anyways. It's no snapshot anymore, but like an half official
> build with some custom version internal to our process.
>
> If you want to upgrade Solr's Lucene, just press "build" on jenkins, wait
> for it to finish, and finally copy-paste the version number from Jenkins
> build description into gradle dependency version file.
>
> Uwe
>
> Uwe
>
>
> Am May 6, 2021 8:13:00 PM UTC schrieb Cassandra Targett <
> casstarg...@gmail.com>:
>>
>> It could be a manual process if that’s what you think is easiest and it
>> could still be on nightlies. It’s just a curl command to push something up:
>> https://nightlies.apache.org/authoring.html. If it is a manual process,
>> I think it would be better for it to be hosted in a place any of us could
>> access, in case you’re busy/something happens to you and we need to pin a
>> different snapshot.
>>
>> I’m not sure I see a reason why Solr couldn’t have our own Jenkins job
>> that built Lucene once a week (or on demand, or whatever cadence that
>> works) and pushes the snapshot to nightlies - we don’t have to use one of
>> Lucene’s jobs, do we?
>>
>> I feel like we could also cascade a new Lucene snapshot build into a
>> “Lucene validation” Solr build that verifies everything is OK with a new
>> snapshot before updating the hosted snapshot on nightlies (although, I
>> don’t actually know how to do that, it just seems like something that could
>> be done).
>>
>> Cassandra
>> On May 6, 2021, 10:13 AM -0500, David Smiley , wrote:
>>
>> I asked on the Lucene dev list about possible use of the Nightlies
>> server.  We don't want to pollute nightlies on a regular basis (I think);
>> this would be an on-demand thing.  As such, I'm not sure a CI server is the
>> best way to approach this vs a manual script publishing to an ASF personal
>> Home space.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Mon, May 3, 2021 at 10:55 AM Cassandra Targett 
>> wrote:
>>
>>> I’m sort of surprised no one has mentioned the
>>> https://nightlies.apache.org/ server, which could be used for this
>>> purpose. Jenkins can push to it, and nothing gets deleted until we decide
>>> to delete it (or overwrite it). Solr Operator builds go there as do drafts
>>> of the Ref Guide pre-publication (in different directories now, but we’ll
>>> fix that eventually).
>>>
>>> Cassandra
>>> On May 3, 2021, 9:15 AM -0500, David Smiley , wrote:
>>>
>>> RE "an external system like Maven" -- we're merely talking about adding
>>> another JAR repo to the list of repos we already have.  Heck, for this
>>> limited purpose, we could even use http://home.apache.org/~dsmiley/
>>>  Note that the Lucene project already uses home dirs of some users for
>>> benchmark data.  If we were talking about adding a Maven *build* then I
>>> would totally appreciate your concern.
>>>
>>> I volunteer to use my space for this purpose.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Sun, May 2, 2021 at 12:17 AM Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> wrote:
>>>
 > Let’s not complicate things.
 By bringing in external systems like Maven, I think we're complicating
 things even though a straight forward way (git submodules) exists.

 On Sat, May 1, 2021 at 5:00 PM Jan Høydahl 
 wrote:

> Sub modules is for organizing internal repos, not for pulling in
> external deps. Let’s not complicate things. And once we switch main to 
> 10.x
> we’d need to use pure jar dependencies in branch_9x to depend upon 
> actually
> released and voted Lucene binaries.
>
> We need some tooling to smoothly work with bleeding edge Lucene
> including local snapshot builds with not yet pushed changes to Lucene. I’d
> rather put in some work to establish such tooling or procedures.
>
> Jan Høydahl
>
> 1. mai 2021 kl. 08:41 skrev Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
>
> 
> > What submodules don't solve is releases - if you're
> > on a
> particular unreleased Lucene version then releasing
> > Solr would still mean 

Test Framework - Zookeeper Watch Limits

[Mike Drob]

[Resending to the correct list]

Hi Devs,

Do folks use the Zookeeper watch limits? What are they useful for
debugging? Can somebody give me an example of when they have been
helpful to you?

The main reason I am asking is because I am looking at switching our
TestZookeeper to use the new 3.7.0 ZooKeeperServerEmbedded feature and
it would greatly simplify my task if I didn't need to worry about the
watch limits.

It's believable that they add value, but they also look like the logic
hasn't been touched since 2014 by the mighty Tim Potter.

Most of the times that I see them come up in the test output, it's
because something else failed anyway and we had lots of other watches
not get cleaned up, and the object release tracker went haywire as
well.

Maybe Mark has some stuff on reference branch that makes these more
constrained as well?

Hoping somebody knows the historical context and use case for these!

Mike

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Solr/Lucene joint development workflow?

[Mike Drob]

Can we commit the security policy addition independently?

On Thu, Jun 3, 2021 at 2:20 PM Michael Gibney 
wrote:

> Ok, I think I've got this now. jars from the mavenLocal() repo are used
> in-place by gradle, and not cached in `${gradle.lib.dir}`. This requires a
> security policy addition to
> `gradle/testing/randomization/policies/solr-tests.policy`:
>
>  grant {
>// Allow reading gradle worker JAR.
>permission java.io.FilePermission "${gradle.worker.jar}", "read";
>// Allow reading from classpath JARs (resources).
>permission java.io.FilePermission "${gradle.user.home}${/}-", "read";
> +  // Allow reading lucene jars from mavenLocal repository
> +  permission java.io.FilePermission
> "${user.home}${/}.m2${/}repository${/}org${/}apache${/}lucene${/}-", "read";
>  };
>
> This, along with the addition of the mavenLocal() repo and the change to
> "9.0.0-SNAPSHOT" lucene version in versions.props (and subsequent update of
> versions.lock file via `gradlew --write-locks`), does the trick.
>
> If I were to take a crack at documenting this workflow, any ideas on the
> best place to do so?
>
> Michael
>
>
> On Wed, Jun 2, 2021 at 9:19 AM Michael Gibney 
> wrote:
>
>> Thanks for digging into this! I can confirm that my
>> lucene-*-9.0.0-SNAPSHOT.jar files have META-INF/services/ with all the
>> expected codecs. I did remove the lucene subfolder from the maven
>> repo, re-ran `gradlew mavenToLocalRepo`, manually verified that the
>> expected META-INF/services/ listed all codecs, and re-ran the Solr
>> build (with `--refresh-dependencies`, and after having added the
>> `mavenLocal()` repo, changed `versions.props` to point at
>> "9.0.0-SNAPSHO", and run `gradlew --write-locks` to update the
>> versions.lock file). I'm still seeing the same exception ...
>>
>> Michael
>>
>> On Wed, Jun 2, 2021 at 8:01 AM Uwe Schindler  wrote:
>> >
>> > FYI,
>> >
>> > I did a recompile locally and checked the JAR files in my local Maven
>> repository ("./gradlew mavenToLocalRepo"): All contain META-INF/services
>> with all Codecs.
>> >
>> > I have no idea what goes wrong on your side. Next I will try to setup
>> mavenLocal() and a SNAPSHOT dependency, too.
>> >
>> > -
>> > Uwe Schindler
>> > Achterdiek 19, D-28357 Bremen
>> 
>> > https://www.thetaphi.de
>> > eMail: u...@thetaphi.de
>> >
>> > > -Original Message-
>> > > From: Uwe Schindler 
>> > > Sent: Wednesday, June 2, 2021 12:40 PM
>> > > To: dev@solr.apache.org
>> > > Subject: RE: Solr/Lucene joint development workflow?
>> > >
>> > > I'd suggest to remove the whole Lucene subfolder from your local
>> Maven repo.
>> > > E.g., rm -rf ~/.m2/repository/org/apache/lucene
>> > >
>> > > Then compile and deploy Lucene again.
>> > >
>> > > Uwe
>> > >
>> > > -
>> > > Uwe Schindler
>> > > Achterdiek 19, D-28357 Bremen
>> 
>> > > https://www.thetaphi.de
>> > > eMail: u...@thetaphi.de
>> > >
>> > > > -Original Message-
>> > > > From: Uwe Schindler 
>> > > > Sent: Wednesday, June 2, 2021 12:38 PM
>> > > > To: 'dev@solr.apache.org' 
>> > > > Subject: RE: Solr/Lucene joint development workflow?
>> > > >
>> > > > Hi,
>> > > >
>> > > > this error helps. It looks like the lucene-core.jar file does not
>> have the META-
>> > > > INF/services. I wonder how Solr works!
>> > > >
>> > > > It would be good to figure out what's going on and why the JAR
>> files in your
>> > > > local repository do not have the required META-INF/services
>> subfolder.
>> > > >
>> > > > Uwe
>> > > >
>> > > > -
>> > > > Uwe Schindler
>> > > > Achterdiek 19, D-28357 Bremen
>> 
>> > > > https://www.thetaphi.de
>> > > > eMail: u...@thetaphi.de
>> > > >
>> > > > > -Original Message-
>> > > > > From: Michael Gibney 
>> > > > > Sent: Wednesday, June 2, 2021 6:05 AM
>> > > > > To: dev@solr.apache.org
>> > > > > Subject: Re: Solr/Lucene joint development workflow?
>> > > > >
>> > > > > Uwe,
>> > > > > Thanks for this advice! I followed this as closely as I could
>> > > > > (including the gradle `--refresh-dependencies` flag), and was
>> again
>> > > > > able to compile (and compileTest) against mavenLocal. But on
>> actually
>> > > > > running tests, afaict all tests fail with the same stacktrace
>> (copied
>> > > > > below). iirc this was the same error I was getting previously
>> with the
>> > > > > SNAPSHOT build (before the introduction of the prerelease
>> dependency).
>> > > > > I'd be grateful for any advice on addressing this remaining error
>> (and
>> > > > > having encountered the error, I continue to be curious whether
>> anyone
>> > > > > has this workflow or something similar working in practice).
>> > > > >
>> > > > > David, your suggestion to add "mavenLocalI()" to Gradle's repo
>> list
>> > > > > seems reason

Re: Solr Reference Impl Branch state

[Mike Drob]

I’ve picked up a few of the improvements and brought them over to main
branch, fixed a few busy waits, improved some concurrent access, minor
stuff really. I’m currently busy trying to digest and experiment with the
jetty changes Mark mentioned earlier, so hopefully that’s another piece
that can come in early. There’s some more components, but I don’t want to
promise anything since it’s all competing with other priorities as well.

Mike

On Tue, Jun 8, 2021 at 5:43 PM Jan Høydahl  wrote:

> +1 to apply same code formatting on the ref branch to be able to compare.
>
> Solr still suffers from sloow and unstable tests. I whsh there were some
> lessons learnt from the ref-branch that could be ported over. I don't know
> if you separated out "Integration" tests from unit tests or something that
> we could bring over. Or got rid of some more bad busy-wait / sleep
> patterns? Or perhaps getting rid of unnecessary CloudCluster
> initializations, unneccessary waiting for cluster state or whatever? If not
> copy-pastable, having the biggest wins written down as short recipies of
> how to fix the test, would help a lot I guess. Or perhaps the test speedup
> all depends on the rest of the improvements made in non-test code... That's
> what it sounds like to me, that you cannot have an improvement in
> isolation, you need to do it all in one go :(
>
> Jan
>
> 8. jun. 2021 kl. 23:42 skrev David Smiley :
>
> Sorry to hear that Mark.  I hope it might be useful for little bits 'n
> pieces.  Occasionally I'm looking at some class, maybe a test, and I
> quickly do a comparison to the ref branch to wonder what you did there.
> There is a grand source code reformatting that is underway; we should apply
> that processing to that branch so that the comparisons remain useful.
>
> Uwe: In light of Mark's comments, I think you should disable and/or remove
> the CI build jobs.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Jun 8, 2021 at 1:27 PM Mark Miller  wrote:
>
>> At this point I doubt the ref branch is going to help anyone much other
>> than me as the need arises from any assignment i may  have or in sharing
>> information with others as the need arises (I put together a presentation
>> on Jetty from it for others on my team a bit ago.)
>>
>> Extracting anything but isolated help for what I may need from it is
>> currently beyond my ability.
>>
>> * It address and fixes thousands of bugs and issues of varying importance
>> and interconnectedness. Where would you even start.
>>
>> * It fully fully embraces, fixes and extends HTTP2 support, async, async
>> IO, Jetty. Doing that at all reasonably requires a tremendous amount of
>> work as each is very sensitive to getting things very close to right all
>> over the map. Even after spending ridiculous amounts of time on core
>> issues, that work is heavy to make solid.
>>
>> * It speeds up and hardens test after test after test. Ridiculous parts
>> and efforts involved, connected to everything else.
>>
>> * It minimizes resources and objects and GC tremendously everywhere,
>> moves more off heap, mmaps transaction log files, parallelizes tons more,
>> and leaves Lucene at the top of garbage generation stack (though Lucene is
>> certainly reasonable there). Again, where do you start.
>>
>> * And then it does some things that build on having everything else below
>> it.
>>
>> Other than a few items and hanging chads, it does or opens up whatever I
>> ever wanted for SolrCloud. And the effort is not simply each of the many
>> many items - it’s the crazy pain staking time and care working through all
>> of the issues and connectedness and surface area for everything.
>>
>> When I get some time to set it up for others, it will offer an alternate
>> view of what Solr could do, but ive imagined for a while it will mostly
>> offer me things and occasionally some sharing and demoing for what some
>> others have been interested in. And as I have things that I have to do, it
>> provides me with a working map and model of what the issues are, what I may
>> need to get around them, and where the current stuff stands.
>>
>> It’s what I would do with SolrCloud. It’s focused on what I think the
>> problems are, what I think matters to the end result. I don’t even often
>> see strong alignment on that with anyone other than silently across
>> timelines with Dat. Always find something I hadn’t noticed from Dat and
>> think, man, that guy is on my page.
>>
>>
>> MRM
>>
>> On Fri, Jun 4, 2021 at 8:51 AM Jan Høydahl  wrote:
>>
>>> Hi,
>>>
>>> We have an upcoming committers meeting soon, and will likely touch on
>>> the topic of Reference Branch.
>>> As we could probably waste the whole meeting on that one topic, I'm
>>> starting this thread to "warm up".
>>>
>>> As for me, I'm totally not up to date on what the status is. I don't
>>> even know if anyone have been looking at the branch at all lately.
>>> *Is there anyone who can give a s

[DISCUSS] Solr Cache with Futures as values

[Mike Drob]

Hi folks,

This is an idea based on a recent prod issue, and while we found
another workaround I think there is some merit to discuss here.

Currently our filter cache is a mapping from queries to docs, and the
result cache is similar although slightly more abstract. When we have
a lot of similar queries come in at the same time, if a particular
filter hasn't been cached yet then it will be computed a bunch of
times in parallel as each query tries to be the one to insert into the
cache.

One option that I've thought about is if instead of inserting results
into the cache directly, we pre-register a future in the cache, and
then use that as a reference to the results. Multiple queries coming
in parallel would all wait for the same result calculation instead of
allocating large arrays each.

The benefits are pretty straightforward - we reduce the amount of
computation done when there are lots of queries coming in, and reduce
the memory allocation pressure.

The complexity might be around handling errors or query timeouts or
cancellations. Or evictions, but I think that would all be manageable.

What do other folks think? Should I write up a SIP for this, since I
think it will be fairly complex, or are there existing solutions that
I should look into first?

Mike

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Solr Cache with Futures as values

[Mike Drob]

Thanks for the pointer, David!

While browsing through that issue, I found this comment left by you
from SOLR-14166

https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L1185

"note: can't use computeIfAbsent because can be recursive"

I don't quite understand what the recursive case means here, can you
elaborate on that? I didn't see any discussion of it in the JIRA
either.

Thanks,
Mike

On Wed, Jul 14, 2021 at 11:03 AM David Smiley  wrote:
>
> Ideally, we could use SolrCache.computeIfAbsent [1] for the filter cache, as 
> is used for some of the other caches.  The best SolrCache is CaffeineCache 
> which works atomically for the same key (just as does ConcurrentHashMap).  
> The problem is that this method on CaffeineCache does not support computing a 
> cache entry that is reentrant, i.e. that which can produce another cache 
> entry when it is computed.  Really, that limitation ought to be elevated to 
> the docs on SolrCache.computeIfAbsent.  Andrzej discovered [1] that some 
> queries could do that, and so he did not update Solr's use of the filter 
> cache to call it.  Please read the thread there and maybe comment further to 
> get the attention of pertinent people.
>
> [1]: https://issues.apache.org/jira/browse/SOLR-13898
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, Jul 13, 2021 at 4:31 PM Mike Drob  wrote:
>>
>> Hi folks,
>>
>> This is an idea based on a recent prod issue, and while we found
>> another workaround I think there is some merit to discuss here.
>>
>> Currently our filter cache is a mapping from queries to docs, and the
>> result cache is similar although slightly more abstract. When we have
>> a lot of similar queries come in at the same time, if a particular
>> filter hasn't been cached yet then it will be computed a bunch of
>> times in parallel as each query tries to be the one to insert into the
>> cache.
>>
>> One option that I've thought about is if instead of inserting results
>> into the cache directly, we pre-register a future in the cache, and
>> then use that as a reference to the results. Multiple queries coming
>> in parallel would all wait for the same result calculation instead of
>> allocating large arrays each.
>>
>> The benefits are pretty straightforward - we reduce the amount of
>> computation done when there are lots of queries coming in, and reduce
>> the memory allocation pressure.
>>
>> The complexity might be around handling errors or query timeouts or
>> cancellations. Or evictions, but I think that would all be manageable.
>>
>> What do other folks think? Should I write up a SIP for this, since I
>> think it will be fairly complex, or are there existing solutions that
>> I should look into first?
>>
>> Mike
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Solr Cache with Futures as values

[Mike Drob]

> It also has has a bulk load API

We currently use this when we need to resize a cache because those
will have the same results but maybe a smaller number. I don't think
we can do it when starting a new cache that needs warming, since the
bulk putAll takes both keys and values rather than keys and a
computational unit. If we move to futures as values then something
like this becomes more possible, I think.

Unrelated, I really struggle figuring out how to test this in a
reproducible fashion. We'd need a filter query that takes a long time
to execute, or even an injectable latch to stall all of the queries
that we can release from the test code. Will fiddle with this some
more.

On Wed, Jul 14, 2021 at 5:46 PM Mark Miller  wrote:
>
> If Caffeine is being used, it might be worthwhile to look into using it’s 
> feature set to do this.
>
> It has the ability to do either async or sync loading - if using sync, 
> modifications will block while an entry is loading.
>
> It also has has a bulk load API, might be interesting for things like auto 
> warming.
>
> - MRM
>
> On Wed, Jul 14, 2021 at 6:18 PM Mike Drob  wrote:
>>
>> Thanks for the pointer, David!
>>
>> While browsing through that issue, I found this comment left by you
>> from SOLR-14166
>>
>> https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L1185
>>
>> "note: can't use computeIfAbsent because can be recursive"
>>
>> I don't quite understand what the recursive case means here, can you
>> elaborate on that? I didn't see any discussion of it in the JIRA
>> either.
>>
>> Thanks,
>> Mike
>>
>> On Wed, Jul 14, 2021 at 11:03 AM David Smiley  wrote:
>> >
>> > Ideally, we could use SolrCache.computeIfAbsent [1] for the filter cache, 
>> > as is used for some of the other caches.  The best SolrCache is 
>> > CaffeineCache which works atomically for the same key (just as does 
>> > ConcurrentHashMap).  The problem is that this method on CaffeineCache does 
>> > not support computing a cache entry that is reentrant, i.e. that which can 
>> > produce another cache entry when it is computed.  Really, that limitation 
>> > ought to be elevated to the docs on SolrCache.computeIfAbsent.  Andrzej 
>> > discovered [1] that some queries could do that, and so he did not update 
>> > Solr's use of the filter cache to call it.  Please read the thread there 
>> > and maybe comment further to get the attention of pertinent people.
>> >
>> > [1]: https://issues.apache.org/jira/browse/SOLR-13898
>> >
>> > ~ David Smiley
>> > Apache Lucene/Solr Search Developer
>> > http://www.linkedin.com/in/davidwsmiley
>> >
>> >
>> > On Tue, Jul 13, 2021 at 4:31 PM Mike Drob  wrote:
>> >>
>> >> Hi folks,
>> >>
>> >> This is an idea based on a recent prod issue, and while we found
>> >> another workaround I think there is some merit to discuss here.
>> >>
>> >> Currently our filter cache is a mapping from queries to docs, and the
>> >> result cache is similar although slightly more abstract. When we have
>> >> a lot of similar queries come in at the same time, if a particular
>> >> filter hasn't been cached yet then it will be computed a bunch of
>> >> times in parallel as each query tries to be the one to insert into the
>> >> cache.
>> >>
>> >> One option that I've thought about is if instead of inserting results
>> >> into the cache directly, we pre-register a future in the cache, and
>> >> then use that as a reference to the results. Multiple queries coming
>> >> in parallel would all wait for the same result calculation instead of
>> >> allocating large arrays each.
>> >>
>> >> The benefits are pretty straightforward - we reduce the amount of
>> >> computation done when there are lots of queries coming in, and reduce
>> >> the memory allocation pressure.
>> >>
>> >> The complexity might be around handling errors or query timeouts or
>> >> cancellations. Or evictions, but I think that would all be manageable.
>> >>
>> >> What do other folks think? Should I write up a SIP for this, since I
>> >> think it will be fairly complex, or are there existing solutions that
>> >> I should look into first?
>> >>
>> >> Mike
>> >>
>> >> -
>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> >> For additional commands, e-mail: dev-h...@solr.apache.org
>> >>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>
> --
> - Mark
>
> http://about.me/markrmiller

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: SolrCloud Alone: Deprecate Standalone Mode

[Mike Drob]

Could we simulate user managed replication with an embedded zookeeper
on the primary and pull replicas on the followers?

On Mon, Aug 9, 2021 at 12:56 PM Jason Gerlowski  wrote:
>
> Hey Marcus,
>
> The places I've worked in the past have all used SolrCloud primarily
> so I can't speak to any specifics, but my impression from reading
> user-list traffic is that a sizable chunk of Solr's user base prefers
> "User-Managed" mode (formerly called "standalone").  Some because they
> don't want to manage a ZooKeeper cluster.  Some because the
> replication model in 'user-managed' fits their needs better.  Some I
> imagine just haven't bothered to update in many years.
>
> I'm absolutely sympathetic to efforts to streamline development and
> reduce collective debt, but it might be tough to displace such a big
> chunk of users.  I'm curious what others think though.  Maybe the
> proportion of 'user-managed' users out there is smaller than I
> imagine.
>
> Jason
>
> On Fri, Aug 6, 2021 at 11:59 PM Marcus Eagan  wrote:
> >
> > Hello again,
> >
> > Has the time come for us to reduce scope to move faster and with more 
> > focus? Even for those not in the cloud, SolrCloud has been the undisputed 
> > performance and usability champ since version 8.0. In version 9.0, I'd like 
> > to propose that the deciders in the community deprecate standalone mode in 
> > favor of SolrCloud.
> >
> > There are a few drivers:
> >
> > We only need to support changes that impact SolrCloud going forward. I know 
> > that this is hard to stomach. But by the time Solr reaches version 10.0, 
> > everyone should have migrated to SolrCloud as there is little reason to 
> > continue to run standalone.
> > The new features keep coming to SolrCloud, but not to standalone. You can 
> > see in a few ways how I embarrassingly discovered this late one night while 
> > trying out a PR. If not careful, users can accidentally start Solr in 
> > standalone mode. Think of all the features that they will see documented 
> > but not in their environment. What a confusing user experience?
> > Last but certainly not least, the number of contributors to the project, 
> > and the velocity of those contributions has dropped. . It does not have to 
> > be that way, though. Two ways are for the community to observe our push for 
> > modernization and improved user experience. Simply eliminating the need to 
> > include the -c flag in the start command would be a huge win for many 
> > engineers.We should make life easier for our users as much as the 
> > maintainers here. We can strive to make the upgrade process from 9 to 10 
> > very simple.
> >
> > I tried to make one step in this direction last year by re-ordering the 
> > README to show the Solr Cloud command before the standalone command. I 
> > believe that patch died on the vine, but I would be excited to revive it to 
> > document this effort when the time is appropriate.
> >
> > Reason not to do it:
> >
> >  Some large company out there might view this move as introducing risk. I 
> > view the risk here as negligible but I welcome any perspective there.
> > Some things I inevitably don't know.
> >
> > What do you all think?
> >
> > Thank you all for your voluntary contributions,
> > --
> > Marcus Eagan
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Solr builds hanging all the time on Jenkins

[Mike Drob]

When I build, I see the following error locally:

> Task :solr:benchmark:forbiddenApisMain
Forbidden method invocation:
java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection
usage to work around access flags fails with SecurityManagers and
likely will not work anymore on runtime classes in Java 9]
  in 
org.apache.solr.bench.index.jmh_generated.CloudIndexing_indexSmallDoc_jmhTest
(CloudIndexing_indexSmallDoc_jmhTest.java:779)

It doesn't fail anything for me, but I wonder if it is related.

On Wed, Aug 11, 2021 at 6:05 AM Uwe Schindler  wrote:
>
> Hi,
>
> Solr "main" builds are hanging all the time on Policeman Jenkins. It looks
> like some test is causing this, because it runs the test suite of Solr Core
> module.
> Does anybody know what could causing this. I am not sure if it is related to
> the new benchmark module, but the problems started around that time. It
> affects all platforms, not only MacOS lik before. Linux and Windows builds
> are hanging, too. I have to kill them manually.
>
> I will for now disable jenkins build on Solr.
>
> Uwe
>
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: SolrCloud Alone: Deprecate Standalone Mode

[Mike Drob]

f the some of the
>> tests could (also optionally) properly isolate themselves within an
>> externally running solr (probably started via cloud.sh with the latest
>> edits. ... develop, cloud.sh, test manually, run tests against same I
>> expect that there are still tests for which that makes no sense of course.
>> This is probably a crazier idea than using an external zookeeper however,
>> where zkChroot should be sufficient to isolate things I think...
>>
>>
>>
>> -Gus
>>
>>
>>
>> On Tue, Aug 10, 2021 at 2:22 PM David Smiley  wrote:
>>
>> Good call-out on perceived complexity due to running 3 ZK nodes.  For
>> many small installations, honestly Solr's embedded ZK is fine.  Also, again
>> for small installations, running ZK alongside Solr (same hardware) is
>> fine.  We shouldn't needlessly shame users away from doing these things as
>> if it's irresponsible.  There's a spectrum of demands on Solr from low to
>> high.  Anyway, I suspect it's increasingly moot with more Docker &
>> Kubernetes being used to reduce the hassles of deploying any service (be it
>> Solr or whatever).  This will only increase going forward.
>>
>>
>>
>> Even if ZK becomes the only mode, I expect many checks in our codebase
>> that conditionally check for ZK to remain.  We want tests that don't care
>> about SolrCloud mode to be fast, and that means not running unnecessary
>> things like ZooKeeper.
>>
>>
>> ~ David Smiley
>>
>> Apache Lucene/Solr Search Developer
>>
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>>
>>
>>
>> On Tue, Aug 10, 2021 at 12:23 PM Gus Heck  wrote:
>>
>> I've met several clients who really didn't want to manage zookeeper as an
>> additional service (I've talked some into it anyway, but it was clearly a
>> key reason they hadn't started/gone cloud). I think it would be far more
>> palatable if it's all "part of solr", doesn't require plumbing the docs of
>> some other project entirely, and requires neither requisitioning additional
>> hardware nor service scripts, monitoring, support that isn't "solr"
>> support... etc... then I think that alleviates some of the pain that folks
>> in small sub-sections of moderate to large orgs feel at the idea of using
>> cloud. These folks face long procurement cycles and disaster/recovery plans
>> etc, despite often having team sizes under 20... or face having to educate
>> large IT departments into handling deployments when they themselves are new
>> (of course that's how some of them wind up hiring folks like me... but
>> that's a barrier too since that has to be approved too).  Also I've met
>> folks who didn't understand that it was possible to have a 1 node "cluster"
>> with zk on the same machine, and had the impression that 5 boxes (2 solr
>> and 3 zk) were absolutely required to run cloud. Which it is of course for
>> high availability with no SPOF, but it is not required if you don't need
>> high availability.
>>
>>
>>
>> I think to sunset "user managed" we need to figure out how to self manage
>> embedded zookeepers, most particularly setup for smaller orgs or lower
>> traffic installs should look like:
>>
>>
>>
>> A) Start Node 1 with zk embedded ... if you only need one node, don't
>> want high availability etc, done.
>>
>> B) Start Node 2 telling it the zk url for node 1. node 2 comes up, offers
>> to participate in zk, but does not because that would make an even number
>>
>> C) Start Node 3 telling it the zk url for node 1. node 1 (node 2 hasn't
>> started zk) node 3 offers to participate in zk, and now with 2 offers
>> pending, both 2 and 3, get up to date on the current state and th join, now
>> the embedded zk cluster is 3 nodes, not one, and no SPOF... as they grow...
>>
>> D) Node 4 - like node 2 but can use zk url of any of 1,2,3
>>
>> E) Node 5 - like node 3, but can use zk url of any 1,2,3
>>
>>
>>
>> Obviously, features for users to set a cap the size of zk clusters, don't
>> need 49 nodes on 50 servers... , ensure they put their data in a convenient
>> place that is well documented, document how to secure the inter-node
>> connections, clarity in the admin UI of what nodes have zk etc.
>>
>>
>>
>> For this embedded zk use case we should document whatever the user needs
>> to know so they don't have to sort through docs at an entir

Re: Check broken links is failing?

[Mike Drob]

I am very against packages split across multiple jars/modules, so would
imagine that a package move is necessary

On Thu, Aug 26, 2021 at 8:04 AM Dawid Weiss  wrote:

> This fails for me locally:
> :solr:documentation:checkBrokenLinks
>
> and yet github actions passed (they do run the same command). I see
> the following offender:
>
>
> [root]/solr/main/solr/documentation/build/site/benchmark/org/apache/solr/bench/MiniClusterState.html
>   BROKEN LINK:
> [root]/solr/main/solr/documentation/build/site/core/org/apache/solr/cloud/MiniSolrCloudCluster.html
>
> which is indeed a link from the benchmark module... but not to the
> core - it's a link to the test framework's project.
>
> I think the same package split may be to blame here somehow - the
> build will pass or fail depending on the order of projects. I don't
> know how this can be reasonably fixed.
>
> D.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: 8.10 release soon?

[Mike Drob]

I'm pretty close to finishing up
https://issues.apache.org/jira/browse/SOLR-1 and would like to try
to get that into 8.10, hopefully this week or next week.

On Mon, Aug 30, 2021 at 12:51 PM Houston Putman  wrote:
>
> SOLR-15599 is almost backported (was merged to main today). I consider this a 
> blocker, but should be good-to-go by tomorrow.
>
> Any comments on the existing blockers?
>
> - SOLR-14438 - Should just be an added line in LICENSE.txt?
> - SOLR-14593 - This has been a blocker for many releases, so I'm not sure we 
> are actually held up here
> - SOLR-15412 - No conversations since the 8.9 release, so again not sure it's 
> actually a blocker.
> - SOLR-15596 - This is important but I don't think there's any work for it 
> yet.
>
> - Houston
>
> On Sat, Aug 28, 2021 at 12:18 PM Michael McCandless 
>  wrote:
>>
>> Yeah +1!  Lucene's first "non floating point" compliant release in a long 
>> time?
>>
>> Mike
>>
>> On Thu, Aug 26, 2021 at 4:09 AM Adrien Grand  wrote:
>>>
>>> +1 to a 8.10 release and cutting a branch next week
>>>
>>> On Tue, Aug 24, 2021 at 8:02 PM Timothy Potter  wrote:

 Hi folks,

 Looks like we have a number of nice enhancements and bug fixes in
 Lucene and Solr for 8.10.

 https://github.com/apache/lucene-solr/blob/branch_8x/lucene/CHANGES.txt
 https://github.com/apache/lucene-solr/blob/branch_8x/solr/CHANGES.txt

 However, there are a few open blockers marked for Solr 8.10, see:
 https://issues.apache.org/jira/browse/SOLR-15596?filter=12350839

 The blockers (SOLR-15596, SOLR-15412, SOLR-14593) are not assigned to
 anyone. Is anyone looking at these? If not, do they need to block the
 8.10 release?

 I propose we should cut the release branch next week but that
 obviously depends on our decision around these open blockers.

 Cheers,
 Timothy Potter

 PS ~ I volunteer to be the Release Manager ;-)

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

>>>
>>>
>>> --
>>> Adrien
>>
>> --
>> Mike McCandless
>>
>> http://blog.mikemccandless.com

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Welcome András Salamon as Solr committer

[Mike Drob]

Welcome and well deserved!

On Wed, Sep 1, 2021 at 5:01 PM Jan Høydahl  wrote:

> Hi all,
>
> Please join me in welcoming András Salamon as the latest Solr committer.
> András, it's tradition for you to introduce yourself with a brief bio.
>
> Congratulations and Welcome!
> Jan
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Mike Drob]

Hi Tim, I'm still working on SOLR-1, the code and benchmarking
both look pretty good, but I've got a few last unit tests that I need
to chase down. Hopefully taken care of by today or tomorrow, I'll be
sure to keep you updated though.


On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter  wrote:
>
> I found https://issues.apache.org/jira/browse/SOLR-15620 while testing
> the schema designer. I haven't built the RC yet, so going to see if I
> can get this in today.
>
> On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter  wrote:
> >
> > NOTICE:
> >
> > Branch branch_8_10 has been cut and versions updated to 8.11 on stable 
> > branch.
> >
> > Please observe the normal rules:
> >
> > * No new features may be committed to the branch.
> >
> > * Documentation patches, build patches and serious bug fixes may be
> >   committed to the branch. However, you should submit all patches you
> >   want to commit to Jira first to give others the chance to review
> >   and possibly vote against the patch. Keep in mind that it is our
> >   main intention to keep the branch as stable as possible.
> >
> > * All patches that are intended for the branch should first be committed
> >   to the unstable branch, merged into the stable branch, and then into
> >   the current release branch.
> >
> > * Normal unstable and stable branch development may continue as usual.
> >   However, if you plan to commit a big change to the unstable branch
> >   while the branch feature freeze is in effect, think twice: can't the
> >   addition wait a couple more days? Merges of bug fixes into the branch
> >   may become more difficult.
> >
> > * Only Jira issues with Fix version 8.10 and priority "Blocker" will delay
> >   a release candidate build.
> > 
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org