SolrCloud creates the data folder in solr.data.home instead of solr.core.name

[Razvan-Daniel Mihai]

It seems like my Email got lost the first time, so  I'll give it a second
try.

-- Forwarded message -
From: Razvan-Daniel Mihai 
Date: Tue, Jun 16, 2020 at 3:28 PM
Subject: SolrCloud creates the data folder in solr.data.home instead of
solr.core.name
To: 


Hello,

I run a kerberized SolrCloud (7.4.0) environment (shipped with Cloudera
6.3.2) and have the problem that all index files for all cores are created
under the same ${solr.solr.home} directory instead of ${solr.core.name} and
thus they are corrupt.

Cloudera uses the HDFSDirectoryFactory by default - but this is largely
unusable for large collections with billions of documents, so we switched
to storing the indexes locally (on /data/solr) a long time ago.

I should probably mention that we have another cluster (CDH6.3.1) which
doesn't have this problem.

Also it might be that there is a Sentry authorization problem (I'm still
investigating) but nevertheless it seems to me like Solr should never ever
use the same data folder for two different cores, so this might also be a
bug in Solr.

Any ideas of what is wrong here?

Thank you,
Razvan

PS:

Here is what it looks like on disk:

$ ls /data/solr/

index  niofs_test_shard1_replica_n1
prod_applogs_20200616_shard2_replica_n2  snapshot_metadata  tlog

[a6709018@cdcdhp10 ~]$ ls /data/solr/niofs_test_shard1_replica_n1/

core.properties

[a6709018@cdcdhp10 ~]$ ls /data/solr/niofs_test_shard1_replica_n1/

core.properties


You can see that there are two core directories (niofs_test
and prod_applogs) and then there are all data-related folders on the same
level. The core folders contain only one file (core.properties).


Here is how Solr is started on one machine. (I removed some security
related properties from the listing):


/usr/lib/jvm/java-openjdk/bin/java -server -XX:NewRatio=3
-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8
-XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4
-XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m
-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50
-XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled
-XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow
-Xlog:gc*:file=/var/log/solr/solr_gc.log:time,uptime:filecount=9,filesize=2
-DzkClientTimeout=15000 -DzkHost=cdcdhp02.bigdatap.de.comdirect.com:2181,
cdcdhp20.bigdatap.de.comdirect.com:2181,
cdcdhp22.bigdatap.de.comdirect.com:2181/solr -Dsolr.log.dir=/var/log/solr
-Djetty.port=8985 -DSTOP.PORT=7985 -DSTOP.KEY=csearch
-Duser.timezone=GMT+0200
-Djetty.home=/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/solr/server
-Dsolr.solr.home=/data/solr -Dsolr.data.home=
-Dsolr.install.dir=/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/solr
-Dsolr.default.confdir=/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/solr/server/solr/configsets/_default/conf
-Xss256k -DwaitForZk=60 -Dsolr.host=cdcdhp10.bigdatap.de.comdirect.com
-DuseCachedStatsBetweenGetMBeanInfoCalls=true
-DdisableSolrFieldCacheMBeanEntryListJmx=true
-Dlog4j.configuration=file:///var/run/cloudera-scm-agent/process/518-solr-SOLR_SERVER/log4j.properties
-Dsolr.log=/var/log/solr -Dsolr.admin.port=8984
-Dsolr.max.connector.thread=1 -Dsolr.solr.home=/data/solr
-Dsolr.authorization.sentry.site=/var/run/cloudera-scm-agent/process/518-solr-SOLR_SERVER/sentry-conf/sentry-site.xml
-Dsolr.sentry.override.plugins=true -jar start.jar --module=https
--lib=/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/solr/server/solr-webapp/webapp/WEB-INF/lib/*
--lib=/var/run/cloudera-scm-agent/process/518-solr-SOLR_SERVER/hadoop-conf

Re: Getting rid of Master/Slave nomenclature in Solr

[Atita Arora]

+1 Noble and Ilan !!



On Thu, Jun 18, 2020 at 7:51 AM Noble Paul  wrote:

> Looking at the code I see a 692 occurrences of the word "slave".
> Mostly variable names and ref guide docs.
>
> The word "slave" is present in the responses as well. Any change in
> the request param/response payload is backward incompatible.
>
> I have no objection to changing the names in ref guide and other
> internal variables. Going ahead with backward incompatible changes is
> painful. If somebody has the appetite to take it up, it's OK
>
> If we must change, master/follower can be a good enough option.
>
> master (noun): A man in charge of an organization or group.
> master(adj) : having or showing very great skill or proficiency.
> master(verb): acquire complete knowledge or skill in (a subject,
> technique, or art).
> master (verb): gain control of; overcome.
>
> I hope nobody has a problem with the term "master"
>
> On Thu, Jun 18, 2020 at 3:19 PM Ilan Ginzburg  wrote:
> >
> > Would master/follower work?
> >
> > Half the rename work while still getting rid of the slavery
> connotation...
> >
> >
> > On Thu 18 Jun 2020 at 07:13, Walter Underwood 
> wrote:
> >
> > > > On Jun 17, 2020, at 4:00 PM, Shawn Heisey 
> wrote:
> > > >
> > > > It has been interesting watching this discussion play out on multiple
> > > open source mailing lists.  On other projects, I have seen a VERY high
> > > level of resistance to these changes, which I find disturbing and
> > > surprising.
> > >
> > > Yes, it is nice to see everyone just pitch in and do it on this list.
> > >
> > > wunder
> > > Walter Underwood
> > > wun...@wunderwood.org
> > > http://observer.wunderwood.org/  (my blog)
> > >
> > >
>
>
>
> --
> -
> Noble Paul
>

Re: Autocommit in SolrCloud with many shards

[Bram Van Dam]

Thanks, just created SOLR-14581 as an entry point.

And sure, beers sound good! ;-)

On 17/06/2020 23:13, Erick Erickson wrote:
> Please raise a JIRA and attach your patch to that….
> 
> Best,
> Erick
> 
> P.S. Buy me some beers sometime if we’re even in the same place...
> 
>> On Jun 17, 2020, at 5:00 PM, Bram Van Dam  wrote:
>>
>> Thanks for pointing that out. I'm attaching a patch for the ref-guide
>> which summarizes what you said. Maybe other people will find this useful
>> as well?
>>
>> Oh and Erick, thanks for your ever thoughtful replies. Given all the
>> hours of your time I've soaked up over the years, you should probably
>> start invoicing me :-)
>>
>> - Bram
>>
>> On 17/06/2020 13:55, Erick Erickson wrote:
>>> Each node has its own timer that starts when it receives an update.
>>> So in your situation, 60 seconds after any give replica gets it’s first
>>> update, all documents that have been received in the interval will
>>> be committed.
>>>
>>> But note several things:
>>>
>>> 1> commits will tend to cluster for a given shard. By that I mean
>>>they’ll tend to happen within a few milliseconds of each other
>>>   ‘cause it doesn’t take that long for an update to get from the
>>>   leader to all the followers.
>>>
>>> 2> this is per replica. So if you host replicas from multiple collections
>>>   on some node, their commits have no relation to each other. And
>>>   say for some reason you transmit exactly one document that lands
>>>   on shard1. Further, say nodeA contains replicas for shard1 and shard2.
>>>   Only the replica for shard1 would commit.
>>>
>>> 3> Solr promises eventual consistency. In this case, due to all the
>>>   timing variables it is not guaranteed that every replica of a single
>>>   shard has the same document available for search at any given time.
>>>   Say doc1 hits the leader at time T and a follower at time T+10ms.
>>>   Say doc2 hits the leader and gets indexed 5ms before the 
>>>   commit is triggered, but for some reason it takes 15ms for it to get
>>>   to the follower. The leader will be able to search doc2, but the
>>>  follower won’t until 60 seconds later.
>>>
>>> Best,
>>> Erick
>>>
 On Jun 17, 2020, at 5:36 AM, Bram Van Dam  wrote:

 'morning :-)

 I'm wondering how autocommits work in Solr.

 Say I have a cluster with many nodes and many colections with many
 shards. If each collection's config has a hard autocommit configured
 every minute, does that mean that SolrCloud (presumably the leader?)
 will dish out commit requests to each node on that schedule? Or does
 each node have its own timed trigger?

 If it's the former, doesn't that mean the load will spike dramatically
 across the whole cluster every minute?

 I tried reading the code, but I don't quite understand the way
 CommitTracker and the UpdateHandlers interact with SolrCloud.

 Thanks,

 - Bram
>>>
>>
>> 
> 

Gettings interestingTerms from solr.MoreLikeThisHandler using SolrJ

[Zander, Sebastian]

Hello solr-fellows,

i'm currently implementing the MoreLikeThis Feature in an e-commerce platform.
I setup my solr.MoreLikeThisHandler in my solrconfig.xml like this:


  


aid, eans, desclong
list
true
10
aid, eans, desclong
0
  


Running the following command in browser 
http://localhost:8983/solr/ecom/mlt?q=aid:1
It returns with 10 docs in his response and a list of interestingTerms.

Running the same via solrj, I can't find the interestingTerms.
I setup my SolrQuery like this:

SolrQuery sq = new SolrQuery();
sq.setRequestHandler("/mlt");
sq.setQuery("aid:" + p_aid);

In the returning QueryResponse I can't find the interestingTerms.
I would really like to grab it on this way, before calling another time.
Any advices? I'm running solr 8.5.2

Thanks in advance,
Sebastian Zander

RE: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr

[Demian Katz]

Regarding people having a problem with the word "master" -- GitHub is changing 
the default branch name away from "master," even in isolation from a "slave" 
pairing... so the terminology seems to be falling out of favor in all contexts. 
See:

https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/

I'm not here to start a debate about the semantics of that, just to provide 
evidence that in some communities, the term "master" is causing concern all by 
itself. If we're going to make the change anyway, it might be best to get it 
over with and pick the most appropriate terminology we can agree upon, rather 
than trying to minimize the amount of change. It's going to be backward 
breaking anyway, so we might as well do it all now rather than risk having to 
go through two separate breaking changes at different points in time.

- Demian

-Original Message-
From: Noble Paul  
Sent: Thursday, June 18, 2020 1:51 AM
To: solr-user@lucene.apache.org
Subject: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr

Looking at the code I see a 692 occurrences of the word "slave".
Mostly variable names and ref guide docs.

The word "slave" is present in the responses as well. Any change in the request 
param/response payload is backward incompatible.

I have no objection to changing the names in ref guide and other internal 
variables. Going ahead with backward incompatible changes is painful. If 
somebody has the appetite to take it up, it's OK

If we must change, master/follower can be a good enough option.

master (noun): A man in charge of an organization or group.
master(adj) : having or showing very great skill or proficiency.
master(verb): acquire complete knowledge or skill in (a subject, technique, or 
art).
master (verb): gain control of; overcome.

I hope nobody has a problem with the term "master"

On Thu, Jun 18, 2020 at 3:19 PM Ilan Ginzburg  wrote:
>
> Would master/follower work?
>
> Half the rename work while still getting rid of the slavery connotation...
>
>
> On Thu 18 Jun 2020 at 07:13, Walter Underwood  wrote:
>
> > > On Jun 17, 2020, at 4:00 PM, Shawn Heisey  wrote:
> > >
> > > It has been interesting watching this discussion play out on 
> > > multiple
> > open source mailing lists.  On other projects, I have seen a VERY 
> > high level of resistance to these changes, which I find disturbing 
> > and surprising.
> >
> > Yes, it is nice to see everyone just pitch in and do it on this list.
> >
> > wunder
> > Walter Underwood
> > wun...@wunderwood.org
> > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobs
> > erver.wunderwood.org%2F&data=02%7C01%7Cdemian.katz%40villanova.e
> > du%7C1eef0604700a442deb7e08d8134b97fb%7C765a8de5cf9444f09cafae5bf8cf
> > a366%7C0%7C0%7C637280562684672329&sdata=0GyK5Tlq0PGsWxl%2FirJOVN
> > VaFCELlEChdxuLJ5RxdQs%3D&reserved=0  (my blog)
> >
> >



--
-
Noble Paul

Re: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr

[Jason Gerlowski]

+1 to rename master/slave, and +1 to choosing terminology distinct
from what's used for SolrCloud.  I could be happy with several of the
proposed options.  Since a good few have been proposed though, maybe
an eventual vote thread is the most organized way to aggregate the
opinions here.

I'm less positive about the prospect of changing the name of our
primary git branch.  Most projects that contributors might come from,
most tutorials out there to learn git, most tools built on top of git
- the majority are going to assume "master" as the main branch.  I
appreciate the change that Github is trying to effect in changing the
default for new projects, but it'll be a long time before that
competes with the huge bulk of projects, documentation, etc. out there
using "master".  Our contributors are smart and I'm sure they'd figure
it out if we used "main" or something else instead, but having a
non-standard git setup would be one more "papercut" in understanding
how to contribute to a project that already makes that harder than it
should.

Jason


On Thu, Jun 18, 2020 at 7:33 AM Demian Katz  wrote:
>
> Regarding people having a problem with the word "master" -- GitHub is 
> changing the default branch name away from "master," even in isolation from a 
> "slave" pairing... so the terminology seems to be falling out of favor in all 
> contexts. See:
>
> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
>
> I'm not here to start a debate about the semantics of that, just to provide 
> evidence that in some communities, the term "master" is causing concern all 
> by itself. If we're going to make the change anyway, it might be best to get 
> it over with and pick the most appropriate terminology we can agree upon, 
> rather than trying to minimize the amount of change. It's going to be 
> backward breaking anyway, so we might as well do it all now rather than risk 
> having to go through two separate breaking changes at different points in 
> time.
>
> - Demian
>
> -Original Message-
> From: Noble Paul 
> Sent: Thursday, June 18, 2020 1:51 AM
> To: solr-user@lucene.apache.org
> Subject: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr
>
> Looking at the code I see a 692 occurrences of the word "slave".
> Mostly variable names and ref guide docs.
>
> The word "slave" is present in the responses as well. Any change in the 
> request param/response payload is backward incompatible.
>
> I have no objection to changing the names in ref guide and other internal 
> variables. Going ahead with backward incompatible changes is painful. If 
> somebody has the appetite to take it up, it's OK
>
> If we must change, master/follower can be a good enough option.
>
> master (noun): A man in charge of an organization or group.
> master(adj) : having or showing very great skill or proficiency.
> master(verb): acquire complete knowledge or skill in (a subject, technique, 
> or art).
> master (verb): gain control of; overcome.
>
> I hope nobody has a problem with the term "master"
>
> On Thu, Jun 18, 2020 at 3:19 PM Ilan Ginzburg  wrote:
> >
> > Would master/follower work?
> >
> > Half the rename work while still getting rid of the slavery connotation...
> >
> >
> > On Thu 18 Jun 2020 at 07:13, Walter Underwood  wrote:
> >
> > > > On Jun 17, 2020, at 4:00 PM, Shawn Heisey  wrote:
> > > >
> > > > It has been interesting watching this discussion play out on
> > > > multiple
> > > open source mailing lists.  On other projects, I have seen a VERY
> > > high level of resistance to these changes, which I find disturbing
> > > and surprising.
> > >
> > > Yes, it is nice to see everyone just pitch in and do it on this list.
> > >
> > > wunder
> > > Walter Underwood
> > > wun...@wunderwood.org
> > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobs
> > > erver.wunderwood.org%2F&data=02%7C01%7Cdemian.katz%40villanova.e
> > > du%7C1eef0604700a442deb7e08d8134b97fb%7C765a8de5cf9444f09cafae5bf8cf
> > > a366%7C0%7C0%7C637280562684672329&sdata=0GyK5Tlq0PGsWxl%2FirJOVN
> > > VaFCELlEChdxuLJ5RxdQs%3D&reserved=0  (my blog)
> > >
> > >
>
>
>
> --
> -
> Noble Paul

Re: Getting rid of Master/Slave nomenclature in Solr

[Mark H. Wood]

Primary / satellite?

-- 
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu


signature.asc
Description: PGP signature

Re: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr

[John Gallagher]

While on the topic of renaming roles, I'd like to propose finding a better
term than "overseer" which has historical slavery connotations as well.
Director, perhaps?


John Gallagher

On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski 
wrote:

> +1 to rename master/slave, and +1 to choosing terminology distinct
> from what's used for SolrCloud.  I could be happy with several of the
> proposed options.  Since a good few have been proposed though, maybe
> an eventual vote thread is the most organized way to aggregate the
> opinions here.
>
> I'm less positive about the prospect of changing the name of our
> primary git branch.  Most projects that contributors might come from,
> most tutorials out there to learn git, most tools built on top of git
> - the majority are going to assume "master" as the main branch.  I
> appreciate the change that Github is trying to effect in changing the
> default for new projects, but it'll be a long time before that
> competes with the huge bulk of projects, documentation, etc. out there
> using "master".  Our contributors are smart and I'm sure they'd figure
> it out if we used "main" or something else instead, but having a
> non-standard git setup would be one more "papercut" in understanding
> how to contribute to a project that already makes that harder than it
> should.
>
> Jason
>
>
> On Thu, Jun 18, 2020 at 7:33 AM Demian Katz 
> wrote:
> >
> > Regarding people having a problem with the word "master" -- GitHub is
> changing the default branch name away from "master," even in isolation from
> a "slave" pairing... so the terminology seems to be falling out of favor in
> all contexts. See:
> >
> >
> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
> >
> > I'm not here to start a debate about the semantics of that, just to
> provide evidence that in some communities, the term "master" is causing
> concern all by itself. If we're going to make the change anyway, it might
> be best to get it over with and pick the most appropriate terminology we
> can agree upon, rather than trying to minimize the amount of change. It's
> going to be backward breaking anyway, so we might as well do it all now
> rather than risk having to go through two separate breaking changes at
> different points in time.
> >
> > - Demian
> >
> > -Original Message-
> > From: Noble Paul 
> > Sent: Thursday, June 18, 2020 1:51 AM
> > To: solr-user@lucene.apache.org
> > Subject: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr
> >
> > Looking at the code I see a 692 occurrences of the word "slave".
> > Mostly variable names and ref guide docs.
> >
> > The word "slave" is present in the responses as well. Any change in the
> request param/response payload is backward incompatible.
> >
> > I have no objection to changing the names in ref guide and other
> internal variables. Going ahead with backward incompatible changes is
> painful. If somebody has the appetite to take it up, it's OK
> >
> > If we must change, master/follower can be a good enough option.
> >
> > master (noun): A man in charge of an organization or group.
> > master(adj) : having or showing very great skill or proficiency.
> > master(verb): acquire complete knowledge or skill in (a subject,
> technique, or art).
> > master (verb): gain control of; overcome.
> >
> > I hope nobody has a problem with the term "master"
> >
> > On Thu, Jun 18, 2020 at 3:19 PM Ilan Ginzburg 
> wrote:
> > >
> > > Would master/follower work?
> > >
> > > Half the rename work while still getting rid of the slavery
> connotation...
> > >
> > >
> > > On Thu 18 Jun 2020 at 07:13, Walter Underwood 
> wrote:
> > >
> > > > > On Jun 17, 2020, at 4:00 PM, Shawn Heisey 
> wrote:
> > > > >
> > > > > It has been interesting watching this discussion play out on
> > > > > multiple
> > > > open source mailing lists.  On other projects, I have seen a VERY
> > > > high level of resistance to these changes, which I find disturbing
> > > > and surprising.
> > > >
> > > > Yes, it is nice to see everyone just pitch in and do it on this list.
> > > >
> > > > wunder
> > > > Walter Underwood
> > > > wun...@wunderwood.org
> > > > https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobs
> > > > erver.wunderwood.org%2F&data=02%7C01%7Cdemian.katz%40villanova.e
> > > > du%7C1eef0604700a442deb7e08d8134b97fb%7C765a8de5cf9444f09cafae5bf8cf
> > > > a366%7C0%7C0%7C637280562684672329&sdata=0GyK5Tlq0PGsWxl%2FirJOVN
> > > > VaFCELlEChdxuLJ5RxdQs%3D&reserved=0  (my blog)
> > > >
> > > >
> >
> >
> >
> > --
> > -
> > Noble Paul
>

Re: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in Solr

[Mike Drob]

I personally think that using Solr cloud terminology for this would be fine
with leader/follower. The leader is the one that accepts updates, followers
cascade the updates somehow. The presence of ZK or election doesn’t really
change this detail.

However, if folks feel that it’s confusing, then I can’t tell them that
they’re not confused. Especially when they’re working with others who have
less Solr experience than we do and are less familiar with the intricacies.

Primary/Replica seems acceptable. Coordinator instead of Overseer seems
acceptable.

Would love to see this in 9.0!

Mike

On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
 wrote:

> While on the topic of renaming roles, I'd like to propose finding a better
> term than "overseer" which has historical slavery connotations as well.
> Director, perhaps?
>
>
> John Gallagher
>
> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski 
> wrote:
>
> > +1 to rename master/slave, and +1 to choosing terminology distinct
> > from what's used for SolrCloud.  I could be happy with several of the
> > proposed options.  Since a good few have been proposed though, maybe
> > an eventual vote thread is the most organized way to aggregate the
> > opinions here.
> >
> > I'm less positive about the prospect of changing the name of our
> > primary git branch.  Most projects that contributors might come from,
> > most tutorials out there to learn git, most tools built on top of git
> > - the majority are going to assume "master" as the main branch.  I
> > appreciate the change that Github is trying to effect in changing the
> > default for new projects, but it'll be a long time before that
> > competes with the huge bulk of projects, documentation, etc. out there
> > using "master".  Our contributors are smart and I'm sure they'd figure
> > it out if we used "main" or something else instead, but having a
> > non-standard git setup would be one more "papercut" in understanding
> > how to contribute to a project that already makes that harder than it
> > should.
> >
> > Jason
> >
> >
> > On Thu, Jun 18, 2020 at 7:33 AM Demian Katz 
> > wrote:
> > >
> > > Regarding people having a problem with the word "master" -- GitHub is
> > changing the default branch name away from "master," even in isolation
> from
> > a "slave" pairing... so the terminology seems to be falling out of favor
> in
> > all contexts. See:
> > >
> > >
> >
> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
> > >
> > > I'm not here to start a debate about the semantics of that, just to
> > provide evidence that in some communities, the term "master" is causing
> > concern all by itself. If we're going to make the change anyway, it might
> > be best to get it over with and pick the most appropriate terminology we
> > can agree upon, rather than trying to minimize the amount of change. It's
> > going to be backward breaking anyway, so we might as well do it all now
> > rather than risk having to go through two separate breaking changes at
> > different points in time.
> > >
> > > - Demian
> > >
> > > -Original Message-
> > > From: Noble Paul 
> > > Sent: Thursday, June 18, 2020 1:51 AM
> > > To: solr-user@lucene.apache.org
> > > Subject: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in
> Solr
> > >
> > > Looking at the code I see a 692 occurrences of the word "slave".
> > > Mostly variable names and ref guide docs.
> > >
> > > The word "slave" is present in the responses as well. Any change in the
> > request param/response payload is backward incompatible.
> > >
> > > I have no objection to changing the names in ref guide and other
> > internal variables. Going ahead with backward incompatible changes is
> > painful. If somebody has the appetite to take it up, it's OK
> > >
> > > If we must change, master/follower can be a good enough option.
> > >
> > > master (noun): A man in charge of an organization or group.
> > > master(adj) : having or showing very great skill or proficiency.
> > > master(verb): acquire complete knowledge or skill in (a subject,
> > technique, or art).
> > > master (verb): gain control of; overcome.
> > >
> > > I hope nobody has a problem with the term "master"
> > >
> > > On Thu, Jun 18, 2020 at 3:19 PM Ilan Ginzburg 
> > wrote:
> > > >
> > > > Would master/follower work?
> > > >
> > > > Half the rename work while still getting rid of the slavery
> > connotation...
> > > >
> > > >
> > > > On Thu 18 Jun 2020 at 07:13, Walter Underwood  >
> > wrote:
> > > >
> > > > > > On Jun 17, 2020, at 4:00 PM, Shawn Heisey 
> > wrote:
> > > > > >
> > > > > > It has been interesting watching this discussion play out on
> > > > > > multiple
> > > > > open source mailing lists.  On other projects, I have seen a VERY
> > > > > high level of resistance to these changes, which I find disturbing
> > > > > and surprising.
> > > > >
> > > > > Yes, it is nice to see everyone just pitch in and do it on this
> list.
> > > > >
> > > > > wunder
> > > > > Wa

Re: Getting rid of Master/Slave nomenclature in Solr

[Walter Underwood]

Actually, the term “master” is a problem, so master/follower doesn’t work.

GitLab is renaming the master branch to main.

Rice University renamed College Masters to College Magisters in 2017.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jun 18, 2020, at 1:03 AM, Atita Arora  wrote:
> 
> +1 Noble and Ilan !!
> 
> 
> 
> On Thu, Jun 18, 2020 at 7:51 AM Noble Paul  wrote:
> 
>> Looking at the code I see a 692 occurrences of the word "slave".
>> Mostly variable names and ref guide docs.
>> 
>> The word "slave" is present in the responses as well. Any change in
>> the request param/response payload is backward incompatible.
>> 
>> I have no objection to changing the names in ref guide and other
>> internal variables. Going ahead with backward incompatible changes is
>> painful. If somebody has the appetite to take it up, it's OK
>> 
>> If we must change, master/follower can be a good enough option.
>> 
>> master (noun): A man in charge of an organization or group.
>> master(adj) : having or showing very great skill or proficiency.
>> master(verb): acquire complete knowledge or skill in (a subject,
>> technique, or art).
>> master (verb): gain control of; overcome.
>> 
>> I hope nobody has a problem with the term "master"
>> 
>> On Thu, Jun 18, 2020 at 3:19 PM Ilan Ginzburg  wrote:
>>> 
>>> Would master/follower work?
>>> 
>>> Half the rename work while still getting rid of the slavery
>> connotation...
>>> 
>>> 
>>> On Thu 18 Jun 2020 at 07:13, Walter Underwood 
>> wrote:
>>> 
> On Jun 17, 2020, at 4:00 PM, Shawn Heisey 
>> wrote:
> 
> It has been interesting watching this discussion play out on multiple
 open source mailing lists.  On other projects, I have seen a VERY high
 level of resistance to these changes, which I find disturbing and
 surprising.
 
 Yes, it is nice to see everyone just pitch in and do it on this list.
 
 wunder
 Walter Underwood
 wun...@wunderwood.org
 http://observer.wunderwood.org/  (my blog)
 
 
>> 
>> 
>> 
>> --
>> -
>> Noble Paul
>> 

Solr 6.6.6 - Shards Whitelist Error

[Ray W]

Hi Everyone,

We upgraded to solr 6.6.6 and a recurring error shows up as "shards
parameter value contained values not in the shards whitelist". I have
attached the error log as well as configurations in solr.xml and
solrconfig.xml. Solr 6.6 documentations doesn't cover setting up a
shards whitelist and it was through release notes
 that we found (CVE-2017-3164 -
configure a host whitelist) was patched in. We've been trying different
combinations in the configuration files and the latest has been this
stackoverflow thread

which
we know may not be correct. If anyone can point out what we're missing that
would be appreciated. Thank you for reading, appreciate the help in
advanced, we're stuck.

-Ray

Error:
org.apache.solr.common.SolrException: The 'shards' parameter value
'solrcloud-dev.vmdata:8983/solr/statistics' contained value(s) not on the
shards whitelist. shardUrl:solrcloud-dev.vmdata:8983/solr/statistics. set
-Dsolr.disable.shardsWhitelist=true to disable shards whitelist checks
at
org.apache.solr.handler.component.HttpShardHandlerFactory$WhitelistHostChecker.lambda$checkWhitelist$1(HttpShardHandlerFactory.java:565)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.solr.handler.component.HttpShardHandlerFactory$WhitelistHostChecker.checkWhitelist(HttpShardHandlerFactory.java:548)
at
org.apache.solr.handler.component.HttpShardHandler.prepDistributed(HttpShardHandler.java:393)
at
org.apache.solr.handler.component.SearchHandler.getAndPrepShardHandler(SearchHandler.java:227)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:265)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:724)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:530)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.j

Re: Getting rid of Master/Slave nomenclature in Solr

[Chris Hostetter]

First off: Forgive me if my comments/questions are redundent or uninformed 
bsaed o nthe larger discussion taking place.  I have not 
caught up on the whole thread before replying -- but that's solely based 
on a lack of time on my part, not a lack of willingness to embrace this 
change.


>From skimming a handful of messages in this thread, there's one aspect of 
the discussion -- particularly in the context of "should we or should we 
not re-use / overlap with 'SolrCloud' terminology" -- that seems (from 
limited review) to be getting overlooked...


Even today, before you start to consider if/what to replace the M & S 
terms with, we've already reach the point where the 
replication/IndexFetcher code can use those tems in confusing/missleading 
ways -- particularly in the context of SolrCloud, recovery, backups, 
etc...

I think a meaningful conversation about retiring the existing terminology 
should probably take into account at least 3 distinct questions:

#1 - what terminology should be used/expected/documented in non-SolrCloud 
usage of explicitly configuring "classic" ReplicationHandler in 
solrconfig.xml ?

#2 - what terminology should be used in the source code of
ReplicationHandler / IndexFetcher related code?

#3 - what terminology should be used in the *log* messages of 
ReplicationHandler / IndexFetcher related code? and (# 3a) should that 
terminology vary based on the type of cluster and when/why/how the 
replication code is being used?


In reverse order: My personal answers would be:

#3a - yes

#3 - I think we can & should tweak the replication code to understand 
the context it's being used in and adjust it's log/error messages accordingly

#2 - w/o digging into the code in depth here, i supect something like 
"remoteSource + localDest" would probably work well in the context of 
"index fetching"  -- or even just simply "fetchSource", w/o any need for a 
'slave' term equivilent because the context is usually just "local".  ... 
i don't think there any contexts where it really matters but 
"replicationSource + replicationDest" code be more generic terms for 
situations where the code isn't specifically a "I am a core that's doing 
fetching" context (alternatively: "remoteSource + localDest" could have 
parity with "localSource + remoteDest" if there are any situations i'm not 
thinking of where we "push" an index)

#1 - this is the least interesting question to me personally (given that 
we are moving away from it in general in favor of solr cloud), but i think 
in the context of how the terms are used in solrconfig.xml even the 
original M & S terms have never really made much sense if you look at 
where/how they are used in the configuration -- particularly in the 
context of the "repeater" use case.  I would suggest as a straw man 
replacing the 'name="master"' config block with a 
'name="provideSnapshots"' block using hte same options; and replace the 
'name="slave"' config block with 'name="fetchSnapshots"' config block, 
using mostly the same options, except replacing 'masterUrl' with 
'sourceUrl'.



-Hoss
http://www.lucidworks.com/

Does 8.5.2 depend on 8.2.0

[Phill Campbell]

I use gradle to build my project. I noticed that in the build the jar it is 
using is 8.2.0. I don’t include that anywhere.
I clear my grade cache and rebuild and I see this:

$find . -name "solr*"
./modules-2/metadata-2.71/descriptors/org.apache.solr/solr-parent
./modules-2/metadata-2.71/descriptors/org.apache.solr/solr-solrj
./modules-2/files-2.1/org.apache.solr/solr-parent
./modules-2/files-2.1/org.apache.solr/solr-parent/8.2.0/347744884a53d0be51aa03dcff13407dc1bbabc3/solr-parent-8.2.0.pom
./modules-2/files-2.1/org.apache.solr/solr-parent/8.5.2/28b292f323144e3c059af8f9dbb5527ef2fb858f/solr-parent-8.5.2.pom
./modules-2/files-2.1/org.apache.solr/solr-solrj
./modules-2/files-2.1/org.apache.solr/solr-solrj/8.2.0/dfe052f181213504c6546f25c0185078886148e8/solr-solrj-8.2.0.pom
./modules-2/files-2.1/org.apache.solr/solr-solrj/8.2.0/5c466f157adf03428765c6e15a3d85a08f540a05/solr-solrj-8.2.0.jar
./modules-2/files-2.1/org.apache.solr/solr-solrj/8.2.0/c247802c64bcb485b637673182061c181393d54e/solr-solrj-8.2.0-sources.jar
./modules-2/files-2.1/org.apache.solr/solr-solrj/8.5.2/35ab7fc3b6bd51440c62edd9ba1a780eec005759/solr-solrj-8.5.2-sources.jar
./modules-2/files-2.1/org.apache.solr/solr-solrj/8.5.2/2cf6f0fccbecf820c62e4e1474073f4216c70356/solr-solrj-8.5.2.jar
./modules-2/files-2.1/org.apache.solr/solr-solrj/8.5.2/43a8ff25b47ca7a230185000a46bf0897aad2e93/solr-solrj-8.5.2.pom


My build.gradle has this:
compile(group: 'org.apache.solr', name: 'solr-solrj', version:'8.5.2')
No where is there a reference to 8.2.0

Any ideas on how this could happen?

I am using the latest Intellij version and Gradle 5.3

Re: Does 8.5.2 depend on 8.2.0

[Chris Hostetter]

: Subject: Does 8.5.2 depend on 8.2.0

No.  The code certainly doesn't, but i suppose it's possible some metadata 
somewhere in some pom file may be broken? 


: My build.gradle has this:
: compile(group: 'org.apache.solr', name: 'solr-solrj', version:'8.5.2')
: No where is there a reference to 8.2.0

it sounds like you are using transitive dependencies (otherwise it 
wouldn't make sense for you to wonder if 8.5.2 depends on 8.2.0) ... is it 
posisble some *other* library you are depending on is depending on 8.2.0 
directly? what does your dependency tree look like?

https://docs.gradle.org/current/userguide/viewing_debugging_dependencies.html


-Hoss
http://www.lucidworks.com/

Re: Does 8.5.2 depend on 8.2.0

[Phill Campbell]

compile(group: 'org.springframework.boot',name: 
'spring-boot-starter-web',version: '2.2.6.RELEASE')
compile(group: 'javax.inject', name: 'javax.inject', version:'1')
compile(group: 'javax.ws.rs', name: 'javax.ws.rs-api', version: '2.1.1')
compile group: 'ch.qos.logback', name: 'logback-core', version: '1.2.3'
compile group: 'com.newrelic.agent.java', name: 'newrelic-api', version: 
'5.11.0'
compile group: 'org.springdoc', name: 'springdoc-openapi-ui', version: '1.3.1'
compile group: 'org.springdoc', name: 'springdoc-openapi-webmvc-core', version: 
‘1.3.1'


I am starting to suspect spring framework.



> On Jun 18, 2020, at 12:09 PM, Chris Hostetter  
> wrote:
> 
> 
> : Subject: Does 8.5.2 depend on 8.2.0
> 
> No.  The code certainly doesn't, but i suppose it's possible some metadata 
> somewhere in some pom file may be broken? 
> 
> 
> : My build.gradle has this:
> : compile(group: 'org.apache.solr', name: 'solr-solrj', version:'8.5.2')
> : No where is there a reference to 8.2.0
> 
> it sounds like you are using transitive dependencies (otherwise it 
> wouldn't make sense for you to wonder if 8.5.2 depends on 8.2.0) ... is it 
> posisble some *other* library you are depending on is depending on 8.2.0 
> directly? what does your dependency tree look like?
> 
> https://docs.gradle.org/current/userguide/viewing_debugging_dependencies.html
> 
> 
> -Hoss
> http://www.lucidworks.com/

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Jan Høydahl]

I support Mike Drob and Trey Grainger. We shuold re-use the leader/replica
terminology from Cloud. Even if you hand-configure a master/slave cluster
and orchestrate what doc goes to which node/shard, and hand-code your shards
parameter, you will still have a cluster where you’d send updates to the leader 
of 
each shard and the replicas would replicate the index from the leader.

Let’s instead find a new good name for the cluster type. Standalone kind of 
works
for me, but I see it can be confused with single-node. We have also discussed
replacing SolrCloud (which is a terrible name) with something more descriptive.

Today: SolrCloud vs Master/slave
Alt A: SolrCloud vs Standalone
Alt B: SolrCloud vs Legacy
Alt C: Clustered vs Independent
Alt D: Clustered vs Manual mode

Jan

> 18. jun. 2020 kl. 15:53 skrev Mike Drob :
> 
> I personally think that using Solr cloud terminology for this would be fine
> with leader/follower. The leader is the one that accepts updates, followers
> cascade the updates somehow. The presence of ZK or election doesn’t really
> change this detail.
> 
> However, if folks feel that it’s confusing, then I can’t tell them that
> they’re not confused. Especially when they’re working with others who have
> less Solr experience than we do and are less familiar with the intricacies.
> 
> Primary/Replica seems acceptable. Coordinator instead of Overseer seems
> acceptable.
> 
> Would love to see this in 9.0!
> 
> Mike
> 
> On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
>  wrote:
> 
>> While on the topic of renaming roles, I'd like to propose finding a better
>> term than "overseer" which has historical slavery connotations as well.
>> Director, perhaps?
>> 
>> 
>> John Gallagher
>> 
>> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski 
>> wrote:
>> 
>>> +1 to rename master/slave, and +1 to choosing terminology distinct
>>> from what's used for SolrCloud.  I could be happy with several of the
>>> proposed options.  Since a good few have been proposed though, maybe
>>> an eventual vote thread is the most organized way to aggregate the
>>> opinions here.
>>> 
>>> I'm less positive about the prospect of changing the name of our
>>> primary git branch.  Most projects that contributors might come from,
>>> most tutorials out there to learn git, most tools built on top of git
>>> - the majority are going to assume "master" as the main branch.  I
>>> appreciate the change that Github is trying to effect in changing the
>>> default for new projects, but it'll be a long time before that
>>> competes with the huge bulk of projects, documentation, etc. out there
>>> using "master".  Our contributors are smart and I'm sure they'd figure
>>> it out if we used "main" or something else instead, but having a
>>> non-standard git setup would be one more "papercut" in understanding
>>> how to contribute to a project that already makes that harder than it
>>> should.
>>> 
>>> Jason
>>> 
>>> 
>>> On Thu, Jun 18, 2020 at 7:33 AM Demian Katz 
>>> wrote:
 
 Regarding people having a problem with the word "master" -- GitHub is
>>> changing the default branch name away from "master," even in isolation
>> from
>>> a "slave" pairing... so the terminology seems to be falling out of favor
>> in
>>> all contexts. See:
 
 
>>> 
>> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
 
 I'm not here to start a debate about the semantics of that, just to
>>> provide evidence that in some communities, the term "master" is causing
>>> concern all by itself. If we're going to make the change anyway, it might
>>> be best to get it over with and pick the most appropriate terminology we
>>> can agree upon, rather than trying to minimize the amount of change. It's
>>> going to be backward breaking anyway, so we might as well do it all now
>>> rather than risk having to go through two separate breaking changes at
>>> different points in time.
 
 - Demian
 
 -Original Message-
 From: Noble Paul 
 Sent: Thursday, June 18, 2020 1:51 AM
 To: solr-user@lucene.apache.org
 Subject: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in
>> Solr
 
 Looking at the code I see a 692 occurrences of the word "slave".
 Mostly variable names and ref guide docs.
 
 The word "slave" is present in the responses as well. Any change in the
>>> request param/response payload is backward incompatible.
 
 I have no objection to changing the names in ref guide and other
>>> internal variables. Going ahead with backward incompatible changes is
>>> painful. If somebody has the appetite to take it up, it's OK
 
 If we must change, master/follower can be a good enough option.
 
 master (noun): A man in charge of an organization or group.
 master(adj) : having or showing very great skill or proficiency.
 master(verb): acquire complete knowledge or skill in (a subject,
>>> technique, or art).
 master (ver

Re: Solr 6.6.6 - Shards Whitelist Error

[Jan Høydahl]

Have you tried starting each node with -Dsolr.disable.shardsWhitelist=true to 
revert back to old behavior?

Jan

> 18. jun. 2020 kl. 17:23 skrev Ray W :
> 
> Hi Everyone,
> 
> We upgraded to solr 6.6.6 and a recurring error shows up as "shards
> parameter value contained values not in the shards whitelist". I have
> attached the error log as well as configurations in solr.xml and
> solrconfig.xml. Solr 6.6 documentations doesn't cover setting up a
> shards whitelist and it was through release notes
>  that we found (CVE-2017-3164 -
> configure a host whitelist) was patched in. We've been trying different
> combinations in the configuration files and the latest has been this
> stackoverflow thread
> 
> which
> we know may not be correct. If anyone can point out what we're missing that
> would be appreciated. Thank you for reading, appreciate the help in
> advanced, we're stuck.
> 
> -Ray
> 
> Error:
> org.apache.solr.common.SolrException: The 'shards' parameter value
> 'solrcloud-dev.vmdata:8983/solr/statistics' contained value(s) not on the
> shards whitelist. shardUrl:solrcloud-dev.vmdata:8983/solr/statistics. set
> -Dsolr.disable.shardsWhitelist=true to disable shards whitelist checks
> at
> org.apache.solr.handler.component.HttpShardHandlerFactory$WhitelistHostChecker.lambda$checkWhitelist$1(HttpShardHandlerFactory.java:565)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.solr.handler.component.HttpShardHandlerFactory$WhitelistHostChecker.checkWhitelist(HttpShardHandlerFactory.java:548)
> at
> org.apache.solr.handler.component.HttpShardHandler.prepDistributed(HttpShardHandler.java:393)
> at
> org.apache.solr.handler.component.SearchHandler.getAndPrepShardHandler(SearchHandler.java:227)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:265)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:724)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:530)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:534)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceCo

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Phill Campbell]

Master - Worker
Master - Peon
Master - Helper
Master - Servant

The term that is not wanted is “slave’. The term “master” is not a problem IMO.

> On Jun 18, 2020, at 3:59 PM, Jan Høydahl  wrote:
> 
> I support Mike Drob and Trey Grainger. We shuold re-use the leader/replica
> terminology from Cloud. Even if you hand-configure a master/slave cluster
> and orchestrate what doc goes to which node/shard, and hand-code your shards
> parameter, you will still have a cluster where you’d send updates to the 
> leader of 
> each shard and the replicas would replicate the index from the leader.
> 
> Let’s instead find a new good name for the cluster type. Standalone kind of 
> works
> for me, but I see it can be confused with single-node. We have also discussed
> replacing SolrCloud (which is a terrible name) with something more 
> descriptive.
> 
> Today: SolrCloud vs Master/slave
> Alt A: SolrCloud vs Standalone
> Alt B: SolrCloud vs Legacy
> Alt C: Clustered vs Independent
> Alt D: Clustered vs Manual mode
> 
> Jan
> 
>> 18. jun. 2020 kl. 15:53 skrev Mike Drob :
>> 
>> I personally think that using Solr cloud terminology for this would be fine
>> with leader/follower. The leader is the one that accepts updates, followers
>> cascade the updates somehow. The presence of ZK or election doesn’t really
>> change this detail.
>> 
>> However, if folks feel that it’s confusing, then I can’t tell them that
>> they’re not confused. Especially when they’re working with others who have
>> less Solr experience than we do and are less familiar with the intricacies.
>> 
>> Primary/Replica seems acceptable. Coordinator instead of Overseer seems
>> acceptable.
>> 
>> Would love to see this in 9.0!
>> 
>> Mike
>> 
>> On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
>>  wrote:
>> 
>>> While on the topic of renaming roles, I'd like to propose finding a better
>>> term than "overseer" which has historical slavery connotations as well.
>>> Director, perhaps?
>>> 
>>> 
>>> John Gallagher
>>> 
>>> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski 
>>> wrote:
>>> 
 +1 to rename master/slave, and +1 to choosing terminology distinct
 from what's used for SolrCloud.  I could be happy with several of the
 proposed options.  Since a good few have been proposed though, maybe
 an eventual vote thread is the most organized way to aggregate the
 opinions here.
 
 I'm less positive about the prospect of changing the name of our
 primary git branch.  Most projects that contributors might come from,
 most tutorials out there to learn git, most tools built on top of git
 - the majority are going to assume "master" as the main branch.  I
 appreciate the change that Github is trying to effect in changing the
 default for new projects, but it'll be a long time before that
 competes with the huge bulk of projects, documentation, etc. out there
 using "master".  Our contributors are smart and I'm sure they'd figure
 it out if we used "main" or something else instead, but having a
 non-standard git setup would be one more "papercut" in understanding
 how to contribute to a project that already makes that harder than it
 should.
 
 Jason
 
 
 On Thu, Jun 18, 2020 at 7:33 AM Demian Katz 
 wrote:
> 
> Regarding people having a problem with the word "master" -- GitHub is
 changing the default branch name away from "master," even in isolation
>>> from
 a "slave" pairing... so the terminology seems to be falling out of favor
>>> in
 all contexts. See:
> 
> 
 
>>> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
> 
> I'm not here to start a debate about the semantics of that, just to
 provide evidence that in some communities, the term "master" is causing
 concern all by itself. If we're going to make the change anyway, it might
 be best to get it over with and pick the most appropriate terminology we
 can agree upon, rather than trying to minimize the amount of change. It's
 going to be backward breaking anyway, so we might as well do it all now
 rather than risk having to go through two separate breaking changes at
 different points in time.
> 
> - Demian
> 
> -Original Message-
> From: Noble Paul 
> Sent: Thursday, June 18, 2020 1:51 AM
> To: solr-user@lucene.apache.org
> Subject: [EXTERNAL] Re: Getting rid of Master/Slave nomenclature in
>>> Solr
> 
> Looking at the code I see a 692 occurrences of the word "slave".
> Mostly variable names and ref guide docs.
> 
> The word "slave" is present in the responses as well. Any change in the
 request param/response payload is backward incompatible.
> 
> I have no objection to changing the names in ref guide and other
 internal variables. Going ahead with backward incompatible changes is
 painful. If somebody has the appetite to take it up

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Rahul Goswami]

I agree with Phill, Noble and Ilan above. The problematic term is "slave"
(not master) which I am all for changing if it causes less regression than
removing BOTH master and slave. Since some people have pointed out Github
changing the "master" terminology, in my personal opinion, it was not a
measured response to addressing the bigger problem we are all trying to
tackle. There is no concept of a "slave" branch, and "master" by itself is
a pretty generic term (Is someone having "mastery" over a skill a bad
thing?). I fear all it would end up achieving in the end with Github is a
mess of broken build scripts at best.
So +1 on "slave" being the problematic term IMO, not "master".

On Thu, Jun 18, 2020 at 8:19 PM Phill Campbell
 wrote:

> Master - Worker
> Master - Peon
> Master - Helper
> Master - Servant
>
> The term that is not wanted is “slave’. The term “master” is not a problem
> IMO.
>
> > On Jun 18, 2020, at 3:59 PM, Jan Høydahl  wrote:
> >
> > I support Mike Drob and Trey Grainger. We shuold re-use the
> leader/replica
> > terminology from Cloud. Even if you hand-configure a master/slave cluster
> > and orchestrate what doc goes to which node/shard, and hand-code your
> shards
> > parameter, you will still have a cluster where you’d send updates to the
> leader of
> > each shard and the replicas would replicate the index from the leader.
> >
> > Let’s instead find a new good name for the cluster type. Standalone kind
> of works
> > for me, but I see it can be confused with single-node. We have also
> discussed
> > replacing SolrCloud (which is a terrible name) with something more
> descriptive.
> >
> > Today: SolrCloud vs Master/slave
> > Alt A: SolrCloud vs Standalone
> > Alt B: SolrCloud vs Legacy
> > Alt C: Clustered vs Independent
> > Alt D: Clustered vs Manual mode
> >
> > Jan
> >
> >> 18. jun. 2020 kl. 15:53 skrev Mike Drob :
> >>
> >> I personally think that using Solr cloud terminology for this would be
> fine
> >> with leader/follower. The leader is the one that accepts updates,
> followers
> >> cascade the updates somehow. The presence of ZK or election doesn’t
> really
> >> change this detail.
> >>
> >> However, if folks feel that it’s confusing, then I can’t tell them that
> >> they’re not confused. Especially when they’re working with others who
> have
> >> less Solr experience than we do and are less familiar with the
> intricacies.
> >>
> >> Primary/Replica seems acceptable. Coordinator instead of Overseer seems
> >> acceptable.
> >>
> >> Would love to see this in 9.0!
> >>
> >> Mike
> >>
> >> On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
> >>  wrote:
> >>
> >>> While on the topic of renaming roles, I'd like to propose finding a
> better
> >>> term than "overseer" which has historical slavery connotations as well.
> >>> Director, perhaps?
> >>>
> >>>
> >>> John Gallagher
> >>>
> >>> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski  >
> >>> wrote:
> >>>
>  +1 to rename master/slave, and +1 to choosing terminology distinct
>  from what's used for SolrCloud.  I could be happy with several of the
>  proposed options.  Since a good few have been proposed though, maybe
>  an eventual vote thread is the most organized way to aggregate the
>  opinions here.
> 
>  I'm less positive about the prospect of changing the name of our
>  primary git branch.  Most projects that contributors might come from,
>  most tutorials out there to learn git, most tools built on top of git
>  - the majority are going to assume "master" as the main branch.  I
>  appreciate the change that Github is trying to effect in changing the
>  default for new projects, but it'll be a long time before that
>  competes with the huge bulk of projects, documentation, etc. out there
>  using "master".  Our contributors are smart and I'm sure they'd figure
>  it out if we used "main" or something else instead, but having a
>  non-standard git setup would be one more "papercut" in understanding
>  how to contribute to a project that already makes that harder than it
>  should.
> 
>  Jason
> 
> 
>  On Thu, Jun 18, 2020 at 7:33 AM Demian Katz <
> demian.k...@villanova.edu>
>  wrote:
> >
> > Regarding people having a problem with the word "master" -- GitHub is
>  changing the default branch name away from "master," even in isolation
> >>> from
>  a "slave" pairing... so the terminology seems to be falling out of
> favor
> >>> in
>  all contexts. See:
> >
> >
> 
> >>>
> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
> >
> > I'm not here to start a debate about the semantics of that, just to
>  provide evidence that in some communities, the term "master" is
> causing
>  concern all by itself. If we're going to make the change anyway, it
> might
>  be best to get it over with and pick the most appropriate terminology
> we
>  can agree upon, ra

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Walter Underwood]

We don’t get to decide whether “master” is a problem. The rest of the world
has already decided that it is a problem.

Our task is to replace the terms “master” and “slave” in Solr.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jun 18, 2020, at 6:50 PM, Rahul Goswami  wrote:
> 
> I agree with Phill, Noble and Ilan above. The problematic term is "slave"
> (not master) which I am all for changing if it causes less regression than
> removing BOTH master and slave. Since some people have pointed out Github
> changing the "master" terminology, in my personal opinion, it was not a
> measured response to addressing the bigger problem we are all trying to
> tackle. There is no concept of a "slave" branch, and "master" by itself is
> a pretty generic term (Is someone having "mastery" over a skill a bad
> thing?). I fear all it would end up achieving in the end with Github is a
> mess of broken build scripts at best.
> So +1 on "slave" being the problematic term IMO, not "master".
> 
> On Thu, Jun 18, 2020 at 8:19 PM Phill Campbell
>  wrote:
> 
>> Master - Worker
>> Master - Peon
>> Master - Helper
>> Master - Servant
>> 
>> The term that is not wanted is “slave’. The term “master” is not a problem
>> IMO.
>> 
>>> On Jun 18, 2020, at 3:59 PM, Jan Høydahl  wrote:
>>> 
>>> I support Mike Drob and Trey Grainger. We shuold re-use the
>> leader/replica
>>> terminology from Cloud. Even if you hand-configure a master/slave cluster
>>> and orchestrate what doc goes to which node/shard, and hand-code your
>> shards
>>> parameter, you will still have a cluster where you’d send updates to the
>> leader of
>>> each shard and the replicas would replicate the index from the leader.
>>> 
>>> Let’s instead find a new good name for the cluster type. Standalone kind
>> of works
>>> for me, but I see it can be confused with single-node. We have also
>> discussed
>>> replacing SolrCloud (which is a terrible name) with something more
>> descriptive.
>>> 
>>> Today: SolrCloud vs Master/slave
>>> Alt A: SolrCloud vs Standalone
>>> Alt B: SolrCloud vs Legacy
>>> Alt C: Clustered vs Independent
>>> Alt D: Clustered vs Manual mode
>>> 
>>> Jan
>>> 
 18. jun. 2020 kl. 15:53 skrev Mike Drob :
 
 I personally think that using Solr cloud terminology for this would be
>> fine
 with leader/follower. The leader is the one that accepts updates,
>> followers
 cascade the updates somehow. The presence of ZK or election doesn’t
>> really
 change this detail.
 
 However, if folks feel that it’s confusing, then I can’t tell them that
 they’re not confused. Especially when they’re working with others who
>> have
 less Solr experience than we do and are less familiar with the
>> intricacies.
 
 Primary/Replica seems acceptable. Coordinator instead of Overseer seems
 acceptable.
 
 Would love to see this in 9.0!
 
 Mike
 
 On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
  wrote:
 
> While on the topic of renaming roles, I'd like to propose finding a
>> better
> term than "overseer" which has historical slavery connotations as well.
> Director, perhaps?
> 
> 
> John Gallagher
> 
> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski >> 
> wrote:
> 
>> +1 to rename master/slave, and +1 to choosing terminology distinct
>> from what's used for SolrCloud.  I could be happy with several of the
>> proposed options.  Since a good few have been proposed though, maybe
>> an eventual vote thread is the most organized way to aggregate the
>> opinions here.
>> 
>> I'm less positive about the prospect of changing the name of our
>> primary git branch.  Most projects that contributors might come from,
>> most tutorials out there to learn git, most tools built on top of git
>> - the majority are going to assume "master" as the main branch.  I
>> appreciate the change that Github is trying to effect in changing the
>> default for new projects, but it'll be a long time before that
>> competes with the huge bulk of projects, documentation, etc. out there
>> using "master".  Our contributors are smart and I'm sure they'd figure
>> it out if we used "main" or something else instead, but having a
>> non-standard git setup would be one more "papercut" in understanding
>> how to contribute to a project that already makes that harder than it
>> should.
>> 
>> Jason
>> 
>> 
>> On Thu, Jun 18, 2020 at 7:33 AM Demian Katz <
>> demian.k...@villanova.edu>
>> wrote:
>>> 
>>> Regarding people having a problem with the word "master" -- GitHub is
>> changing the default branch name away from "master," even in isolation
> from
>> a "slave" pairing... so the terminology seems to be falling out of
>> favor
> in
>> all contexts. See:
>>> 
>>> 
>> 
> 
>> https://www.cnet.com/news/microsofts-gi

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Trey Grainger]

>
> Let’s instead find a new good name for the cluster type. Standalone kind
> of works
> for me, but I see it can be confused with single-node.

Yeah, I've typically referred to it as "standalone", but I don't think it's
descriptive enough. I can see why some people have been calling it
"master/slave" mode in lieu of a more descriptive alternative. I think a
new name (other than "standalone" or "legacy") would be superb.

We have also discussed replacing SolrCloud (which is a terrible name) with
> something more descriptive.

Today: SolrCloud vs Master/slave
> Alt A: SolrCloud vs Standalone
> Alt B: SolrCloud vs Legacy
> Alt C: Clustered vs Independent
> Alt D: Clustered vs Manual mode


+1 SolrCloud is even less descriptive and IMHO just sounds silly at this
point.

re: "Clustered" vs Independent/Manual. The thing I don't like about that is
that you typically have clusters in both modes. I think the key distinction
is whether Solr "manages" the cluster automatically for you or whether you
manage it manually yourself.

What do you think about:
Alt E: "Managed Clustering" vs. "Unmanaged Clustering" Mode
Alt F:  "Managed Clustering" vs. "Manual Clustering" Mode
?

I think I prefer option F.

Trey Grainger
Founder, Searchkernel
https://searchkernel.com

On Thu, Jun 18, 2020 at 5:59 PM Jan Høydahl  wrote:

> I support Mike Drob and Trey Grainger. We shuold re-use the leader/replica
> terminology from Cloud. Even if you hand-configure a master/slave cluster
> and orchestrate what doc goes to which node/shard, and hand-code your
> shards
> parameter, you will still have a cluster where you’d send updates to the
> leader of
> each shard and the replicas would replicate the index from the leader.
>
> Let’s instead find a new good name for the cluster type. Standalone kind
> of works
> for me, but I see it can be confused with single-node. We have also
> discussed
> replacing SolrCloud (which is a terrible name) with something more
> descriptive.
>
> Today: SolrCloud vs Master/slave
> Alt A: SolrCloud vs Standalone
> Alt B: SolrCloud vs Legacy
> Alt C: Clustered vs Independent
> Alt D: Clustered vs Manual mode
>
> Jan
>
> > 18. jun. 2020 kl. 15:53 skrev Mike Drob :
> >
> > I personally think that using Solr cloud terminology for this would be
> fine
> > with leader/follower. The leader is the one that accepts updates,
> followers
> > cascade the updates somehow. The presence of ZK or election doesn’t
> really
> > change this detail.
> >
> > However, if folks feel that it’s confusing, then I can’t tell them that
> > they’re not confused. Especially when they’re working with others who
> have
> > less Solr experience than we do and are less familiar with the
> intricacies.
> >
> > Primary/Replica seems acceptable. Coordinator instead of Overseer seems
> > acceptable.
> >
> > Would love to see this in 9.0!
> >
> > Mike
> >
> > On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
> >  wrote:
> >
> >> While on the topic of renaming roles, I'd like to propose finding a
> better
> >> term than "overseer" which has historical slavery connotations as well.
> >> Director, perhaps?
> >>
> >>
> >> John Gallagher
> >>
> >> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski 
> >> wrote:
> >>
> >>> +1 to rename master/slave, and +1 to choosing terminology distinct
> >>> from what's used for SolrCloud.  I could be happy with several of the
> >>> proposed options.  Since a good few have been proposed though, maybe
> >>> an eventual vote thread is the most organized way to aggregate the
> >>> opinions here.
> >>>
> >>> I'm less positive about the prospect of changing the name of our
> >>> primary git branch.  Most projects that contributors might come from,
> >>> most tutorials out there to learn git, most tools built on top of git
> >>> - the majority are going to assume "master" as the main branch.  I
> >>> appreciate the change that Github is trying to effect in changing the
> >>> default for new projects, but it'll be a long time before that
> >>> competes with the huge bulk of projects, documentation, etc. out there
> >>> using "master".  Our contributors are smart and I'm sure they'd figure
> >>> it out if we used "main" or something else instead, but having a
> >>> non-standard git setup would be one more "papercut" in understanding
> >>> how to contribute to a project that already makes that harder than it
> >>> should.
> >>>
> >>> Jason
> >>>
> >>>
> >>> On Thu, Jun 18, 2020 at 7:33 AM Demian Katz  >
> >>> wrote:
> 
>  Regarding people having a problem with the word "master" -- GitHub is
> >>> changing the default branch name away from "master," even in isolation
> >> from
> >>> a "slave" pairing... so the terminology seems to be falling out of
> favor
> >> in
> >>> all contexts. See:
> 
> 
> >>>
> >>
> https://www.cnet.com/news/microsofts-github-is-removing-coding-terms-like-master-and-slave/
> 
>  I'm not here to start a debate about the semantics of that, just to
> >>> provide evidence that in some commun

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Walter Underwood]

I sometimes describe them as tightly-coupled and loosely-coupled. There is a 
vast
difference in the amount of shared state in the two kinds of clusters. Old 
school
clusters are essentially a REST system. The primary server knows nothing 
about the leeches. The replication only assumes that the trusted resposiitory
is a later generation of the same index.

Solr Cloud has massive amounts of shared state. Just last week, we couldn’t
delete some replicas in two of the shards. I finally found a long queue at the
overseer and started rebooting server processes in those two shards. It fixed 
whatever state was broken, but it was pretty much turning it off and on again.
That kind of thing just cannot happen with master/slave.

Not sure about “manual”. We do a lot more manual management of our Solr
Cloud clusters. Scaling out the master/slave cluster is stupid simple. Bring
up a clone of a current slave, add it to the load balancer, and walk away.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jun 18, 2020, at 7:40 PM, Trey Grainger  wrote:
> 
>> 
>> Let’s instead find a new good name for the cluster type. Standalone kind
>> of works
>> for me, but I see it can be confused with single-node.
> 
> Yeah, I've typically referred to it as "standalone", but I don't think it's
> descriptive enough. I can see why some people have been calling it
> "master/slave" mode in lieu of a more descriptive alternative. I think a
> new name (other than "standalone" or "legacy") would be superb.
> 
> We have also discussed replacing SolrCloud (which is a terrible name) with
>> something more descriptive.
> 
> Today: SolrCloud vs Master/slave
>> Alt A: SolrCloud vs Standalone
>> Alt B: SolrCloud vs Legacy
>> Alt C: Clustered vs Independent
>> Alt D: Clustered vs Manual mode
> 
> 
> +1 SolrCloud is even less descriptive and IMHO just sounds silly at this
> point.
> 
> re: "Clustered" vs Independent/Manual. The thing I don't like about that is
> that you typically have clusters in both modes. I think the key distinction
> is whether Solr "manages" the cluster automatically for you or whether you
> manage it manually yourself.
> 
> What do you think about:
> Alt E: "Managed Clustering" vs. "Unmanaged Clustering" Mode
> Alt F:  "Managed Clustering" vs. "Manual Clustering" Mode
> ?
> 
> I think I prefer option F.
> 
> Trey Grainger
> Founder, Searchkernel
> https://searchkernel.com
> 
> On Thu, Jun 18, 2020 at 5:59 PM Jan Høydahl  wrote:
> 
>> I support Mike Drob and Trey Grainger. We shuold re-use the leader/replica
>> terminology from Cloud. Even if you hand-configure a master/slave cluster
>> and orchestrate what doc goes to which node/shard, and hand-code your
>> shards
>> parameter, you will still have a cluster where you’d send updates to the
>> leader of
>> each shard and the replicas would replicate the index from the leader.
>> 
>> Let’s instead find a new good name for the cluster type. Standalone kind
>> of works
>> for me, but I see it can be confused with single-node. We have also
>> discussed
>> replacing SolrCloud (which is a terrible name) with something more
>> descriptive.
>> 
>> Today: SolrCloud vs Master/slave
>> Alt A: SolrCloud vs Standalone
>> Alt B: SolrCloud vs Legacy
>> Alt C: Clustered vs Independent
>> Alt D: Clustered vs Manual mode
>> 
>> Jan
>> 
>>> 18. jun. 2020 kl. 15:53 skrev Mike Drob :
>>> 
>>> I personally think that using Solr cloud terminology for this would be
>> fine
>>> with leader/follower. The leader is the one that accepts updates,
>> followers
>>> cascade the updates somehow. The presence of ZK or election doesn’t
>> really
>>> change this detail.
>>> 
>>> However, if folks feel that it’s confusing, then I can’t tell them that
>>> they’re not confused. Especially when they’re working with others who
>> have
>>> less Solr experience than we do and are less familiar with the
>> intricacies.
>>> 
>>> Primary/Replica seems acceptable. Coordinator instead of Overseer seems
>>> acceptable.
>>> 
>>> Would love to see this in 9.0!
>>> 
>>> Mike
>>> 
>>> On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
>>>  wrote:
>>> 
 While on the topic of renaming roles, I'd like to propose finding a
>> better
 term than "overseer" which has historical slavery connotations as well.
 Director, perhaps?
 
 
 John Gallagher
 
 On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski 
 wrote:
 
> +1 to rename master/slave, and +1 to choosing terminology distinct
> from what's used for SolrCloud.  I could be happy with several of the
> proposed options.  Since a good few have been proposed though, maybe
> an eventual vote thread is the most organized way to aggregate the
> opinions here.
> 
> I'm less positive about the prospect of changing the name of our
> primary git branch.  Most projects that contributors might come from,
> most tutorials out there to learn git, most tools built on top of git
> - the

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Oakley, Craig (NIH/NLM/NCBI) [C]]

Trying think of a term that both fresh (and yet sort of standard already) and 
appropriate: how about IndexFetcher instead of "Slave"? And then "Master" could 
be "FetchedIndex" or "FetchedSource"

I think it could be beneficial to broaden the range of candidates.

From: Walter Underwood 
Sent: Thursday, June 18, 2020 10:34 PM
To: solr-user@lucene.apache.org 
Subject: Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

We don’t get to decide whether “master” is a problem. The rest of the world
has already decided that it is a problem.

Our task is to replace the terms “master” and “slave” in Solr.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jun 18, 2020, at 6:50 PM, Rahul Goswami  wrote:
>
> I agree with Phill, Noble and Ilan above. The problematic term is "slave"
> (not master) which I am all for changing if it causes less regression than
> removing BOTH master and slave. Since some people have pointed out Github
> changing the "master" terminology, in my personal opinion, it was not a
> measured response to addressing the bigger problem we are all trying to
> tackle. There is no concept of a "slave" branch, and "master" by itself is
> a pretty generic term (Is someone having "mastery" over a skill a bad
> thing?). I fear all it would end up achieving in the end with Github is a
> mess of broken build scripts at best.
> So +1 on "slave" being the problematic term IMO, not "master".
>
> On Thu, Jun 18, 2020 at 8:19 PM Phill Campbell
>  wrote:
>
>> Master - Worker
>> Master - Peon
>> Master - Helper
>> Master - Servant
>>
>> The term that is not wanted is “slave’. The term “master” is not a problem
>> IMO.
>>
>>> On Jun 18, 2020, at 3:59 PM, Jan Høydahl  wrote:
>>>
>>> I support Mike Drob and Trey Grainger. We shuold re-use the
>> leader/replica
>>> terminology from Cloud. Even if you hand-configure a master/slave cluster
>>> and orchestrate what doc goes to which node/shard, and hand-code your
>> shards
>>> parameter, you will still have a cluster where you’d send updates to the
>> leader of
>>> each shard and the replicas would replicate the index from the leader.
>>>
>>> Let’s instead find a new good name for the cluster type. Standalone kind
>> of works
>>> for me, but I see it can be confused with single-node. We have also
>> discussed
>>> replacing SolrCloud (which is a terrible name) with something more
>> descriptive.
>>>
>>> Today: SolrCloud vs Master/slave
>>> Alt A: SolrCloud vs Standalone
>>> Alt B: SolrCloud vs Legacy
>>> Alt C: Clustered vs Independent
>>> Alt D: Clustered vs Manual mode
>>>
>>> Jan
>>>
 18. jun. 2020 kl. 15:53 skrev Mike Drob :

 I personally think that using Solr cloud terminology for this would be
>> fine
 with leader/follower. The leader is the one that accepts updates,
>> followers
 cascade the updates somehow. The presence of ZK or election doesn’t
>> really
 change this detail.

 However, if folks feel that it’s confusing, then I can’t tell them that
 they’re not confused. Especially when they’re working with others who
>> have
 less Solr experience than we do and are less familiar with the
>> intricacies.

 Primary/Replica seems acceptable. Coordinator instead of Overseer seems
 acceptable.

 Would love to see this in 9.0!

 Mike

 On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
  wrote:

> While on the topic of renaming roles, I'd like to propose finding a
>> better
> term than "overseer" which has historical slavery connotations as well.
> Director, perhaps?
>
>
> John Gallagher
>
> On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski >>
> wrote:
>
>> +1 to rename master/slave, and +1 to choosing terminology distinct
>> from what's used for SolrCloud.  I could be happy with several of the
>> proposed options.  Since a good few have been proposed though, maybe
>> an eventual vote thread is the most organized way to aggregate the
>> opinions here.
>>
>> I'm less positive about the prospect of changing the name of our
>> primary git branch.  Most projects that contributors might come from,
>> most tutorials out there to learn git, most tools built on top of git
>> - the majority are going to assume "master" as the main branch.  I
>> appreciate the change that Github is trying to effect in changing the
>> default for new projects, but it'll be a long time before that
>> competes with the huge bulk of projects, documentation, etc. out there
>> using "master".  Our contributors are smart and I'm sure they'd figure
>> it out if we used "main" or something else instead, but having a
>> non-standard git setup would be one more "papercut" in understanding
>> how to contribute to a project that already makes that harder than it
>> should.
>>
>> Jason
>>
>>
>> On Thu, Jun 18, 2

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Thomas Corthals]

Since "overseer" is also problematic, I'd like to propose "orchestrator" as
an alternative.

Thomas

Op vr 19 jun. 2020 04:34 schreef Walter Underwood :

> We don’t get to decide whether “master” is a problem. The rest of the world
> has already decided that it is a problem.
>
> Our task is to replace the terms “master” and “slave” in Solr.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
> > On Jun 18, 2020, at 6:50 PM, Rahul Goswami 
> wrote:
> >
> > I agree with Phill, Noble and Ilan above. The problematic term is "slave"
> > (not master) which I am all for changing if it causes less regression
> than
> > removing BOTH master and slave. Since some people have pointed out Github
> > changing the "master" terminology, in my personal opinion, it was not a
> > measured response to addressing the bigger problem we are all trying to
> > tackle. There is no concept of a "slave" branch, and "master" by itself
> is
> > a pretty generic term (Is someone having "mastery" over a skill a bad
> > thing?). I fear all it would end up achieving in the end with Github is a
> > mess of broken build scripts at best.
> > So +1 on "slave" being the problematic term IMO, not "master".
> >
> > On Thu, Jun 18, 2020 at 8:19 PM Phill Campbell
> >  wrote:
> >
> >> Master - Worker
> >> Master - Peon
> >> Master - Helper
> >> Master - Servant
> >>
> >> The term that is not wanted is “slave’. The term “master” is not a
> problem
> >> IMO.
> >>
> >>> On Jun 18, 2020, at 3:59 PM, Jan Høydahl 
> wrote:
> >>>
> >>> I support Mike Drob and Trey Grainger. We shuold re-use the
> >> leader/replica
> >>> terminology from Cloud. Even if you hand-configure a master/slave
> cluster
> >>> and orchestrate what doc goes to which node/shard, and hand-code your
> >> shards
> >>> parameter, you will still have a cluster where you’d send updates to
> the
> >> leader of
> >>> each shard and the replicas would replicate the index from the leader.
> >>>
> >>> Let’s instead find a new good name for the cluster type. Standalone
> kind
> >> of works
> >>> for me, but I see it can be confused with single-node. We have also
> >> discussed
> >>> replacing SolrCloud (which is a terrible name) with something more
> >> descriptive.
> >>>
> >>> Today: SolrCloud vs Master/slave
> >>> Alt A: SolrCloud vs Standalone
> >>> Alt B: SolrCloud vs Legacy
> >>> Alt C: Clustered vs Independent
> >>> Alt D: Clustered vs Manual mode
> >>>
> >>> Jan
> >>>
>  18. jun. 2020 kl. 15:53 skrev Mike Drob :
> 
>  I personally think that using Solr cloud terminology for this would be
> >> fine
>  with leader/follower. The leader is the one that accepts updates,
> >> followers
>  cascade the updates somehow. The presence of ZK or election doesn’t
> >> really
>  change this detail.
> 
>  However, if folks feel that it’s confusing, then I can’t tell them
> that
>  they’re not confused. Especially when they’re working with others who
> >> have
>  less Solr experience than we do and are less familiar with the
> >> intricacies.
> 
>  Primary/Replica seems acceptable. Coordinator instead of Overseer
> seems
>  acceptable.
> 
>  Would love to see this in 9.0!
> 
>  Mike
> 
>  On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
>   wrote:
> 
> > While on the topic of renaming roles, I'd like to propose finding a
> >> better
> > term than "overseer" which has historical slavery connotations as
> well.
> > Director, perhaps?
> >
> >
> > John Gallagher
> >
> > On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski <
> gerlowsk...@gmail.com
> >>>
> > wrote:
> >
> >> +1 to rename master/slave, and +1 to choosing terminology distinct
> >> from what's used for SolrCloud.  I could be happy with several of
> the
> >> proposed options.  Since a good few have been proposed though, maybe
> >> an eventual vote thread is the most organized way to aggregate the
> >> opinions here.
> >>
> >> I'm less positive about the prospect of changing the name of our
> >> primary git branch.  Most projects that contributors might come
> from,
> >> most tutorials out there to learn git, most tools built on top of
> git
> >> - the majority are going to assume "master" as the main branch.  I
> >> appreciate the change that Github is trying to effect in changing
> the
> >> default for new projects, but it'll be a long time before that
> >> competes with the huge bulk of projects, documentation, etc. out
> there
> >> using "master".  Our contributors are smart and I'm sure they'd
> figure
> >> it out if we used "main" or something else instead, but having a
> >> non-standard git setup would be one more "papercut" in understanding
> >> how to contribute to a project that already makes that harder than
> it
> >> should.
> >>
> >> Jason
> >>
> >>
> >> On Thu, Jun 18, 2020 at 7:33 

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[gnandre]

What about blacklist and whitelist for shards? May I suggest blocklist and
safelist?

On Fri, Jun 19, 2020, 1:45 AM Thomas Corthals  wrote:

> Since "overseer" is also problematic, I'd like to propose "orchestrator" as
> an alternative.
>
> Thomas
>
> Op vr 19 jun. 2020 04:34 schreef Walter Underwood :
>
> > We don’t get to decide whether “master” is a problem. The rest of the
> world
> > has already decided that it is a problem.
> >
> > Our task is to replace the terms “master” and “slave” in Solr.
> >
> > wunder
> > Walter Underwood
> > wun...@wunderwood.org
> > http://observer.wunderwood.org/  (my blog)
> >
> > > On Jun 18, 2020, at 6:50 PM, Rahul Goswami 
> > wrote:
> > >
> > > I agree with Phill, Noble and Ilan above. The problematic term is
> "slave"
> > > (not master) which I am all for changing if it causes less regression
> > than
> > > removing BOTH master and slave. Since some people have pointed out
> Github
> > > changing the "master" terminology, in my personal opinion, it was not a
> > > measured response to addressing the bigger problem we are all trying to
> > > tackle. There is no concept of a "slave" branch, and "master" by itself
> > is
> > > a pretty generic term (Is someone having "mastery" over a skill a bad
> > > thing?). I fear all it would end up achieving in the end with Github
> is a
> > > mess of broken build scripts at best.
> > > So +1 on "slave" being the problematic term IMO, not "master".
> > >
> > > On Thu, Jun 18, 2020 at 8:19 PM Phill Campbell
> > >  wrote:
> > >
> > >> Master - Worker
> > >> Master - Peon
> > >> Master - Helper
> > >> Master - Servant
> > >>
> > >> The term that is not wanted is “slave’. The term “master” is not a
> > problem
> > >> IMO.
> > >>
> > >>> On Jun 18, 2020, at 3:59 PM, Jan Høydahl 
> > wrote:
> > >>>
> > >>> I support Mike Drob and Trey Grainger. We shuold re-use the
> > >> leader/replica
> > >>> terminology from Cloud. Even if you hand-configure a master/slave
> > cluster
> > >>> and orchestrate what doc goes to which node/shard, and hand-code your
> > >> shards
> > >>> parameter, you will still have a cluster where you’d send updates to
> > the
> > >> leader of
> > >>> each shard and the replicas would replicate the index from the
> leader.
> > >>>
> > >>> Let’s instead find a new good name for the cluster type. Standalone
> > kind
> > >> of works
> > >>> for me, but I see it can be confused with single-node. We have also
> > >> discussed
> > >>> replacing SolrCloud (which is a terrible name) with something more
> > >> descriptive.
> > >>>
> > >>> Today: SolrCloud vs Master/slave
> > >>> Alt A: SolrCloud vs Standalone
> > >>> Alt B: SolrCloud vs Legacy
> > >>> Alt C: Clustered vs Independent
> > >>> Alt D: Clustered vs Manual mode
> > >>>
> > >>> Jan
> > >>>
> >  18. jun. 2020 kl. 15:53 skrev Mike Drob :
> > 
> >  I personally think that using Solr cloud terminology for this would
> be
> > >> fine
> >  with leader/follower. The leader is the one that accepts updates,
> > >> followers
> >  cascade the updates somehow. The presence of ZK or election doesn’t
> > >> really
> >  change this detail.
> > 
> >  However, if folks feel that it’s confusing, then I can’t tell them
> > that
> >  they’re not confused. Especially when they’re working with others
> who
> > >> have
> >  less Solr experience than we do and are less familiar with the
> > >> intricacies.
> > 
> >  Primary/Replica seems acceptable. Coordinator instead of Overseer
> > seems
> >  acceptable.
> > 
> >  Would love to see this in 9.0!
> > 
> >  Mike
> > 
> >  On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
> >   wrote:
> > 
> > > While on the topic of renaming roles, I'd like to propose finding a
> > >> better
> > > term than "overseer" which has historical slavery connotations as
> > well.
> > > Director, perhaps?
> > >
> > >
> > > John Gallagher
> > >
> > > On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski <
> > gerlowsk...@gmail.com
> > >>>
> > > wrote:
> > >
> > >> +1 to rename master/slave, and +1 to choosing terminology distinct
> > >> from what's used for SolrCloud.  I could be happy with several of
> > the
> > >> proposed options.  Since a good few have been proposed though,
> maybe
> > >> an eventual vote thread is the most organized way to aggregate the
> > >> opinions here.
> > >>
> > >> I'm less positive about the prospect of changing the name of our
> > >> primary git branch.  Most projects that contributors might come
> > from,
> > >> most tutorials out there to learn git, most tools built on top of
> > git
> > >> - the majority are going to assume "master" as the main branch.  I
> > >> appreciate the change that Github is trying to effect in changing
> > the
> > >> default for new projects, but it'll be a long time before that
> > >> competes with the huge bulk of projects, documentation, etc. ou

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[gnandre]

Another alternative for master-slave nodes might be parent-child nodes.
This was adopted in Python too afaik.

On Fri, Jun 19, 2020, 2:07 AM gnandre  wrote:

> What about blacklist and whitelist for shards? May I suggest blocklist and
> safelist?
>
> On Fri, Jun 19, 2020, 1:45 AM Thomas Corthals 
> wrote:
>
>> Since "overseer" is also problematic, I'd like to propose "orchestrator"
>> as
>> an alternative.
>>
>> Thomas
>>
>> Op vr 19 jun. 2020 04:34 schreef Walter Underwood > >:
>>
>> > We don’t get to decide whether “master” is a problem. The rest of the
>> world
>> > has already decided that it is a problem.
>> >
>> > Our task is to replace the terms “master” and “slave” in Solr.
>> >
>> > wunder
>> > Walter Underwood
>> > wun...@wunderwood.org
>> > http://observer.wunderwood.org/  (my blog)
>> >
>> > > On Jun 18, 2020, at 6:50 PM, Rahul Goswami 
>> > wrote:
>> > >
>> > > I agree with Phill, Noble and Ilan above. The problematic term is
>> "slave"
>> > > (not master) which I am all for changing if it causes less regression
>> > than
>> > > removing BOTH master and slave. Since some people have pointed out
>> Github
>> > > changing the "master" terminology, in my personal opinion, it was not
>> a
>> > > measured response to addressing the bigger problem we are all trying
>> to
>> > > tackle. There is no concept of a "slave" branch, and "master" by
>> itself
>> > is
>> > > a pretty generic term (Is someone having "mastery" over a skill a bad
>> > > thing?). I fear all it would end up achieving in the end with Github
>> is a
>> > > mess of broken build scripts at best.
>> > > So +1 on "slave" being the problematic term IMO, not "master".
>> > >
>> > > On Thu, Jun 18, 2020 at 8:19 PM Phill Campbell
>> > >  wrote:
>> > >
>> > >> Master - Worker
>> > >> Master - Peon
>> > >> Master - Helper
>> > >> Master - Servant
>> > >>
>> > >> The term that is not wanted is “slave’. The term “master” is not a
>> > problem
>> > >> IMO.
>> > >>
>> > >>> On Jun 18, 2020, at 3:59 PM, Jan Høydahl 
>> > wrote:
>> > >>>
>> > >>> I support Mike Drob and Trey Grainger. We shuold re-use the
>> > >> leader/replica
>> > >>> terminology from Cloud. Even if you hand-configure a master/slave
>> > cluster
>> > >>> and orchestrate what doc goes to which node/shard, and hand-code
>> your
>> > >> shards
>> > >>> parameter, you will still have a cluster where you’d send updates to
>> > the
>> > >> leader of
>> > >>> each shard and the replicas would replicate the index from the
>> leader.
>> > >>>
>> > >>> Let’s instead find a new good name for the cluster type. Standalone
>> > kind
>> > >> of works
>> > >>> for me, but I see it can be confused with single-node. We have also
>> > >> discussed
>> > >>> replacing SolrCloud (which is a terrible name) with something more
>> > >> descriptive.
>> > >>>
>> > >>> Today: SolrCloud vs Master/slave
>> > >>> Alt A: SolrCloud vs Standalone
>> > >>> Alt B: SolrCloud vs Legacy
>> > >>> Alt C: Clustered vs Independent
>> > >>> Alt D: Clustered vs Manual mode
>> > >>>
>> > >>> Jan
>> > >>>
>> >  18. jun. 2020 kl. 15:53 skrev Mike Drob :
>> > 
>> >  I personally think that using Solr cloud terminology for this
>> would be
>> > >> fine
>> >  with leader/follower. The leader is the one that accepts updates,
>> > >> followers
>> >  cascade the updates somehow. The presence of ZK or election doesn’t
>> > >> really
>> >  change this detail.
>> > 
>> >  However, if folks feel that it’s confusing, then I can’t tell them
>> > that
>> >  they’re not confused. Especially when they’re working with others
>> who
>> > >> have
>> >  less Solr experience than we do and are less familiar with the
>> > >> intricacies.
>> > 
>> >  Primary/Replica seems acceptable. Coordinator instead of Overseer
>> > seems
>> >  acceptable.
>> > 
>> >  Would love to see this in 9.0!
>> > 
>> >  Mike
>> > 
>> >  On Thu, Jun 18, 2020 at 8:25 AM John Gallagher
>> >   wrote:
>> > 
>> > > While on the topic of renaming roles, I'd like to propose finding
>> a
>> > >> better
>> > > term than "overseer" which has historical slavery connotations as
>> > well.
>> > > Director, perhaps?
>> > >
>> > >
>> > > John Gallagher
>> > >
>> > > On Thu, Jun 18, 2020 at 8:48 AM Jason Gerlowski <
>> > gerlowsk...@gmail.com
>> > >>>
>> > > wrote:
>> > >
>> > >> +1 to rename master/slave, and +1 to choosing terminology
>> distinct
>> > >> from what's used for SolrCloud.  I could be happy with several of
>> > the
>> > >> proposed options.  Since a good few have been proposed though,
>> maybe
>> > >> an eventual vote thread is the most organized way to aggregate
>> the
>> > >> opinions here.
>> > >>
>> > >> I'm less positive about the prospect of changing the name of our
>> > >> primary git branch.  Most projects that contributors might come
>> > from,
>> > >> most tutorials out there to learn git, most tools buil