Re: question about solrCloud joining

[Mikhail Khludnev]

Ok. Still hard to follow. Can you clarify which collection you run these
queries on?
Collection name (url segment before /select) is more significant than any
port (jvm) identity.

On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang 
wrote:

> Hi Mikhail
>
> Thanks for your response,  but question is not related to "title:Get Out",
> maybe i did not describe clearly.
>
> I knew solrCloud joining is not working in index which is splited to
> multiple shards.
>
> but why i run "*{!join from=director_id fromIndex=movies
> to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards)  i
> got exception "SolrCloud join: multiple shards not yet supported movies"
>
> but when run on 8983, i got result but it is incorrect without above
> exception. i think should get same exception no matter run joining on 8983
> or 8984.
>
> Not sure my explanation is clear?
>
> Please kindly let me know if you have any question.
>
> Thanks!
>
> Lisheng
>
>
>
> Mikhail Khludnev  于2019年8月21日周三 上午4:41写道：
>
> > Hello, Lisheng.
> > I barely follow, but couldn't the space symbol in "title:Get Out"
> > cause the problem
> > ?
> > Check debugQuery and nested query in local param.
> >
> >
> > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang 
> > wrote:
> >
> > > Hi Erick
> > >
> > > Thanks for your quick response and remaining me about attachment issue.
> > >
> > > Yes, i run on 2 different jvms that not related to if they are on same
> > > machine or not.
> > >
> > > let me describe my scenario, i have two collection:
> > >
> > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> 8984.
> > >
> > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on
> > 8983
> > > 2. movies: 2 shard, 1 replica/shardshard1 is on 8983, shard2 is on
> > > 8984.
> > >
> > > collection movieDirectors has 2 docs:
> > > {
> > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > 1642343781358370816
> > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > 1642343828930166784
> > > }
> > > collection movies has 2 docs too:
> > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > 1642343781358370816
> > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > 1642343828930166784
> > > }
> > > everything is ok when i run query with "{!join from=id
> > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983
> and
> > > 8984, i can got expected result:
> > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > "params":{
> > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > "response":{"numFound
> > > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk", "
> > > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > > "title":"Get
> > > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > > but when i run "{!join from=director_id fromIndex=movies
> > > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > >  if i filter by "title:Get Out", i got nothing.  i understood "Get Out"
> > is
> > > not exist in 8983.
> > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> > "params":{
> > > "
> > > q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> > > to=id}title:\"Dunkirk\"", "_":"1566261450613"}},
> > "response":{"numFound":1,"
> > > start":0,"docs":[ { "id":"1", "name":"Christopher Nolan",
> > "has_oscar":true,
> > > "_version_":1642343436642156544}] }}
> > >
> > > but question is coming, when i run "{!join from=director_id
> > > fromIndex=movies to=id}title:"Dunkirk"" on 8984, i got "SolrCloud join:
> > > multiple shards not yet supported movies"
> > > no matter what filter value is.
> > >
> > > i found following code:
> > >
> > > private static String findLocalReplicaForFromIndex(ZkController
> > > zkController, String fromIndex) {
> > >   String fromReplica = null;
> > >
> > >   String nodeName = zkController.getNodeName();
> > >   for (Slice slice :
> > >
> > >
> >
> zkController.getClusterState().getCollection(fromIndex).getActiveSlicesArr())
> > > {
> > > if (fromReplica != null)
> > >   throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > >   "SolrCloud join: multiple shards not yet supported " +
> > > fromIndex);
> > > for (Replica replica : slice.getReplicas()) {
> > >   if (replica.getNodeName().equals(nodeName)) {
> > > fromReplica = replica.getStr(ZkStateReader.CORE_NAME_PROP);
> > > // found local replica, but is it Active?
> > > if (replica.getState() != Replica.State.ACTIVE)
> > >   throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
> > >   "SolrCloud join: "+fromIndex+" has a local replica
> > > ("+fromReplica+
> > >   ") on "+nodeName+", but it is "+replica.getState());
> > >
> > > break;
> > >   }
> > > }
> > >   }
> > >
> > >   if (fromReplica == null)
> > > 

Re: question about solrCloud joining

[Lisheng Wang]

Hi  Mikhail,

okay.

below is 2 requests:

both are select from "movieDirectors" collection join "movies" collection
which has 2 shards.

http://localhost:8983/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
http://localhost:8984/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*

first request can get result without Exception, response is following

{ "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{ "
q":"*:*", "fq":"{!join from=director_id fromIndex=movies
to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
"_version_":1642343436642156544}] }}

second request will get Exception
{ "responseHeader":{ "zkConnected":true, "status":400, "QTime":29, "params":{
"q":"*:*", "fq":"{!join from=director_id fromIndex=movies
to=id}title:\"Dunkirk\"", "_":"1566261620152"}}, "error":{ "metadata":[
"error-class","org.apache.solr.common.SolrException", "root-error-class",
"org.apache.solr.common.SolrException"], "msg":"SolrCloud join: multiple
shards not yet supported movies", "code":400}}

i don't know why get 2 different result when you request from different
node, i think both should get Exception with "SolrCloud join: multiple
shards not yet supported movies".

Best,
Lisheng


Mikhail Khludnev  于2019年8月21日周三 下午3:19写道：

> Ok. Still hard to follow. Can you clarify which collection you run these
> queries on?
> Collection name (url segment before /select) is more significant than any
> port (jvm) identity.
>
> On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang 
> wrote:
>
> > Hi Mikhail
> >
> > Thanks for your response,  but question is not related to "title:Get
> Out",
> > maybe i did not describe clearly.
> >
> > I knew solrCloud joining is not working in index which is splited to
> > multiple shards.
> >
> > but why i run "*{!join from=director_id fromIndex=movies
> > to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards)
> i
> > got exception "SolrCloud join: multiple shards not yet supported movies"
> >
> > but when run on 8983, i got result but it is incorrect without above
> > exception. i think should get same exception no matter run joining on
> 8983
> > or 8984.
> >
> > Not sure my explanation is clear?
> >
> > Please kindly let me know if you have any question.
> >
> > Thanks!
> >
> > Lisheng
> >
> >
> >
> > Mikhail Khludnev  于2019年8月21日周三 上午4:41写道：
> >
> > > Hello, Lisheng.
> > > I barely follow, but couldn't the space symbol in "title:Get Out"
> > > cause the problem
> > > ?
> > > Check debugQuery and nested query in local param.
> > >
> > >
> > > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang 
> > > wrote:
> > >
> > > > Hi Erick
> > > >
> > > > Thanks for your quick response and remaining me about attachment
> issue.
> > > >
> > > > Yes, i run on 2 different jvms that not related to if they are on
> same
> > > > machine or not.
> > > >
> > > > let me describe my scenario, i have two collection:
> > > >
> > > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> > 8984.
> > > >
> > > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is on
> > > 8983
> > > > 2. movies: 2 shard, 1 replica/shardshard1 is on 8983, shard2 is
> on
> > > > 8984.
> > > >
> > > > collection movieDirectors has 2 docs:
> > > > {
> > > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > 1642343781358370816
> > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > 1642343828930166784
> > > > }
> > > > collection movies has 2 docs too:
> > > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > 1642343781358370816
> > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > 1642343828930166784
> > > > }
> > > > everything is ok when i run query with "{!join from=id
> > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both 8983
> > and
> > > > 8984, i can got expected result:
> > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > > "params":{
> > > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > > "response":{"numFound
> > > > ":2,"start":0,"maxScore":1.0,"docs":[ { "id":"1", "title":"Dunkirk",
> "
> > > > director_id":"1", "_version_":1642343781358370816}, { "id":"2",
> > > > "title":"Get
> > > > Out", "director_id":"2", "_version_":1642343828930166784}] }}
> > > > but when i run "{!join from=director_id fromIndex=movies
> > > > to=id}title:"Dunkirk"" on 8983 got 1 doc,
> > > >  if i filter by "title:Get Out", i got nothing.  i understood "Get
> Out"
> > > is
> > > > not exist in 8983.
> > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3,
> > > "params":{
> > > > "
> > > > q":"*:*", "fq":"{!join from=director_id f

Re: Problems with restricting access to users using Basic auth

[Salmaan Rashid Syed]

Hi Jason,

Thanks for your prompt reply.

Your code does address few of my concerns like restricting *solr-user* from
accessing the dashboard and from executing other request methods apart from
*"update"* and *"read"*.

But I am still able to access other collections such as *"Collection3",
"Collection4"* and so on, apart from the intended two collection entered in
the code. I can give *"update"* and *"read" *requests to these external
Collections which solr-user should not be able to do.

Moreover solr-user can look at the
*http://localhost:8983/solr/admin/authentication
* link which lists the
users and their *SHA256* coded passwords. How can I hide this and restrict
access to other collections?

Thanks and regards
Salmaan


On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski 
wrote:

> Hi Salmaan,
>
> Solr's RuleBasedAuthorizationPlugin allows requests through if none of
> the specified permissions apply.  I think that's what you're running
> into in your example above.  If you want to lockdown a particular API
> (or set of APIs) then you need to explicitly add a permission that
> restricts those APIs to a particular role.
>
> One way to get the behavior that it sounds like you're looking for
> would be to add a catch-all permission at the bottom of your
> permissions list that restricts all other APIs to "admin".  This would
> look a bit like:
>
>  "permissions":[
> {
> "name":"security-edit",
> "role":"admin"
> },
> {
> "collection": ["Collection1", "Collection2"],
> "name": ["update", "read"],
> "role": "dev"
> },
> {
> "name": "all",
> "role": "admin"
> }
> ]
>
> Hope that helps get you started.
>
> Best,
>
> Jason
>
> On Tue, Aug 20, 2019 at 3:19 AM Salmaan Rashid Syed
>  wrote:
> >
> > Hi Solr Users,
> >
> > I want to create a user that has restricted access to Solr. I did the
> > follwowing:-
> >
> >
> >1. {
> >2. "authentication":{
> >3."blockUnknown": true,
> >4."class":"solr.BasicAuthPlugin",
> >5."credentials":{
> >6. "solr-admin":
> >"2IUJD9dxRhxSXaJGdMP5z8ggSn4I285Ty9GCWeRNMUg=
> > /sSNJJufPtj4baRizoJshJawFsWvopvZSqZpQ/Nwd78="
> >,
> >7. "solr-user":
> >"p+XwOh15p/rvFltv2LXP1CwtbvwBgGlC9qcDKxV73B4=
> > DcNsjfA6Wf16V1XKT+YraosSFQ5Cr3eRUX6BQnx9XKA="
> >
> >8.  }
> >9. },
> >10. "authorization":{
> >11."class":"solr.RuleBasedAuthorizationPlugin",
> >12."user-role":{"solr-admin":"admin", "solr-user":"dev"},
> >13."permissions":[
> >14.   {
> >15."name":"security-edit",
> >16."role":"admin"
> >17.   },
> >18.   {
> >19. "collection": ["Collection1", "Collection2"],
> >20. "name": ["update", "read"],
> >21. "role": "dev"
> >22.   }
> >23.   ]
> >24. }}
> >
> >
> > But when Login intot the Solr admin dash-board using Solr-user
> credentials,
> > I can read, select, write, update, delete collections and do all sorts of
> > things like a solr-admin can do.
> >
> > I want solr-user to be able to access only *Collection1* and
> *Collection2*
> > and be able to only *update *and *read*. He should not be able to access
> > other collections and do anything apart from the above mentioned role.
> >
> > Where am I exactly going wrong?
> >
> > Thanks and Regards,
> > Salmaan
>

StackOverflowError leader election on 8.2.0

[Markus Jelsma]

Hello,

Looking this up i found SOLR-5692, but that was solved a lifetime ago, so just 
checking if this is a familiar error and one i missing in Jira:

A client's Solr 8.2.0 cluster brought us the next StackOverflowError while 
running 8.2.0 on Java 8:

Exception in thread "coreZkRegister-1-thread-3" java.lang.StackOverflowError
at 
org.apache.logging.log4j.ThreadContext.getImmutableContext(ThreadContext.java:352)
at 
org.apache.logging.log4j.core.impl.ThreadContextDataInjector$ForDefaultThreadContextMap.injectContextData(ThreadContextDataInjector.java:66)
at 
org.apache.logging.log4j.core.impl.Log4jLogEvent.createContextData(Log4jLogEvent.java:473)
at 
org.apache.logging.log4j.core.impl.Log4jLogEvent.(Log4jLogEvent.java:331)
at 
org.apache.logging.log4j.core.impl.DefaultLogEventFactory.createEvent(DefaultLogEventFactory.java:54)
at 
org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:404)
at 
org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
at 
org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)
at 
org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125)
at 
org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108)
at 
org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2007)
at 
org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1866)
at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:179)
at org.apache.solr.update.PeerSync.sync(PeerSync.java:172)
at 
org.apache.solr.cloud.SyncStrategy.syncWithReplicas(SyncStrategy.java:187)
at 
org.apache.solr.cloud.SyncStrategy.syncReplicas(SyncStrategy.java:131)
at org.apache.solr.cloud.SyncStrategy.sync(SyncStrategy.java:109)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:400)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)

. it repeats hundreds of times

at 
org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1479)
at org.apache.solr.cloud.ZkController.register(ZkController.java:1219)
at org.apache.solr.cloud.ZkController.register(ZkController.java:1171)
at 
org.apache.solr.core.ZkContain

Re: 8.2.0 getting warning - unable to load jetty,​ not starting JettyAdminServer

[Martijn Koster]

Hi Arnold,

It’s hard to say without seeing exactly what you’re doing and exactly what 
you’re seeing.
Simplify it first, ie remove your custom plugins and related config and see if 
the problem reproduces still, then try without cloud mode and see it it 
reproduces still. Then create an issue on 
https://github.com/docker-solr/docker-solr/issues 
, labelled as a question, 
with the exact command you run and its full output, and attach your zipped-up 
project directory (Dockerfile, config files and plugins, and full docker log 
output).

— Martijn

> On 20 Aug 2019, at 19:26, Arnold Bronley  wrote:
> 
> Hi,
> 
> I am using 8.2.0-slim version. I wrap it in my own image by specifying some
> additional settings in Dockerfile (all it does is specify a custom Solr
> home, copy my config files and custom Solr plugins to container and boot in
> SolrCloud mode).
> All things same, if I just change version from 8.2.0-slim to 8.1.1-slim
> then I do not get any such warning.
> 
> On Tue, Aug 20, 2019 at 5:01 AM Furkan KAMACI 
> wrote:
> 
>> Hi Arnold,
>> 
>> Such errors may arise due to file permission issues. I can run latest
>> version without of Solr via docker image without any errors. Could you
>> write which steps do you follow to run Solr docker?
>> 
>> Kind Regards,
>> Furkan KAMACI
>> 
>> On Tue, Aug 20, 2019 at 1:25 AM Arnold Bronley 
>> wrote:
>> 
>>> Hi,
>>> 
>>> I am getting following warning in Solr admin UI logs. I did not get this
>>> warning in Solr 8.1.1
>>> Please note that I am using Solr docker slim image from here -
>>> https://hub.docker.com/_/solr/
>>> 
>>> Unable to load jetty,​ not starting JettyAdminServer
>>> 
>> 

Re: Problems with restricting access to users using Basic auth

[Jason Gerlowski]

The "all" permissions _should_ block solr-user from accessing all of
those resources, and I believe it does in newer versions of Solr.
There was a bug with it that was fixed a few versions back though- it
sounds like you might be running into that. (see
https://issues.apache.org/jira/browse/SOLR-13355) What version of Solr
are you using?

Jason



On Wed, Aug 21, 2019 at 5:21 AM Salmaan Rashid Syed
 wrote:
>
> Hi Jason,
>
> Thanks for your prompt reply.
>
> Your code does address few of my concerns like restricting *solr-user* from
> accessing the dashboard and from executing other request methods apart from
> *"update"* and *"read"*.
>
> But I am still able to access other collections such as *"Collection3",
> "Collection4"* and so on, apart from the intended two collection entered in
> the code. I can give *"update"* and *"read" *requests to these external
> Collections which solr-user should not be able to do.
>
> Moreover solr-user can look at the
> *http://localhost:8983/solr/admin/authentication
> * link which lists the
> users and their *SHA256* coded passwords. How can I hide this and restrict
> access to other collections?
>
> Thanks and regards
> Salmaan
>
>
> On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski 
> wrote:
>
> > Hi Salmaan,
> >
> > Solr's RuleBasedAuthorizationPlugin allows requests through if none of
> > the specified permissions apply.  I think that's what you're running
> > into in your example above.  If you want to lockdown a particular API
> > (or set of APIs) then you need to explicitly add a permission that
> > restricts those APIs to a particular role.
> >
> > One way to get the behavior that it sounds like you're looking for
> > would be to add a catch-all permission at the bottom of your
> > permissions list that restricts all other APIs to "admin".  This would
> > look a bit like:
> >
> >  "permissions":[
> > {
> > "name":"security-edit",
> > "role":"admin"
> > },
> > {
> > "collection": ["Collection1", "Collection2"],
> > "name": ["update", "read"],
> > "role": "dev"
> > },
> > {
> > "name": "all",
> > "role": "admin"
> > }
> > ]
> >
> > Hope that helps get you started.
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Aug 20, 2019 at 3:19 AM Salmaan Rashid Syed
> >  wrote:
> > >
> > > Hi Solr Users,
> > >
> > > I want to create a user that has restricted access to Solr. I did the
> > > follwowing:-
> > >
> > >
> > >1. {
> > >2. "authentication":{
> > >3."blockUnknown": true,
> > >4."class":"solr.BasicAuthPlugin",
> > >5."credentials":{
> > >6. "solr-admin":
> > >"2IUJD9dxRhxSXaJGdMP5z8ggSn4I285Ty9GCWeRNMUg=
> > > /sSNJJufPtj4baRizoJshJawFsWvopvZSqZpQ/Nwd78="
> > >,
> > >7. "solr-user":
> > >"p+XwOh15p/rvFltv2LXP1CwtbvwBgGlC9qcDKxV73B4=
> > > DcNsjfA6Wf16V1XKT+YraosSFQ5Cr3eRUX6BQnx9XKA="
> > >
> > >8.  }
> > >9. },
> > >10. "authorization":{
> > >11."class":"solr.RuleBasedAuthorizationPlugin",
> > >12."user-role":{"solr-admin":"admin", "solr-user":"dev"},
> > >13."permissions":[
> > >14.   {
> > >15."name":"security-edit",
> > >16."role":"admin"
> > >17.   },
> > >18.   {
> > >19. "collection": ["Collection1", "Collection2"],
> > >20. "name": ["update", "read"],
> > >21. "role": "dev"
> > >22.   }
> > >23.   ]
> > >24. }}
> > >
> > >
> > > But when Login intot the Solr admin dash-board using Solr-user
> > credentials,
> > > I can read, select, write, update, delete collections and do all sorts of
> > > things like a solr-admin can do.
> > >
> > > I want solr-user to be able to access only *Collection1* and
> > *Collection2*
> > > and be able to only *update *and *read*. He should not be able to access
> > > other collections and do anything apart from the above mentioned role.
> > >
> > > Where am I exactly going wrong?
> > >
> > > Thanks and Regards,
> > > Salmaan
> >

Re: Problems with restricting access to users using Basic auth

[Salmaan Rashid Syed]

Hi Jason,

I am using version 7.6 of Solr.

Thanks,
Salmaan



On Wed, Aug 21, 2019 at 4:12 PM Jason Gerlowski 
wrote:

> The "all" permissions _should_ block solr-user from accessing all of
> those resources, and I believe it does in newer versions of Solr.
> There was a bug with it that was fixed a few versions back though- it
> sounds like you might be running into that. (see
> https://issues.apache.org/jira/browse/SOLR-13355) What version of Solr
> are you using?
>
> Jason
>
>
>
> On Wed, Aug 21, 2019 at 5:21 AM Salmaan Rashid Syed
>  wrote:
> >
> > Hi Jason,
> >
> > Thanks for your prompt reply.
> >
> > Your code does address few of my concerns like restricting *solr-user*
> from
> > accessing the dashboard and from executing other request methods apart
> from
> > *"update"* and *"read"*.
> >
> > But I am still able to access other collections such as *"Collection3",
> > "Collection4"* and so on, apart from the intended two collection entered
> in
> > the code. I can give *"update"* and *"read" *requests to these external
> > Collections which solr-user should not be able to do.
> >
> > Moreover solr-user can look at the
> > *http://localhost:8983/solr/admin/authentication
> > * link which lists the
> > users and their *SHA256* coded passwords. How can I hide this and
> restrict
> > access to other collections?
> >
> > Thanks and regards
> > Salmaan
> >
> >
> > On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski 
> > wrote:
> >
> > > Hi Salmaan,
> > >
> > > Solr's RuleBasedAuthorizationPlugin allows requests through if none of
> > > the specified permissions apply.  I think that's what you're running
> > > into in your example above.  If you want to lockdown a particular API
> > > (or set of APIs) then you need to explicitly add a permission that
> > > restricts those APIs to a particular role.
> > >
> > > One way to get the behavior that it sounds like you're looking for
> > > would be to add a catch-all permission at the bottom of your
> > > permissions list that restricts all other APIs to "admin".  This would
> > > look a bit like:
> > >
> > >  "permissions":[
> > > {
> > > "name":"security-edit",
> > > "role":"admin"
> > > },
> > > {
> > > "collection": ["Collection1", "Collection2"],
> > > "name": ["update", "read"],
> > > "role": "dev"
> > > },
> > > {
> > > "name": "all",
> > > "role": "admin"
> > > }
> > > ]
> > >
> > > Hope that helps get you started.
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > On Tue, Aug 20, 2019 at 3:19 AM Salmaan Rashid Syed
> > >  wrote:
> > > >
> > > > Hi Solr Users,
> > > >
> > > > I want to create a user that has restricted access to Solr. I did the
> > > > follwowing:-
> > > >
> > > >
> > > >1. {
> > > >2. "authentication":{
> > > >3."blockUnknown": true,
> > > >4."class":"solr.BasicAuthPlugin",
> > > >5."credentials":{
> > > >6. "solr-admin":
> > > >"2IUJD9dxRhxSXaJGdMP5z8ggSn4I285Ty9GCWeRNMUg=
> > > > /sSNJJufPtj4baRizoJshJawFsWvopvZSqZpQ/Nwd78="
> > > >,
> > > >7. "solr-user":
> > > >"p+XwOh15p/rvFltv2LXP1CwtbvwBgGlC9qcDKxV73B4=
> > > > DcNsjfA6Wf16V1XKT+YraosSFQ5Cr3eRUX6BQnx9XKA="
> > > >
> > > >8.  }
> > > >9. },
> > > >10. "authorization":{
> > > >11."class":"solr.RuleBasedAuthorizationPlugin",
> > > >12."user-role":{"solr-admin":"admin", "solr-user":"dev"},
> > > >13."permissions":[
> > > >14.   {
> > > >15."name":"security-edit",
> > > >16."role":"admin"
> > > >17.   },
> > > >18.   {
> > > >19. "collection": ["Collection1", "Collection2"],
> > > >20. "name": ["update", "read"],
> > > >21. "role": "dev"
> > > >22.   }
> > > >23.   ]
> > > >24. }}
> > > >
> > > >
> > > > But when Login intot the Solr admin dash-board using Solr-user
> > > credentials,
> > > > I can read, select, write, update, delete collections and do all
> sorts of
> > > > things like a solr-admin can do.
> > > >
> > > > I want solr-user to be able to access only *Collection1* and
> > > *Collection2*
> > > > and be able to only *update *and *read*. He should not be able to
> access
> > > > other collections and do anything apart from the above mentioned
> role.
> > > >
> > > > Where am I exactly going wrong?
> > > >
> > > > Thanks and Regards,
> > > > Salmaan
> > >
>

Re: Problems with restricting access to users using Basic auth

[Salmaan Rashid Syed]

Hi Jason,

Is there a way to fix this in version 7.6?

Or is it mandatory to upgrade to other versions?

If I have to upgrade to a higher version, then what is the best way to do
this without effecting the current configuration and indexed data?

Thanks,
Salmaan



On Wed, Aug 21, 2019 at 4:13 PM Salmaan Rashid Syed <
salmaan.ras...@mroads.com> wrote:

> Hi Jason,
>
> I am using version 7.6 of Solr.
>
> Thanks,
> Salmaan
>
>
>
> On Wed, Aug 21, 2019 at 4:12 PM Jason Gerlowski 
> wrote:
>
>> The "all" permissions _should_ block solr-user from accessing all of
>> those resources, and I believe it does in newer versions of Solr.
>> There was a bug with it that was fixed a few versions back though- it
>> sounds like you might be running into that. (see
>> https://issues.apache.org/jira/browse/SOLR-13355) What version of Solr
>> are you using?
>>
>> Jason
>>
>>
>>
>> On Wed, Aug 21, 2019 at 5:21 AM Salmaan Rashid Syed
>>  wrote:
>> >
>> > Hi Jason,
>> >
>> > Thanks for your prompt reply.
>> >
>> > Your code does address few of my concerns like restricting *solr-user*
>> from
>> > accessing the dashboard and from executing other request methods apart
>> from
>> > *"update"* and *"read"*.
>> >
>> > But I am still able to access other collections such as *"Collection3",
>> > "Collection4"* and so on, apart from the intended two collection
>> entered in
>> > the code. I can give *"update"* and *"read" *requests to these external
>> > Collections which solr-user should not be able to do.
>> >
>> > Moreover solr-user can look at the
>> > *http://localhost:8983/solr/admin/authentication
>> > * link which lists the
>> > users and their *SHA256* coded passwords. How can I hide this and
>> restrict
>> > access to other collections?
>> >
>> > Thanks and regards
>> > Salmaan
>> >
>> >
>> > On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski 
>> > wrote:
>> >
>> > > Hi Salmaan,
>> > >
>> > > Solr's RuleBasedAuthorizationPlugin allows requests through if none of
>> > > the specified permissions apply.  I think that's what you're running
>> > > into in your example above.  If you want to lockdown a particular API
>> > > (or set of APIs) then you need to explicitly add a permission that
>> > > restricts those APIs to a particular role.
>> > >
>> > > One way to get the behavior that it sounds like you're looking for
>> > > would be to add a catch-all permission at the bottom of your
>> > > permissions list that restricts all other APIs to "admin".  This would
>> > > look a bit like:
>> > >
>> > >  "permissions":[
>> > > {
>> > > "name":"security-edit",
>> > > "role":"admin"
>> > > },
>> > > {
>> > > "collection": ["Collection1", "Collection2"],
>> > > "name": ["update", "read"],
>> > > "role": "dev"
>> > > },
>> > > {
>> > > "name": "all",
>> > > "role": "admin"
>> > > }
>> > > ]
>> > >
>> > > Hope that helps get you started.
>> > >
>> > > Best,
>> > >
>> > > Jason
>> > >
>> > > On Tue, Aug 20, 2019 at 3:19 AM Salmaan Rashid Syed
>> > >  wrote:
>> > > >
>> > > > Hi Solr Users,
>> > > >
>> > > > I want to create a user that has restricted access to Solr. I did
>> the
>> > > > follwowing:-
>> > > >
>> > > >
>> > > >1. {
>> > > >2. "authentication":{
>> > > >3."blockUnknown": true,
>> > > >4."class":"solr.BasicAuthPlugin",
>> > > >5."credentials":{
>> > > >6. "solr-admin":
>> > > >"2IUJD9dxRhxSXaJGdMP5z8ggSn4I285Ty9GCWeRNMUg=
>> > > > /sSNJJufPtj4baRizoJshJawFsWvopvZSqZpQ/Nwd78="
>> > > >,
>> > > >7. "solr-user":
>> > > >"p+XwOh15p/rvFltv2LXP1CwtbvwBgGlC9qcDKxV73B4=
>> > > > DcNsjfA6Wf16V1XKT+YraosSFQ5Cr3eRUX6BQnx9XKA="
>> > > >
>> > > >8.  }
>> > > >9. },
>> > > >10. "authorization":{
>> > > >11."class":"solr.RuleBasedAuthorizationPlugin",
>> > > >12."user-role":{"solr-admin":"admin", "solr-user":"dev"},
>> > > >13."permissions":[
>> > > >14.   {
>> > > >15."name":"security-edit",
>> > > >16."role":"admin"
>> > > >17.   },
>> > > >18.   {
>> > > >19. "collection": ["Collection1", "Collection2"],
>> > > >20. "name": ["update", "read"],
>> > > >21. "role": "dev"
>> > > >22.   }
>> > > >23.   ]
>> > > >24. }}
>> > > >
>> > > >
>> > > > But when Login intot the Solr admin dash-board using Solr-user
>> > > credentials,
>> > > > I can read, select, write, update, delete collections and do all
>> sorts of
>> > > > things like a solr-admin can do.
>> > > >
>> > > > I want solr-user to be able to access only *Collection1* and
>> > > *Collection2*
>> > > > and be able to only *update *and *read*. He should not be able to
>> access
>> > > > other collections and do anything apart from the above mentioned
>> role.
>> > > >
>> > > > Where am I exactly going wrong?
>> > > >
>> > > > Thanks and Regards,
>> > > > Salma

Re: Problems with restricting access to users using Basic auth

[Jason Gerlowski]

Ah, ok.  SOLR-13355 still affects 7.6, so that explains why you're
seeing this behavior.

You could upgrade to get the new behavior, but you don't need to-
there's a workaround.  You just need to add a few extra rules to your
security.json.  The problem in SOLR-13355 is that the "all" permission
isn't being considered for APIs that are covered by other predefined
permissions.  So the workaround is to add a permission rule for each
of the predefined permissions, locking them down to the "admin" role.
It really bloats security.json, but should do the job.  So your
security.json should have a permissions section that looks like the
JSON below:

{"name": "dev-read", "collection": ["collection1", "collection2"],
"role": "dev"},
{"name": "security-edit", "role": "admin"},
{"name": "security-read", "role": "admin"},
{"name": "schema-edit", "role": "admin"},
{"name": "schema-read", "role": "admin"},
{"name": "config-edit", "role": "admin"},
{"name": "config-read", "role": "admin"},
{"name": "core-admin-edit", "role": "admin"},
{"name": "core-admin-read", "role": "admin"},
{"name": "collection-api-edit", "role": "admin"},
{"name": "collection-api-read", "role": "admin"},
{"name": "read", "role": "admin"},
{"name": "update", "role": "admin"},
{"name": "all", "role": "admin"}

Hope that helps.  Let me know if that still has any problems for you.

Jason

On Wed, Aug 21, 2019 at 6:48 AM Salmaan Rashid Syed
 wrote:
>
> Hi Jason,
>
> Is there a way to fix this in version 7.6?
>
> Or is it mandatory to upgrade to other versions?
>
> If I have to upgrade to a higher version, then what is the best way to do
> this without effecting the current configuration and indexed data?
>
> Thanks,
> Salmaan
>
>
>
> On Wed, Aug 21, 2019 at 4:13 PM Salmaan Rashid Syed <
> salmaan.ras...@mroads.com> wrote:
>
> > Hi Jason,
> >
> > I am using version 7.6 of Solr.
> >
> > Thanks,
> > Salmaan
> >
> >
> >
> > On Wed, Aug 21, 2019 at 4:12 PM Jason Gerlowski 
> > wrote:
> >
> >> The "all" permissions _should_ block solr-user from accessing all of
> >> those resources, and I believe it does in newer versions of Solr.
> >> There was a bug with it that was fixed a few versions back though- it
> >> sounds like you might be running into that. (see
> >> https://issues.apache.org/jira/browse/SOLR-13355) What version of Solr
> >> are you using?
> >>
> >> Jason
> >>
> >>
> >>
> >> On Wed, Aug 21, 2019 at 5:21 AM Salmaan Rashid Syed
> >>  wrote:
> >> >
> >> > Hi Jason,
> >> >
> >> > Thanks for your prompt reply.
> >> >
> >> > Your code does address few of my concerns like restricting *solr-user*
> >> from
> >> > accessing the dashboard and from executing other request methods apart
> >> from
> >> > *"update"* and *"read"*.
> >> >
> >> > But I am still able to access other collections such as *"Collection3",
> >> > "Collection4"* and so on, apart from the intended two collection
> >> entered in
> >> > the code. I can give *"update"* and *"read" *requests to these external
> >> > Collections which solr-user should not be able to do.
> >> >
> >> > Moreover solr-user can look at the
> >> > *http://localhost:8983/solr/admin/authentication
> >> > * link which lists the
> >> > users and their *SHA256* coded passwords. How can I hide this and
> >> restrict
> >> > access to other collections?
> >> >
> >> > Thanks and regards
> >> > Salmaan
> >> >
> >> >
> >> > On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski 
> >> > wrote:
> >> >
> >> > > Hi Salmaan,
> >> > >
> >> > > Solr's RuleBasedAuthorizationPlugin allows requests through if none of
> >> > > the specified permissions apply.  I think that's what you're running
> >> > > into in your example above.  If you want to lockdown a particular API
> >> > > (or set of APIs) then you need to explicitly add a permission that
> >> > > restricts those APIs to a particular role.
> >> > >
> >> > > One way to get the behavior that it sounds like you're looking for
> >> > > would be to add a catch-all permission at the bottom of your
> >> > > permissions list that restricts all other APIs to "admin".  This would
> >> > > look a bit like:
> >> > >
> >> > >  "permissions":[
> >> > > {
> >> > > "name":"security-edit",
> >> > > "role":"admin"
> >> > > },
> >> > > {
> >> > > "collection": ["Collection1", "Collection2"],
> >> > > "name": ["update", "read"],
> >> > > "role": "dev"
> >> > > },
> >> > > {
> >> > > "name": "all",
> >> > > "role": "admin"
> >> > > }
> >> > > ]
> >> > >
> >> > > Hope that helps get you started.
> >> > >
> >> > > Best,
> >> > >
> >> > > Jason
> >> > >
> >> > > On Tue, Aug 20, 2019 at 3:19 AM Salmaan Rashid Syed
> >> > >  wrote:
> >> > > >
> >> > > > Hi Solr Users,
> >> > > >
> >> > > > I want to create a user that has restricted access to Solr. I did
> >> the
> >> > > > follwowing:-
> >> > > >
> >> > > >
> >> > > >1. {
> >> > > >2. "authentication":{
> >> > > >3."bl

Re: question about solrCloud joining

[Mikhail Khludnev]

I'm not sure, but it might be an issue. It make sense to add negative test
and assert the exception at
https://github.com/apache/lucene-solr/blob/master/solr/core/src/test/org/apache/solr/cloud/DistribJoinFromCollectionTest.java

On Wed, Aug 21, 2019 at 10:37 AM Lisheng Wang 
wrote:

> Hi  Mikhail,
>
> okay.
>
> below is 2 requests:
>
> both are select from "movieDirectors" collection join "movies" collection
> which has 2 shards.
>
>
> http://localhost:8983/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
>
> http://localhost:8984/solr/movieDirectors/select?fq=%7B!join%20from%3Ddirector_id%20fromIndex%3Dmovies%20to%3Did%7Dtitle%3A%22Dunkirk%22&q=*%3A*
>
> first request can get result without Exception, response is following
>
> { "responseHeader":{ "zkConnected":true, "status":0, "QTime":3, "params":{
> "
> q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261450613"}}, "response":{"numFound":1,"
> start":0,"docs":[ { "id":"1", "name":"Christopher Nolan", "has_oscar":true,
> "_version_":1642343436642156544}] }}
>
> second request will get Exception
> { "responseHeader":{ "zkConnected":true, "status":400, "QTime":29,
> "params":{
> "q":"*:*", "fq":"{!join from=director_id fromIndex=movies
> to=id}title:\"Dunkirk\"", "_":"1566261620152"}}, "error":{ "metadata":[
> "error-class","org.apache.solr.common.SolrException", "root-error-class",
> "org.apache.solr.common.SolrException"], "msg":"SolrCloud join: multiple
> shards not yet supported movies", "code":400}}
>
> i don't know why get 2 different result when you request from different
> node, i think both should get Exception with "SolrCloud join: multiple
> shards not yet supported movies".
>
> Best,
> Lisheng
>
>
> Mikhail Khludnev  于2019年8月21日周三 下午3:19写道：
>
> > Ok. Still hard to follow. Can you clarify which collection you run these
> > queries on?
> > Collection name (url segment before /select) is more significant than any
> > port (jvm) identity.
> >
> > On Wed, Aug 21, 2019 at 5:14 AM Lisheng Wang 
> > wrote:
> >
> > > Hi Mikhail
> > >
> > > Thanks for your response,  but question is not related to "title:Get
> > Out",
> > > maybe i did not describe clearly.
> > >
> > > I knew solrCloud joining is not working in index which is splited to
> > > multiple shards.
> > >
> > > but why i run "*{!join from=director_id fromIndex=movies
> > > to=id}title:"Dunkirk"*" on 8984 (fromIndex=movies, movies has 2 shards)
> > i
> > > got exception "SolrCloud join: multiple shards not yet supported
> movies"
> > >
> > > but when run on 8983, i got result but it is incorrect without above
> > > exception. i think should get same exception no matter run joining on
> > 8983
> > > or 8984.
> > >
> > > Not sure my explanation is clear?
> > >
> > > Please kindly let me know if you have any question.
> > >
> > > Thanks!
> > >
> > > Lisheng
> > >
> > >
> > >
> > > Mikhail Khludnev  于2019年8月21日周三 上午4:41写道：
> > >
> > > > Hello, Lisheng.
> > > > I barely follow, but couldn't the space symbol in "title:Get Out"
> > > > cause the problem
> > > > ?
> > > > Check debugQuery and nested query in local param.
> > > >
> > > >
> > > > On Tue, Aug 20, 2019 at 6:35 PM Lisheng Wang <
> wanglishen...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Erick
> > > > >
> > > > > Thanks for your quick response and remaining me about attachment
> > issue.
> > > > >
> > > > > Yes, i run on 2 different jvms that not related to if they are on
> > same
> > > > > machine or not.
> > > > >
> > > > > let me describe my scenario, i have two collection:
> > > > >
> > > > > i start 2 nodes on my laptop on 2 different JVM, ports are 8983 and
> > > 8984.
> > > > >
> > > > > 1. movieDirectors: 1 shard, 2 replica, master is on 8984, slave is
> on
> > > > 8983
> > > > > 2. movies: 2 shard, 1 replica/shardshard1 is on 8983, shard2 is
> > on
> > > > > 8984.
> > > > >
> > > > > collection movieDirectors has 2 docs:
> > > > > {
> > > > > "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > > 1642343781358370816
> > > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > > 1642343828930166784
> > > > > }
> > > > > collection movies has 2 docs too:
> > > > > { "id":"1", "title":"Dunkirk", "director_id":"1", "_version_":
> > > > > 1642343781358370816
> > > > > }, { "id":"2", "title":"Get Out", "director_id":"2", "_version_":
> > > > > 1642343828930166784
> > > > > }
> > > > > everything is ok when i run query with "{!join from=id
> > > > > fromIndex=movieDirectors to=director_id}has_oscar:true" on both
> 8983
> > > and
> > > > > 8984, i can got expected result:
> > > > > { "responseHeader":{ "zkConnected":true, "status":0, "QTime":79,
> > > > "params":{
> > > > > "q":"*:*", "fq":"{!join from=id fromIndex=movieDirectors
> > > > > to=director_id}has_oscar:true", "_":"1566313944099"}},
> > > > > "response":{"numFound
> > > > > ":2,"start":0,"maxScore":1.0,"docs":[ { "i

Re: StackOverflowError leader election on 8.2.0

[Mikhail Khludnev]

>  Looking this up i found SOLR-5692, but that was solved a lifetime ago,

It wasn't.
https://issues.apache.org/jira/browse/SOLR-5692?focusedCommentId=14556876&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14556876

On Wed, Aug 21, 2019 at 1:29 PM Markus Jelsma 
wrote:

> Hello,
>
> Looking this up i found SOLR-5692, but that was solved a lifetime ago, so
> just checking if this is a familiar error and one i missing in Jira:
>
> A client's Solr 8.2.0 cluster brought us the next StackOverflowError while
> running 8.2.0 on Java 8:
>
> Exception in thread "coreZkRegister-1-thread-3"
> java.lang.StackOverflowError
> at
> org.apache.logging.log4j.ThreadContext.getImmutableContext(ThreadContext.java:352)
> at
> org.apache.logging.log4j.core.impl.ThreadContextDataInjector$ForDefaultThreadContextMap.injectContextData(ThreadContextDataInjector.java:66)
> at
> org.apache.logging.log4j.core.impl.Log4jLogEvent.createContextData(Log4jLogEvent.java:473)
> at
> org.apache.logging.log4j.core.impl.Log4jLogEvent.(Log4jLogEvent.java:331)
> at
> org.apache.logging.log4j.core.impl.DefaultLogEventFactory.createEvent(DefaultLogEventFactory.java:54)
> at
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:404)
> at
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
> at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
> at
> org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2170)
> at
> org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2125)
> at
> org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2108)
> at
> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2007)
> at
> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1866)
> at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:179)
> at org.apache.solr.update.PeerSync.sync(PeerSync.java:172)
> at
> org.apache.solr.cloud.SyncStrategy.syncWithReplicas(SyncStrategy.java:187)
> at
> org.apache.solr.cloud.SyncStrategy.syncReplicas(SyncStrategy.java:131)
> at org.apache.solr.cloud.SyncStrategy.sync(SyncStrategy.java:109)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:400)
> at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
> at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
> at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
> at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
> at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
> at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
> at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
> at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:137)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:309)
> at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:218)
>
> . it repeats hundreds of times
>
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:703)
> at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:449)
> at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:172)
> 

Solr master issue

[Akreeti Agarwal]

Hi,

I am facing issue on my solr master, getting this in logs:

2019-08-19 22:29:55.573 ERROR (qtp59559151-101) [   ] o.a.s.s.HttpSolrCall 
null:org.apache.solr.common.SolrException: SolrCore 'sitecore_contents_index' 
is not available due to init failure: Error opening new searcher
at 
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
at 
org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.(SolrCore.java:820)
at org.apache.solr.core.SolrCore.(SolrCore.java:658)
at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)
at 
org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
at 
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
at 
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at 
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1696)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1807)
at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:914)
at org.apache.solr.core.SolrCore.(SolrCore.java:793)
... 10 more
Caused by: java.nio.file.NoSuchFileException: 
/solr-m/server/solr/sitecore_contents_index_rebuild/data/index/_6lps.si
at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at 
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:237)
at 
org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:187)
at 
org.apache.lucene.store.Directory.openChecksumInput(Director

Re: Problems with restricting access to users using Basic auth

[Salmaan Rashid Syed]

Hi Jason,

Unfortunately, the above mentioned workaround does not work for me.

I modified your script a little bit as follow and it still doesn't work.

 {"name": "update", "collection": ["collection1", "collection2"],
"role": "dev"},
 {"name": "read", "collection": ["collection1", "collection2"],
"role": "dev"},
{"name": "security-edit", "role": "admin"},
{"name": "security-read", "role": "admin"},
{"name": "schema-edit", "role": "admin"},
{"name": "schema-read", "role": "admin"},
{"name": "config-edit", "role": "admin"},
{"name": "config-read", "role": "admin"},
{"name": "core-admin-edit", "role": "admin"},
{"name": "core-admin-read", "role": "admin"},
{"name": "collection-api-edit", "role": "admin"},
{"name": "collection-api-read", "role": "admin"},
{"name": "read", "role": "admin"},
{"name": "update", "role": "admin"},

{"name": "all", "role": "admin"}


I can still use "*update"* and "*read"* methods into other collections
using *solr-user *login credentials.

Any help is highly appreciated.

Regards,
Salmaan

On Wed, Aug 21, 2019 at 4:33 PM Jason Gerlowski 
wrote:

> Ah, ok.  SOLR-13355 still affects 7.6, so that explains why you're
> seeing this behavior.
>
> You could upgrade to get the new behavior, but you don't need to-
> there's a workaround.  You just need to add a few extra rules to your
> security.json.  The problem in SOLR-13355 is that the "all" permission
> isn't being considered for APIs that are covered by other predefined
> permissions.  So the workaround is to add a permission rule for each
> of the predefined permissions, locking them down to the "admin" role.
> It really bloats security.json, but should do the job.  So your
> security.json should have a permissions section that looks like the
> JSON below:
>
> {"name": "dev-read", "collection": ["collection1", "collection2"],
> "role": "dev"},
> {"name": "security-edit", "role": "admin"},
> {"name": "security-read", "role": "admin"},
> {"name": "schema-edit", "role": "admin"},
> {"name": "schema-read", "role": "admin"},
> {"name": "config-edit", "role": "admin"},
> {"name": "config-read", "role": "admin"},
> {"name": "core-admin-edit", "role": "admin"},
> {"name": "core-admin-read", "role": "admin"},
> {"name": "collection-api-edit", "role": "admin"},
> {"name": "collection-api-read", "role": "admin"},
> {"name": "read", "role": "admin"},
> {"name": "update", "role": "admin"},
> {"name": "all", "role": "admin"}
>
> Hope that helps.  Let me know if that still has any problems for you.
>
> Jason
>
> On Wed, Aug 21, 2019 at 6:48 AM Salmaan Rashid Syed
>  wrote:
> >
> > Hi Jason,
> >
> > Is there a way to fix this in version 7.6?
> >
> > Or is it mandatory to upgrade to other versions?
> >
> > If I have to upgrade to a higher version, then what is the best way to do
> > this without effecting the current configuration and indexed data?
> >
> > Thanks,
> > Salmaan
> >
> >
> >
> > On Wed, Aug 21, 2019 at 4:13 PM Salmaan Rashid Syed <
> > salmaan.ras...@mroads.com> wrote:
> >
> > > Hi Jason,
> > >
> > > I am using version 7.6 of Solr.
> > >
> > > Thanks,
> > > Salmaan
> > >
> > >
> > >
> > > On Wed, Aug 21, 2019 at 4:12 PM Jason Gerlowski  >
> > > wrote:
> > >
> > >> The "all" permissions _should_ block solr-user from accessing all of
> > >> those resources, and I believe it does in newer versions of Solr.
> > >> There was a bug with it that was fixed a few versions back though- it
> > >> sounds like you might be running into that. (see
> > >> https://issues.apache.org/jira/browse/SOLR-13355) What version of
> Solr
> > >> are you using?
> > >>
> > >> Jason
> > >>
> > >>
> > >>
> > >> On Wed, Aug 21, 2019 at 5:21 AM Salmaan Rashid Syed
> > >>  wrote:
> > >> >
> > >> > Hi Jason,
> > >> >
> > >> > Thanks for your prompt reply.
> > >> >
> > >> > Your code does address few of my concerns like restricting
> *solr-user*
> > >> from
> > >> > accessing the dashboard and from executing other request methods
> apart
> > >> from
> > >> > *"update"* and *"read"*.
> > >> >
> > >> > But I am still able to access other collections such as
> *"Collection3",
> > >> > "Collection4"* and so on, apart from the intended two collection
> > >> entered in
> > >> > the code. I can give *"update"* and *"read" *requests to these
> external
> > >> > Collections which solr-user should not be able to do.
> > >> >
> > >> > Moreover solr-user can look at the
> > >> > *http://localhost:8983/solr/admin/authentication
> > >> > * link which
> lists the
> > >> > users and their *SHA256* coded passwords. How can I hide this and
> > >> restrict
> > >> > access to other collections?
> > >> >
> > >> > Thanks and regards
> > >> > Salmaan
> > >> >
> > >> >
> > >> > On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski <
> gerlowsk...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi Salmaan,
> > >> > >
> > >> > > Solr's RuleBasedAuthorizationPlugin allows requests through if
> none of
> > >> > > the specified permissions apply.  I think th

Re: Solr master issue

[Atita Arora]

I think it would be useful to provide additional details like solr version,
when did you start seeing the problem? Did you upgrade lately?
Did you change anything?

On Wed, Aug 21, 2019 at 2:17 PM Akreeti Agarwal  wrote:

> Hi,
>
> I am facing issue on my solr master, getting this in logs:
>
> 2019-08-19 22:29:55.573 ERROR (qtp59559151-101) [   ] o.a.s.s.HttpSolrCall
> null:org.apache.solr.common.SolrException: SolrCore
> 'sitecore_contents_index' is not available due to init failure: Error
> opening new searcher
> at
> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
> at
> org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
> at
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.eclipse.jetty.io
> .AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.(SolrCore.java:820)
> at org.apache.solr.core.SolrCore.(SolrCore.java:658)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)
> at
> org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
> at
> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
> at
> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at
> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1696)
> at
> org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1807)
> at
> org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:914)
> at org.apache.solr.core.SolrCore.(SolrCore.java:793)
> ... 10 more
> Caused by: java.nio.file.NoSuchFileException:
> /solr-m/server/solr/sitecore_contents_index_rebuild/data/index/_6lps.si
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
> at java.nio.channels.FileChannel.open(F

RE: Invalid Date in Date Math String

[Grant Killian]

I happened to see this, Ritesh, so let me offer that this issue (#178247) is 
addressed in the Sitecore 9 version: 
https://dev.sitecore.net/Downloads/Sitecore%20Experience%20Platform/90/Sitecore%20Experience%20Platform%2090%20Update2/Release%20Notes.
 That fix, however, resolves the issue ONLY for Sitecore specific DateTime 
fields (__smallcreateddate, __smallupdateddate). Apparently, an additional 
patch is necessary to address other DateTime fields with Solr. This patch 
linked here may resolve the issue: 
https://sitecore.box.com/s/y4oct5rpf1022hrez3n4gb5fdkc7obyp. Otherwise, open a 
support ticket with Sitecore and they'll get you squared away.

Thanks,

Grant

-Original Message-
From: Ritesh Kumar (Avanade) 
Sent: Tuesday, August 20, 2019 7:39 PM
To: solr-user@lucene.apache.org
Subject: RE: Invalid Date in Date Math String

Thanks Eric for the response!

Regards
Ritesh


-Original Message-
From: Erick Erickson 
Sent: Tuesday, August 20, 2019 4:34 PM
To: solr-user@lucene.apache.org
Subject: Re: Invalid Date in Date Math String

You probably need to talk to the SiteCore people. The Apache project doesn’t 
have much insight into Sitecore, nor does it maintain the .net version of a 
Solr client…

Best,
Erick

> On Aug 20, 2019, at 7:16 PM, Ritesh Kumar (Avanade) 
>  wrote:
>
> Hello,
>
> We have been using solr 6.6.3 with Sitecore 9, however currently we seem to 
> be observing the below issue while rebuilding the index "Invalid Date in Date 
> Math String"
> We already applied the patch located at the below location but it did not fix 
> the issue:
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2FSitecoreSupport%2FSitecore.Support.178247%2Freleases%2Ftag%2F
> 8.2.2.0&data=02%7C01%7Cv-kurite%40microsoft.com%7Cf8ba7751ef5b413c
> c3de08d725c6da9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63701940
> 8314524415&sdata=Y6rsPNdcpjIvgPFAGVsLYFJqW01fjkcGsxUqKloyvYc%3D&am
> p;reserved=0
>
> Please let me know if anyone was able to fix this issue with any other
> method. (Note: I have updated my Sitecore support ticket as well with
> the same details)
>
> 5980 22:10:20 WARN  [Index=xxx _index] Crawler : AddRecursive
> DoItemAdd failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
> Exception: SolrNet.Exceptions.SolrConnectionException
> Message:name="responseHeader">400 name="QTime">325 name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException  
> name="root-error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException  name="msg">7 Async exceptions during distributed update:
> Error from server at x.x.x.x:8983/solr/_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/_index_shard2_replica2/update?update.distrib
> =TOLEADER&distrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%
> _index_shard1_replica3%2F&wt=javabin&version=2
> Remote error message: Invalid Date in Date Math String:'20180730T164926Z'
> Error from server at
> x.x.x.x:8983/solr/x_index_shard1_replica2: Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=
> TOLEADER&distrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2F&wt=javabin&version=2
> Remote error message: Invalid Date in Date Math String:'20170901T183400Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=
> TOLEADER&distrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2F&wt=javabin&version=2
> Remote error message: Invalid Date in Date Math String:'20171026T181027Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=
> TOLEADER&distrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2F&wt=javabin&version=2
> Remote error message: Invalid Date in Date Math String:'20171028T07

Re: Solr master issue

[Andrea Gazzarini]

I bet this is the problem:

java.nio.file.NoSuchFileException: /solr-m/server/solr/sitecore_c

Do you have any idea about why Solr us not finding that datafile?

Andrea

On Wed, 21 Aug 2019, 14:41 Atita Arora,  wrote:

> I think it would be useful to provide additional details like solr version,
> when did you start seeing the problem? Did you upgrade lately?
> Did you change anything?
>
> On Wed, Aug 21, 2019 at 2:17 PM Akreeti Agarwal  wrote:
>
> > Hi,
> >
> > I am facing issue on my solr master, getting this in logs:
> >
> > 2019-08-19 22:29:55.573 ERROR (qtp59559151-101) [   ]
> o.a.s.s.HttpSolrCall
> > null:org.apache.solr.common.SolrException: SolrCore
> > 'sitecore_contents_index' is not available due to init failure: Error
> > opening new searcher
> > at
> > org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
> > at
> > org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
> > at
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> > at
> org.eclipse.jetty.server.Server.handle(Server.java:499)
> > at
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> > at
> >
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> > at org.eclipse.jetty.io
> > .AbstractConnection$2.run(AbstractConnection.java:540)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> > at java.lang.Thread.run(Thread.java:748)
> > Caused by: org.apache.solr.common.SolrException: Error opening new
> searcher
> > at
> org.apache.solr.core.SolrCore.(SolrCore.java:820)
> > at
> org.apache.solr.core.SolrCore.(SolrCore.java:658)
> > at
> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)
> > at
> > org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
> > at
> > org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
> > at
> > org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
> > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> >
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > ... 1 more
> > Caused by: org.apache.solr.common.SolrException: Error opening new
> searcher
> > at
> > org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1696)
> > at
> > org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1807)
> > at
> > org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:914)
> > at
> org.apache.solr.core.SolrCore.(SolrCore.java:793)
> > ... 10 more
> > Caused by: java.nio.file.NoSuchFileException:
> > /solr-m/server/solr/sitecore_contents_index_rebuild/data/

Re: Solr master issue

[Erick Erickson]

You’d also probably get more information from the Sitecore folks, Apache Solr 
is used by Sitecore, but doesn’t know much about Sitecore issues.

Offhand I’d check if your disk is at or near full. Going just from the 
“rebuild” mention, looks like the index is being recreated, in which case there 
needs to be _at least_ as much free space on the disk as the index consumes.

> On Aug 21, 2019, at 9:26 AM, Andrea Gazzarini  wrote:
> 
> I bet this is the problem:
> 
> java.nio.file.NoSuchFileException: /solr-m/server/solr/sitecore_c
> 
> Do you have any idea about why Solr us not finding that datafile?
> 
> Andrea
> 
> On Wed, 21 Aug 2019, 14:41 Atita Arora,  wrote:
> 
>> I think it would be useful to provide additional details like solr version,
>> when did you start seeing the problem? Did you upgrade lately?
>> Did you change anything?
>> 
>> On Wed, Aug 21, 2019 at 2:17 PM Akreeti Agarwal  wrote:
>> 
>>> Hi,
>>> 
>>> I am facing issue on my solr master, getting this in logs:
>>> 
>>> 2019-08-19 22:29:55.573 ERROR (qtp59559151-101) [   ]
>> o.a.s.s.HttpSolrCall
>>> null:org.apache.solr.common.SolrException: SolrCore
>>> 'sitecore_contents_index' is not available due to init failure: Error
>>> opening new searcher
>>>at
>>> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
>>>at
>>> org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
>>>at
>>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
>>>at
>>> 
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
>>>at
>>> 
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
>>>at
>>> 
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>>>at
>>> 
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>>>at
>>> 
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>>>at
>>> 
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>>>at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>>>at
>>> 
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>>>at
>>> 
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>>>at
>> org.eclipse.jetty.server.Server.handle(Server.java:499)
>>>at
>>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>>>at
>>> 
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>>>at org.eclipse.jetty.io
>>> .AbstractConnection$2.run(AbstractConnection.java:540)
>>>at
>>> 
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>>>at
>>> 
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>>>at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.solr.common.SolrException: Error opening new
>> searcher
>>>at
>> org.apache.solr.core.SolrCore.(SolrCore.java:820)
>>>at
>> org.apache.solr.core.SolrCore.(SolrCore.java:658)
>>>at
>>> org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)
>>>at
>>> org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
>>>at
>>> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
>>>at
>>> org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
>>>at
>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>at
>>> 
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>>>at
>>> 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>at
>>> 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>... 1 more
>>> Caused by: org.apache.solr.common.SolrException: Error opening new
>> searcher
>>> 

RE: Solr master issue

[Akreeti Agarwal]

SOLR version is : solr-5.5.5

Thanks & Regards,
Akreeti Agarwal


-Original Message-
From: Akreeti Agarwal  
Sent: Wednesday, August 21, 2019 5:47 PM
To: solr-user@lucene.apache.org
Subject: Solr master issue

Hi,

I am facing issue on my solr master, getting this in logs:

2019-08-19 22:29:55.573 ERROR (qtp59559151-101) [   ] o.a.s.s.HttpSolrCall 
null:org.apache.solr.common.SolrException: SolrCore 'sitecore_contents_index' 
is not available due to init failure: Error opening new searcher
at 
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1071)
at 
org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.(SolrCore.java:820)
at org.apache.solr.core.SolrCore.(SolrCore.java:658)
at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:820)
at 
org.apache.solr.core.CoreContainer.access$000(CoreContainer.java:90)
at 
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:473)
at 
org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:464)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at 
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1696)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1807)
at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:914)
at org.apache.solr.core.SolrCore.(SolrCore.java:793)
... 10 more
Caused by: java.nio.file.NoSuchFileException: 
/solr-m/server/solr/sitecore_contents_index_rebuild/data/index/_6lps.si
at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at 
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at 
org.apache.lucene.store.MMapDirectory.

RE: Invalid Date in Date Math String

[Ritesh Kumar (Avanade)]

Hi Grant,

Thank you for the response!

The Sitecore version is 9.0 update 2 and we already have applied the patch you 
mentioned below (which I was referring earlier) but still the issue (random 
times) would be observed sometimes. 

Note: We have to restart the solr-zk ensemble + Sitecore VM to have a 
successful rebuild without this error.

Thanks
Ritesh


-Original Message-
From: Grant Killian  
Sent: Wednesday, August 21, 2019 6:12 AM
To: solr-user@lucene.apache.org
Subject: RE: Invalid Date in Date Math String

I happened to see this, Ritesh, so let me offer that this issue (#178247) is 
addressed in the Sitecore 9 version: 
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.sitecore.net%2FDownloads%2FSitecore%2520Experience%2520Platform%2F90%2FSitecore%2520Experience%2520Platform%252090%2520Update2%2FRelease%2520Notes&data=02%7C01%7Cv-kurite%40microsoft.com%7C63a7c4eb202c4debc1d908d726392e39%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C637019899347402934&sdata=7CWVpIIIpketown%2BwomAo%2FMa15ceteUZIKwmeYdo4sw%3D&reserved=0.
 That fix, however, resolves the issue ONLY for Sitecore specific DateTime 
fields (__smallcreateddate, __smallupdateddate). Apparently, an additional 
patch is necessary to address other DateTime fields with Solr. This patch 
linked here may resolve the issue: 
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsitecore.box.com%2Fs%2Fy4oct5rpf1022hrez3n4gb5fdkc7obyp&data=02%7C01%7Cv-kurite%40microsoft.com%7C63a7c4eb202c4debc1d908d726392e39%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C637019899347402934&sdata=OT4RtWsZkJISG26Ow4Vaq11Y6yl45fs1b7c459E4Q8Q%3D&reserved=0.
 Otherwise, open a support ticket with Sitecore and they'll get you squared 
away.

Thanks,

Grant

-Original Message-
From: Ritesh Kumar (Avanade) 
Sent: Tuesday, August 20, 2019 7:39 PM
To: solr-user@lucene.apache.org
Subject: RE: Invalid Date in Date Math String

Thanks Eric for the response!

Regards
Ritesh


-Original Message-
From: Erick Erickson 
Sent: Tuesday, August 20, 2019 4:34 PM
To: solr-user@lucene.apache.org
Subject: Re: Invalid Date in Date Math String

You probably need to talk to the SiteCore people. The Apache project doesn’t 
have much insight into Sitecore, nor does it maintain the .net version of a 
Solr client…

Best,
Erick

> On Aug 20, 2019, at 7:16 PM, Ritesh Kumar (Avanade) 
>  wrote:
>
> Hello,
>
> We have been using solr 6.6.3 with Sitecore 9, however currently we seem to 
> be observing the below issue while rebuilding the index "Invalid Date in Date 
> Math String"
> We already applied the patch located at the below location but it did not fix 
> the issue:
> https://gith
> ub.com%2FSitecoreSupport%2FSitecore.Support.178247%2Freleases%2Ftag%2F
> 8.2.2.0&data=02%7C01%7Cv-kurite%40microsoft.com%7Cf8ba7751ef5b413c
> c3de08d725c6da9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63701940
> 8314524415&sdata=Y6rsPNdcpjIvgPFAGVsLYFJqW01fjkcGsxUqKloyvYc%3D&am
> p;reserved=0
>
> Please let me know if anyone was able to fix this issue with any other 
> method. (Note: I have updated my Sitecore support ticket as well with 
> the same details)
>
> 5980 22:10:20 WARN  [Index=xxx _index] Crawler : AddRecursive 
> DoItemAdd failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
> Exception: SolrNet.Exceptions.SolrConnectionException
> Message:name="responseHeader">400 name="QTime">325 name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException  
> name="root-error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException  name="msg">7 Async exceptions during distributed update:
> Error from server at x.x.x.x:8983/solr/_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/_index_shard2_replica2/update?update.distrib
> =TOLEADER&distrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%
> _index_shard1_replica3%2F&wt=javabin&version=2
> Remote error message: Invalid Date in Date Math String:'20180730T164926Z'
> Error from server at
> x.x.x.x:8983/solr/x_index_shard1_replica2: Bad Reques

Re: sorl 8.1.1 all of sudden requiring https

[Russell Bahr]

Hi Jorn,
I was finally able to get the entire cluster back up on http.  I ended up
having to reload the collection from the Web interface on each individual
server and then restart the members 1 at a time until 1 of them came up as
master and was able to locate and load the core.
I am still not sure why it went into such a weird state in the first place
but I guess good for troubleshooting incase something goes awry in the
future.
I am now going to set up HTTPs and make sure all is working then will
continue with testing and tuning.
Thank you for your help and guidance.
Russ

*Manzama*a MODERN GOVERNANCE company

Russell Bahr
Lead Infrastructure Engineer

USA & CAN Office: +1 (541) 306 3271
USA & CAN Support: +1 (541) 706 9393
UK Office & Support: +44 (0)203 282 1633
AUS Office & Support: +61 (0) 2 8417 2339

543 NW York Drive, Suite 100, Bend, OR 97703

LinkedIn  | Twitter
 | Facebook
 | YouTube



On Tue, Aug 20, 2019 at 11:49 PM Jörn Franke  wrote:

> This is strange that it does not work. Have you checked the logs if there
> is a different error message?
> Here is the guide:
> https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html
>
> Please do not use self signed certificates, but certificates signed by the
> CA of your company.
>
> > Am 21.08.2019 um 01:00 schrieb Russell Bahr :
> >
> > I was able to change this setting in Zookeeper, and also using the Solr
> > REST interface. But even after restarting and verifying that the
> urlScheme
> > in clusterprops.json had http and not https, each of the servers still
> are
> > trying to talk to eachother and on the Cloud Graph are still showing as
> > https.  I am going to try to set up security and see if things are back
> to
> > working properly after I do that.  Is there a step by step guide or video
> > how to on setting up ssl on solr cloud with external zookeeper?  I have
> > searched and not found anything for 6.6 or newer to follow.  hopefully
> > something specific to 8.1.  Can you recommend something?
> >
> >
> > *Manzama*a MODERN GOVERNANCE company
> >
> > Russell Bahr
> > Lead Infrastructure Engineer
> >
> > USA & CAN Office: +1 (541) 306 3271
> > USA & CAN Support: +1 (541) 706 9393
> > UK Office & Support: +44 (0)203 282 1633
> > AUS Office & Support: +61 (0) 2 8417 2339
> >
> > 543 NW York Drive, Suite 100, Bend, OR 97703
> >
> > LinkedIn  | Twitter
> >  | Facebook
> >  | YouTube
> > 
> >
> >
> >> On Tue, Aug 20, 2019 at 2:31 PM Russell Bahr  wrote:
> >>
> >> I am a very new to both solr and zookeeper.  I have inherited the
> existing
> >> clusters from my predecessor without any training or explanation before
> he
> >> left.  I have modeled the new zookeepers and solr cloud off of the basic
> >> configuration that was in place in the existing cluster and zookeepers.
> >>
> >> I loaded my collection into zookeeper, but did not set any security in
> >> zookeeper.  is this possibly a bug in zookeeper then?  how would I
> change
> >> it back to http for the time being to complete my other testing?  If I
> do
> >> enable HTTPs will I have to rebuild anyway?
> >>
> >> *Manzama*
> >> a MODERN GOVERNANCE company
> >>
> >> Russell Bahr
> >> Lead Infrastructure Engineer
> >>
> >> USA & CAN Office: +1 (541) 306 3271
> >> USA & CAN Support: +1 (541) 706 9393
> >> UK Office & Support: +44 (0)203 282 1633
> >> AUS Office & Support: +61 (0) 2 8417 2339
> >>
> >> 543 NW York Drive, Suite 100, Bend, OR 97703
> >>
> >> LinkedIn  | Twitter
> >>  | Facebook
> >>  | YouTube
> >> 
> >>
> >>
> >>> On Tue, Aug 20, 2019 at 1:49 PM Jörn Franke 
> wrote:
> >>>
> >>> You should definitely enable HTTPs even if it is not exposed to the
> >>> Internet. Even within your own company network it is a good security
> >>> practice to enable HTTPs.
> >>> About your error. This is due to a setting in Zookeeper:
> >>>
> >>>
> https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html#EnablingSSL-ConfigureZooKeeper
> >>>
> >>> I do not recommend to do any setting for SolrCloud directly in
> Zookeeper,
> >>> but use the corresponding Solr REST interfaces, e.g. for urlScheme in
> >>> clusterprops.json
> >>>
> >>>
> https://lucene.apache.org/solr/guide/8_1/collections-api.html#clusterprop-parameters
> >>>
>  Am 20.08.2019 um 21:46 schrieb Russell Bahr :
> 
>  Hi All,
>  I have a solr 8.1.1 cloud that I set up and have been testing to
> replace
>  our solr 4.10.4 cloud implementation.
> 
>  I have been testing on the solr 8.1.1 cloud for a couple of weeks now
> >

Solr process getting killed suddenly

[Adriano Rogério de O . Carolino de Melo]

Hi, does anybody know why Solr Java process is terminated with no reason?
The OOM script does not run. In the server logs does not show anything.
Solr is running with 25g of Java heap and only using 20%.

-- 
*Adriano Melo*
Tel.: (83) 98875-1868

Re: Solr process getting killed suddenly

[Anshum Gupta]

Hi Adriano,

Can you provide more information around what are you doing? Answers to
questions like the following would be very useful for anyone who might try
to help you here:

1. What version of Solr are you using?
2. Is this vanilla, or are you using something custom w.r.t. the code as
well as config.
3. How are you running Solr? Do you see any logs at all, if so, can you
share those?

These are just some of the questions you can provide the answer to. Feel
free to add to these.

Also, is there a specific reason for you to run w/ a 25G heap ?

On Wed, Aug 21, 2019 at 3:52 PM Adriano Rogério de O. Carolino de Melo <
carol...@gmail.com> wrote:

> Hi, does anybody know why Solr Java process is terminated with no reason?
> The OOM script does not run. In the server logs does not show anything.
> Solr is running with 25g of Java heap and only using 20%.
>
> --
> *Adriano Melo*
> Tel.: (83) 98875-1868
>


-- 
Anshum Gupta

Re: Solr process getting killed suddenly

[Shawn Heisey]

On 8/21/2019 4:44 PM, Adriano Rogério de O. Carolino de Melo wrote:

Hi, does anybody know why Solr Java process is terminated with no reason?


What OS is it running on?  Maybe it's the OS killing it.  How did you 
install/start Solr, and what version is it?



The OOM script does not run. In the server logs does not show anything.
Solr is running with 25g of Java heap and only using 20%.


Can you share the GC logs that Solr writes?  Another piece of 
information that would be extremely useful is the screenshot described here:


https://cwiki.apache.org/confluence/display/solr/SolrPerformanceProblems#SolrPerformanceProblems-Askingforhelponamemory/performanceissue

Be sure that the listing is sorted according to the description there. 
There are instructions for Windows as well as most other operating systems.


You will need to use a file sharing site to share logs and screenshots. 
The mailing list almost never allows attachments through.


Thanks,
Shawn

Solr 8 Nested Documents Highlighting

[Eichstädt , Konrad]

Hi Everybody,

we are using Apache Solr strongly with nested documents feature.  But in 
version 8 we noticed that the highlighting which has been worked in Version 7 
doesn't work properly in Version 8 . It works only for root parent element but 
not for childs? Is this an known open issue or should I open an ticket which 
describes the problem in more detail?

Many Thanks

Best regards

Konrad

Konrad Eichstädt

Abt. Informations- und Datenmanagement
IDM 2.2 Management fachspezifischer Nachweissysteme
Softwarearchitekt

Staatsbibliothek zu Berlin - Preußischer Kulturbesitz
Potsdamer Straße 33
10785 Berlin
Tel: +49 030 266 43

Re: Solr 7.6 JoinQueryParser with Muti Threaded Facet stops solr core with Exceptions

[harjagsbby]

Few deatils added. 

public void close() {
int count = refCount.decrementAndGet();
if (count > 0) return; // close is called often, and only actually
closes if nothing is using it.
if (count < 0) {
  log.error("Too many close [count:{}] on {}. Please report this
exception to solr-user@lucene.apache.org", count, this );
  assert false : "Too many closes on SolrCore";
  return;
}
log.info("{} CLOSING SolrCore {}", logid, this);

// stop reporting metrics
try {
  coreMetricManager.close();
} catch (Throwable e) {
  SolrException.log(log, e);
  if (e instanceof  Error) {
throw (Error) e;
  }
}}

The above code has refCount in SolrCore.java this atomicRef is in negative
when we do multi threaded facets and joins.

Can anyone explain the semaphore implemenattion here. What would be scenario
when this can go below 0. I can clearly see each thread of multithreaded
facet trying to close the core.

Every Thread of facet tries to call createWeight of JoinQParserPlugin and
attaches closeHooks which getscalled from SolrRequestInfo.clearRequestInfo()
.

if (fromRef != null) {
  final RefCounted ref = fromRef;
  info.addCloseHook(new Closeable() {
@Override
public void close() {
  ref.decref();
}
  });
}

info.addCloseHook(new Closeable() {
  @Override
  public void close() {
fromCore.close();
  }
});



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7.6 JoinQueryParser with Muti Threaded Facet stops solr core with Exceptions

[harjagsbby]

Few deatils added. 

public void close() {
int count = refCount.decrementAndGet();
if (count > 0) return; // close is called often, and only actually
closes if nothing is using it.
if (count < 0) {
  log.error("Too many close [count:{}] on {}. Please report this
exception to solr-user@lucene.apache.org", count, this );
  assert false : "Too many closes on SolrCore";
  return;
}
log.info("{} CLOSING SolrCore {}", logid, this);

// stop reporting metrics
try {
  coreMetricManager.close();
} catch (Throwable e) {
  SolrException.log(log, e);
  if (e instanceof  Error) {
throw (Error) e;
  }
}}

The above code has refCount in SolrCore.java this atomicRef is in negative
when we do multi threaded facets and joins.

Can anyone explain the semaphore implemenattion here. What would be scenario
when this can go below 0. I can clearly see each thread of multithreaded
facet trying to close the core.

Every Thread of facet tries to call createWeight of JoinQParserPlugin and
attaches closeHooks which getscalled from SolrRequestInfo.clearRequestInfo()
.

if (fromRef != null) {
  final RefCounted ref = fromRef;
  info.addCloseHook(new Closeable() {
@Override
public void close() {
  ref.decref();
}
  });
}

info.addCloseHook(new Closeable() {
  @Override
  public void close() {
fromCore.close();
  }
});



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Problems with restricting access to users using Basic auth

[Salmaan Rashid Syed]

Hi,

Any suggestions as to what can be done?

Regards,
Salmaan


On Wed, Aug 21, 2019 at 4:33 PM Jason Gerlowski 
wrote:

> Ah, ok.  SOLR-13355 still affects 7.6, so that explains why you're
> seeing this behavior.
>
> You could upgrade to get the new behavior, but you don't need to-
> there's a workaround.  You just need to add a few extra rules to your
> security.json.  The problem in SOLR-13355 is that the "all" permission
> isn't being considered for APIs that are covered by other predefined
> permissions.  So the workaround is to add a permission rule for each
> of the predefined permissions, locking them down to the "admin" role.
> It really bloats security.json, but should do the job.  So your
> security.json should have a permissions section that looks like the
> JSON below:
>
> {"name": "dev-read", "collection": ["collection1", "collection2"],
> "role": "dev"},
> {"name": "security-edit", "role": "admin"},
> {"name": "security-read", "role": "admin"},
> {"name": "schema-edit", "role": "admin"},
> {"name": "schema-read", "role": "admin"},
> {"name": "config-edit", "role": "admin"},
> {"name": "config-read", "role": "admin"},
> {"name": "core-admin-edit", "role": "admin"},
> {"name": "core-admin-read", "role": "admin"},
> {"name": "collection-api-edit", "role": "admin"},
> {"name": "collection-api-read", "role": "admin"},
> {"name": "read", "role": "admin"},
> {"name": "update", "role": "admin"},
> {"name": "all", "role": "admin"}
>
> Hope that helps.  Let me know if that still has any problems for you.
>
> Jason
>
> On Wed, Aug 21, 2019 at 6:48 AM Salmaan Rashid Syed
>  wrote:
> >
> > Hi Jason,
> >
> > Is there a way to fix this in version 7.6?
> >
> > Or is it mandatory to upgrade to other versions?
> >
> > If I have to upgrade to a higher version, then what is the best way to do
> > this without effecting the current configuration and indexed data?
> >
> > Thanks,
> > Salmaan
> >
> >
> >
> > On Wed, Aug 21, 2019 at 4:13 PM Salmaan Rashid Syed <
> > salmaan.ras...@mroads.com> wrote:
> >
> > > Hi Jason,
> > >
> > > I am using version 7.6 of Solr.
> > >
> > > Thanks,
> > > Salmaan
> > >
> > >
> > >
> > > On Wed, Aug 21, 2019 at 4:12 PM Jason Gerlowski  >
> > > wrote:
> > >
> > >> The "all" permissions _should_ block solr-user from accessing all of
> > >> those resources, and I believe it does in newer versions of Solr.
> > >> There was a bug with it that was fixed a few versions back though- it
> > >> sounds like you might be running into that. (see
> > >> https://issues.apache.org/jira/browse/SOLR-13355) What version of
> Solr
> > >> are you using?
> > >>
> > >> Jason
> > >>
> > >>
> > >>
> > >> On Wed, Aug 21, 2019 at 5:21 AM Salmaan Rashid Syed
> > >>  wrote:
> > >> >
> > >> > Hi Jason,
> > >> >
> > >> > Thanks for your prompt reply.
> > >> >
> > >> > Your code does address few of my concerns like restricting
> *solr-user*
> > >> from
> > >> > accessing the dashboard and from executing other request methods
> apart
> > >> from
> > >> > *"update"* and *"read"*.
> > >> >
> > >> > But I am still able to access other collections such as
> *"Collection3",
> > >> > "Collection4"* and so on, apart from the intended two collection
> > >> entered in
> > >> > the code. I can give *"update"* and *"read" *requests to these
> external
> > >> > Collections which solr-user should not be able to do.
> > >> >
> > >> > Moreover solr-user can look at the
> > >> > *http://localhost:8983/solr/admin/authentication
> > >> > * link which
> lists the
> > >> > users and their *SHA256* coded passwords. How can I hide this and
> > >> restrict
> > >> > access to other collections?
> > >> >
> > >> > Thanks and regards
> > >> > Salmaan
> > >> >
> > >> >
> > >> > On Wed, Aug 21, 2019 at 5:07 AM Jason Gerlowski <
> gerlowsk...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi Salmaan,
> > >> > >
> > >> > > Solr's RuleBasedAuthorizationPlugin allows requests through if
> none of
> > >> > > the specified permissions apply.  I think that's what you're
> running
> > >> > > into in your example above.  If you want to lockdown a particular
> API
> > >> > > (or set of APIs) then you need to explicitly add a permission that
> > >> > > restricts those APIs to a particular role.
> > >> > >
> > >> > > One way to get the behavior that it sounds like you're looking for
> > >> > > would be to add a catch-all permission at the bottom of your
> > >> > > permissions list that restricts all other APIs to "admin".  This
> would
> > >> > > look a bit like:
> > >> > >
> > >> > >  "permissions":[
> > >> > > {
> > >> > > "name":"security-edit",
> > >> > > "role":"admin"
> > >> > > },
> > >> > > {
> > >> > > "collection": ["Collection1", "Collection2"],
> > >> > > "name": ["update", "read"],
> > >> > > "role": "dev"
> > >> > > },
> > >> > > {
> > >> > > "name": "all",
> > >> > > "role": "admin"
> > >> > > }
> > >> > > ]
> > >