Yes, I see the same issue. I'll update the JIRA and drill down. Thanks.
On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <ans...@anshumgupta.net>
wrote:
> To restart solr, you should instead use something like:
> bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> or
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
>
> I've seen others report the same exception but never ran into this one
> myself. Let me try this out.
>
>
>
> On Mon, Nov 23, 2015 at 2:55 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> craig.oak...@nih.gov> wrote:
>
>> FWIW
>>
>> I am getting fairly consistent results that if I follow the SOLR-8326
>> procedure just up through the step of "solr-5.3.1/bin/solr start -e cloud
>> -z localhost:2181": if I then stop just one node (either "./solr stop -p
>> 7574" or "./solr stop -p 8983") and then restart that same node (using the
>> command suggested by "solr-5.3.1/bin/solr start -e cloud -z
>> localhost:2181"), then the solr.log for the stopped-and-restarted node gets
>> such stack traces as
>> ERROR - 2015-11-23 21:49:28.663; [c:gettingstarted s:shard2 r:core_node3
>> x:gettingstarted_shard2_replica2] org.apache.solr.common.SolrException;
>> Error while trying to recover.:java.util.concurrent.ExecutionException:
>> org.apache.http.ParseException: Invalid content type:
>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
>> at
>> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>> Caused by: org.apache.http.ParseException: Invalid content type:
>> at org.apache.http.entity.ContentType.parse(ContentType.java:273)
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
>> at
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> While the node which stayed up the whole time starts getting such stack
>> traces as
>> ERROR - 2015-11-23 21:57:46.019; [c:gettingstarted s:shard2 r:core_node3
>> x:gettingstarted_shard2_replica2]
>> org.apache.solr.security.PKIAuthenticationPlugin; Invalid time r?
>> java.lang.NumberFormatException: For input string: "r?"
>> at
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>> at java.lang.Long.parseLong(Long.java:589)
>> at java.lang.Long.parseLong(Long.java:631)
>> at
>> org.apache.solr.security.PKIAuthenticationPlugin.doAuthenticate(PKIAuthenticationPlugin.java:128)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:252)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:186)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>> at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>> at org.eclipse.jetty.server.Server.handle(Server.java:499)
>> at
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>> at
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>> at
>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> In this case the string is just "r?", but usually it is a longer string
>> of control characters.
>>
>> If I shutdown _both_ nodes and restart _one_, and then allow it to be
>> "Waiting until we see more replicas up" until it recognizes itself as
>> leader, and _then_ restart the other node -- in this case it successfully
>> starts.
>>
>> Is there some necessary environment tweaking? The symptoms seem similar
>> whether I use the security.json from SOLR-8326 or the security.json from
>> the Wiki (with the comma repositioned).
>>
>>
>>
>> -----Original Message-----
>> From: Oakley, Craig (NIH/NLM/NCBI) [C]
>> Sent: Friday, November 20, 2015 6:59 PM
>> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
>> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
>>
>> Thanks
>>
>> It seems to work when there is no security.json, so perhaps there's some
>> typo in the initial version.
>>
>> I notice that the version you sent is different from the documentation at
>> cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
>> in that the Wiki version has "permissions" before "user-role": I also
>> notice that (at least as of right this moment) the Wiki version has a comma
>> at the end of '"user-role":{"solr":"admin"},' even though it is at the end:
>> and I notice that the Wiki version seems to lack a comma between the
>> "permissions" section and the "user-role" section. I just now also noticed
>> that the version you sent has '"user-role":{"solr":["admin"]}' (with square
>> brackets) whereas the Wiki does not have square brackets.
>>
>> The placement of the comma definitely looks wrong in the Wiki at the
>> moment (though perhaps someone might correct the Wiki before too long).
>> Other than that, I don’t know whether the order and/or the square brackets
>> make a difference. I can try with different permutations.
>>
>> Thanks again
>>
>> P.S. for the record, the Wiki currently has
>> {
>> "authentication":{
>> "class":"solr.BasicAuthPlugin",
>> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
>> },
>> "authorization":{
>> "class":"solr.RuleBasedAuthorizationPlugin",
>> "permissions":[{"name":"security-edit",
>> "role":"admin"}]
>> "user-role":{"solr":"admin"},
>> }}
>>
>> -----Original Message-----
>> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
>> Sent: Friday, November 20, 2015 6:18 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>>
>> This seems unrelated and more like a user error somewhere. Can you just
>> follow the steps, without any security settings i.e. not even uploading
>> security.json and see if you still see this? Sorry, but I don't have
>> access
>> to the code right now, I try and look at this later tonight.
>>
>> On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
>> craig.oak...@nih.gov> wrote:
>>
>> > Thank you for opening SOLR-8326
>> >
>> > As a side note, in the procedure you listed, even before adding the
>> > collection-admin-edit authorization, I'm already hitting trouble:
>> stopping
>> > and restarting a node results in the following
>> >
>> > INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
>> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
>> > Publishing state of core solr8326_shard2_replica1 as recovering, leader
>> is
>> > http://{IP-address-redacted}:8983/solr/solr8326_shard2_replica2/ and I
>> am
>> > http://{IP-address-redacted}:7574/solr/solr8326_shard2_replica1/
>> > INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4
>> > x:solr8326_shard2_replica1] org.apache.solr.cloud.ZkController;
>> publishing
>> > state=recovering
>> > INFO - 2015-11-20 22:48:41.278; [c:solr8326 s:shard1 r:core_node3
>> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
>> > Publishing state of core solr8326_shard1_replica1 as recovering, leader
>> is
>> > http://{IP-address-redacted}:8983/solr/solr8326_shard1_replica2/ and I
>> am
>> > http://{IP-address-redacted}:7574/solr/solr8326_shard1_replica1/
>> > INFO - 2015-11-20 22:48:41.280; [c:solr8326 s:shard1 r:core_node3
>> > x:solr8326_shard1_replica1] org.apache.solr.cloud.ZkController;
>> publishing
>> > state=recovering
>> > INFO - 2015-11-20 22:48:41.282; [c:solr8326 s:shard2 r:core_node4
>> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
>> Sending
>> > prep recovery command to http://{IP-address-redacted}:8983/solr;
>> > WaitForState:
>> >
>> action=PREPRECOVERY&core=solr8326_shard2_replica2&nodeName={IP-address-redacted}%3A7574_solr&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
>> > INFO - 2015-11-20 22:48:41.289; [ ]
>> > org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change:
>> > WatchedEvent state:SyncConnected type:NodeDataChanged
>> > path:/collections/solr8326/state.json for collection solr8326 has
>> occurred
>> > - updating... (live nodes size: 2)
>> > INFO - 2015-11-20 22:48:41.290; [c:solr8326 s:shard1 r:core_node3
>> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
>> Sending
>> > prep recovery command to http://{IP-address-redacted}:8983/solr;
>> > WaitForState:
>> >
>> action=PREPRECOVERY&core=solr8326_shard1_replica2&nodeName={IP-address-redacted}%3A7574_solr&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
>> > INFO - 2015-11-20 22:48:41.291; [ ]
>> > org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326
>> to
>> > ver 25
>> > ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4
>> > x:solr8326_shard2_replica1] org.apache.solr.common.SolrException; Error
>> > while trying to recover.:java.util.concurrent.ExecutionException:
>> > org.apache.http.ParseException: Invalid content type:
>> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> > at
>> >
>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
>> > at
>> >
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
>> > at
>> > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>> > Caused by: org.apache.http.ParseException: Invalid content type:
>> > at
>> org.apache.http.entity.ContentType.parse(ContentType.java:273)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
>> >
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> >
>> > INFO - 2015-11-20 22:48:41.298; [ ]
>> > org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change:
>> > WatchedEvent state:SyncConnected type:NodeDataChanged
>> > path:/collections/solr8326/state.json for collection solr8326 has
>> occurred
>> > - updating... (live nodes size: 2)
>> > ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4
>> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy;
>> > Recovery failed - trying again... (4)
>> > INFO - 2015-11-20 22:48:41.300; [c:solr8326 s:shard2 r:core_node4
>> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait
>> > 32.0 seconds before trying to recover again (5)
>> > ERROR - 2015-11-20 22:48:41.300; [c:solr8326 s:shard1 r:core_node3
>> > x:solr8326_shard1_replica1] org.apache.solr.common.SolrException; Error
>> > while trying to recover.:java.util.concurrent.ExecutionException:
>> > org.apache.http.ParseException: Invalid content type:
>> > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> > at
>> >
>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598)
>> > at
>> >
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361)
>> > at
>> > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>> > Caused by: org.apache.http.ParseException: Invalid content type:
>> > at
>> org.apache.http.entity.ContentType.parse(ContentType.java:273)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
>> >
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > at java.lang.Thread.run(Thread.java:745)
>> >
>> > ERROR - 2015-11-20 22:48:41.318; [c:solr8326 s:shard1 r:core_node3
>> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy;
>> > Recovery failed - trying again... (4)
>> > INFO - 2015-11-20 22:48:41.318; [ ]
>> > org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326
>> to
>> > ver 26
>> > INFO - 2015-11-20 22:48:41.319; [c:solr8326 s:shard1 r:core_node3
>> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; Wait
>> > 32.0 seconds before trying to recover again (5)
>> >
>> >
>> > I would not be surprised if this were to be some unrelated issue (the
>> > symptoms are quite different)
>> >
>> >
>> >
>> > Thanks again
>> >
>> >
>> > -----Original Message-----
>> > From: Anshum Gupta [mailto:ans...@anshumgupta.net]
>> > Sent: Friday, November 20, 2015 1:31 PM
>> > To: solr-user@lucene.apache.org
>> > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>> >
>> > Collections API were available before November of 2014, if that is when
>> you
>> > took the class. However, it was only with Solr 5.0 (released in Feb
>> 2015)
>> > that the only supported mechanism to create a collection was restricted
>> to
>> > Collections API.
>> >
>> > Here are the list of steps that you'd need to run to see that things are
>> > fine for you without the read permission:
>> > * Untar and setup Solr, don't start it yet
>> > * Start clean zookeeper
>> > * Put the security.json in zk, without anything other than a
>> security-edit
>> > permission. Find the content of the file below. Upload it using your
>> own zk
>> > client or through the solr script:
>> > > solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost
>> localhost:2181
>> > -cmd putfile /security.json ~/security.json
>> >
>> > security.json:
>> >
>> >
>> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> >
>> >
>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":{"solr":["admin"]},"permissions":[{"name":"security-edit","role":"admin"}]}}
>> >
>> > * Start solr:
>> > > solr-5.3.1/bin/solr start -e cloud -z localhost:2181
>> >
>> > You would need to key in a few things e.g. #nodes and ports, leave them
>> at
>> > the default values of 2 nodes and 8983/7574, unless you want to run
>> Solr on
>> > a different port. Then let it create a default collection to just make
>> sure
>> > that everything works fine.
>> >
>> > * Add the collection-admin-edit command:
>> > > curl --user solr:SolrRocks
>> > http://localhost:8983/solr/admin/authorization
>> > -H 'Content-type:application/json' -d '{"set-permission" :
>> > {"name":"collection-admin-edit", "role":"admin"}}'
>> >
>> > At this point, everything should be working fine. Restarting the nodes
>> > should also work fine. You can try 2 things at this point:
>> > 1. Create a new collection with 1 shard and 1 replica and then try
>> adding a
>> > replica, here's how:
>> > > curl --user solr:SolrRocks
>> >
>> >
>> http://localhost:8983/solr/admin/collections?action=CREATE&name=testcollection&collection.configName=gettingstarted&numShards=1
>> >
>> > > curl --user solr:SolrRocks
>> >
>> >
>> http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=testcollection&shard=shard1
>> >
>> > This should work fine.
>> >
>> > 2. After this, try restarting the solr cluster. Here's how you can do
>> so,
>> > assuming you didn't change any of the defaults and you are running zk on
>> > localhost:2181. If not, just change those values below:
>> > > bin/solr stop -all
>> >
>> > After this, check that Solr was actually stopped. I'd also suggest you
>> tail
>> > the logs on both nodes when they are coming up to see any errors, if
>> any.
>> > The logs would be here: example/cloud/node1/logs/solr.log
>> > and example/cloud/node2/logs/solr.log
>> >
>> > > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z
>> localhost:2181
>> > > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z
>> localhost:2181
>> >
>> > If you get to this checkpoint fine, try adding a read permission.
>> > Add a permission:
>> > > curl --user solr:SolrRocks
>> > http://localhost:8983/solr/admin/authorization
>> > -H 'Content-type:application/json' -d '{"set-permission" :
>> {"name":"read",
>> > "role":"read"}}'
>> >
>> > Add a user:
>> > > curl --user solr:SolrRocks
>> > http://localhost:8983/solr/admin/authentication
>> > -H 'Content-type:application/json' -d '{"set-user" :
>> > {"solrread":"solrRocks"}}'
>> >
>> > Assign a role to the user:
>> > >curl --user solr:SolrRocks
>> http://localhost:8983/solr/admin/authorization
>> > -H 'Content-type:application/json' -d '{"set-user-role" :
>> > {"solrread":["read"]}}'
>> >
>> > After this, you should start having issues with ADDREPLICA.
>> > Also, as you would at this point have a collection with a shard that
>> has a
>> > replication factor > 1 (remember the ADDREPLICA we did earlier), you
>> would
>> > have issues when you restart the cluster again using the steps I
>> mentioned
>> > above.
>> >
>> >
>> > Can you confirm this? I guess I'll just use this text to create a new
>> JIRA
>> > now.
>> >
>> >
>> > On Fri, Nov 20, 2015 at 10:04 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
>> > craig.oak...@nih.gov> wrote:
>> >
>> > > Thank you again for the reply.
>> > >
>> > > Below is the Email I was about to send prior to your reply a moment
>> ago:
>> > > shall I try again without "read" in the security.json?
>> > >
>> > >
>> > >
>> > > The Collections API method was not discussed in the "Unleashed" class
>> at
>> > > the conference in DC in 2014 (probably because it was not yet
>> available),
>> > > so I was using the method I knew.
>> > >
>> > > I have now tried again using admin/collections?action=CREATE (using
>> > > different port numbers to avoid confusion from the failed previous
>> > > attempts: the previously created nodes had been shutdown and their
>> > > core.properties files renamed so as not to be discovered), but the
>> > results
>> > > are the same:
>> > > INFO - 2015-11-20 16:56:25.283; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
>> Starting
>> > > Replication Recovery.
>> > > INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Begin
>> > > buffering updates.
>> > > INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Starting to
>> > > buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
>> > > INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
>> > Attempting
>> > > to replicate from http://
>> > > {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1/.
>> > > ERROR - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.common.SolrException; Error
>> > while
>> > > trying to
>> > >
>> >
>> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> > > Error from server at http://
>> > {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1:
>> > > Expected mime type application/octet-stream but got text/html. <html>
>> > > <head>
>> > > <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>> > > <title>Error 401 Unauthorized request, Response code: 401</title>
>> > > </head>
>> > > <body><h2>HTTP ERROR 401</h2>
>> > > <p>Problem accessing /solr/xmpl3_shard1_replica1/update. Reason:
>> > > <pre> Unauthorized request, Response code:
>> > > 401</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>
>> > >
>> > > </body>
>> > > </html>
>> > >
>> > > at
>> > >
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
>> > > at
>> > >
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
>> > > at
>> > >
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
>> > > at
>> > > org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
>> > > at
>> > > org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
>> > > at
>> > >
>> >
>> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
>> > > at
>> > >
>> >
>> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
>> > > at
>> > >
>> >
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
>> > > at
>> > > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>> > >
>> > > INFO - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Dropping
>> > > buffered updates FSUpdateLog{state=BUFFERING, tlog=null}
>> > > ERROR - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy;
>> Recovery
>> > > failed - trying again... (2)
>> > > INFO - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
>> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Wait
>> 8.0
>> > > seconds before trying to recover again (3)
>> > >
>> > >
>> > > Below is a list of the steps I took.
>> > >
>> > > ./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl3
>> > > ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd putfile
>> /security.json
>> > > ~/solr/security151119a.json
>> > > ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd upconfig -confdir
>> > > ../../solr/configsets/basic_configs/conf -confname xmpl3
>> > > cd ../../../bin/
>> > > ./solr -c -p 4695 -d
>> ~dbman/solr/straight531outofbox/solr-5.3.1/server/
>> > -z
>> > > localhost:4545/solr/xmpl3 -s
>> > > ~dbman/solr/straight531outofbox/solr-5.3.1/example/solr
>> > > ./solr -c -p 4685 -d
>> ~dbman/solr/straight531outofbox/solr-5.3.1/server/
>> > -z
>> > > localhost:4545/solr/xmpl3 -s
>> > > ~dbman/solr/straight531outofbox/solr-5.3.1/server/solr
>> > > curl -u solr:SolrRocks '
>> > >
>> >
>> http://nosqltest11:4685/solr/admin/collections?action=CREATE&name=xmpl3&numShards=1&replicationFactor=1&createNodeSet={IP-address-redacted}:4685_solr
>> > > '
>> > > curl -u solr:SolrRocks '
>> > >
>> >
>> http://nosqltest11:4685/solr/admin/collections?action=ADDREPLICA&collection=xmpl3&shard=shard1&node={IP-address-redacted}:4695_solr&wt=json&indent=true
>> > > '
>> > >
>> > >
>> > >
>> > >
>> > > Can you provide a list of steps to take in an out-of-the-box directory
>> > > tree whereby ADDREPLICA _will_ work with security.json already in
>> place?
>> > >
>> > >
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: Anshum Gupta [mailto:ans...@anshumgupta.net]
>> > > Sent: Thursday, November 19, 2015 3:44 PM
>> > > To: solr-user@lucene.apache.org
>> > > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>> > >
>> > > I'll try out what you did later in the day, as soon as I get time but
>> why
>> > > exactly are you creating cores manually? Seems like you manually
>> create a
>> > > core and the try to add a replica. Can you try using the Collections
>> API
>> > to
>> > > create a collection?
>> > >
>> > > Starting Solr 5.0, the only supported way to create a new collection
>> is
>> > via
>> > > the Collections API. Creating a core would lead to a collection
>> creation
>> > > but that's not really supported. It was just something that was done
>> when
>> > > there were no Collections API.
>> > >
>> > >
>> > > On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
>> > > craig.oak...@nih.gov> wrote:
>> > >
>> > > > I tried again with the following security.json, but the results were
>> > the
>> > > > same:
>> > > >
>> > > > {
>> > > > "authentication":{
>> > > > "class":"solr.BasicAuthPlugin",
>> > > > "credentials":{
>> > > > "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> > > > Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
>> > > > "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
>> > > > 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
>> > > > "":{"v":9}},
>> > > > "authorization":{
>> > > > "class":"solr.RuleBasedAuthorizationPlugin",
>> > > > "user-role":{
>> > > > "solr":[
>> > > > "admin",
>> > > > "read",
>> > > > "xmpladmin",
>> > > > "xmplgen",
>> > > > "xmplsel"],
>> > > > "solruser":[
>> > > > "read",
>> > > > "xmplgen",
>> > > > "xmplsel"]},
>> > > > "permissions":[
>> > > > {
>> > > > "name":"security-edit",
>> > > > "role":"admin"},
>> > > > {
>> > > > "name":"xmpl_admin",
>> > > > "collection":"xmpl",
>> > > > "path":"/admin/*",
>> > > > "role":"xmpladmin"},
>> > > > {
>> > > > "name":"xmpl_sel",
>> > > > "collection":"xmpl",
>> > > > "path":"/select/*",
>> > > > "role":null},
>> > > > {
>> > > > "name":"all-admin",
>> > > > "collection":null,
>> > > > "path":"/*",
>> > > > "role":"xmplgen"},
>> > > > {
>> > > > "name":"all-core-handlers",
>> > > > "path":"/*",
>> > > > "role":"xmplgen"}],
>> > > > "":{"v":42}}}
>> > > >
>> > > > -----Original Message-----
>> > > > From: Oakley, Craig (NIH/NLM/NCBI) [C]
>> > > > Sent: Thursday, November 19, 2015 1:46 PM
>> > > > To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
>> > > > Subject: RE: Re:Re: Implementing security.json is breaking
>> ADDREPLICA
>> > > >
>> > > > I note that the thread called "Security Problems" (most recent post
>> by
>> > > > Nobel Paul) seems like it may help with much of what I'm trying to
>> do.
>> > I
>> > > > will see to what extent that may help.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Anshum Gupta
>> > >
>> >
>> >
>> >
>> > --
>> > Anshum Gupta
>> >
>>
>>
>>
>> --
>> Anshum Gupta
>>
>
>
>
> --
> Anshum Gupta
>
--
Anshum Gupta
