bq: I don't suppose there is an ETA for 5.4? Actually, 5.4 is probably in the works within the next month. I'm not the one cutting the release, but there's some rumors that a label will be cut this week, then the "usual" process is a week or two (sometimes more if bugs are flushed out) before the official release.
Call it the first of the year for safety's sake, but that's a guess. Best, Erick On Tue, Nov 24, 2015 at 10:22 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <craig.oak...@nih.gov> wrote: > Thanks for the reply, > > I don't suppose there is an ETA for 5.4? > > > Thanks again > > -----Original Message----- > From: Anshum Gupta [mailto:ans...@anshumgupta.net] > Sent: Tuesday, November 24, 2015 12:31 PM > To: solr-user@lucene.apache.org > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA > > Yes, it certainly is a PKI issue. > > On Tue, Nov 24, 2015 at 7:59 AM, Oakley, Craig (NIH/NLM/NCBI) [C] < > craig.oak...@nih.gov> wrote: > >> Thank you for the reply >> >> Trying those exact commands, I'm still getting the same issue >> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/solr-5.3.1.tgz >> tar xvzf /net/sybdev11/export/home/sybase/Distr/Solr/zookeeper-3.4.6.tar.gz >> cd zookeeper-3.4.6/ >> bin/zkServer.sh start zoo_sample.cfg >> cd .. >> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 >> -cmd putfile /security.json >> PREVsolr-5.3.1/server/scripts/cloud-scripts/security.json >> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 >> -cmd list >> solr-5.3.1/bin/solr start -e cloud -z localhost:2181 >> cd solr-5.3.1/ >> bin/solr stop -p 7574 >> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181 >> tail -f example/cloud/node2/logs/solr.log >> >> The -cmd list shows >> / (2) >> DATA: >> >> /zookeeper (1) >> DATA: >> >> /security.json (0) >> DATA: >> >> >> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= >> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role": >> {"solr":["admin"]} >> >> ,"permissions":[ >> {"name":"security-edit","role":"admin"} >> >> ]}} >> >> >> While the output of tail contains >> ERROR - 2015-11-24 10:45:54.796; [c:gettingstarted s:shard1 r:core_node4 >> x:gettingstarted_shard1_replica1] org.apache.solr.common.SolrException; >> Error while trying to recover.:java.util.concurrent.ExecutionException: >> org.apache.http.ParseException: Invalid content type: >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> at >> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598) >> at >> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361) >> at >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227) >> Caused by: org.apache.http.ParseException: Invalid content type: >> at org.apache.http.entity.ContentType.parse(ContentType.java:273) >> at >> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512) >> at >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270) >> at >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> >> -----Original Message----- >> From: Anshum Gupta [mailto:ans...@anshumgupta.net] >> Sent: Monday, November 23, 2015 7:24 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA >> >> Yes, I see the same issue. I'll update the JIRA and drill down. Thanks. >> >> On Mon, Nov 23, 2015 at 4:18 PM, Anshum Gupta <ans...@anshumgupta.net> >> wrote: >> >> > To restart solr, you should instead use something like: >> > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181 >> > or >> > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181 >> > >> > I've seen others report the same exception but never ran into this one >> > myself. Let me try this out. >> > >> > >> > >> > On Mon, Nov 23, 2015 at 2:55 PM, Oakley, Craig (NIH/NLM/NCBI) [C] < >> > craig.oak...@nih.gov> wrote: >> > >> >> FWIW >> >> >> >> I am getting fairly consistent results that if I follow the SOLR-8326 >> >> procedure just up through the step of "solr-5.3.1/bin/solr start -e >> cloud >> >> -z localhost:2181": if I then stop just one node (either "./solr stop -p >> >> 7574" or "./solr stop -p 8983") and then restart that same node (using >> the >> >> command suggested by "solr-5.3.1/bin/solr start -e cloud -z >> >> localhost:2181"), then the solr.log for the stopped-and-restarted node >> gets >> >> such stack traces as >> >> ERROR - 2015-11-23 21:49:28.663; [c:gettingstarted s:shard2 r:core_node3 >> >> x:gettingstarted_shard2_replica2] org.apache.solr.common.SolrException; >> >> Error while trying to recover.:java.util.concurrent.ExecutionException: >> >> org.apache.http.ParseException: Invalid content type: >> >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> >> at >> >> >> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598) >> >> at >> >> >> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361) >> >> at >> >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227) >> >> Caused by: org.apache.http.ParseException: Invalid content type: >> >> at >> org.apache.http.entity.ContentType.parse(ContentType.java:273) >> >> at >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512) >> >> at >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270) >> >> at >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266) >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> >> at >> >> >> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210) >> >> at >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> >> at >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> >> at java.lang.Thread.run(Thread.java:745) >> >> >> >> While the node which stayed up the whole time starts getting such stack >> >> traces as >> >> ERROR - 2015-11-23 21:57:46.019; [c:gettingstarted s:shard2 r:core_node3 >> >> x:gettingstarted_shard2_replica2] >> >> org.apache.solr.security.PKIAuthenticationPlugin; Invalid time r? >> >> java.lang.NumberFormatException: For input string: "r?" >> >> at >> >> >> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) >> >> at java.lang.Long.parseLong(Long.java:589) >> >> at java.lang.Long.parseLong(Long.java:631) >> >> at >> >> >> org.apache.solr.security.PKIAuthenticationPlugin.doAuthenticate(PKIAuthenticationPlugin.java:128) >> >> at >> >> >> org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:252) >> >> at >> >> >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:186) >> >> at >> >> >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) >> >> at >> >> >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >> >> at >> >> >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >> >> at >> >> >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >> >> at >> >> >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >> >> at >> >> >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >> >> at >> >> >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >> >> at >> >> >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >> >> at >> >> >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >> >> at >> >> >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) >> >> at >> >> >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) >> >> at >> >> >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) >> >> at >> >> >> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) >> >> at >> >> >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> >> at org.eclipse.jetty.server.Server.handle(Server.java:499) >> >> at >> >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) >> >> at >> >> >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) >> >> at >> >> >> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) >> >> at >> >> >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >> >> at >> >> >> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >> >> at java.lang.Thread.run(Thread.java:745) >> >> >> >> In this case the string is just "r?", but usually it is a longer string >> >> of control characters. >> >> >> >> If I shutdown _both_ nodes and restart _one_, and then allow it to be >> >> "Waiting until we see more replicas up" until it recognizes itself as >> >> leader, and _then_ restart the other node -- in this case it >> successfully >> >> starts. >> >> >> >> Is there some necessary environment tweaking? The symptoms seem similar >> >> whether I use the security.json from SOLR-8326 or the security.json from >> >> the Wiki (with the comma repositioned). >> >> >> >> >> >> >> >> -----Original Message----- >> >> From: Oakley, Craig (NIH/NLM/NCBI) [C] >> >> Sent: Friday, November 20, 2015 6:59 PM >> >> To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org> >> >> Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA >> >> >> >> Thanks >> >> >> >> It seems to work when there is no security.json, so perhaps there's some >> >> typo in the initial version. >> >> >> >> I notice that the version you sent is different from the documentation >> at >> >> >> cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins >> >> in that the Wiki version has "permissions" before "user-role": I also >> >> notice that (at least as of right this moment) the Wiki version has a >> comma >> >> at the end of '"user-role":{"solr":"admin"},' even though it is at the >> end: >> >> and I notice that the Wiki version seems to lack a comma between the >> >> "permissions" section and the "user-role" section. I just now also >> noticed >> >> that the version you sent has '"user-role":{"solr":["admin"]}' (with >> square >> >> brackets) whereas the Wiki does not have square brackets. >> >> >> >> The placement of the comma definitely looks wrong in the Wiki at the >> >> moment (though perhaps someone might correct the Wiki before too long). >> >> Other than that, I don’t know whether the order and/or the square >> brackets >> >> make a difference. I can try with different permutations. >> >> >> >> Thanks again >> >> >> >> P.S. for the record, the Wiki currently has >> >> { >> >> "authentication":{ >> >> "class":"solr.BasicAuthPlugin", >> >> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= >> >> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="} >> >> }, >> >> "authorization":{ >> >> "class":"solr.RuleBasedAuthorizationPlugin", >> >> "permissions":[{"name":"security-edit", >> >> "role":"admin"}] >> >> "user-role":{"solr":"admin"}, >> >> }} >> >> >> >> -----Original Message----- >> >> From: Anshum Gupta [mailto:ans...@anshumgupta.net] >> >> Sent: Friday, November 20, 2015 6:18 PM >> >> To: solr-user@lucene.apache.org >> >> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA >> >> >> >> This seems unrelated and more like a user error somewhere. Can you just >> >> follow the steps, without any security settings i.e. not even uploading >> >> security.json and see if you still see this? Sorry, but I don't have >> >> access >> >> to the code right now, I try and look at this later tonight. >> >> >> >> On Fri, Nov 20, 2015 at 3:07 PM, Oakley, Craig (NIH/NLM/NCBI) [C] < >> >> craig.oak...@nih.gov> wrote: >> >> >> >> > Thank you for opening SOLR-8326 >> >> > >> >> > As a side note, in the procedure you listed, even before adding the >> >> > collection-admin-edit authorization, I'm already hitting trouble: >> >> stopping >> >> > and restarting a node results in the following >> >> > >> >> > INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4 >> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; >> >> > Publishing state of core solr8326_shard2_replica1 as recovering, >> leader >> >> is >> >> > http://{IP-address-redacted}:8983/solr/solr8326_shard2_replica2/ and >> I >> >> am >> >> > http://{IP-address-redacted}:7574/solr/solr8326_shard2_replica1/ >> >> > INFO - 2015-11-20 22:48:41.275; [c:solr8326 s:shard2 r:core_node4 >> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.ZkController; >> >> publishing >> >> > state=recovering >> >> > INFO - 2015-11-20 22:48:41.278; [c:solr8326 s:shard1 r:core_node3 >> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; >> >> > Publishing state of core solr8326_shard1_replica1 as recovering, >> leader >> >> is >> >> > http://{IP-address-redacted}:8983/solr/solr8326_shard1_replica2/ and >> I >> >> am >> >> > http://{IP-address-redacted}:7574/solr/solr8326_shard1_replica1/ >> >> > INFO - 2015-11-20 22:48:41.280; [c:solr8326 s:shard1 r:core_node3 >> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.ZkController; >> >> publishing >> >> > state=recovering >> >> > INFO - 2015-11-20 22:48:41.282; [c:solr8326 s:shard2 r:core_node4 >> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; >> >> Sending >> >> > prep recovery command to http://{IP-address-redacted}:8983/solr; >> >> > WaitForState: >> >> > >> >> >> action=PREPRECOVERY&core=solr8326_shard2_replica2&nodeName={IP-address-redacted}%3A7574_solr&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true >> >> > INFO - 2015-11-20 22:48:41.289; [ ] >> >> > org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change: >> >> > WatchedEvent state:SyncConnected type:NodeDataChanged >> >> > path:/collections/solr8326/state.json for collection solr8326 has >> >> occurred >> >> > - updating... (live nodes size: 2) >> >> > INFO - 2015-11-20 22:48:41.290; [c:solr8326 s:shard1 r:core_node3 >> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; >> >> Sending >> >> > prep recovery command to http://{IP-address-redacted}:8983/solr; >> >> > WaitForState: >> >> > >> >> >> action=PREPRECOVERY&core=solr8326_shard1_replica2&nodeName={IP-address-redacted}%3A7574_solr&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true >> >> > INFO - 2015-11-20 22:48:41.291; [ ] >> >> > org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326 >> >> to >> >> > ver 25 >> >> > ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4 >> >> > x:solr8326_shard2_replica1] org.apache.solr.common.SolrException; >> Error >> >> > while trying to recover.:java.util.concurrent.ExecutionException: >> >> > org.apache.http.ParseException: Invalid content type: >> >> > at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> >> > at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> >> > at >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598) >> >> > at >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361) >> >> > at >> >> > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227) >> >> > Caused by: org.apache.http.ParseException: Invalid content type: >> >> > at >> >> org.apache.http.entity.ContentType.parse(ContentType.java:273) >> >> > at >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512) >> >> > at >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270) >> >> > at >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266) >> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> >> > at >> >> > >> >> >> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210) >> >> > at >> >> > >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> >> > at >> >> > >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> >> > at java.lang.Thread.run(Thread.java:745) >> >> > >> >> > INFO - 2015-11-20 22:48:41.298; [ ] >> >> > org.apache.solr.common.cloud.ZkStateReader$8; A cluster state change: >> >> > WatchedEvent state:SyncConnected type:NodeDataChanged >> >> > path:/collections/solr8326/state.json for collection solr8326 has >> >> occurred >> >> > - updating... (live nodes size: 2) >> >> > ERROR - 2015-11-20 22:48:41.298; [c:solr8326 s:shard2 r:core_node4 >> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; >> >> > Recovery failed - trying again... (4) >> >> > INFO - 2015-11-20 22:48:41.300; [c:solr8326 s:shard2 r:core_node4 >> >> > x:solr8326_shard2_replica1] org.apache.solr.cloud.RecoveryStrategy; >> Wait >> >> > 32.0 seconds before trying to recover again (5) >> >> > ERROR - 2015-11-20 22:48:41.300; [c:solr8326 s:shard1 r:core_node3 >> >> > x:solr8326_shard1_replica1] org.apache.solr.common.SolrException; >> Error >> >> > while trying to recover.:java.util.concurrent.ExecutionException: >> >> > org.apache.http.ParseException: Invalid content type: >> >> > at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> >> > at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> >> > at >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:598) >> >> > at >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:361) >> >> > at >> >> > org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227) >> >> > Caused by: org.apache.http.ParseException: Invalid content type: >> >> > at >> >> org.apache.http.entity.ContentType.parse(ContentType.java:273) >> >> > at >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:512) >> >> > at >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:270) >> >> > at >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:266) >> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> >> > at >> >> > >> >> >> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:210) >> >> > at >> >> > >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> >> > at >> >> > >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> >> > at java.lang.Thread.run(Thread.java:745) >> >> > >> >> > ERROR - 2015-11-20 22:48:41.318; [c:solr8326 s:shard1 r:core_node3 >> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; >> >> > Recovery failed - trying again... (4) >> >> > INFO - 2015-11-20 22:48:41.318; [ ] >> >> > org.apache.solr.common.cloud.ZkStateReader; Updating data for solr8326 >> >> to >> >> > ver 26 >> >> > INFO - 2015-11-20 22:48:41.319; [c:solr8326 s:shard1 r:core_node3 >> >> > x:solr8326_shard1_replica1] org.apache.solr.cloud.RecoveryStrategy; >> Wait >> >> > 32.0 seconds before trying to recover again (5) >> >> > >> >> > >> >> > I would not be surprised if this were to be some unrelated issue (the >> >> > symptoms are quite different) >> >> > >> >> > >> >> > >> >> > Thanks again >> >> > >> >> > >> >> > -----Original Message----- >> >> > From: Anshum Gupta [mailto:ans...@anshumgupta.net] >> >> > Sent: Friday, November 20, 2015 1:31 PM >> >> > To: solr-user@lucene.apache.org >> >> > Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA >> >> > >> >> > Collections API were available before November of 2014, if that is >> when >> >> you >> >> > took the class. However, it was only with Solr 5.0 (released in Feb >> >> 2015) >> >> > that the only supported mechanism to create a collection was >> restricted >> >> to >> >> > Collections API. >> >> > >> >> > Here are the list of steps that you'd need to run to see that things >> are >> >> > fine for you without the read permission: >> >> > * Untar and setup Solr, don't start it yet >> >> > * Start clean zookeeper >> >> > * Put the security.json in zk, without anything other than a >> >> security-edit >> >> > permission. Find the content of the file below. Upload it using your >> >> own zk >> >> > client or through the solr script: >> >> > > solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost >> >> localhost:2181 >> >> > -cmd putfile /security.json ~/security.json >> >> > >> >> > security.json: >> >> > >> >> > >> >> >> {"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= >> >> > >> >> > >> >> >> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":{"solr":["admin"]},"permissions":[{"name":"security-edit","role":"admin"}]}} >> >> > >> >> > * Start solr: >> >> > > solr-5.3.1/bin/solr start -e cloud -z localhost:2181 >> >> > >> >> > You would need to key in a few things e.g. #nodes and ports, leave >> them >> >> at >> >> > the default values of 2 nodes and 8983/7574, unless you want to run >> >> Solr on >> >> > a different port. Then let it create a default collection to just make >> >> sure >> >> > that everything works fine. >> >> > >> >> > * Add the collection-admin-edit command: >> >> > > curl --user solr:SolrRocks >> >> > http://localhost:8983/solr/admin/authorization >> >> > -H 'Content-type:application/json' -d '{"set-permission" : >> >> > {"name":"collection-admin-edit", "role":"admin"}}' >> >> > >> >> > At this point, everything should be working fine. Restarting the nodes >> >> > should also work fine. You can try 2 things at this point: >> >> > 1. Create a new collection with 1 shard and 1 replica and then try >> >> adding a >> >> > replica, here's how: >> >> > > curl --user solr:SolrRocks >> >> > >> >> > >> >> >> http://localhost:8983/solr/admin/collections?action=CREATE&name=testcollection&collection.configName=gettingstarted&numShards=1 >> >> > >> >> > > curl --user solr:SolrRocks >> >> > >> >> > >> >> >> http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=testcollection&shard=shard1 >> >> > >> >> > This should work fine. >> >> > >> >> > 2. After this, try restarting the solr cluster. Here's how you can do >> >> so, >> >> > assuming you didn't change any of the defaults and you are running zk >> on >> >> > localhost:2181. If not, just change those values below: >> >> > > bin/solr stop -all >> >> > >> >> > After this, check that Solr was actually stopped. I'd also suggest you >> >> tail >> >> > the logs on both nodes when they are coming up to see any errors, if >> >> any. >> >> > The logs would be here: example/cloud/node1/logs/solr.log >> >> > and example/cloud/node2/logs/solr.log >> >> > >> >> > > bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z >> >> localhost:2181 >> >> > > bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z >> >> localhost:2181 >> >> > >> >> > If you get to this checkpoint fine, try adding a read permission. >> >> > Add a permission: >> >> > > curl --user solr:SolrRocks >> >> > http://localhost:8983/solr/admin/authorization >> >> > -H 'Content-type:application/json' -d '{"set-permission" : >> >> {"name":"read", >> >> > "role":"read"}}' >> >> > >> >> > Add a user: >> >> > > curl --user solr:SolrRocks >> >> > http://localhost:8983/solr/admin/authentication >> >> > -H 'Content-type:application/json' -d '{"set-user" : >> >> > {"solrread":"solrRocks"}}' >> >> > >> >> > Assign a role to the user: >> >> > >curl --user solr:SolrRocks >> >> http://localhost:8983/solr/admin/authorization >> >> > -H 'Content-type:application/json' -d '{"set-user-role" : >> >> > {"solrread":["read"]}}' >> >> > >> >> > After this, you should start having issues with ADDREPLICA. >> >> > Also, as you would at this point have a collection with a shard that >> >> has a >> >> > replication factor > 1 (remember the ADDREPLICA we did earlier), you >> >> would >> >> > have issues when you restart the cluster again using the steps I >> >> mentioned >> >> > above. >> >> > >> >> > >> >> > Can you confirm this? I guess I'll just use this text to create a new >> >> JIRA >> >> > now. >> >> > >> >> > >> >> > On Fri, Nov 20, 2015 at 10:04 AM, Oakley, Craig (NIH/NLM/NCBI) [C] < >> >> > craig.oak...@nih.gov> wrote: >> >> > >> >> > > Thank you again for the reply. >> >> > > >> >> > > Below is the Email I was about to send prior to your reply a moment >> >> ago: >> >> > > shall I try again without "read" in the security.json? >> >> > > >> >> > > >> >> > > >> >> > > The Collections API method was not discussed in the "Unleashed" >> class >> >> at >> >> > > the conference in DC in 2014 (probably because it was not yet >> >> available), >> >> > > so I was using the method I knew. >> >> > > >> >> > > I have now tried again using admin/collections?action=CREATE (using >> >> > > different port numbers to avoid confusion from the failed previous >> >> > > attempts: the previously created nodes had been shutdown and their >> >> > > core.properties files renamed so as not to be discovered), but the >> >> > results >> >> > > are the same: >> >> > > INFO - 2015-11-20 16:56:25.283; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; >> >> Starting >> >> > > Replication Recovery. >> >> > > INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; >> Begin >> >> > > buffering updates. >> >> > > INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Starting >> to >> >> > > buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} >> >> > > INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; >> >> > Attempting >> >> > > to replicate from http:// >> >> > > {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1/. >> >> > > ERROR - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.common.SolrException; Error >> >> > while >> >> > > trying to >> >> > > >> >> > >> >> >> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: >> >> > > Error from server at http:// >> >> > {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1: >> >> > > Expected mime type application/octet-stream but got text/html. >> <html> >> >> > > <head> >> >> > > <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> >> >> > > <title>Error 401 Unauthorized request, Response code: 401</title> >> >> > > </head> >> >> > > <body><h2>HTTP ERROR 401</h2> >> >> > > <p>Problem accessing /solr/xmpl3_shard1_replica1/update. Reason: >> >> > > <pre> Unauthorized request, Response code: >> >> > > 401</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/> >> >> > > >> >> > > </body> >> >> > > </html> >> >> > > >> >> > > at >> >> > > >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528) >> >> > > at >> >> > > >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234) >> >> > > at >> >> > > >> >> > >> >> >> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226) >> >> > > at >> >> > > >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) >> >> > > at >> >> > > >> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152) >> >> > > at >> >> > > >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207) >> >> > > at >> >> > > >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147) >> >> > > at >> >> > > >> >> > >> >> >> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437) >> >> > > at >> >> > > >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227) >> >> > > >> >> > > INFO - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Dropping >> >> > > buffered updates FSUpdateLog{state=BUFFERING, tlog=null} >> >> > > ERROR - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; >> >> Recovery >> >> > > failed - trying again... (2) >> >> > > INFO - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2 >> >> > > x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; >> Wait >> >> 8.0 >> >> > > seconds before trying to recover again (3) >> >> > > >> >> > > >> >> > > Below is a list of the steps I took. >> >> > > >> >> > > ./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl3 >> >> > > ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd putfile >> >> /security.json >> >> > > ~/solr/security151119a.json >> >> > > ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd upconfig -confdir >> >> > > ../../solr/configsets/basic_configs/conf -confname xmpl3 >> >> > > cd ../../../bin/ >> >> > > ./solr -c -p 4695 -d >> >> ~dbman/solr/straight531outofbox/solr-5.3.1/server/ >> >> > -z >> >> > > localhost:4545/solr/xmpl3 -s >> >> > > ~dbman/solr/straight531outofbox/solr-5.3.1/example/solr >> >> > > ./solr -c -p 4685 -d >> >> ~dbman/solr/straight531outofbox/solr-5.3.1/server/ >> >> > -z >> >> > > localhost:4545/solr/xmpl3 -s >> >> > > ~dbman/solr/straight531outofbox/solr-5.3.1/server/solr >> >> > > curl -u solr:SolrRocks ' >> >> > > >> >> > >> >> >> http://nosqltest11:4685/solr/admin/collections?action=CREATE&name=xmpl3&numShards=1&replicationFactor=1&createNodeSet={IP-address-redacted}:4685_solr >> >> > > ' >> >> > > curl -u solr:SolrRocks ' >> >> > > >> >> > >> >> >> http://nosqltest11:4685/solr/admin/collections?action=ADDREPLICA&collection=xmpl3&shard=shard1&node={IP-address-redacted}:4695_solr&wt=json&indent=true >> >> > > ' >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > Can you provide a list of steps to take in an out-of-the-box >> directory >> >> > > tree whereby ADDREPLICA _will_ work with security.json already in >> >> place? >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > -----Original Message----- >> >> > > From: Anshum Gupta [mailto:ans...@anshumgupta.net] >> >> > > Sent: Thursday, November 19, 2015 3:44 PM >> >> > > To: solr-user@lucene.apache.org >> >> > > Subject: Re: Re:Re: Implementing security.json is breaking >> ADDREPLICA >> >> > > >> >> > > I'll try out what you did later in the day, as soon as I get time >> but >> >> why >> >> > > exactly are you creating cores manually? Seems like you manually >> >> create a >> >> > > core and the try to add a replica. Can you try using the Collections >> >> API >> >> > to >> >> > > create a collection? >> >> > > >> >> > > Starting Solr 5.0, the only supported way to create a new collection >> >> is >> >> > via >> >> > > the Collections API. Creating a core would lead to a collection >> >> creation >> >> > > but that's not really supported. It was just something that was done >> >> when >> >> > > there were no Collections API. >> >> > > >> >> > > >> >> > > On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] < >> >> > > craig.oak...@nih.gov> wrote: >> >> > > >> >> > > > I tried again with the following security.json, but the results >> were >> >> > the >> >> > > > same: >> >> > > > >> >> > > > { >> >> > > > "authentication":{ >> >> > > > "class":"solr.BasicAuthPlugin", >> >> > > > "credentials":{ >> >> > > > "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= >> >> > > > Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=", >> >> > > > "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE= >> >> > > > 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="}, >> >> > > > "":{"v":9}}, >> >> > > > "authorization":{ >> >> > > > "class":"solr.RuleBasedAuthorizationPlugin", >> >> > > > "user-role":{ >> >> > > > "solr":[ >> >> > > > "admin", >> >> > > > "read", >> >> > > > "xmpladmin", >> >> > > > "xmplgen", >> >> > > > "xmplsel"], >> >> > > > "solruser":[ >> >> > > > "read", >> >> > > > "xmplgen", >> >> > > > "xmplsel"]}, >> >> > > > "permissions":[ >> >> > > > { >> >> > > > "name":"security-edit", >> >> > > > "role":"admin"}, >> >> > > > { >> >> > > > "name":"xmpl_admin", >> >> > > > "collection":"xmpl", >> >> > > > "path":"/admin/*", >> >> > > > "role":"xmpladmin"}, >> >> > > > { >> >> > > > "name":"xmpl_sel", >> >> > > > "collection":"xmpl", >> >> > > > "path":"/select/*", >> >> > > > "role":null}, >> >> > > > { >> >> > > > "name":"all-admin", >> >> > > > "collection":null, >> >> > > > "path":"/*", >> >> > > > "role":"xmplgen"}, >> >> > > > { >> >> > > > "name":"all-core-handlers", >> >> > > > "path":"/*", >> >> > > > "role":"xmplgen"}], >> >> > > > "":{"v":42}}} >> >> > > > >> >> > > > -----Original Message----- >> >> > > > From: Oakley, Craig (NIH/NLM/NCBI) [C] >> >> > > > Sent: Thursday, November 19, 2015 1:46 PM >> >> > > > To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org> >> >> > > > Subject: RE: Re:Re: Implementing security.json is breaking >> >> ADDREPLICA >> >> > > > >> >> > > > I note that the thread called "Security Problems" (most recent >> post >> >> by >> >> > > > Nobel Paul) seems like it may help with much of what I'm trying to >> >> do. >> >> > I >> >> > > > will see to what extent that may help. >> >> > > > >> >> > > >> >> > > >> >> > > >> >> > > -- >> >> > > Anshum Gupta >> >> > > >> >> > >> >> > >> >> > >> >> > -- >> >> > Anshum Gupta >> >> > >> >> >> >> >> >> >> >> -- >> >> Anshum Gupta >> >> >> > >> > >> > >> > -- >> > Anshum Gupta >> > >> >> >> >> -- >> Anshum Gupta >> > > > > -- > Anshum Gupta
