Collections API were available before November of 2014, if that is when you
took the class. However, it was only with Solr 5.0 (released in Feb 2015)
that the only supported mechanism to create a collection was restricted to
Collections API.
Here are the list of steps that you'd need to run to see that things are
fine for you without the read permission:
* Untar and setup Solr, don't start it yet
* Start clean zookeeper
* Put the security.json in zk, without anything other than a security-edit
permission. Find the content of the file below. Upload it using your own zk
client or through the solr script:
> solr-5.3.1/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181
-cmd putfile /security.json ~/security.json
security.json:
{"authentication":{"class":"solr.BasicAuthPlugin","credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},"authorization":{"class":"solr.RuleBasedAuthorizationPlugin","user-role":{"solr":["admin"]},"permissions":[{"name":"security-edit","role":"admin"}]}}
* Start solr:
> solr-5.3.1/bin/solr start -e cloud -z localhost:2181
You would need to key in a few things e.g. #nodes and ports, leave them at
the default values of 2 nodes and 8983/7574, unless you want to run Solr on
a different port. Then let it create a default collection to just make sure
that everything works fine.
* Add the collection-admin-edit command:
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization
-H 'Content-type:application/json' -d '{"set-permission" :
{"name":"collection-admin-edit", "role":"admin"}}'
At this point, everything should be working fine. Restarting the nodes
should also work fine. You can try 2 things at this point:
1. Create a new collection with 1 shard and 1 replica and then try adding a
replica, here's how:
> curl --user solr:SolrRocks
http://localhost:8983/solr/admin/collections?action=CREATE&name=testcollection&collection.configName=gettingstarted&numShards=1
> curl --user solr:SolrRocks
http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=testcollection&shard=shard1
This should work fine.
2. After this, try restarting the solr cluster. Here's how you can do so,
assuming you didn't change any of the defaults and you are running zk on
localhost:2181. If not, just change those values below:
> bin/solr stop -all
After this, check that Solr was actually stopped. I'd also suggest you tail
the logs on both nodes when they are coming up to see any errors, if any.
The logs would be here: example/cloud/node1/logs/solr.log
and example/cloud/node2/logs/solr.log
> bin/solr start -c -p 8983 -s "example/cloud/node1/solr" -z localhost:2181
> bin/solr start -c -p 7574 -s "example/cloud/node2/solr" -z localhost:2181
If you get to this checkpoint fine, try adding a read permission.
Add a permission:
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization
-H 'Content-type:application/json' -d '{"set-permission" : {"name":"read",
"role":"read"}}'
Add a user:
> curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication
-H 'Content-type:application/json' -d '{"set-user" :
{"solrread":"solrRocks"}}'
Assign a role to the user:
>curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization
-H 'Content-type:application/json' -d '{"set-user-role" :
{"solrread":["read"]}}'
After this, you should start having issues with ADDREPLICA.
Also, as you would at this point have a collection with a shard that has a
replication factor > 1 (remember the ADDREPLICA we did earlier), you would
have issues when you restart the cluster again using the steps I mentioned
above.
Can you confirm this? I guess I'll just use this text to create a new JIRA
now.
On Fri, Nov 20, 2015 at 10:04 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:
> Thank you again for the reply.
>
> Below is the Email I was about to send prior to your reply a moment ago:
> shall I try again without "read" in the security.json?
>
>
>
> The Collections API method was not discussed in the "Unleashed" class at
> the conference in DC in 2014 (probably because it was not yet available),
> so I was using the method I knew.
>
> I have now tried again using admin/collections?action=CREATE (using
> different port numbers to avoid confusion from the failed previous
> attempts: the previously created nodes had been shutdown and their
> core.properties files renamed so as not to be discovered), but the results
> are the same:
> INFO - 2015-11-20 16:56:25.283; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Starting
> Replication Recovery.
> INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Begin
> buffering updates.
> INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Starting to
> buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> INFO - 2015-11-20 16:56:25.284; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Attempting
> to replicate from http://
> {IP-address-redacted}:4685/solr/xmpl3_shard1_replica1/.
> ERROR - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.common.SolrException; Error while
> trying to
> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at
> http://{IP-address-redacted}:4685/solr/xmpl3_shard1_replica1:
> Expected mime type application/octet-stream but got text/html. <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
> <title>Error 401 Unauthorized request, Response code: 401</title>
> </head>
> <body><h2>HTTP ERROR 401</h2>
> <p>Problem accessing /solr/xmpl3_shard1_replica1/update. Reason:
> <pre> Unauthorized request, Response code:
> 401</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>
>
> </body>
> </html>
>
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:528)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
> at
> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:207)
> at
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
> at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
> at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>
> INFO - 2015-11-20 16:56:25.292; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.update.UpdateLog; Dropping
> buffered updates FSUpdateLog{state=BUFFERING, tlog=null}
> ERROR - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Recovery
> failed - trying again... (2)
> INFO - 2015-11-20 16:56:25.293; [c:xmpl3 s:shard1 r:core_node2
> x:xmpl3_shard1_replica2] org.apache.solr.cloud.RecoveryStrategy; Wait 8.0
> seconds before trying to recover again (3)
>
>
> Below is a list of the steps I took.
>
> ./zkcli.sh --zkhost localhost:4545 -cmd makepath /solr/xmpl3
> ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd putfile /security.json
> ~/solr/security151119a.json
> ./zkcli.sh --zkhost localhost:4545/solr/xmpl3 -cmd upconfig -confdir
> ../../solr/configsets/basic_configs/conf -confname xmpl3
> cd ../../../bin/
> ./solr -c -p 4695 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
> localhost:4545/solr/xmpl3 -s
> ~dbman/solr/straight531outofbox/solr-5.3.1/example/solr
> ./solr -c -p 4685 -d ~dbman/solr/straight531outofbox/solr-5.3.1/server/ -z
> localhost:4545/solr/xmpl3 -s
> ~dbman/solr/straight531outofbox/solr-5.3.1/server/solr
> curl -u solr:SolrRocks '
> http://nosqltest11:4685/solr/admin/collections?action=CREATE&name=xmpl3&numShards=1&replicationFactor=1&createNodeSet={IP-address-redacted}:4685_solr
> '
> curl -u solr:SolrRocks '
> http://nosqltest11:4685/solr/admin/collections?action=ADDREPLICA&collection=xmpl3&shard=shard1&node={IP-address-redacted}:4695_solr&wt=json&indent=true
> '
>
>
>
>
> Can you provide a list of steps to take in an out-of-the-box directory
> tree whereby ADDREPLICA _will_ work with security.json already in place?
>
>
>
>
> -----Original Message-----
> From: Anshum Gupta [mailto:ans...@anshumgupta.net]
> Sent: Thursday, November 19, 2015 3:44 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Re:Re: Implementing security.json is breaking ADDREPLICA
>
> I'll try out what you did later in the day, as soon as I get time but why
> exactly are you creating cores manually? Seems like you manually create a
> core and the try to add a replica. Can you try using the Collections API to
> create a collection?
>
> Starting Solr 5.0, the only supported way to create a new collection is via
> the Collections API. Creating a core would lead to a collection creation
> but that's not really supported. It was just something that was done when
> there were no Collections API.
>
>
> On Thu, Nov 19, 2015 at 12:36 PM, Oakley, Craig (NIH/NLM/NCBI) [C] <
> craig.oak...@nih.gov> wrote:
>
> > I tried again with the following security.json, but the results were the
> > same:
> >
> > {
> > "authentication":{
> > "class":"solr.BasicAuthPlugin",
> > "credentials":{
> > "solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> > Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c=",
> > "solruser":"VgZX1TAMNHT2IJikoGdKtxQdXc+MbNwfqzf89YqcLEE=
> > 37pPWQ9v4gciIKHuTmFmN0Rv66rnlMOFEWfEy9qjJfY="},
> > "":{"v":9}},
> > "authorization":{
> > "class":"solr.RuleBasedAuthorizationPlugin",
> > "user-role":{
> > "solr":[
> > "admin",
> > "read",
> > "xmpladmin",
> > "xmplgen",
> > "xmplsel"],
> > "solruser":[
> > "read",
> > "xmplgen",
> > "xmplsel"]},
> > "permissions":[
> > {
> > "name":"security-edit",
> > "role":"admin"},
> > {
> > "name":"xmpl_admin",
> > "collection":"xmpl",
> > "path":"/admin/*",
> > "role":"xmpladmin"},
> > {
> > "name":"xmpl_sel",
> > "collection":"xmpl",
> > "path":"/select/*",
> > "role":null},
> > {
> > "name":"all-admin",
> > "collection":null,
> > "path":"/*",
> > "role":"xmplgen"},
> > {
> > "name":"all-core-handlers",
> > "path":"/*",
> > "role":"xmplgen"}],
> > "":{"v":42}}}
> >
> > -----Original Message-----
> > From: Oakley, Craig (NIH/NLM/NCBI) [C]
> > Sent: Thursday, November 19, 2015 1:46 PM
> > To: 'solr-user@lucene.apache.org' <solr-user@lucene.apache.org>
> > Subject: RE: Re:Re: Implementing security.json is breaking ADDREPLICA
> >
> > I note that the thread called "Security Problems" (most recent post by
> > Nobel Paul) seems like it may help with much of what I'm trying to do. I
> > will see to what extent that may help.
> >
>
>
>
> --
> Anshum Gupta
>
--
Anshum Gupta
