Kevin & Noble, I've manually verified the fix for SOLR-8000, but not yet for SOLR-8004.
I reproduced the initial problem with reloading security.json after restarting both Solr and ZooKeeper. I verified using zkcli.sh that ZooKeeper does retain the changes to the file after using /solr/admin/authorization, and that therefore the problem was Solr. After building solr-5.3.1-SNAPSHOT.tgz with ant package (because I don't know how to give parameters to ant server), I expanded it, copied in the core data, and then started it. I was prompted for a password, and it let me in once the password was given. I'll probably get to SOLR-8004 shortly, since I have both environments built and working. It also occurs to me that it might be better to forbid all permissions and grant specific permissions to specific roles. Is there a comprehensive list of the permissions available? On Tue, Sep 8, 2015 at 1:07 PM, Kevin Lee <kgle...@yahoo.com.invalid> wrote: > Thanks Dan! Please let us know what you find. I’m interested to know if > this is an issue with anyone else’s setup or if I have an issue in my local > configuration that is still preventing it to work on start/restart. > > - Kevin > > > On Sep 5, 2015, at 8:45 AM, Dan Davis <dansm...@gmail.com> wrote: > > > > Kevin & Noble, > > > > I'll take it on to test this. I've built from source before, and I've > > wanted this authorization capability for awhile. > > > > On Fri, Sep 4, 2015 at 9:59 AM, Kevin Lee <kgle...@yahoo.com.invalid> > wrote: > > > >> Noble, > >> > >> Does SOLR-8000 need to be re-opened? Has anyone else been able to test > >> the restart fix? > >> > >> At startup, these are the log messages that say there is no security > >> configuration and the plugins aren’t being used even though > security.json > >> is in Zookeeper: > >> 2015-09-04 08:06:21.205 INFO (main) [ ] o.a.s.c.CoreContainer > Security > >> conf doesn't exist. Skipping setup for authorization module. > >> 2015-09-04 08:06:21.205 INFO (main) [ ] o.a.s.c.CoreContainer No > >> authentication plugin used. > >> > >> Thanks, > >> Kevin > >> > >>> On Sep 4, 2015, at 5:47 AM, Noble Paul <noble.p...@gmail.com> wrote: > >>> > >>> There are no download links for 5.3.x branch till we do a bug fix > >> release > >>> > >>> If you wish to download the trunk nightly (which is not same as 5.3.0) > >>> check here > >> > https://builds.apache.org/job/Solr-Artifacts-trunk/lastSuccessfulBuild/artifact/solr/package/ > >>> > >>> If you wish to get the binaries for 5.3 branch you will have to make it > >>> (you will need to install svn and ant) > >>> > >>> Here are the steps > >>> > >>> svn checkout > >> http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_3/ > >>> cd lucene_solr_5_3/solr > >>> ant server > >>> > >>> > >>> > >>> On Fri, Sep 4, 2015 at 4:11 PM, davidphilip cherian > >>> <davidphilipcher...@gmail.com> wrote: > >>>> Hi Kevin/Noble, > >>>> > >>>> What is the download link to take the latest? What are the steps to > >> compile > >>>> it, test and use? > >>>> We also have a use case to have this feature in solr too. Therefore, > >> wanted > >>>> to test and above info would help a lot to get started. > >>>> > >>>> Thanks. > >>>> > >>>> > >>>> On Fri, Sep 4, 2015 at 1:45 PM, Kevin Lee <kgle...@yahoo.com.invalid> > >> wrote: > >>>> > >>>>> Thanks, I downloaded the source and compiled it and replaced the jar > >> file > >>>>> in the dist and solr-webapp’s WEB-INF/lib directory. It does seem to > >> be > >>>>> protecting the Collections API reload command now as long as I upload > >> the > >>>>> security.json after startup of the Solr instances. If I shutdown and > >> bring > >>>>> the instances back up, the security is no longer in place and I have > to > >>>>> upload the security.json again for it to take effect. > >>>>> > >>>>> - Kevin > >>>>> > >>>>>> On Sep 3, 2015, at 10:29 PM, Noble Paul <noble.p...@gmail.com> > wrote: > >>>>>> > >>>>>> Both these are committed. If you could test with the latest 5.3 > branch > >>>>>> it would be helpful > >>>>>> > >>>>>> On Wed, Sep 2, 2015 at 5:11 PM, Noble Paul <noble.p...@gmail.com> > >> wrote: > >>>>>>> I opened a ticket for the same > >>>>>>> https://issues.apache.org/jira/browse/SOLR-8004 > >>>>>>> > >>>>>>> On Wed, Sep 2, 2015 at 1:36 PM, Kevin Lee > <kgle...@yahoo.com.invalid > >>> > >>>>> wrote: > >>>>>>>> I’ve found that completely exiting Chrome or Firefox and opening > it > >>>>> back up re-prompts for credentials when they are required. It was > >>>>> re-prompting with the /browse path where authentication was working > >> each > >>>>> time I completely exited and started the browser again, however it > >> won’t > >>>>> re-prompt unless you exit completely and close all running instances > >> so I > >>>>> closed all instances each time to test. > >>>>>>>> > >>>>>>>> However, to make sure I ran it via the command line via curl as > >>>>> suggested and it still does not give any authentication error when > >> trying > >>>>> to issue the command via curl. I get a success response from all the > >> Solr > >>>>> instances that the reload was successful. > >>>>>>>> > >>>>>>>> Not sure why the pre-canned permissions aren’t working, but the > one > >> to > >>>>> the request handler at the /browse path is. > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Sep 1, 2015, at 11:03 PM, Noble Paul <noble.p...@gmail.com> > >> wrote: > >>>>>>>>> > >>>>>>>>> " However, after uploading the new security.json and restarting > the > >>>>>>>>> web browser," > >>>>>>>>> > >>>>>>>>> The browser remembers your login , So it is unlikely to prompt > for > >> the > >>>>>>>>> credentials again. > >>>>>>>>> > >>>>>>>>> Why don't you try the RELOAD operation using command line (curl) > ? > >>>>>>>>> > >>>>>>>>> On Tue, Sep 1, 2015 at 10:31 PM, Kevin Lee > >> <kgle...@yahoo.com.invalid> > >>>>> wrote: > >>>>>>>>>> The restart issues aside, I’m trying to lockdown usage of the > >>>>> Collections API, but that also does not seem to be working either. > >>>>>>>>>> > >>>>>>>>>> Here is my security.json. I’m using the “collection-admin-edit” > >>>>> permission and assigning it to the “adminRole”. However, after > >> uploading > >>>>> the new security.json and restarting the web browser, it doesn’t seem > >> to be > >>>>> requiring credentials when calling the RELOAD action on the > Collections > >>>>> API. The only thing that seems to work is the custom permission > >> “browse” > >>>>> which is requiring authentication before allowing me to pull up the > >> page. > >>>>> Am I using the permissions correctly for the > >> RuleBasedAuthorizationPlugin? > >>>>>>>>>> > >>>>>>>>>> { > >>>>>>>>>> "authentication":{ > >>>>>>>>>> "class":"solr.BasicAuthPlugin", > >>>>>>>>>> "credentials": { > >>>>>>>>>> "admin”:”<pass> <salt>", > >>>>>>>>>> "user": ”<pass> <salt>" > >>>>>>>>>> } > >>>>>>>>>> }, > >>>>>>>>>> "authorization":{ > >>>>>>>>>> "class":"solr.RuleBasedAuthorizationPlugin", > >>>>>>>>>> "permissions": [ > >>>>>>>>>> { > >>>>>>>>>> "name":"security-edit", > >>>>>>>>>> "role":"adminRole" > >>>>>>>>>> }, > >>>>>>>>>> { > >>>>>>>>>> "name":"collection-admin-edit”, > >>>>>>>>>> "role":"adminRole" > >>>>>>>>>> }, > >>>>>>>>>> { > >>>>>>>>>> "name":"browse", > >>>>>>>>>> "collection": "inventory", > >>>>>>>>>> "path": "/browse", > >>>>>>>>>> "role":"browseRole" > >>>>>>>>>> } > >>>>>>>>>> ], > >>>>>>>>>> "user-role": { > >>>>>>>>>> "admin": [ > >>>>>>>>>> "adminRole", > >>>>>>>>>> "browseRole" > >>>>>>>>>> ], > >>>>>>>>>> "user": [ > >>>>>>>>>> "browseRole" > >>>>>>>>>> ] > >>>>>>>>>> } > >>>>>>>>>> } > >>>>>>>>>> } > >>>>>>>>>> > >>>>>>>>>> Also tried adding the permission using the Authorization API, > but > >> no > >>>>> effect, still isn’t protecting the Collections API from being invoked > >>>>> without a username password. I do see in the Solr logs that it sees > >> the > >>>>> updates because it outputs the messages “Updating /security.json …”, > >>>>> “Security node changed”, “Initializing authorization plugin: > >>>>> solr.RuleBasedAuthorizationPlugin” and “Authentication plugin class > >>>>> obtained from ZK: solr.BasicAuthPlugin”. > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> Kevin > >>>>>>>>>> > >>>>>>>>>>> On Sep 1, 2015, at 12:31 AM, Noble Paul <noble.p...@gmail.com> > >>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> I'm investigating why restarts or first time start does not > read > >> the > >>>>>>>>>>> security.json > >>>>>>>>>>> > >>>>>>>>>>> On Tue, Sep 1, 2015 at 1:00 PM, Noble Paul < > noble.p...@gmail.com > >>> > >>>>> wrote: > >>>>>>>>>>>> I removed that statement > >>>>>>>>>>>> > >>>>>>>>>>>> "If activating the authorization plugin doesn't protect the > >> admin > >>>>> ui, > >>>>>>>>>>>> how does one protect access to it?" > >>>>>>>>>>>> > >>>>>>>>>>>> One does not need to protect the admin UI. You only need to > >> protect > >>>>>>>>>>>> the relevant API calls . I mean it's OK to not protect the CSS > >> and > >>>>>>>>>>>> HTML stuff. But if you perform an action to create a core or > >> do a > >>>>>>>>>>>> query through admin UI , it automatically will prompt you for > >>>>>>>>>>>> credentials (if those APIs are protected) > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, Sep 1, 2015 at 12:41 PM, Kevin Lee > >>>>> <kgle...@yahoo.com.invalid> wrote: > >>>>>>>>>>>>> Thanks for the clarification! > >>>>>>>>>>>>> > >>>>>>>>>>>>> So is the wiki page incorrect at > >>>>>>>>>>>>> > >>>>> > >> > https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin > >>>>> which says that the admin ui will require authentication once the > >>>>> authorization plugin is activated? > >>>>>>>>>>>>> > >>>>>>>>>>>>> "An authorization plugin is also available to configure Solr > >> with > >>>>> permissions to perform various activities in the system. Once > >> activated, > >>>>> access to the Solr Admin UI and all requests will need to be > >> authenticated > >>>>> and users will be required to have the proper authorization for all > >>>>> requests, including using the Admin UI and making any API calls." > >>>>>>>>>>>>> > >>>>>>>>>>>>> If activating the authorization plugin doesn't protect the > >> admin > >>>>> ui, how does one protect access to it? > >>>>>>>>>>>>> > >>>>>>>>>>>>> Also, the issue I'm having is not just at restart. According > >> to > >>>>> the docs security.json should be uploaded to Zookeeper before > starting > >> any > >>>>> of the Solr instances. However, I tried to upload security.json > before > >>>>> starting any of the Solr instances, but it would not pick up the > >> security > >>>>> config until after the Solr instances are already running and then > >>>>> uploading the security.json again. I can see in the logs at startup > >> that > >>>>> the Solr instances don't see any plugin enabled even though > >> security.json > >>>>> is already in zookeeper and then after they are started and the > >>>>> security.json is uploaded again I see it reconfigure to use the > plugin. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> Kevin > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Aug 31, 2015, at 11:22 PM, Noble Paul < > >> noble.p...@gmail.com> > >>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Admin UI is not protected by any of these permissions. Only > if > >>>>> you try > >>>>>>>>>>>>>> to perform a protected operation , it asks for a password. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I'll investigate the restart problem and report my findings > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Tue, Sep 1, 2015 at 3:10 AM, Kevin Lee > >>>>> <kgle...@yahoo.com.invalid> wrote: > >>>>>>>>>>>>>>> Anyone else running into any issues trying to get the > >>>>> authentication and authorization plugins in 5.3 working? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Aug 29, 2015, at 2:30 AM, Kevin Lee > >>>>> <kgle...@yahoo.com.INVALID> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I’m trying to use the new basic auth plugin for Solr 5.3 > and > >>>>> it doesn’t seem to be working quite right. Not sure if I’m missing > >> steps > >>>>> or there is a bug. I am able to get it to protect access to a URL > >> under a > >>>>> collection, but am unable to get it to secure access to the Admin UI. > >> In > >>>>> addition, after stopping the Solr and Zookeeper instances, the > >>>>> security.json is still in Zookeeper, however Solr is allowing access > to > >>>>> everything again like the security configuration isn’t in place. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Contents of security.json taken from wiki page, but edited > >> to > >>>>> produce valid JSON. Had to move comma after 3rd from last “}” up to > >> just > >>>>> after the last “]”. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> { > >>>>>>>>>>>>>>>> "authentication":{ > >>>>>>>>>>>>>>>> "class":"solr.BasicAuthPlugin", > >>>>>>>>>>>>>>>> > >>>>> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= > >>>>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="} > >>>>>>>>>>>>>>>> }, > >>>>>>>>>>>>>>>> "authorization":{ > >>>>>>>>>>>>>>>> "class":"solr.RuleBasedAuthorizationPlugin", > >>>>>>>>>>>>>>>> "permissions":[{"name":"security-edit", > >>>>>>>>>>>>>>>> "role":"admin"}], > >>>>>>>>>>>>>>>> "user-role":{"solr":"admin"} > >>>>>>>>>>>>>>>> }} > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Here are the steps I followed: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Upload security.json to zookeeper > >>>>>>>>>>>>>>>> ./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 > >>>>> -cmd putfile /security.json ~/solr/security.json > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Use zkCli.sh from Zookeeper to ensure the security.json is > >> in > >>>>> Zookeeper at /security.json. It is there and looks like what was > >>>>> originally uploaded. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Start Solr Instances > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Attempt to create a permission, however get the following > >>>>> error: > >>>>>>>>>>>>>>>> { > >>>>>>>>>>>>>>>> "responseHeader":{ > >>>>>>>>>>>>>>>> "status":400, > >>>>>>>>>>>>>>>> "QTime":0}, > >>>>>>>>>>>>>>>> "error":{ > >>>>>>>>>>>>>>>> "msg":"No authorization plugin configured", > >>>>>>>>>>>>>>>> "code":400}} > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Upload security.json again. > >>>>>>>>>>>>>>>> ./zkcli.sh -z localhost:2181,localhost:2182,localhost:2183 > >>>>> -cmd putfile /security.json ~/solr/security.json > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Issue the following to try to create the permission again > >> and > >>>>> this time it’s successful. > >>>>>>>>>>>>>>>> // Create a permission for mysearch endpoint > >>>>>>>>>>>>>>>> curl --user solr:SolrRocks -H > >>>>> 'Content-type:application/json' -d '{"set-permission": > >>>>> {"name":"mycollection-search","collection": > >>>>> “mycollection","path":”/mysearch","role": "search-user"}}' > >>>>> http://localhost:8983/solr/admin/authorization > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> { > >>>>>>>>>>>>>>>> "responseHeader":{ > >>>>>>>>>>>>>>>> "status":0, > >>>>>>>>>>>>>>>> "QTime":7}} > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Issue the following commands to add users > >>>>>>>>>>>>>>>> curl --user solr:SolrRocks > >>>>> http://localhost:8983/solr/admin/authentication -H > >>>>> 'Content-type:application/json' -d '{"set-user": {"admin" : > “password" > >> }}’ > >>>>>>>>>>>>>>>> curl --user solr:SolrRocks > >>>>> http://localhost:8983/solr/admin/authentication -H > >>>>> 'Content-type:application/json' -d '{"set-user": {"user" : “password" > >> }}' > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Issue the following command to add permission to users > >>>>>>>>>>>>>>>> curl -u solr:SolrRocks -H 'Content-type:application/json' > -d > >>>>> '{ "set-user-role" : {"admin": ["search-user", "admin"]}}' > >>>>> http://localhost:8983/solr/admin/authorization > >>>>>>>>>>>>>>>> curl -u solr:SolrRocks -H 'Content-type:application/json' > -d > >>>>> '{ "set-user-role" : {"user": ["search-user"]}}' > >>>>> http://localhost:8983/solr/admin/authorization > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> After executing the above, access to /mysearch is > protected > >>>>> until I restart the Solr and Zookeeper instances. However, the admin > >> UI is > >>>>> never protected like the Wiki page says it should be once activated. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>> > >> > https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin > >>>>> < > >>>>> > >> > https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin > >>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Why does the authentication and authorization plugin not > >> stay > >>>>> activated after restart and why is the Admin UI never protected? Am > I > >>>>> missing any steps? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Kevin > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> ----------------------------------------------------- > >>>>>>>>>>>>>> Noble Paul > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> -- > >>>>>>>>>>>> ----------------------------------------------------- > >>>>>>>>>>>> Noble Paul > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> ----------------------------------------------------- > >>>>>>>>>>> Noble Paul > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> ----------------------------------------------------- > >>>>>>>>> Noble Paul > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> ----------------------------------------------------- > >>>>>>> Noble Paul > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> ----------------------------------------------------- > >>>>>> Noble Paul > >>>>> > >>>>> > >>> > >>> > >>> > >>> -- > >>> ----------------------------------------------------- > >>> Noble Paul > >> > >> > >
