Re: Solr 5.4 leader selection

[Ishan Chattopadhyaya]

Maybe, you could have a look at REBALANCELEADERS to see if this solves your
usecase?
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-RebalanceLeaders

On Thu, Dec 24, 2015 at 3:54 PM,  wrote:

> Hi guys ,
> i have a solr cluster of 3 nodes using solr cloud , and a collection of 3
> shards , below is what's seen in cloud
> Node 1 : shard1_replica1   ,  shard2_replica2Node 2 : shard2_replica1
>   ,  shard3_replica2Node 3 : shard1_replica2, shard3_replica1
>
> Leaders :
> Shard 1 :Node 1Shard 2 :Node 2Shard 3 :Node 1
> i want to make shard 3's leader Node 3 to not make node 1 busy with
> leading both shard 1 and shard 3 , or solr automatically make load balance
> through the 3 nodes ?
> Thanks in advance
> --  Best Regards,
>  -- Tarek Abouzeid

Re: Kerberos ticket not renewing when storing index on Kerberized HDFS

[Ishan Chattopadhyaya]

Not sure how reliably renewals are taken care of in the context of
kerberized HDFS, but here's my 10-15 minute analysis.
Seems to me that the auto renewal thread is not spawned [0]. This relies on
kinit.
Not sure if having a login configuration with renewTGT is sufficient (which
seems to be passed in by default, unless there's a jaas config being
explicitly passed in with renewTGT=false). As per the last comments from
Devraj & Owen [1] kinit based logins have worked more reliably.

If you can rule out any setup issues, I suggest you file a JIRA and someone
who has worked on the HdfsDirectoryFactory would be able to suggest better.
Thanks,
Ishan

[0] -
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-common/2.7.1/org/apache/hadoop/security/UserGroupInformation.java#UserGroupInformation.spawnAutoRenewalThreadForUserCreds%28%29

[1] - https://issues.apache.org/jira/browse/HADOOP-6656

On Fri, Jan 8, 2016 at 10:21 PM, Andrew Bumstead <
andrew.bumst...@bigdatapartnership.com> wrote:

> Hello,
>
> I have Solr Cloud configured to stores its index files on a Kerberized HDFS
> (I followed documentation at
> https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS),
> and
> have been able to index some documents with the files being written to the
> HDFS as expected. However, it appears that some time after starting, Solr
> is unable to connect to HDFS as it no longer has a valid Kerberos TGT. The
> time-frame of this occurring is consistent with my default Kerberos ticket
> lifetime of 24 hours, so it appears as though Solr is not renewing its
> Kerberos ticket upon expiry. A restart of Solr resolves the issue again for
> 24 hours.
>
> Is there any configuration I can add to make Solr automatically renew its
> ticket or is this an issue with Solr?
>
> The following is the stack trace I am getting in Solr.
>
> java.io.IOException: Failed on local exception: java.io.IOException:
> Couldn't setup connection for solr/sandbox.hortonworks@hortonworks.com
> to sandbox.hortonworks.com/10.0.2.15:8020; Host Details : local host is: "
> sandbox.hortonworks.com/10.0.2.15"; destination host is: "
> sandbox.hortonworks.com":8020;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> at org.apache.hadoop.ipc.Client.call(Client.java:1472)
> at org.apache.hadoop.ipc.Client.call(Client.java:1399)
> at
>
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
> at com.sun.proxy.$Proxy10.renewLease(Unknown Source)
> at
>
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.renewLease(ClientNamenodeProtocolTranslatorPB.java:571)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy11.renewLease(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:879)
> at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:417)
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:442)
> at
> org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:298)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Couldn't setup connection for solr/
> sandbox.hortonworks@hortonworks.com to
> sandbox.hortonworks.com/10.0.2.15:8020
> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:672)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
>
> org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:643)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:730)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1521)
> at org.apache.hadoop.ipc.Client.call(Client.java:1438)
> ... 16 more
> Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused
> by GSSException: No valid credentials provided (Mechanism level: Failed to
> find any Kerberos tgt)]
> at
>
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
> at
>
> org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413)
> at
>
> org.apache.hadoop.ipc.Client$Connection

Re: Rolling upgrade to 5.4 from 5.0 - "bug" caused by leader changes - is there a workaround?

[Ishan Chattopadhyaya]

Perhaps you could stay on 5.4.1 RC2, since that is what 5.4.1 will be
(unless there are last moment issues).

On Wed, Jan 20, 2016 at 7:50 PM, Michael Joyner  wrote:

> Unfortunately, it really couldn't wait.
>
> I did a rolling upgrade to the 5.4.1RC2 then downgraded everything to
> 5.4.0 and so far everything seems fine.
>
> Couldn't take the cluster down.
>
>
> On 01/19/2016 05:03 PM, Anshum Gupta wrote:
>
>> If you can wait, I'd suggest to be on the bug fix release. It should be
>> out
>> around the weekend.
>>
>> On Tue, Jan 19, 2016 at 1:48 PM, Michael Joyner 
>> wrote:
>>
>> ok,
>>>
>>> I just found the 5.4.1 RC2 download, it seems to work ok for a rolling
>>> upgrade.
>>>
>>> I will see about downgrading back to 5.4.0 afterwards to be on an
>>> official
>>> release ...
>>>
>>>
>>>
>>> On 01/19/2016 04:27 PM, Michael Joyner wrote:
>>>
>>> Hello all,

 I downloaded 5.4 and started doing a rolling upgrade from a 5.0
 solrcloud
 cluster and discovered that there seems to be a compatibility issue
 where
 doing a rolling upgrade from pre-5.4 which causes the 5.4 to fail with
 unable to determine leader errors.

 Is there a work around that does not require taking the cluster down to
 upgrade to 5.4? Should I just stay with 5.3 for now? I need to implement
 programmatic schema changes in our collection via solrj, and based on
 what
 I'm reading this is a very new feature and requires the latest (or near
 latest) solrcloud.

 Thanks!

 -Mike


>>>
>>
>

Re: Solr-kerbarose URL not accessible

[Ishan Chattopadhyaya]

This appears to be Cloudera search specific. The kerberos support in Solr
is similar to, but not identical with, the kerberos support in Cloudera's
Search. Maybe you could check with Cloudera's support?


On Fri, Feb 12, 2016 at 8:06 PM, Shawn Heisey  wrote:

> On 2/12/2016 4:28 AM, vidya wrote:
> >   When I am trying to access my solrCloud web UI page, deployed in
> cloudera
> > cluster, I have encountered with the error "DEFECTED TOKENS DETECTED" .
> Find
> > the attachment of the error that is added here. It is because of
> kerbarose
> > installed on cluster.
> >
> > Is there any other way that I can access solr in this scenario with
> > kerbarose installed ?
> > Writing a java program helps in any way? While writing a java program
> also,
> > i have to give connection to solr URL with port or zookeeper host
> variable.
> > Will that java program work out?
>
> Thismailing list will filter out most attachments.  It looks like you're
> accessing the list through the Nabble forum, but I still don't see any
> files even when I visit the Nabble website.  Without your attachment(s),
> I can't see the problem, so I cannot offer any advice.
>
> The best option is to place the relevant data on a website like gist or
> dropbox and provide a link to that information.
>
> The error message you have described does not appear in the Solr source
> code, so it must be coming from whatever Kerberos software is being used
> for the authentication, or maybe from the customizations that Cloudera
> has made to Solr in their search product.
>
> I keep coming back to the fact that I can't actually see the full error
> message and Java stacktrace, because your attachments are not available.
>
> Thanks,
> Shawn
>
>

Re: how to update billions of docs

[Ishan Chattopadhyaya]

Hi Mohsin,
There's some work in progress for in-place updates to docValued fields,
https://issues.apache.org/jira/browse/SOLR-5944. Can you try the latest
patch there (or ping me if you need a git branch)?
It would be nice to know how fast the updates go for your usecase with that
patch. Please note that for that patch, both the version field and the
updated field needs to have stored=false, indexed=false, docValues=true.
Regards,
Ishan


On Thu, Mar 17, 2016 at 10:55 PM, Jack Krupansky 
wrote:

> It would be nice to have a wiki/doc for "Bulk Field Update" that listed all
> of these techniques and tricks.
>
> And, of course, it would be so much better to have an explicit Lucene
> feature for this. It could work in the background like merge and process
> one segment at a time as efficiently as possible.
>
> Have several modes:
>
> 1. Set a field of all documents to explicit value.
> 2. Set a field of query documents to an explicit value.
> 3. Increment by n.
> 4. Add new field to all document, or maybe by query.
> 5. Delete existing field for all documents.
> 6. Delete field value for all documents or a specified query.
>
>
> -- Jack Krupansky
>
> On Thu, Mar 17, 2016 at 12:31 PM, Ken Krugler  >
> wrote:
>
> > As others noted, currently updating a field means deleting and inserting
> > the entire document.
> >
> > Depending on how you use the field, you might be able to create another
> > core/container with that one field (plus the key field), and use join
> > support.
> >
> > Note that https://issues.apache.org/jira/browse/LUCENE-6352 is an
> > improvement, which looks like it's in the 5.x code line, though I don't
> see
> > a fix version.
> >
> > -- Ken
> >
> > > From: Mohsin Beg Beg
> > > Sent: March 16, 2016 3:52:47pm PDT
> > > To: solr-user@lucene.apache.org
> > > Subject: how to update billions of docs
> > >
> > > Hi,
> > >
> > > I have a requirement to replace a value of a field in 100B's of docs in
> > 100's of cores.
> > > The field is multiValued=false docValues=true type=StrField stored=true
> > indexed=true.
> > >
> > > Atomic Updates performance is on the order of 5K docs per sec per core
> > in solr 5.3 (other fields are quite big).
> > >
> > > Any suggestions ?
> > >
> > > -Mohsin
> >
> >
> > --
> > Ken Krugler
> > +1 530-210-6378
> > http://www.scaleunlimited.com
> > custom big data solutions & training
> > Hadoop, Cascading, Cassandra & Solr
> >
> >
> >
> >
> >
> >
>

Re: Sleep function query a big security hole?

[Ishan Chattopadhyaya]

+1, I agree. Opened https://issues.apache.org/jira/browse/SOLR-8099
Thanks,
Ishan

On Sun, Sep 27, 2015 at 5:22 AM, Doug Turnbull <
dturnb...@opensourceconnections.com> wrote:

> Relevant code
>
> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.solr/solr-core/5.2.0/org/apache/solr/search/ValueSourceParser.java#126
>
> On Saturday, September 26, 2015, Doug Turnbull <
> dturnb...@opensourceconnections.com> wrote:
>
> > I noticed a while back that "sleep" is a function query. Which I
> > believe means I can make the current query thread sleep for as long as I
> > like.
> >
> > I'm guessing an attacker could use this to starve Solr of threads,
> running
> > a denial of service attack by running multiple queries with sleeps in
> them.
> >
> > Is this a concern? I realize there may be test purposes to sleep a
> > function query, but I'm trying to think if there's really practical
> purpose
> > to having sleep here.
> >
> > Best,
> > -Doug
> >
> >
> > --
> > *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> > , LLC | 240.476.9983
> > Author: Relevant Search 
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless
> > of whether attachments are marked as such.
> >
> >
>
> --
> *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> , LLC | 240.476.9983
> Author: Relevant Search 
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>

Re: Find records with no values in solr.LatLongType fied type

[Ishan Chattopadhyaya]

There's also a function, exists(), which might work here, and result in a
neater query.
e.g. something like: q=*:* -exists(usrlatlong_0_coordinate)
Haven't tried it, though.
https://cwiki.apache.org/confluence/display/solr/Function+Queries#FunctionQueries-AvailableFunctions

On Wed, Sep 30, 2015 at 8:17 PM, Kamal Kishore Aggarwal <
kkroyal@gmail.com> wrote:

> Thanks Erick..it worked..
>
> On Wed, Sep 16, 2015 at 9:21 PM, Erick Erickson 
> wrote:
>
> > Top level queries need a *:* in front, something like
> > q=*:* -usrlatlong_0_coordinate:[* TO *]
> >
> > I just took a quick check and just using usrlatlong:[* TO *]
> > encounters a parse error.
> >
> > P.S. It would help if you told us what you _did_ receive
> > when you tried your options. Parse errors? All docs?
> >
> > Best,
> > Erick
> >
> > On Mon, Sep 14, 2015 at 10:58 PM, Kamal Kishore Aggarwal
> >  wrote:
> > > Hi,
> > >
> > > I am working on solr 4.8,1. I am trying to find the docs where
> > latlongtype
> > > have null values.
> > >
> > > I have tried using these, but not getting the results :
> > >
> > > 1) http://localhost:8984/solr/IM-Search/select?q.alt=-usrlatlong:[' '
> > TO *]
> > >
> > > 2) http://localhost:8984/solr/IM-Search/select?q.alt=-usrlatlong:[* TO
> > *]
> > >
> > > Here's the configurations :
> > >>  > >> subFieldSuffix="_coordinate"/>
> > >>  stored="true"
> > >> required="false" multiValued="false" />
> > >
> > >
> > > Please help.
> >
>

Re: Passing Basic Auth info to HttpSolrClient

[Ishan Chattopadhyaya]

In latest Solr release, you can use the basic auth plugins for
authentication instead of doing something at the Jetty level.
https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin
Right at the end, there's a note on how to use SolrJ with this.

Also, there exists: https://issues.apache.org/jira/browse/SOLR-8053 which
is due in Solr 5.4 release.

On Wed, Sep 30, 2015 at 7:28 PM, Davis, Daniel (NIH/NLM) [C] <
daniel.da...@nih.gov> wrote:

> HttpSolrClient can accept the Apache Commons HttpClient in its constructor:
>
> https://lucene.apache.org/solr/5_3_1/solr-solrj/org/apache/solr/client/solrj/impl/HttpSolrClient.html
>
> You can use the HttpClientBuilder (
> http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html),
> to build an HttpClient that does Basic Authentication, and then pass client
> to the SolrHttpClient constructor.
>
> A search on "HttpClientBuilder Basic Authentication" returned many hits,
> but here's one of them:
>
> http://www.baeldung.com/httpclient-4-basic-authentication
>
> Hope this helps,
>
> Dan Davis, Systems/Applications Architect (Contractor),
> Office of Computer and Communications Systems,
> National Library of Medicine, NIH
>
>
> -Original Message-
> From: Steven White [mailto:swhite4...@gmail.com]
> Sent: Tuesday, September 29, 2015 8:13 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Passing Basic Auth info to HttpSolrClient
>
> Hi,
>
> Re-posting to see if anyone can help.  If my question is not clear, let me
> know.
>
> Thanks!
>
> Steve
>
> On Mon, Sep 28, 2015 at 5:15 PM, Steven White 
> wrote:
>
> > Hi,
> >
> > I'm using HttpSolrClient to connect to Solr.  Everything works until
> > when I enabled basic authentication in Jetty.  My question is, how do
> > I pass to SolrJ the basic auth info. so that I don't get a 401 error?
> >
> > Thanks in advance
> >
> > Steve
> >
>

Re: Is it impossible to update an index that is undergoing an optimize?

[Ishan Chattopadhyaya]

On Sat, Nov 7, 2015 at 9:09 AM, Yonik Seeley  wrote:

> On Fri, Nov 6, 2015 at 10:20 PM, Shawn Heisey  wrote:
> >  Is there a decent API for getting uniqueKey?
>
> Not off the top of my head.
> I deeply regret making it configurable and not just using "id" ;-)
>

Maybe this?
https://cwiki.apache.org/confluence/display/solr/Schema+API#SchemaAPI-ListUniqueKey


>
> -Yonik
>

Re: DELETEREPLICA command shouldn't delete de last replica of a shard

[Ishan Chattopadhyaya]

On Sun, Nov 8, 2015 at 11:20 PM, Erick Erickson 
wrote:

> I see your point. In the custom routing case where it _is_ reasonable
> to delete all replicas in a shard, DELETESHARD does the trick.
>
> It's reasonable to raise a JIRA I think so we have a record of the
> discussion/decision.
>

There is a related, but maybe not exactly same, JIRA:
https://issues.apache.org/jira/browse/SOLR-5209


>
> Erick
>
> On Sun, Nov 8, 2015 at 9:13 AM, Yago Riveiro 
> wrote:
> > I don't know if this behaviour makes sense but in my IMHO the last
> replica of
> > a shard should be removed only by the DELETESHARD command ...
> >
> > If you don't not notice about this behaviour, the result is the deletion
> of
> > the whole shard, and as a consequence, the deletion of the shard in the
> > clusterstate.json ... with all the pain associated.
> >
> >
> >
> > -
> > Best regards
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/DELETEREPLICA-command-shouldn-t-delete-de-last-replica-of-a-shard-tp4239054.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Jetty Vs Tomcat (Performance issue)

[Ishan Chattopadhyaya]

Also, what are the specific performance issues you are observing?

On Mon, Nov 16, 2015 at 6:41 PM, Upayavira  wrote:

> Just to be sure, are you installing Solr inside a different Jetty, or
> using the Jetty that comes with Solr?
>
> You would be expected to use the one installed and managed by Solr.
>
> Upayavira
>
> On Mon, Nov 16, 2015, at 11:58 AM, Behzad Qureshi wrote:
> > Hi All,
> >
> > I am using Tomcat server with solr 4.10.3. I want to shift to Jetty as
> > replacement of Tomcat server but I am not getting any good results with
> > respect to performance. I have tried solr 4.10.3 on both Jetty 8 and
> > Jetty
> > 9 with java 8. Below are configurations I have used.
> >
> > Can anyone please tell me if I am missing anything?
> >
> > *Jetty:*
> >
> > Xms: 5GB
> > Xmx: 50GB
> > Xss: 256MB
> >
> >
> > *Tomcat:*
> >
> > Xms: 5GB
> > Xmx: 50GB
> > Xss: Default
> >
> >
> > *Index Size:*
> >
> > 1TB (20 cores)
> >
> >
> >
> > --
> >
> > Regards,
> >
> > Behzad Qureshi
>

Re: Authorization API versus zkcli.sh

[Ishan Chattopadhyaya]

As I see things, if you're using Basic Authentication and/or Rule Based
Authorization, then the security.json could be copied over to start things.

On Sat, Dec 5, 2015 at 3:38 AM, Oakley, Craig (NIH/NLM/NCBI) [C] <
craig.oak...@nih.gov> wrote:

> Looking through
> cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
> one notices that security.json is initially created by zkcli.sh, and then
> modified by means of the Authentication API and the Authorization API. By
> and large, this sounds like a good way to accomplish such tasks, assuming
> that these APIs do some error checking to prevent corruption of
> security.json
>
> I was wondering about cases where one is cloning an existing Solr
> instance, such as when creating an instance in Amazon Cloud. If one has a
> security.json that has been thoroughly tried and successfully tested on
> another Solr instance, is it possible / safe / not-un-recommended to use
> zkcli.sh to load the full security.json (as extracted via zkcli.sh from the
> Zookeeper of the thoroughly tested existing instance)? Or would the
> official verdict be that the only acceptable way to create security.json is
> to load a minimal version with zkcli.sh and then to build the remaining
> components with the Authentication API and the Authorization API (in a
> script, if one wants to automate the process: although such a script would
> have to include plain-text passwords)?
>
> I figured there is no harm in asking.
>

Re: How to config security.json?

[Ishan Chattopadhyaya]

FYI,
http://lucidworks.com/blog/2015/08/17/securing-solr-basic-auth-permission-rules/

On Thu, Nov 19, 2015 at 4:21 PM, Byzen Ma  wrote:

> Hi, I'm not quite familar with security.json. I want to achieve these
> implementations. (1)Anyone who wants to do read/select/query action should
> be required passwd and username, namely authentication, no matter from
> Admin
> UI and solrj, especially from Admin UI! For I need to restrict strangers to
> reach my solr. (2)Anyone who wants to update, includes delete, create and
> so
> on, also needs passwd and username same as (1). I config the security.json
> as follows:
>
>
>
> {
>
>   "authentication":{
>
> "class":"solr.BasicAuthPlugin",
>
> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}},
>
>   "authorization":{
>
> "class":"solr.RuleBasedAuthorizationPlugin",
>
> "user-role":{"solr":"admin"},
>
> "permissions":[
>
>   {
>
> "name":"security-edit",
>
> "role":"admin"},
>
>   {
>
> "name":"read",
>
> "role":"admin"},
>
>   {
>
> "name":"update",
>
> "role":"admin"}],
>
> "":{"v":3}}}
>
>
>
> However, it does't work! More than that, an error happened again and again
> when node2 recover from node1. I start my solrcloud by commandLine with
> "solr start -e cloud" and set all the configs as default. How can I set the
> security.json to achieve my goals? Here are the server logs:
>
>
>
> ERROR (RecoveryThread-gettingstarted_shard2_replica2) [c:gettingstarted
> s:shard2 r:core_node2 x:gettingstarted_shard2_replica2]
> o.a.s.c.RecoveryStrategy Error while trying to
>
> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException
> : Error from server at
> http://(myhost):7574/solr/gettingstarted_shard2_replica1: Expected mime
> type
> application/octet-stream but got text/html. 
>
> 
>
> 
>
> Error 401 Unauthorized request, Response code: 401
>
> 
>
> HTTP ERROR 401
>
> Problem accessing /solr/gettingstarted_shard2_replica1/update. Reason:
>
> Unauthorized request, Response code:
> 401Powered by Jetty://
>
>
>
> 
>
> 
>
>
>
>  at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClien
> t.java:528)
>
>  at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java
> :234)
>
>  at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java
> :226)
>
>  at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
>
>  at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
>
>  at
>
> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:
> 207)
>
>  at
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
>
>  at
>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
>
>  at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>
> Error while trying to
>
> recover:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException
> : Error from server at
> http://(myhost):7574/solr/gettingstarted_shard2_replica1: Expected mime
> type
> application/octet-stream but got text/html. 
>
> 
>
> 
>
> Error 401 Unauthorized request, Response code: 401
>
> 
>
> HTTP ERROR 401
>
> Problem accessing /solr/gettingstarted_shard2_replica1/update. Reason:
>
> Unauthorized request, Response code:
> 401Powered by Jetty://
>
>
>
> 
>
> 
>
>
>
>  at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClien
> t.java:528)
>
>  at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java
> :234)
>
>  at
>
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java
> :226)
>
>  at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
>
>  at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152)
>
>  at
>
> org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:
> 207)
>
>  at
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:147)
>
>  at
>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:437)
>
>  at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:227)
>
> 1531614 ERROR (qtp5264648-21) [c:gettingstarted s:shard1 r:core_node1
> x:gettingstarted_shard1_replica2] o.a.s.h.a.ShowFileRequestHandler Can not
> find: /configs/gettingstarted/admin-extra.menu-top.html
>
> 1531614 ERROR (qtp5264648-16) [c:gettingstarted s:shard1 r:core_node1
> x:gettingstarted_shard1_replica2] o.a.s.h.a.ShowFileRequestHandler Can not
> find: /configs/gettingstarted/admin-extra.menu-bottom.html
>
> 1531661 ERROR (qtp5264648-14) [c:gettingstarted s:shard1 r:core_node1
> x:gettingstarted_shard1_replica2] o.a.s.h.a.ShowFileRequest

Re: Sold 5.3 with Basic Authentication in a not SolrCloud environment

[Ishan Chattopadhyaya]

At the moment, the Basic Authentication doesn't work with standalone Solr
(non-SolrCloud). Kerberos authentication works.
If you want support for Basic Authentication for standalone Solr, please
feel free to open a JIRA and contribute a patch.

On Wed, Sep 16, 2015 at 3:30 PM, Dominique Bejean  wrote:

> Hi,
>
> The wiki explains how to upload the security.json file to Zk (
>
> https://cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
> ).
>
> However, is it possible to use authentication and authorization plugin in a
> not SolrCloud environment ? If yes, where has to be located the
> security.json file ?
>
> Thank you.
>
> Dominique
>

Re: secure solr 5.3.1

[Ishan Chattopadhyaya]

Right, as Bosco said, this has been tested well and supported on SolrCloud.
It should be possible to run it in standalone mode, but it is not something
that has been well test yet.

On Tue, Dec 8, 2015 at 11:02 PM, Don Bosco Durai  wrote:

> It was tested and meant to work only in SolrCloud mode.
>
>
>
>
>
>
> On Tue, Dec 8, 2015 at 9:30 AM -0800, "kostali hassan" <
> med.has.kost...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
>
>- Kerberos authentication
>:
>work in SolrCloud or standalone mode but the documentation is not clear
>-
>
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin?focusedCommentId=61331746#comment-61331746
>
>
> 2015-12-08 17:14 GMT+00:00 Don Bosco Durai :
>
> > Not sure exactly what you mean here. Even if you are running in
> SolrCloud,
> > you can access it using URL. So there won't be any change on the client
> > side.
> > Bosco
> >
> >
> >
> >
> >
> >
> > On Tue, Dec 8, 2015 at 2:03 AM -0800, "kostali hassan" <
> > med.has.kost...@gmail.com> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > if I run solr in SolrCloud mode , my web hosting shoud be Cloud web
> > hosting? or dont need a web server having cloud..?
> >
> > 2015-12-08 1:58 GMT+00:00 Don Bosco Durai :
> >
> > > Have you considered running your Solr as SolrCloud with embedded
> > zookeeper?
> > >
> > > If you do, you have multiple options. Basic Auth, Kerberos and
> > > authorization support.
> > >
> > >
> > > Bosco
> > >
> > >
> > >
> > >
> > >
> > > On 12/7/15, 7:03 AM, "kostali hassan"  wrote:
> > >
> > > >How I shoud secure my server of solr 5 .3.1 in  single-node Mode. I Am
> > > >searching for the best way to secure my server solr but I found only
> for
> > > >cloud mode.
> > >
> > >
> >
> >
> >
> >
> >
> >
>
>
>
>
>
>

Re: secure solr 5.3.1

[Ishan Chattopadhyaya]

The kerberos plugin is available for use with Solr out of the box. The two
resources which Bosco mentioned should get you up and running.

On Wed, Dec 9, 2015 at 11:34 PM, Don Bosco Durai  wrote:

> There are two resources available:
>
>
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin
>
>
>
> https://cwiki.apache.org/confluence/display/RANGER/How+to+configure+Solr+Cloud+with+Kerberos+for+Ranger+0.5
>
>
> Bosco
>
>
>
>
>
> On 12/9/15, 3:14 AM, "kostali hassan"  wrote:
>
> >how I setting up Solr to use Kerberos ? i have to dowload kerberos and put
> >the plug-in implementation in the classpath(/server/solr).
> >
> >2015-12-08 22:19 GMT+00:00 Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
> >
> >> Right, as Bosco said, this has been tested well and supported on
> SolrCloud.
> >> It should be possible to run it in standalone mode, but it is not
> something
> >> that has been well test yet.
> >>
> >> On Tue, Dec 8, 2015 at 11:02 PM, Don Bosco Durai 
> wrote:
> >>
> >> > It was tested and meant to work only in SolrCloud mode.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Tue, Dec 8, 2015 at 9:30 AM -0800, "kostali hassan" <
> >> > med.has.kost...@gmail.com> wrote:
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >- Kerberos authentication
> >> >:
> >> >work in SolrCloud or standalone mode but the documentation is not
> >> clear
> >> >-
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin?focusedCommentId=61331746#comment-61331746
> >> >
> >> >
> >> > 2015-12-08 17:14 GMT+00:00 Don Bosco Durai :
> >> >
> >> > > Not sure exactly what you mean here. Even if you are running in
> >> > SolrCloud,
> >> > > you can access it using URL. So there won't be any change on the
> client
> >> > > side.
> >> > > Bosco
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > On Tue, Dec 8, 2015 at 2:03 AM -0800, "kostali hassan" <
> >> > > med.has.kost...@gmail.com> wrote:
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > if I run solr in SolrCloud mode , my web hosting shoud be Cloud web
> >> > > hosting? or dont need a web server having cloud..?
> >> > >
> >> > > 2015-12-08 1:58 GMT+00:00 Don Bosco Durai :
> >> > >
> >> > > > Have you considered running your Solr as SolrCloud with embedded
> >> > > zookeeper?
> >> > > >
> >> > > > If you do, you have multiple options. Basic Auth, Kerberos and
> >> > > > authorization support.
> >> > > >
> >> > > >
> >> > > > Bosco
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On 12/7/15, 7:03 AM, "kostali hassan"  wrote:
> >> > > >
> >> > > > >How I shoud secure my server of solr 5 .3.1 in  single-node
> Mode. I
> >> Am
> >> > > > >searching for the best way to secure my server solr but I found
> only
> >> > for
> >> > > > >cloud mode.
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
>
>

Re: secure solr 5.3.1

[Ishan Chattopadhyaya]

What exactly is your confusion? Do you have access to a KDC?

Briefly:
Login to your KDC server, do a kadmin.local:
Then,
addprinc HTTP/192.168.0.107
ktadd -k /tmp/107.keytab HTTP/192.168.0.107

Then copy the keytab file to your solr node to the appropriate places.


On Thu, Dec 10, 2015 at 12:08 AM, kostali hassan 
wrote:

> I folow this two resources and Iam stuck in
>
>- Create service principals and keytab files.
>
>
> 2015-12-09 18:06 GMT+00:00 Ishan Chattopadhyaya  >:
>
> > The kerberos plugin is available for use with Solr out of the box. The
> two
> > resources which Bosco mentioned should get you up and running.
> >
> > On Wed, Dec 9, 2015 at 11:34 PM, Don Bosco Durai 
> wrote:
> >
> > > There are two resources available:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin
> > >
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/RANGER/How+to+configure+Solr+Cloud+with+Kerberos+for+Ranger+0.5
> > >
> > >
> > > Bosco
> > >
> > >
> > >
> > >
> > >
> > > On 12/9/15, 3:14 AM, "kostali hassan" 
> wrote:
> > >
> > > >how I setting up Solr to use Kerberos ? i have to dowload kerberos and
> > put
> > > >the plug-in implementation in the classpath(/server/solr).
> > > >
> > > >2015-12-08 22:19 GMT+00:00 Ishan Chattopadhyaya <
> > > ichattopadhy...@gmail.com>:
> > > >
> > > >> Right, as Bosco said, this has been tested well and supported on
> > > SolrCloud.
> > > >> It should be possible to run it in standalone mode, but it is not
> > > something
> > > >> that has been well test yet.
> > > >>
> > > >> On Tue, Dec 8, 2015 at 11:02 PM, Don Bosco Durai 
> > > wrote:
> > > >>
> > > >> > It was tested and meant to work only in SolrCloud mode.
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Tue, Dec 8, 2015 at 9:30 AM -0800, "kostali hassan" <
> > > >> > med.has.kost...@gmail.com> wrote:
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >- Kerberos authentication
> > > >> >:
> > > >> >work in SolrCloud or standalone mode but the documentation is
> not
> > > >> clear
> > > >> >-
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin?focusedCommentId=61331746#comment-61331746
> > > >> >
> > > >> >
> > > >> > 2015-12-08 17:14 GMT+00:00 Don Bosco Durai :
> > > >> >
> > > >> > > Not sure exactly what you mean here. Even if you are running in
> > > >> > SolrCloud,
> > > >> > > you can access it using URL. So there won't be any change on the
> > > client
> > > >> > > side.
> > > >> > > Bosco
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Tue, Dec 8, 2015 at 2:03 AM -0800, "kostali hassan" <
> > > >> > > med.has.kost...@gmail.com> wrote:
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > if I run solr in SolrCloud mode , my web hosting shoud be Cloud
> > web
> > > >> > > hosting? or dont need a web server having cloud..?
> > > >> > >
> > > >> > > 2015-12-08 1:58 GMT+00:00 Don Bosco Durai :
> > > >> > >
> > > >> > > > Have you considered running your Solr as SolrCloud with
> embedded
> > > >> > > zookeeper?
> > > >> > > >
> > > >> > > > If you do, you have multiple options. Basic Auth, Kerberos and
> > > >> > > > authorization support.
> > > >> > > >
> > > >> > > >
> > > >> > > > Bosco
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On 12/7/15, 7:03 AM, "kostali hassan"  wrote:
> > > >> > > >
> > > >> > > > >How I shoud secure my server of solr 5 .3.1 in  single-node
> > > Mode. I
> > > >> Am
> > > >> > > > >searching for the best way to secure my server solr but I
> found
> > > only
> > > >> > for
> > > >> > > > >cloud mode.
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > >
> > >
> >
>

Re: secure solr 5.3.1

[Ishan Chattopadhyaya]

Alternatively, you could also try using BasicAuth, which doesn't require
this additional setup of a KDC.

On Thu, Dec 10, 2015 at 1:04 AM, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I don't have much personal experience with setting up a kerberos server on
> a Windows machine, but I remember things being painful when I tried and
> failed once. If you have an option to use a VM, I suggest try setting up
> the KDC in a GNU/Linux VM (through VirtualBox).
> In that case, make sure the Windows host is able to access the ports of
> the guest machine. Better still, setup GNU/Linux VMs for Solr too.
>
> On Thu, Dec 10, 2015 at 12:39 AM, kostali hassan <
> med.has.kost...@gmail.com> wrote:
>
>> I install MIT Kerberos for Windows 4.0.1
>>
>> 2015-12-09 19:05 GMT+00:00 Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>>
>> > What exactly is your confusion? Do you have access to a KDC?
>> >
>> > Briefly:
>> > Login to your KDC server, do a kadmin.local:
>> > Then,
>> > addprinc HTTP/192.168.0.107
>> > ktadd -k /tmp/107.keytab HTTP/192.168.0.107
>> >
>> > Then copy the keytab file to your solr node to the appropriate places.
>> >
>> >
>> > On Thu, Dec 10, 2015 at 12:08 AM, kostali hassan <
>> > med.has.kost...@gmail.com>
>> > wrote:
>> >
>> > > I folow this two resources and Iam stuck in
>> > >
>> > >- Create service principals and keytab files.
>> > >
>> > >
>> > > 2015-12-09 18:06 GMT+00:00 Ishan Chattopadhyaya <
>> > ichattopadhy...@gmail.com
>> > > >:
>> > >
>> > > > The kerberos plugin is available for use with Solr out of the box.
>> The
>> > > two
>> > > > resources which Bosco mentioned should get you up and running.
>> > > >
>> > > > On Wed, Dec 9, 2015 at 11:34 PM, Don Bosco Durai 
>> > > wrote:
>> > > >
>> > > > > There are two resources available:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/RANGER/How+to+configure+Solr+Cloud+with+Kerberos+for+Ranger+0.5
>> > > > >
>> > > > >
>> > > > > Bosco
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On 12/9/15, 3:14 AM, "kostali hassan" 
>> > > wrote:
>> > > > >
>> > > > > >how I setting up Solr to use Kerberos ? i have to dowload
>> kerberos
>> > and
>> > > > put
>> > > > > >the plug-in implementation in the classpath(/server/solr).
>> > > > > >
>> > > > > >2015-12-08 22:19 GMT+00:00 Ishan Chattopadhyaya <
>> > > > > ichattopadhy...@gmail.com>:
>> > > > > >
>> > > > > >> Right, as Bosco said, this has been tested well and supported
>> on
>> > > > > SolrCloud.
>> > > > > >> It should be possible to run it in standalone mode, but it is
>> not
>> > > > > something
>> > > > > >> that has been well test yet.
>> > > > > >>
>> > > > > >> On Tue, Dec 8, 2015 at 11:02 PM, Don Bosco Durai <
>> > bo...@apache.org>
>> > > > > wrote:
>> > > > > >>
>> > > > > >> > It was tested and meant to work only in SolrCloud mode.
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > On Tue, Dec 8, 2015 at 9:30 AM -0800, "kostali hassan" <
>> > > > > >> > med.has.kost...@gmail.com> wrote:
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >

Re: secure solr 5.3.1

[Ishan Chattopadhyaya]

I don't have much personal experience with setting up a kerberos server on
a Windows machine, but I remember things being painful when I tried and
failed once. If you have an option to use a VM, I suggest try setting up
the KDC in a GNU/Linux VM (through VirtualBox).
In that case, make sure the Windows host is able to access the ports of the
guest machine. Better still, setup GNU/Linux VMs for Solr too.

On Thu, Dec 10, 2015 at 12:39 AM, kostali hassan 
wrote:

> I install MIT Kerberos for Windows 4.0.1
>
> 2015-12-09 19:05 GMT+00:00 Ishan Chattopadhyaya  >:
>
> > What exactly is your confusion? Do you have access to a KDC?
> >
> > Briefly:
> > Login to your KDC server, do a kadmin.local:
> > Then,
> > addprinc HTTP/192.168.0.107
> > ktadd -k /tmp/107.keytab HTTP/192.168.0.107
> >
> > Then copy the keytab file to your solr node to the appropriate places.
> >
> >
> > On Thu, Dec 10, 2015 at 12:08 AM, kostali hassan <
> > med.has.kost...@gmail.com>
> > wrote:
> >
> > > I folow this two resources and Iam stuck in
> > >
> > >- Create service principals and keytab files.
> > >
> > >
> > > 2015-12-09 18:06 GMT+00:00 Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com
> > > >:
> > >
> > > > The kerberos plugin is available for use with Solr out of the box.
> The
> > > two
> > > > resources which Bosco mentioned should get you up and running.
> > > >
> > > > On Wed, Dec 9, 2015 at 11:34 PM, Don Bosco Durai 
> > > wrote:
> > > >
> > > > > There are two resources available:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/RANGER/How+to+configure+Solr+Cloud+with+Kerberos+for+Ranger+0.5
> > > > >
> > > > >
> > > > > Bosco
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On 12/9/15, 3:14 AM, "kostali hassan" 
> > > wrote:
> > > > >
> > > > > >how I setting up Solr to use Kerberos ? i have to dowload kerberos
> > and
> > > > put
> > > > > >the plug-in implementation in the classpath(/server/solr).
> > > > > >
> > > > > >2015-12-08 22:19 GMT+00:00 Ishan Chattopadhyaya <
> > > > > ichattopadhy...@gmail.com>:
> > > > > >
> > > > > >> Right, as Bosco said, this has been tested well and supported on
> > > > > SolrCloud.
> > > > > >> It should be possible to run it in standalone mode, but it is
> not
> > > > > something
> > > > > >> that has been well test yet.
> > > > > >>
> > > > > >> On Tue, Dec 8, 2015 at 11:02 PM, Don Bosco Durai <
> > bo...@apache.org>
> > > > > wrote:
> > > > > >>
> > > > > >> > It was tested and meant to work only in SolrCloud mode.
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > On Tue, Dec 8, 2015 at 9:30 AM -0800, "kostali hassan" <
> > > > > >> > med.has.kost...@gmail.com> wrote:
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> >- Kerberos authentication
> > > > > >> >:
> > > > > >> >work in SolrCloud or standalone mode but the documentation
> is
> > > not
> > > > > >> clear
> > > > > >> >-
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.or

Re: Re: secure solr 5.3.1

[Ishan Chattopadhyaya]

1. Please set up your Kerberos server. (KDC)
2. Create principals (usernames) and keytab files as mentioned in the
document.
3. Copy the keytab files to your Solr machines
4. Mention all parameters in your bin/solr.in.sh file.
5. Start Solr using a separate parameter,
"-DauthenticationPlugin=org.apache.solr.security.KerberosPlugin". You can
put this parameter also in your bin/solr.in.sh file.

All the best,
Regards,
Ishan


On Thu, Dec 10, 2015 at 5:19 PM, kostali hassan 
wrote:

> Iam looking to secure my solr runing in standalone Mode within windows ;the
> kerberose plugin is only able to secure solr in standalone mode. how create
> principale and here password.
>
> 2015-12-10 9:35 GMT+00:00 kostali hassan :
>
> > Iam looking to secure my solr runing in standalone Mode the kerberose
> > plugin is only able to secure solr in standalone mode.
> >
> > 2015-12-09 23:00 GMT+00:00 kostali hassan :
> >
> >> -- Message transféré --
> >> De : "Ishan Chattopadhyaya" 
> >> Date : 9 déc. 2015 19:39
> >> Objet : Re: secure solr 5.3.1
> >> À : 
> >> Cc :
> >>
> >> I don't have much personal experience with setting up a kerberos server
> on
> >> a Windows machine, but I remember things being painful when I tried and
> >> failed once. If you have an option to use a VM, I suggest try setting up
> >> the KDC in a GNU/Linux VM (through VirtualBox).
> >> In that case, make sure the Windows host is able to access the ports of
> >> the
> >> guest machine. Better still, setup GNU/Linux VMs for Solr too.
> >>
> >> On Thu, Dec 10, 2015 at 12:39 AM, kostali hassan <
> >> med.has.kost...@gmail.com>
> >> wrote:
> >>
> >> > I install MIT Kerberos for Windows 4.0.1
> >> >
> >> > 2015-12-09 19:05 GMT+00:00 Ishan Chattopadhyaya <
> >> ichattopadhy...@gmail.com
> >> > >:
> >> >
> >> > > What exactly is your confusion? Do you have access to a KDC?
> >> > >
> >> > > Briefly:
> >> > > Login to your KDC server, do a kadmin.local:
> >> > > Then,
> >> > > addprinc HTTP/192.168.0.107
> >> > > ktadd -k /tmp/107.keytab HTTP/192.168.0.107
> >> > >
> >> > > Then copy the keytab file to your solr node to the appropriate
> places.
> >> > >
> >> > >
> >> > > On Thu, Dec 10, 2015 at 12:08 AM, kostali hassan <
> >> > > med.has.kost...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > I folow this two resources and Iam stuck in
> >> > > >
> >> > > >- Create service principals and keytab files.
> >> > > >
> >> > > >
> >> > > > 2015-12-09 18:06 GMT+00:00 Ishan Chattopadhyaya <
> >> > > ichattopadhy...@gmail.com
> >> > > > >:
> >> > > >
> >> > > > > The kerberos plugin is available for use with Solr out of the
> box.
> >> > The
> >> > > > two
> >> > > > > resources which Bosco mentioned should get you up and running.
> >> > > > >
> >> > > > > On Wed, Dec 9, 2015 at 11:34 PM, Don Bosco Durai <
> >> bo...@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > > There are two resources available:
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/RANGER/How+to+configure+Solr+Cloud+with+Kerberos+for+Ranger+0.5
> >> > > > > >
> >> > > > > >
> >> > > > > > Bosco
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On 12/9/15, 3:14 AM, "kostali hassan" <
> >> med.has.kost.

Re: how to secure standalone solr

[Ishan Chattopadhyaya]

Hi Daniel,
That sounds good. It is a custom solution, which is a way to secure just
about any server. I think Noble's point was about out of the box, community
supported, way of securing Solr.
Regards,
Ishan

On Mon, Dec 14, 2015 at 9:26 PM, Davis, Daniel (NIH/NLM) [C] <
daniel.da...@nih.gov> wrote:

> Wait a second. There are other sorts of ways to secure Solr that don't
> work with any sort role-based security control.   What I do is place a
> reverse-proxy in front of Apache Solr on port 80, and have that reverse
> proxy use CAS authentication.  I also have a list of "valid-users" who may
> use the Solr admin UI.
>
> Then, I have port 8983 open in my port-based host firewall (iptables), but
> it only allows the hosts that need to talk directly to Solr.  Firewalls
> prevent the other accesses.  Many security wrappers such as mod_auth_cas,
> which works with Apache httpd, can set a request header such as REMOTE_USER
> to the username of the individual who has authenticated with the wrapper.
>  In fact, I'm hoping security.json can eventually be made to work with such
> a header.
>
> -Original Message-
> From: Noble Paul [mailto:noble.p...@gmail.com]
> Sent: Friday, December 11, 2015 8:12 PM
> To: solr-user@lucene.apache.org
> Subject: Re: how to secure standalone solr
>
> For standalone Solr , Kerberos is the only option for authentication.
> If you have  a SolrCloud setup, you have other options
>
>
> https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin
>
> https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin
>
> On Fri, Dec 11, 2015 at 11:02 PM, Don Bosco Durai 
> wrote:
> >>Anyone told me how to secure standalone solr .
> > Recently there were few discussion on this. In short, it is not tested
> and there doesn’t seem to a plan to test it.
> >
> >>1.)using Kerberos Plugin is a good practice or any other else.
> > The answer depends how you are using it. Where you are deploying it, who
> is accessing it, whether you want to restrict by access type (read/write),
> what authentication environment (LDAP/AD, Kerberos, etc) you already have.
> >
> > Depending upon your use cases and environment, you may have one or more
> options.
> >
> > Bosco
> >
> >
> >
> >
> >
> >
> > On 12/11/15, 4:27 AM, "Mugeesh Husain"  wrote:
> >
> >>Hello,
> >>
> >>Anyone told me how to secure standalone solr .
> >>
> >>1.)using Kerberos Plugin is a good practice or any other else.
> >>
> >>
> >>
> >>--
> >>View this message in context:
> >>http://lucene.472066.n3.nabble.com/how-to-secure-standalone-solr-tp424
> >>4866.html Sent from the Solr - User mailing list archive at
> >>Nabble.com.
> >
>
>
>
> --
> -
> Noble Paul
>

Re: Writing Solr Custom Components

[Ishan Chattopadhyaya]

http://home.apache.org/~ctargett/RefGuidePOC/jekyll-full/adding-custom-plugins-in-solrcloud-mode.html

On Tue, Oct 4, 2016 at 12:23 PM, John Bickerstaff 
wrote:

> All,
>
> I'm looking for information on writing custom Solr components.  A quick
> search showed nothing really recent and before I dig deeper, I thought I'd
> ask the community for anything you are aware of.
>
> Thanks
>

[ANNOUNCE] Apache Solr 6.6.0 released

[Ishan Chattopadhyaya]

6 June 2017, Apache Solr 6.6.0 availableSolr is the popular, blazing
fast, open source NoSQL search platform from the Apache Lucene
project. Its major features include powerful full-text search, hit
highlighting, faceted search and analytics, rich document parsing,
geospatial search, extensive REST APIs as well as parallel SQL. Solr
is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.Solr
6.6.0 is available for immediate download at: .
http://lucene.apache.org/solr/mirrors-solr-latest-redir.htmlPlease
read CHANGES.txt for a full list of new features and changes: .
https://lucene.apache.org/solr/6_6_0/changes/Changes.htmlSolr 6.6
Release Highlights:Fields and field types:* Payload support with
payload() value source and {!payload_score} and {!payload_check}
query parsers* Solr support for SimpleTextCodec, via   in solrconfig.xml (per-field
specification in the schema is not possible)* Multi-field support to
TermsComponent when requesting terms' statistics* Grouping support
with PointFields* CollapseQParser support with PointFields*
ExpandComponent support with PointFieldsNew API:* UPLOAD command
(Config Set API) for uploading zipped configsets* MOVEREPLICA command
(Collections API) for moving a replica across nodes* LISTALIASES
command (Collections API) to return a list of all collection aliases*
STATUS command (Core Admin API) to emit collection details of each
coreScript:* Basic authentication can be enabled/disabled using
bin/solr|bin/solr.cmd* ls command to ZkCLI for listing only
sub-directories* bin/solr script now prints warning when available
system entropy is lower than 300* solr.in.sh installed by install
script should be writable by solr userFaceting:* Variance and Standard
Deviation aggregators for the JSON Facet API* JSON Faceting now
supports a query time 'join' domain change optionStreaming expressions
and evaluators:* CartesianProductStream, which turns a single tuple
with a multi-valued field into N tuples, one for each value in the
multi-valued field* Basic math Streaming Evaluators* UUID Streaming
Evaluator* Date/time Stream Evaluators* analyze Stream Evaluator to
support streaming NLP* shuffle Streaming Expression* search Streaming
Expression should work in non-SolrCloud mode* echo Streaming
Expression* eval Streaming Expression* timeseries Streaming
Expression* let, get and tuple Streaming Expressions* Correlation
Stream Evaluator* stats Streaming Expression should work in
non-SolrCloud mode* regress and predict Stream Evaluators* covariance
Stream Evaluator* convolution Stream Evaluator* normalize Stream
EvaluatorExamples:* Solr default/example uses
WordDelimiterGraphFilterFactory and SynonymGraphFilterFactory* New
DataImportHandler 'atom' example, replacing broken 'rss' example*
Redone DataImportHandler 'tika' example, removing all unused and
irrelevant definitionsMetrics:* Expose cache statistics using metrics
API* Improvements to metric reporters and API: support for "regex"
parameter in /admin/metrics, "enabled" flag in reporter
configurations, correct handling of "serviceUrl" in SolrJmxReporter,
better handling of service clients for JMX, Ganglia and Graphite
reportersUpgrades/deprecations/removals:* Upgraded Metrics library to
3.2.2, Zookeeper to 3.4.10* Deprecated LatLonType, GeoHashField,
SpatialPointVectorFieldType, and SpatialTermQueryPrefixTreeFieldType.
Instead, switch to LatLonPointSpatialField or
SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField*
Deprecated PostingsSolrHighlighter. Use UnifiedSolrHighlighter
instead.* Removed Solr contribs: map-reduce, morphlines-core and
morphlines-cellSolrJ:* CloudSolrClient can now be initialized using
the base URL of a Solr instance instead of ZooKeeper hosts* SolrJ:
Added SolrParams.toLocalParamsString() and
ClientUtils.encodeLocalParamVal* LBHttpSolrClient.doRequest is now
always wrapped in a Mapped Diagnostic Context (MDC)Others:* New
AtomicUpdateProcessor to convert normal update operations to atomic
update operations* totalTermFreq support to TermsComponent* Hide
keystore and truststore passwords from /admin/info/* outputs*
Configurability for thread pool size to recoveryExecutor* Introducing
sort=childfield(field) asc for searching by {!parent}* Transient core
cache is now pluggable* Renamed getSortWithinGroup to
getWithinGroupSort in search.grouping.CommandOptimizations:*
facet.heatmap is now significantly faster when the docset (base query)
matches everything and there are no deleted docs. It is also faster
when the docset matches a small fraction of the index or none* Reduced
heap consumption for filter({!join ... score=...}) * JSON Facet API
now uses hyper-log-log++ for determining the number of buckets when
merging requests from a multi-shard distributed request* Better
ZkStateWriter batching* Using cache for DistributedQueue in case of
single-consumerFurther deta

Re: [ANNOUNCE] Apache Solr 6.6.0 released

[Ishan Chattopadhyaya]

6 June 2017, Apache Solr 6.6.0 available

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search and analytics, rich document
parsing, geospatial search, extensive REST APIs as well as parallel SQL.
Solr is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.

Solr 6.6.0 is available for immediate download at:

 . http://lucene.apache.org/solr/mirrors-solr-latest-redir.html

Please read CHANGES.txt for a full list of new features and changes:

 . https://lucene.apache.org/solr/6_6_0/changes/Changes.html

Solr 6.6 Release Highlights:

Fields and field types:
* Payload support with payload() value source and {!payload_score} and
{!payload_check}
  query parsers
* Solr support for SimpleTextCodec, via 
  in solrconfig.xml (per-field specification in the schema is not possible)
* Multi-field support to TermsComponent when requesting terms' statistics
* Grouping support with PointFields
* CollapseQParser support with PointFields
* ExpandComponent support with PointFields

New API:
* UPLOAD command (Config Set API) for uploading zipped configsets
* MOVEREPLICA command (Collections API) for moving a replica across nodes
* LISTALIASES command (Collections API) to return a list of all collection
aliases
* STATUS command (Core Admin API) to emit collection details of each core

Script:
* Basic authentication can be enabled/disabled using bin/solr|bin/solr.cmd
* ls command to ZkCLI for listing only sub-directories
* bin/solr script now prints warning when available system entropy is lower
than 300
* solr.in.sh installed by install script should be writable by solr user

Faceting:
* Variance and Standard Deviation aggregators for the JSON Facet API
* JSON Faceting now supports a query time 'join' domain change option

Streaming expressions and evaluators:
* CartesianProductStream, which turns a single tuple with a multi-valued
field into N tuples, one for each value in the multi-valued field
* Basic math Streaming Evaluators
* UUID Streaming Evaluator
* Date/time Stream Evaluators
* analyze Stream Evaluator to support streaming NLP
* shuffle Streaming Expression
* search Streaming Expression should work in non-SolrCloud mode
* echo Streaming Expression
* eval Streaming Expression
* timeseries Streaming Expression
* let, get and tuple Streaming Expressions
* Correlation Stream Evaluator
* stats Streaming Expression should work in non-SolrCloud mode
* regress and predict Stream Evaluators
* covariance Stream Evaluator
* convolution Stream Evaluator
* normalize Stream Evaluator

Examples:
* Solr default/example uses WordDelimiterGraphFilterFactory and
SynonymGraphFilterFactory
* New DataImportHandler 'atom' example, replacing broken 'rss' example
* Redone DataImportHandler 'tika' example, removing all unused and
irrelevant definitions

Metrics:
* Expose cache statistics using metrics API
* Improvements to metric reporters and API: support for "regex" parameter
in /admin/metrics, "enabled" flag in reporter configurations, correct
handling of
"serviceUrl" in SolrJmxReporter, better handling of service clients for
JMX, Ganglia and Graphite reporters

Upgrades/deprecations/removals:
* Upgraded Metrics library to 3.2.2, Zookeeper to 3.4.10
* Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and
SpatialTermQueryPrefixTreeFieldType. Instead, switch to
LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType or
RptWithGeometrySpatialField
* Deprecated PostingsSolrHighlighter. Use UnifiedSolrHighlighter instead.
* Removed Solr contribs: map-reduce, morphlines-core and morphlines-cell

SolrJ:
* CloudSolrClient can now be initialized using the base URL of a Solr
instance instead of ZooKeeper hosts
* SolrJ: Added SolrParams.toLocalParamsString() and
ClientUtils.encodeLocalParamVal
* LBHttpSolrClient.doRequest is now always wrapped in a Mapped Diagnostic
Context (MDC)

Others:
* New AtomicUpdateProcessor to convert normal update operations to atomic
update operations
* totalTermFreq support to TermsComponent
* Hide keystore and truststore passwords from /admin/info/* outputs
* Configurability for thread pool size to recoveryExecutor
* Introducing sort=childfield(field) asc for searching by {!parent}
* Transient core cache is now pluggable
* Renamed getSortWithinGroup to getWithinGroupSort in
search.grouping.Command

Optimizations:
* facet.heatmap is now significantly faster when the docset (base query)
matches everything and there are no deleted docs. It is also faster when
the docset matches a small fraction of the index or none
* Reduced heap consumption for filter({!join ... score=...})
* JSON Facet API now uses hyper-log-log++ for determining the number of
buckets when merging requests from a multi-shard distributed request
* Better ZkS

Re: [ANNOUNCE] Apache Solr 6.6.0 released

[Ishan Chattopadhyaya]

Thanks Alexandre! Gmail sent it out as HTML formatted (and looked fine on
Gmail), but the mailing list bot must've messed it up trying to convert it
to text only.


On Wed, Jun 7, 2017 at 8:27 PM, Alexandre Rafalovitch 
wrote:

> That did not format well. At least for me.
> 
> http://www.solr-start.com/ - Resources for Solr users, new and experienced
>
>
> On 7 June 2017 at 10:40, Ishan Chattopadhyaya 
> wrote:
> > 6 June 2017, Apache Solr 6.6.0 availableSolr is the popular, blazing
> > fast, open source NoSQL search platform from the Apache Lucene
> > project. Its major features include powerful full-text search, hit
> > highlighting, faceted search and analytics, rich document parsing,
> > geospatial search, extensive REST APIs as well as parallel SQL. Solr
> > is enterprise grade, secure and highly scalable, providing fault
> > tolerant distributed search and indexing, and powers the search and
> > navigation features of many of the world's largest internet sites.Solr
> > 6.6.0 is available for immediate download at: .
> > http://lucene.apache.org/solr/mirrors-solr-latest-redir.htmlPlease
> > read CHANGES.txt for a full list of new features and changes: .
> > https://lucene.apache.org/solr/6_6_0/changes/Changes.htmlSolr 6.6
> > Release Highlights:Fields and field types:* Payload support with
> > payload() value source and {!payload_score} and {!payload_check}
> > query parsers* Solr support for SimpleTextCodec, via  > class="solr.SimpleTextCodecFactory"/>  in solrconfig.xml (per-field
> > specification in the schema is not possible)* Multi-field support to
> > TermsComponent when requesting terms' statistics* Grouping support
> > with PointFields* CollapseQParser support with PointFields*
> > ExpandComponent support with PointFieldsNew API:* UPLOAD command
> > (Config Set API) for uploading zipped configsets* MOVEREPLICA command
> > (Collections API) for moving a replica across nodes* LISTALIASES
> > command (Collections API) to return a list of all collection aliases*
> > STATUS command (Core Admin API) to emit collection details of each
> > coreScript:* Basic authentication can be enabled/disabled using
> > bin/solr|bin/solr.cmd* ls command to ZkCLI for listing only
> > sub-directories* bin/solr script now prints warning when available
> > system entropy is lower than 300* solr.in.sh installed by install
> > script should be writable by solr userFaceting:* Variance and Standard
> > Deviation aggregators for the JSON Facet API* JSON Faceting now
> > supports a query time 'join' domain change optionStreaming expressions
> > and evaluators:* CartesianProductStream, which turns a single tuple
> > with a multi-valued field into N tuples, one for each value in the
> > multi-valued field* Basic math Streaming Evaluators* UUID Streaming
> > Evaluator* Date/time Stream Evaluators* analyze Stream Evaluator to
> > support streaming NLP* shuffle Streaming Expression* search Streaming
> > Expression should work in non-SolrCloud mode* echo Streaming
> > Expression* eval Streaming Expression* timeseries Streaming
> > Expression* let, get and tuple Streaming Expressions* Correlation
> > Stream Evaluator* stats Streaming Expression should work in
> > non-SolrCloud mode* regress and predict Stream Evaluators* covariance
> > Stream Evaluator* convolution Stream Evaluator* normalize Stream
> > EvaluatorExamples:* Solr default/example uses
> > WordDelimiterGraphFilterFactory and SynonymGraphFilterFactory* New
> > DataImportHandler 'atom' example, replacing broken 'rss' example*
> > Redone DataImportHandler 'tika' example, removing all unused and
> > irrelevant definitionsMetrics:* Expose cache statistics using metrics
> > API* Improvements to metric reporters and API: support for "regex"
> > parameter in /admin/metrics, "enabled" flag in reporter
> > configurations, correct handling of "serviceUrl" in SolrJmxReporter,
> > better handling of service clients for JMX, Ganglia and Graphite
> > reportersUpgrades/deprecations/removals:* Upgraded Metrics library to
> > 3.2.2, Zookeeper to 3.4.10* Deprecated LatLonType, GeoHashField,
> > SpatialPointVectorFieldType, and SpatialTermQueryPrefixTreeFieldType.
> > Instead, switch to LatLonPointSpatialField or
> > SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField*
> > Deprecated PostingsSolrHighlighter. Use UnifiedSolrHighlighter
> > instead.* Removed Solr contribs: map-reduce, morphlines-core and
> > morphlines-cellSolrJ:* CloudSolrClient can now be initialized using
> > the base URL of a S

[ANNOUNCE] Apache Solr 6.6.2 released

[Ishan Chattopadhyaya]

18 October 2017, Apache Solr™ 6.6.2 available

The Lucene PMC is pleased to announce the release of Apache Solr 6.6.2

Solr is the popular, blazing fast, open source NoSQL search platform from
the
Apache Lucene project. Its major features include powerful full-text
search,
hit highlighting, faceted search and analytics, rich document parsing,
geospatial search, extensive REST APIs as well as parallel SQL. Solr is
enterprise grade, secure and highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features
of many of the world's largest internet sites.

This release includes a critical security fix and a bugfix. Details:

* Fix for a 0-day exploit (CVE-2017-12629), details:
https://s.apache.org/FJDl.
  RunExecutableListener has been disabled by default (can be enabled by
  -Dsolr.enableRunExecutableListener=true) and resolving external entities
in
  the XML query parser (defType=xmlparser or {!xmlparser ... }) is disabled
by
  default.

* Fix a bug where Solr was attempting to load the same core twice (Error
message:
  "Lock held by this virtual machine").

Furthermore, this release includes Apache Lucene 6.6.2 which includes one
security
fix since the 6.6.1 release.

The release is available for immediate download at:

  http://www.apache.org/dyn/closer.lua/lucene/solr/6.6.2

Please read CHANGES.txt for a detailed list of changes:

  https://lucene.apache.org/solr/6_6_2/changes/Changes.html

Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

Re: [ANNOUNCE] Apache Solr 6.6.2 released

[Ishan Chattopadhyaya]

Hi Bernd,

> I get something like "permissionViolation=true", even after login!!!
This is due to the security sensitive nature of those issues. Only PMC
members have access to those issues at the moment.
Please revisit those issues after Solr 5.5.5 release to see if you can
access them.

> Is SOLR going to be closed source?
No

> Do we have to pay for seeing the issues? ;-)
No

Regards,
Ishan


On Wed, Oct 18, 2017 at 2:23 PM, Bernd Fehling <
bernd.fehl...@uni-bielefeld.de> wrote:

> Thanks,
> but I tried to access the mentioned issues of
> https://lucene.apache.org/solr/6_6_2/changes/Changes.html
>
> https://issues.apache.org/jira/browse/SOLR-11477
> https://issues.apache.org/jira/browse/SOLR-11482
>
> I get something like "permissionViolation=true", even after login!!!
>
> Is SOLR going to be closed source?
>
> Do we have to pay for seeing the issues? ;-)
>
> Regards
> Bernd
>
>
> Am 18.10.2017 um 10:29 schrieb Ishan Chattopadhyaya:
> > 18 October 2017, Apache Solr™ 6.6.2 available
> >
> > The Lucene PMC is pleased to announce the release of Apache Solr 6.6.2
> >
> > Solr is the popular, blazing fast, open source NoSQL search platform from
> > the
> > Apache Lucene project. Its major features include powerful full-text
> > search,
> > hit highlighting, faceted search and analytics, rich document parsing,
> > geospatial search, extensive REST APIs as well as parallel SQL. Solr is
> > enterprise grade, secure and highly scalable, providing fault tolerant
> > distributed search and indexing, and powers the search and navigation
> > features
> > of many of the world's largest internet sites.
> >
> > This release includes a critical security fix and a bugfix. Details:
> >
> > * Fix for a 0-day exploit (CVE-2017-12629), details:
> > https://s.apache.org/FJDl.
> >   RunExecutableListener has been disabled by default (can be enabled by
> >   -Dsolr.enableRunExecutableListener=true) and resolving external
> entities
> > in
> >   the XML query parser (defType=xmlparser or {!xmlparser ... }) is
> disabled
> > by
> >   default.
> >
> > * Fix a bug where Solr was attempting to load the same core twice (Error
> > message:
> >   "Lock held by this virtual machine").
> >
> > Furthermore, this release includes Apache Lucene 6.6.2 which includes one
> > security
> > fix since the 6.6.1 release.
> >
> > The release is available for immediate download at:
> >
> >   http://www.apache.org/dyn/closer.lua/lucene/solr/6.6.2
> >
> > Please read CHANGES.txt for a detailed list of changes:
> >
> >   https://lucene.apache.org/solr/6_6_2/changes/Changes.html
> >
> > Please report any feedback to the mailing lists
> > (http://lucene.apache.org/solr/discussion.html)
> >
> > Note: The Apache Software Foundation uses an extensive mirroring
> > network for distributing releases. It is possible that the mirror you
> > are using may not have replicated the release yet. If that is the
> > case, please try another mirror. This also goes for Maven access.
> >
>

Re: zero-day exploit security issue

[Ishan Chattopadhyaya]

There will be a 5.5.5 release soon. 6.6.2 has just been released.

On Mon, Oct 16, 2017 at 8:17 PM, Keith L  wrote:

> Additionally, it looks like the commits are public on github. Is this
> backported to 5.5.x too? Users that are still on 5x might want to backport
> some of the issues themselves since is not officially supported anymore.
>
> On Mon, Oct 16, 2017 at 10:11 AM Mike Drob  wrote:
>
> > Given that the already public nature of the disclosure, does it make
> sense
> > to make the work being done public prior to release as well?
> >
> > Normally security fixes are kept private while the vulnerabilities are
> > private, but that's not the case here...
> >
> > On Mon, Oct 16, 2017 at 1:20 AM, Shalin Shekhar Mangar <
> > shalinman...@gmail.com> wrote:
> >
> > > Yes, there is but it is private i.e. only the Apache Lucene PMC
> > > members can see it. This is standard for all security issues in Apache
> > > land. The fixes for this issue has been applied to the release
> > > branches and the Solr 7.1.0 release candidate is already up for vote.
> > > Barring any unforeseen circumstances, a 7.1.0 release with the fixes
> > > should be expected this week.
> > >
> > > On Fri, Oct 13, 2017 at 8:14 PM, Xie, Sean  wrote:
> > > > Is there a tracking to address this issue for SOLR 6.6.x and 7.x?
> > > >
> > > > https://lucene.apache.org/solr/news.html#12-october-
> > > 2017-please-secure-your-apache-solr-servers-since-a-
> > > zero-day-exploit-has-been-reported-on-a-public-mailing-list
> > > >
> > > > Sean
> > > >
> > > > Confidentiality Notice::  This email, including attachments, may
> > include
> > > non-public, proprietary, confidential or legally privileged
> information.
> > > If you are not an intended recipient or an authorized agent of an
> > intended
> > > recipient, you are hereby notified that any dissemination, distribution
> > or
> > > copying of the information contained in or transmitted with this e-mail
> > is
> > > unauthorized and strictly prohibited.  If you have received this email
> in
> > > error, please notify the sender by replying to this message and
> > permanently
> > > delete this e-mail, its attachments, and any copies of it immediately.
> > You
> > > should not retain, copy or use this e-mail or any attachment for any
> > > purpose, nor disclose all or any part of the contents to any other
> > person.
> > > Thank you.
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Shalin Shekhar Mangar.
> > >
> >
>

Re: [ANNOUNCE] Apache Solr 6.6.2 released

[Ishan Chattopadhyaya]

You can now access them. Thanks for your patience and understanding.

On 18 Oct 2017 3:15 pm, "Ishan Chattopadhyaya" 
wrote:

> Hi Bernd,
>
> > I get something like "permissionViolation=true", even after login!!!
> This is due to the security sensitive nature of those issues. Only PMC
> members have access to those issues at the moment.
> Please revisit those issues after Solr 5.5.5 release to see if you can
> access them.
>
> > Is SOLR going to be closed source?
> No
>
> > Do we have to pay for seeing the issues? ;-)
> No
>
> Regards,
> Ishan
>
>
> On Wed, Oct 18, 2017 at 2:23 PM, Bernd Fehling <
> bernd.fehl...@uni-bielefeld.de> wrote:
>
>> Thanks,
>> but I tried to access the mentioned issues of
>> https://lucene.apache.org/solr/6_6_2/changes/Changes.html
>>
>> https://issues.apache.org/jira/browse/SOLR-11477
>> https://issues.apache.org/jira/browse/SOLR-11482
>>
>> I get something like "permissionViolation=true", even after login!!!
>>
>> Is SOLR going to be closed source?
>>
>> Do we have to pay for seeing the issues? ;-)
>>
>> Regards
>> Bernd
>>
>>
>> Am 18.10.2017 um 10:29 schrieb Ishan Chattopadhyaya:
>> > 18 October 2017, Apache Solr™ 6.6.2 available
>> >
>> > The Lucene PMC is pleased to announce the release of Apache Solr 6.6.2
>> >
>> > Solr is the popular, blazing fast, open source NoSQL search platform
>> from
>> > the
>> > Apache Lucene project. Its major features include powerful full-text
>> > search,
>> > hit highlighting, faceted search and analytics, rich document parsing,
>> > geospatial search, extensive REST APIs as well as parallel SQL. Solr is
>> > enterprise grade, secure and highly scalable, providing fault tolerant
>> > distributed search and indexing, and powers the search and navigation
>> > features
>> > of many of the world's largest internet sites.
>> >
>> > This release includes a critical security fix and a bugfix. Details:
>> >
>> > * Fix for a 0-day exploit (CVE-2017-12629), details:
>> > https://s.apache.org/FJDl.
>> >   RunExecutableListener has been disabled by default (can be enabled by
>> >   -Dsolr.enableRunExecutableListener=true) and resolving external
>> entities
>> > in
>> >   the XML query parser (defType=xmlparser or {!xmlparser ... }) is
>> disabled
>> > by
>> >   default.
>> >
>> > * Fix a bug where Solr was attempting to load the same core twice (Error
>> > message:
>> >   "Lock held by this virtual machine").
>> >
>> > Furthermore, this release includes Apache Lucene 6.6.2 which includes
>> one
>> > security
>> > fix since the 6.6.1 release.
>> >
>> > The release is available for immediate download at:
>> >
>> >   http://www.apache.org/dyn/closer.lua/lucene/solr/6.6.2
>> >
>> > Please read CHANGES.txt for a detailed list of changes:
>> >
>> >   https://lucene.apache.org/solr/6_6_2/changes/Changes.html
>> >
>> > Please report any feedback to the mailing lists
>> > (http://lucene.apache.org/solr/discussion.html)
>> >
>> > Note: The Apache Software Foundation uses an extensive mirroring
>> > network for distributing releases. It is possible that the mirror you
>> > are using may not have replicated the release yet. If that is the
>> > case, please try another mirror. This also goes for Maven access.
>> >
>>
>
>

Re: update a document without changing anything

[Ishan Chattopadhyaya]

Maybe do an "inc" of 0 to a numeric field for every document.
https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents

On Wed, Nov 23, 2016 at 2:13 PM, Dorian Hoxha 
wrote:

> Hello searcherers,
>
> So, I have document that is fully stored. Then I make small change in
> schema. And now I have to reinsert every document. But I'm afraid of doing
> a get+insert, because something else may change the document in the
> meantime. So I want to do an "update" of nothing, so internally on the
> master-shard, the document is updated without changes. Maybe an update with
> no modifiers ?
>
> Thank You!
>

[ANNOUNCE] Apache Solr 6.4.2 released

[Ishan Chattopadhyaya]

7 March 2017, Apache Solr 6.4.2 available

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search and analytics, rich document
parsing, geospatial search, extensive REST APIs as well as parallel SQL.
Solr is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.

Solr 6.4.2 is available for immediate download at:

   -

   http://lucene.apache.org/solr/mirrors-solr-latest-redir.html

Please read CHANGES.txt for a full list of new features and changes:

   -

   https://lucene.apache.org/solr/6_4_2/changes/Changes.html

Solr 6.4.2 contains 4 bug fixes since the 6.4.1 release:

   -

   Serious performance degradation in Solr 6.4 due to the metrics
   collection. IndexWriter metrics collection turned off by default, directory
   level metrics collection completely removed (until a better design is
   found)
   -

   Transaction log replay can hit an NullPointerException due to new
   Metrics code
   -

   NullPointerException in CloudSolrClient when reading stale alias
   -

   UnifiedHighlighter and PostingsHighlighter bug in PrefixQuery and
   TermRangeQuery for multi-byte text

Further details of changes are available in the change log available at:
http://lucene.apache.org/solr/6_4_2/changes/Changes.html

Please report any feedback to the mailing lists (http://lucene.apache.org/
solr/discussion.html)
Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases. It is possible that the mirror you are using may
not have replicated the release yet. If that is the case, please try
another mirror. This also applies to Maven access.

Re: [ANNOUNCE] Apache Solr 6.4.2 released

[Ishan Chattopadhyaya]

Hi Bernd,
Can you please double check?

I downloaded the 6.4.2 tarball and see that they have 6.4.2:

[ishan@ishanvps solr-6.4.2]$ grep -rn "luceneMatchVersion" *|grep
solrconfig.xml
CHANGES.txt:1474:   or
your luceneMatchVersion in the solrconfig.xml is less than 6.0
docs/changes/Changes.html:1694: or your luceneMatchVersion in the
solrconfig.xml is less than 6.0
example/files/conf/solrconfig.xml:38:
6.4.2
example/example-DIH/solr/tika/conf/solrconfig.xml:38:
6.4.2
example/example-DIH/solr/rss/conf/solrconfig.xml:38:
6.4.2
example/example-DIH/solr/mail/conf/solrconfig.xml:38:
6.4.2
example/example-DIH/solr/db/conf/solrconfig.xml:38:
6.4.2
example/example-DIH/solr/solr/conf/solrconfig.xml:38:
6.4.2
server/solr/configsets/basic_configs/conf/solrconfig.xml:38:
6.4.2
server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml:38:
6.4.2
server/solr/configsets/data_driven_schema_configs/conf/solrconfig.xml:38:
6.4.2


Maybe you downloaded the 6.4.1 version by mistake?
Thanks,
Ishan


On Thu, Mar 9, 2017 at 10:19 AM, Shawn Heisey  wrote:

> On 3/8/2017 2:36 AM, Bernd Fehling wrote:
> > Shouldn't in server/solr/configsets/.../solrconfig.xml
> > 6.4.1
> > really read
> > 6.4.2
> >
> > May be something for package builder for future releases?
>
> That does look like it got overlooked, and is generally something that
> SHOULD be changed with each new version, but in this case, changing
> between those two version numbers will have zero effect.  It is against
> project policy to make significant changes in a bugfix release (where
> third version number changes).
>
> Any change that's significant enough to be controlled by a
> luceneMatchVersion check would only be allowed a minor or major release.
>
> Thanks,
> Shawn
>
>

Re: in-place updates

[Ishan Chattopadhyaya]

> For in-place updates, the documentation states that only the fields being
> modified are updated, but does that mean that all other fields don't need
> to be stored?

Correct, in general there's no need to store the other fields. However,
there's a niche case where if a simultaneous DeleteByQuery command and
in-place update command get re-ordered on a replica, and it ends up
deleting the document that is required to be updated, the replica will
fetch the document from leader. At that point, non-stored fields can be
lost. So, if you don't use DeleteByQuery commands you can safely stop
storing other fields.

> we are calculating authority-like and popularity
> scores for our documents

This is the ideal usecase for in-place updates.

On Thu, Mar 30, 2017 at 1:13 AM, Elaine Cario  wrote:

> I need some clarity on atomic vs in-place updates.  For atomic I understand
> that all fields need to be stored, either explicitly or through docValues,
> since the entire document is re-indexed.
>
> For in-place updates, the documentation states that only the fields being
> modified are updated, but does that mean that all other fields don't need
> to be stored?
>
> My real problem is that we are calculating authority-like and popularity
> scores for our documents in an external process, and would actually prefer
> to use something like ExternalFileField for those values.  But EFF is not
> really SolrCloud-friendly, and we have a fairly mature indexing process in
> place already.  It might be more palatable for us to do an in-place update,
> rather than deal with the operational issues of dropping files into every
> shard of a collection, but only IF we don't need to change our existing
> schemas to store everything which might not be stored today.
>
> Thanks in advance.
>

Re: Is CloudSolrClient thread-safe?

[Ishan Chattopadhyaya]

> I use SolrJ 6.4.0 and SolrCloud 6.4.0
Please upgrade to 6.4.2 immediately. 6.4.0 has major performance problems.

On Wed, Mar 29, 2017 at 12:12 AM, Mikhail Ibraheem <
mikhail.ibrah...@oracle.com> wrote:

> Thanks Shawn so much.
> I use SolrJ 6.4.0 and SolrCloud 6.4.0
> The code is very simple:
> I have Spring singleton bean to get one and only one instance of
> solrCloudClient as:
>
>   @Value("${zkHost.url:rws00dtr.us.oracle.com:2181,rws00dtr.
> us.oracle.com:2182,rws00dtr.us.oracle.com:2183}")
>   private String zkHost;
>   @Bean
>   public CloudSolrClient solrClient() {
>  Builder builder = new CloudSolrClient.Builder().withZkHost(zkHost);
> CloudSolrClient solrClient = builder.build();
> return solrClient;
>   }
>
> And I use this code for all functionalities, pivot, faceting, queries,
> indexing and so on.
> The exception is intermittent and that is why I think it is related to
> thread-safe issue with the class.
>
> Thanks
> Mikhail
>
> -Original Message-
> From: Shawn Heisey [mailto:apa...@elyograg.org]
> Sent: 28 مارس, 2017 04:26 م
> To: solr-user@lucene.apache.org
> Subject: Re: Is CloudSolrClient thread-safe?
>
> On 3/28/2017 8:13 AM, Mikhail Ibraheem wrote:
> > I have a project with solr and spring. I am have only one instance of
> CloudSolrClient at the context (singleton).
> >
> > This is because I believe that it should be only one instance to load
> balance between the nodes.
>
> Yes, the intent is to be fully threadsafe.  You should know that
> HttpClient, the class which the SolrClient implementations use under the
> hood, has a default limitation of only allowing two simultaneous
> connections to a given route, which I believe means a host/port combination.
>
> Increasing this is possible, but it must be done by creating a custom
> HttpClient object.  Here's some code that I use to build such a client,
> then use that client to create an HttpSolrClient.  The same thing can be
> done with CloudSolrClient:
>
>   RequestConfig rc = RequestConfig.custom().setConnectTimeout(15000)
>   .setSocketTimeout(Const.SOCKET_TIMEOUT).build();
>   httpClient = HttpClients.custom().setDefaultRequestConfig(rc)
>
> .setMaxConnPerRoute(300).setMaxConnTotal(5000).disableAutomaticRetries()
>   .build();
>   client = new HttpSolrClient(serverBaseUrl, httpClient);
>
> > I have big issue which doesn't occur all the times, it is intermittent:
> >
> > java.lang.ClassCastException: java.lang.String cannot be cast to
> > org.apache.solr.common.SolrDocumentList
>
> That is odd.  What is the exact version of SolrJ that you are using, and
> the version of the SolrCloud cluster that it is talking to?  Can you share
> your SolrJ code?
>
> Thanks,
> Shawn
>
>

Re: DistributedUpdateProcessorFactory was explicitly disabled from this updateRequestProcessorChain

[Ishan Chattopadhyaya]

Why are you adding these update processors (esp. the
AddSchemaFieldsUpdateProcessor) after DistributedUpdateProcessor? Try
adding them before DUP, and it has a better chance to work.

On Wed, Apr 12, 2017 at 3:44 PM, Pratik Thaker <
pratik.tha...@smartstreamrdu.com> wrote:

> Hi All,
>
> I am facing this issue since very long, can you please provide your
> suggestion on it ?
>
> Regards,
> Pratik Thaker
>
> -Original Message-
> From: Pratik Thaker [mailto:pratik.tha...@smartstreamrdu.com]
> Sent: 09 February 2017 21:24
> To: 'solr-user@lucene.apache.org'
> Subject: RE: DistributedUpdateProcessorFactory was explicitly disabled
> from this updateRequestProcessorChain
>
> Hi Friends,
>
> Can you please try to give me some details about below issue ?
>
> Regards,
> Pratik Thaker
>
> From: Pratik Thaker
> Sent: 07 February 2017 17:12
> To: 'solr-user@lucene.apache.org'
> Subject: DistributedUpdateProcessorFactory was explicitly disabled from
> this updateRequestProcessorChain
>
> Hi All,
>
> I am using SOLR Cloud 6.0
>
> I am receiving below exception very frequently in solr logs,
>
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
> RunUpdateProcessor has received an AddUpdateCommand containing a document
> that appears to still contain Atomic document update operations, most
> likely because DistributedUpdateProcessorFactory was explicitly disabled
> from this updateRequestProcessorChain
> at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(
> RunUpdateProcessorFactory.java:63)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessor
> Factory$AddSchemaFieldsUpdateProcessor.processAdd(
> AddSchemaFieldsUpdateProcessorFactory.java:335)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.
> processAdd(FieldMutatingUpdateProcessor.java:117)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.
> processAdd(FieldMutatingUpdateProcessor.java:117)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.
> processAdd(FieldMutatingUpdateProcessor.java:117)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.
> processAdd(FieldMutatingUpdateProcessor.java:117)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.FieldNameMutatingUpdateProcess
> orFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.
> processAdd(FieldMutatingUpdateProcessor.java:117)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.DistributedUpdateProcessor.
> doLocalAdd(DistributedUpdateProcessor.java:936)
> at org.apache.solr.update.processor.DistributedUpdateProcessor.
> versionAdd(DistributedUpdateProcessor.java:1091)
> at org.apache.solr.update.processor.DistributedUpdateProcessor.
> processAdd(DistributedUpdateProcessor.java:714)
> at org.apache.solr.update.processor.UpdateRequestProcessor.
> processAdd(UpdateRequestProcessor.java:48)
> at org.apache.solr.update.processor.AbstractDefaultValueUpdateProc
> essorFactory$DefaultValueUpdateProcessor.processAdd(
> AbstractDefaultValueUpdateProcessorFactory.java:93)
> at org.apache.solr.handler.loader.JavabinLoader$1.update(
> JavabinLoader.java:97)
>
> Can you please help me with the root cause ? Below is the snapshot of
> solrconfig,
>
> 
> 
> 
>
> 
>
> 
> 
>   [^\w-\.]
>   _
> 
> 
> 
> 
> 
>   
> -MM-dd'T'HH:mm:ss.SSSZ
> -MM-dd'T'HH:mm:ss,SSSZ
> -MM-dd'T'HH:mm:ss.SSS
> -MM-dd'T'HH:mm:ss,SSS
> -MM-dd'T'HH:mm:ssZ
> -MM-dd'T'HH:mm:ss
> -MM-dd'T'HH:mmZ
> -MM-dd'T'HH:mm
> -MM-dd HH:mm:ss.SSSZ
> -MM-dd HH:mm:ss,SSSZ
> -MM-dd HH:mm:ss.SSS
> -MM-dd HH:mm:ss,SSS
> -MM-dd HH:mm:ssZ
> -MM-dd HH:mm:ss
> -MM-dd HH:mmZ
> -MM-dd HH:mm
> -MM-dd
>   
> 
> 
>   strings
>  

Re: Atomic Updates

[Ishan Chattopadhyaya]

> Hmm, interesting. I can imagine that as long as you're updating
> docValues fields, the other_text field would be there. But the instant
> you updated a non-docValues field (text_field in your example) the
> other_text field would disappear

I can confirm this. When in-place updates to DV fields are done, the rest
of the fields remain as they were.

On Thu, Apr 27, 2017 at 4:33 AM, Erick Erickson 
wrote:

> Hmm, interesting. I can imagine that as long as you're updating
> docValues fields, the other_text field would be there. But the instant
> you updated a non-docValues field (text_field in your example) the
> other_text field would disappear.
>
> I DO NOT KNOW this for a fact, but I'm asking people who do.
>
> On Wed, Apr 26, 2017 at 2:13 PM, Dorian Hoxha 
> wrote:
> > There are In Place Updates, but according to docs they stll shouldn't
> work
> > in your case:
> > https://cwiki.apache.org/confluence/display/solr/
> Updating+Parts+of+Documents
> >
> > On Wed, Apr 26, 2017 at 10:36 PM, Chris Ulicny  wrote:
> >
> >> That's the thing I'm curious about though. As I mentioned in the first
> >> post, I've already tried a few tests, and the value seems to still be
> >> present after an atomic update.
> >>
> >> I haven't exhausted all possible atomic updates, but 'set' and 'add'
> seem
> >> to preserve the non-stored text field.
> >>
> >> Thanks,
> >> Chris
> >>
> >> On Wed, Apr 26, 2017 at 4:07 PM Dorian Hoxha 
> >> wrote:
> >>
> >> > You'll lose the data in that field. Try doing a commit and it should
> >> > happen.
> >> >
> >> > On Wed, Apr 26, 2017 at 9:50 PM, Chris Ulicny 
> wrote:
> >> >
> >> > > Thanks Shawn, I didn't realize docValues were enabled by default
> now.
> >> > > That's very convenient and probably makes a lot of the schemas we've
> >> been
> >> > > making excessively verbose.
> >> > >
> >> > > This is on 6.3.0. Do you know what the first version was that they
> >> added
> >> > > the docValues by default for non-Text field?
> >> > >
> >> > > However, that shouldn't apply to this since I'm concerned with a
> >> > non-stored
> >> > > TextField without docValues enabled.
> >> > >
> >> > > Best,
> >> > > Chris
> >> > >
> >> > > On Wed, Apr 26, 2017 at 3:36 PM Shawn Heisey 
> >> > wrote:
> >> > >
> >> > > > On 4/25/2017 1:40 PM, Chris Ulicny wrote:
> >> > > > > Hello all,
> >> > > > >
> >> > > > > Suppose I have the following fields in a document and populate
> all
> >> 4
> >> > > > fields
> >> > > > > for every document.
> >> > > > >
> >> > > > > id: uniqueKey, indexed and stored
> >> > > > > integer_field: indexed and stored
> >> > > > > text_field: indexed and stored
> >> > > > > othertext_field: indexed but not stored
> >> > > > >
> >> > > > > No default values, multivalues, docvalues, copyfields, or any
> other
> >> > > > > properties set.
> >> > > >
> >> > > > You didn't indicate the Solr version.  In recent Solr versions,
> most
> >> > > > field classes other than TextField have docValues enabled by
> default,
> >> > > > even if the config is not mentioned on the field, and in those
> >> > versions,
> >> > > > docValues will take the place of stored if stored is false.
> >> > > >
> >> > > > Thanks,
> >> > > > Shawn
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>

Re: solrJ bug related to solrJ 4.10 for having both incremental partial update and child document on the same solr document!

[Ishan Chattopadhyaya]

Hi A.Nazemian,
Thanks for your findings. I think I was able to reproduce what you
reported. Please have a look at the tests for SOLR-6596 and add your
comments there. :-)
Regards,
Ishan

> Did anybody test that?
> Best regards.

> On Mon, Sep 29, 2014 at 2:05 PM, Ali Nazemian 
> wrote:

>> I also check both solr log and solr console. There is no error inside
>> that, it seems that every thing is fine! But actually there is not
any
>> child document after executing process.
>>
>>
>> On Mon, Sep 29, 2014 at 1:47 PM, Ali Nazemian 
>> wrote:
>>
>>> Dear all,
>>> Hi,
>>> Right now I face with the strange problem related to solJ client:
>>> When I use only incremental partial update. The incremental partial
>>> update works fine. When I use only the add child documents. It works
>>> perfectly and the child documents added successfully. But when I
have both
>>> of them in solrInputDocument the adding child documents did not
work. I
>>> think that the solr.add(document) method can not work when you have
both
>>> incremental partial update and child document in your solr document.
So
>>> probably it is a bug related to solj. Would you please consider this
>>> situation?
>>> Thank you very much.
>>> Best regards.
>>>
>>> --
>>> A.Nazemian

Apache Solr 7.7.1 released

[Ishan Chattopadhyaya]

1 March 2019, Apache Solr™ 7.7.1 available

The Lucene PMC is pleased to announce the release of Apache Solr 7.7.1

Solr is the popular, blazing fast, open source NoSQL search platform from
the
Apache Lucene project. Its major features include powerful full-text
search,
hit highlighting, faceted search and analytics, rich document parsing,
geospatial search, extensive REST APIs as well as parallel SQL. Solr is
enterprise grade, secure and highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features
of many of the world's largest internet sites.

This release includes 2 bug fixes since the 7.7.0 release:

 * Bugfix for ClassCastException when URPs try to read a String field which
returns
   a ByteArrayUTF8CHarSequence (a regression in release 7.7.0).

 * Bugfix: Autoscaling based replica placement was broken out of the box.
Solr 7.5
   enabled autoscaling based replica placement by default but in the
absence of default
   cluster policies, autoscaling can place more than 1 replica of the same
shard on the
   same node. Also, the maxShardsPerNode and createNodeSet was not
respected.

The release is available for immediate download at:

  http://www.apache.org/dyn/closer.lua/lucene/solr/7.7.1

Please read CHANGES.txt for a detailed list of changes:

  https://lucene.apache.org/solr/7_7_1/changes/Changes.html

Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

Apache Lucene 7.7.1 released

[Ishan Chattopadhyaya]

1 March 2019, Apache Solr™ 7.7.1 available

The Lucene PMC is pleased to announce the release of Apache Solr 7.7.1

Solr is the popular, blazing fast, open source NoSQL search platform from
the
Apache Lucene project. Its major features include powerful full-text
search,
hit highlighting, faceted search and analytics, rich document parsing,
geospatial search, extensive REST APIs as well as parallel SQL. Solr is
enterprise grade, secure and highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features
of many of the world's largest internet sites.

This release includes 2 bug fixes since the 7.7.0 release:

 * Bugfix for ClassCastException when URPs try to read a String field which
returns
   a ByteArrayUTF8CHarSequence (a regression in release 7.7.0).

 * Bugfix: Autoscaling based replica placement was broken out of the box.
Solr 7.5
   enabled autoscaling based replica placement by default but in the
absence of default
   cluster policies, autoscaling can place more than 1 replica of the same
shard on the
   same node. Also, the maxShardsPerNode and createNodeSet was not
respected.

The release is available for immediate download at:

  http://www.apache.org/dyn/closer.lua/lucene/solr/7.7.1

Please read CHANGES.txt for a detailed list of changes:

  https://lucene.apache.org/solr/7_7_1/changes/Changes.html

Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

Apache Solr 7.7.1 released

[Ishan Chattopadhyaya]

1 March 2019, Apache Solr™ 7.7.1 available

The Lucene PMC is pleased to announce the release of Apache Solr 7.7.1

Solr is the popular, blazing fast, open source NoSQL search platform from
the
Apache Lucene project. Its major features include powerful full-text
search,
hit highlighting, faceted search and analytics, rich document parsing,
geospatial search, extensive REST APIs as well as parallel SQL. Solr is
enterprise grade, secure and highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features
of many of the world's largest internet sites.

This release includes 2 bug fixes since the 7.7.0 release:

 * Bugfix for ClassCastException when URPs try to read a String field which
returns
   a ByteArrayUTF8CHarSequence (a regression in release 7.7.0).

 * Bugfix: Autoscaling based replica placement was broken out of the box.
Solr 7.5
   enabled autoscaling based replica placement by default but in the
absence of default
   cluster policies, autoscaling can place more than 1 replica of the same
shard on the
   same node. Also, the maxShardsPerNode and createNodeSet was not
respected.

The release is available for immediate download at:

  http://www.apache.org/dyn/closer.lua/lucene/solr/7.7.1

Please read CHANGES.txt for a detailed list of changes:

  https://lucene.apache.org/solr/7_7_1/changes/Changes.html

Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

[ANNOUNCE] Apache Solr 6.6.6 released

[Ishan Chattopadhyaya]

5 April 2019, Apache Solr™ 6.6.6 available

The Lucene PMC is pleased to announce the release of Apache Solr 6.6.6.

Solr is the popular, blazing fast, open source NoSQL search platform from the
Apache Lucene project. Its major features include powerful full-text search,
hit highlighting, faceted search and analytics, rich document parsing,
geospatial search, extensive REST APIs as well as parallel SQL. Solr is
enterprise grade, secure and highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation features
of many of the world's largest internet sites.

This release contains the following important bug fixes:

 * Fix memory leak (upon collection reload or ZooKeeper session
expiry) in ZkIndexSchemaReader.
 * Fix for Rule-based Authorization skipping authorization if querying
node host the collection
 * (CVE-2017-3164) Make it possible to configure a host whitelist for
distributed search

The release is available for immediate download at:

https://www-us.apache.org/dist/lucene/solr/6.6.6/

Please read CHANGES.txt for a detailed list of changes:

  https://lucene.apache.org/solr/6_6_6/changes/Changes.html

Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

[ANNOUNCE] Apache Solr 6.6.4 released

[Ishan Chattopadhyaya]

18 May 2018, Apache Solr™ 6.6.4 available

The Lucene PMC is pleased to announce the release of Apache Solr 6.6.4

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search and analytics, rich document
parsing, geospatial search, extensive REST APIs as well as parallel SQL.
Solr is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.

This release includes 1 bug fix since the 6.6.3 release:

* Do not allow to use absolute URIs for including other files in
solrconfig.xml and schema parsing

The release is available for immediate download at:

http://www.apache.org/dyn/closer.lua/lucene/solr/6.6.4

Please read CHANGES.txt for a detailed list of changes:

https://lucene.apache.org/solr/6_6_4/changes/Changes.html

Please report any feedback to the mailing lists (
http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases. It is possible that the mirror you are using may
not have replicated the release yet. If that is the case, please try
another mirror. This also goes for Maven access.

[ANNOUNCE] Apache Solr 6.6.4 released

[Ishan Chattopadhyaya]

18 May 2018, Apache Solr™ 6.6.4 available

The Lucene PMC is pleased to announce the release of Apache Solr 6.6.4

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search and analytics, rich document
parsing, geospatial search, extensive REST APIs as well as parallel SQL.
Solr is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.

This release includes 1 bug fix since the 6.6.3 release:

* Do not allow to use absolute URIs for including other files in
solrconfig.xml and schema parsing

The release is available for immediate download at:

http://www.apache.org/dyn/closer.lua/lucene/solr/6.6.4

Please read CHANGES.txt for a detailed list of changes:

https://lucene.apache.org/solr/6_6_4/changes/Changes.html

Please report any feedback to the mailing lists (
http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases. It is possible that the mirror you are using may
not have replicated the release yet. If that is the case, please try
another mirror. This also goes for Maven access.

[ANNOUNCE] Apache Lucene 6.6.5 released

[Ishan Chattopadhyaya]

3 July 2018, Apache Lucene™ 6.6.5 available

The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.5.

Apache Lucene is a high-performance, full-featured text search engine
library written entirely in Java. It is a technology suitable for nearly
any application that requires full-text search, especially cross-platform.

This release contains one bug fix. The release is available for immediate
download at:
http://lucene.apache.org/core/mirrors-core-latest-redir.html

Further details of changes are available in the change log available at:
http://lucene.apache.org/core/6_6_5/changes/Changes.html

Please report any feedback to the mailing lists (
http://lucene.apache.org/core/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases. It is possible that the mirror you are using may
not have replicated the release yet. If that is the case, please try
another mirror. This also applies to Maven access.

[ANNOUNCE] Apache Solr 6.6.5 released

[Ishan Chattopadhyaya]

03 July 2018, Apache Solr™ 6.6.5 available

The Lucene PMC is pleased to announce the release of Apache Solr 6.6.5

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search and analytics, rich document
parsing, geospatial search, extensive REST APIs as well as parallel SQL.
Solr is enterprise grade, secure and highly scalable, providing fault
tolerant distributed search and indexing, and powers the search and
navigation features of many of the world's largest internet sites.

This release includes the following changes:

* Ability to disable configset upload via -Dconfigset.upload.enabled=false
 startup parameter
* Referal to external resources in various config files now disallowed

The release is available for immediate download at:

http://www.apache.org/dyn/closer.lua/lucene/solr/6.6.5

Please read CHANGES.txt for a detailed list of changes:

https://lucene.apache.org/solr/6_6_5/changes/Changes.html

Please report any feedback to the mailing lists (
http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases. It is possible that the mirror you are using may
not have replicated the release yet. If that is the case, please try
another mirror. This also goes for Maven access.

Re: Getting authenticated user inside DocTransformer plugin

[Ishan Chattopadhyaya]

This is a problem, indeed. I've opened
https://issues.apache.org/jira/browse/SOLR-14491 to address this.

On Mon, Feb 3, 2020 at 11:08 PM mosheB  wrote:

> We are using Solr's kerberos authentication plugin and we are trying to
> implement field-level filtering based on the authenticated user and
> DocTransformer class:
>
> public class FieldAclTransformerFactory extends TransformerFactory {
> @Override
> public DocTransformer create(String field, SolrParams params,
> SolrQueryRequest req) {
> String user = req.getUserPrincipal().getName();
> return new FieldAclTransformer(user);
> }
> }
> //
> public class FieldAclTransformer extends DocTransformer {
> String user;
> public FieldAclTransformer(String user) {
> this.user = user;
> }
>
> @Override
> public void transform(SolrDocument doc, int docid, float score) {
> //filter fields according to applicative logic, based on
> the authenticated
> user.
> }
> }
>
> For simplicity, we do not use authorization plugin (here is our complete
> security.json file):
> {
> "authentication":{
> "class": "org.apache.solr.security.KerberosPlugin"
> }
> }
>
> During develop phase plugin was tested against collection with single shard
> and everything worked as expected (Solr 8.3.1).
> After moving to production, plugin failed. During debug we saw that the
> reason is that SOME shards were getting incorrect user from
> /req.getUserPrincipal().getName()/: instead of the ORIGINAL user, Solr's
> SPN
> is returned.
> Our best guess is that failing requests are the distributed requests (the
> requests the are routed from the node that received the original request),
> and indeed, if we add `/distrib=false/` to our request plugin wasnt
> failing.
>
> So, back to the question... is this a bug in solr, or is that just not way
> we suppose to get the authenticated user?
> Thanks.
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Ishan Chattopadhyaya]

Rather than getting rid of the terminology, we should get rid of the
standalone mode Solr altogether. I totally understand that SolrCloud is
broken in many ways today, but we should attempt to fix it and have it as
the only mode in Solr.

On Wed, 24 Jun, 2020, 8:17 pm Mike Drob,  wrote:

> Brend,
>
> I appreciate that you are trying to examine this issue from multiple sides
> and consider future implications, but I don’t think that is a stirring
> argument. By analogy, if we are out of eggs and my wife asks me to go to
> the store to get some, refusing to do so on the basis that she might call
> me while I’m there and also ask me to get milk would not be reasonable.
>
> What will come next may be an interesting question philosophically, but we
> are not discussing abstract concepts here. There is a concrete issue
> identified, and we’re soliciting input in how best to address it.
>
> Thank you for the suggestion of "guide/follower"
>
> Mike
>
> On Wed, Jun 24, 2020 at 6:30 AM Bernd Fehling <
> bernd.fehl...@uni-bielefeld.de> wrote:
>
> > I'm following this thread now for a while and I can understand
> > the wish to change some naming/wording/speech in one or the other
> > programs but I always get back to the one question:
> > "Is it the weapon which kills people or the hand controlled by
> > the mind which fires the weapon?"
> >
> > The thread started with slave - slavery, then turned over to master
> > and followed by leader (for me as a german... you know).
> > What will come next?
> >
> > And more over, we now discuss about changes in the source code and
> > due to this there need to be changes to the documentation.
> > What about the books people wrote about this programs and source code,
> > should we force this authors to rewrite their books?
> > May be we should file a request to all web search engines to reject
> > all stored content about these "banned" words?
> > And contact all web hosters about providing bad content.
> >
> > To sum things up, within my 40 years of computer science and writing
> > programs I have never had a nanosecond any thoughts about words
> > like master, slave, leader, ... other than thinking about computers
> > and programming.
> >
> > Just my 2 cents.
> >
> > For what it is worth, I tend to guide/follower if there "must be" any
> > changes.
> >
> > Bernd
> >
>

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Ishan Chattopadhyaya]

I would like to know under which situations (except for the various bugs
that will be fixed eventually) would a SolrCloud solution not suffice.
AFAICT, pull replicas and tlog replicas can provide similar replication
strategies commonly used with standalone Solr. I understand that running ZK
is an overhead and SolrCloud isn't best written when it comes to handling
ZK, but that can be improved.

And for those users who just want a single node Solr, they can just start
Solr with embedded ZK. It won't practically make difference.

On Sun, 28 Jun, 2020, 3:45 pm Ilan Ginzburg,  wrote:

> I disagree Ishan. We shouldn't get rid of standalone mode.
> I see three layers in Solr:
>
>1. Lucene (the actual search libraries)
>2. The server infra ("standalone Solr" basically)
>3. Cluster management (SolrCloud)
>
> There's value in using lower layers without higher ones.
> SolrCloud is a good solution for some use cases but there are others that
> need a search server and for which SolrCloud is not a good fit and will
> likely never be. If standalone mode is no longer available, such use cases
> will have to turn to something other than Solr (or fork and go their own
> way).
>
> Ilan
>
> On Sat, Jun 27, 2020 at 9:39 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
> > Rather than getting rid of the terminology, we should get rid of the
> > standalone mode Solr altogether. I totally understand that SolrCloud is
> > broken in many ways today, but we should attempt to fix it and have it as
> > the only mode in Solr.
> >
> > On Wed, 24 Jun, 2020, 8:17 pm Mike Drob,  wrote:
> >
> > > Brend,
> > >
> > > I appreciate that you are trying to examine this issue from multiple
> > sides
> > > and consider future implications, but I don’t think that is a stirring
> > > argument. By analogy, if we are out of eggs and my wife asks me to go
> to
> > > the store to get some, refusing to do so on the basis that she might
> call
> > > me while I’m there and also ask me to get milk would not be reasonable.
> > >
> > > What will come next may be an interesting question philosophically, but
> > we
> > > are not discussing abstract concepts here. There is a concrete issue
> > > identified, and we’re soliciting input in how best to address it.
> > >
> > > Thank you for the suggestion of "guide/follower"
> > >
> > > Mike
> > >
> > > On Wed, Jun 24, 2020 at 6:30 AM Bernd Fehling <
> > > bernd.fehl...@uni-bielefeld.de> wrote:
> > >
> > > > I'm following this thread now for a while and I can understand
> > > > the wish to change some naming/wording/speech in one or the other
> > > > programs but I always get back to the one question:
> > > > "Is it the weapon which kills people or the hand controlled by
> > > > the mind which fires the weapon?"
> > > >
> > > > The thread started with slave - slavery, then turned over to master
> > > > and followed by leader (for me as a german... you know).
> > > > What will come next?
> > > >
> > > > And more over, we now discuss about changes in the source code and
> > > > due to this there need to be changes to the documentation.
> > > > What about the books people wrote about this programs and source
> code,
> > > > should we force this authors to rewrite their books?
> > > > May be we should file a request to all web search engines to reject
> > > > all stored content about these "banned" words?
> > > > And contact all web hosters about providing bad content.
> > > >
> > > > To sum things up, within my 40 years of computer science and writing
> > > > programs I have never had a nanosecond any thoughts about words
> > > > like master, slave, leader, ... other than thinking about computers
> > > > and programming.
> > > >
> > > > Just my 2 cents.
> > > >
> > > > For what it is worth, I tend to guide/follower if there "must be" any
> > > > changes.
> > > >
> > > > Bernd
> > > >
> > >
> >
>

Re: [EXTERNAL] Getting rid of Master/Slave nomenclature in Solr

[Ishan Chattopadhyaya]

Cost of maintaining feature parity across the two modes is an overhead.
Security plugins, package manager (that doesn't work in standalone), UI,
etc. Our codebase is littered with checks to ascertain if we're zkAware.
There are massive benefits to maintainability if standalone mode were to go
away. Of course, provided all usecases that could be solved using
standalone can also be solved using SolrCloud. At that point, I'd love for
us to get rid of the term "SolrCloud".

On Sun, 28 Jun, 2020, 3:59 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> I would like to know under which situations (except for the various bugs
> that will be fixed eventually) would a SolrCloud solution not suffice.
> AFAICT, pull replicas and tlog replicas can provide similar replication
> strategies commonly used with standalone Solr. I understand that running ZK
> is an overhead and SolrCloud isn't best written when it comes to handling
> ZK, but that can be improved.
>
> And for those users who just want a single node Solr, they can just start
> Solr with embedded ZK. It won't practically make difference.
>
> On Sun, 28 Jun, 2020, 3:45 pm Ilan Ginzburg,  wrote:
>
>> I disagree Ishan. We shouldn't get rid of standalone mode.
>> I see three layers in Solr:
>>
>>1. Lucene (the actual search libraries)
>>2. The server infra ("standalone Solr" basically)
>>3. Cluster management (SolrCloud)
>>
>> There's value in using lower layers without higher ones.
>> SolrCloud is a good solution for some use cases but there are others that
>> need a search server and for which SolrCloud is not a good fit and will
>> likely never be. If standalone mode is no longer available, such use cases
>> will have to turn to something other than Solr (or fork and go their own
>> way).
>>
>> Ilan
>>
>> On Sat, Jun 27, 2020 at 9:39 PM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>> > Rather than getting rid of the terminology, we should get rid of the
>> > standalone mode Solr altogether. I totally understand that SolrCloud is
>> > broken in many ways today, but we should attempt to fix it and have it
>> as
>> > the only mode in Solr.
>> >
>> > On Wed, 24 Jun, 2020, 8:17 pm Mike Drob,  wrote:
>> >
>> > > Brend,
>> > >
>> > > I appreciate that you are trying to examine this issue from multiple
>> > sides
>> > > and consider future implications, but I don’t think that is a stirring
>> > > argument. By analogy, if we are out of eggs and my wife asks me to go
>> to
>> > > the store to get some, refusing to do so on the basis that she might
>> call
>> > > me while I’m there and also ask me to get milk would not be
>> reasonable.
>> > >
>> > > What will come next may be an interesting question philosophically,
>> but
>> > we
>> > > are not discussing abstract concepts here. There is a concrete issue
>> > > identified, and we’re soliciting input in how best to address it.
>> > >
>> > > Thank you for the suggestion of "guide/follower"
>> > >
>> > > Mike
>> > >
>> > > On Wed, Jun 24, 2020 at 6:30 AM Bernd Fehling <
>> > > bernd.fehl...@uni-bielefeld.de> wrote:
>> > >
>> > > > I'm following this thread now for a while and I can understand
>> > > > the wish to change some naming/wording/speech in one or the other
>> > > > programs but I always get back to the one question:
>> > > > "Is it the weapon which kills people or the hand controlled by
>> > > > the mind which fires the weapon?"
>> > > >
>> > > > The thread started with slave - slavery, then turned over to master
>> > > > and followed by leader (for me as a german... you know).
>> > > > What will come next?
>> > > >
>> > > > And more over, we now discuss about changes in the source code and
>> > > > due to this there need to be changes to the documentation.
>> > > > What about the books people wrote about this programs and source
>> code,
>> > > > should we force this authors to rewrite their books?
>> > > > May be we should file a request to all web search engines to reject
>> > > > all stored content about these "banned" words?
>> > > > And contact all web hosters about providing bad content.
>> > > >
>> > > > To sum things up, within my 40 years of computer science and writing
>> > > > programs I have never had a nanosecond any thoughts about words
>> > > > like master, slave, leader, ... other than thinking about computers
>> > > > and programming.
>> > > >
>> > > > Just my 2 cents.
>> > > >
>> > > > For what it is worth, I tend to guide/follower if there "must be"
>> any
>> > > > changes.
>> > > >
>> > > > Bernd
>> > > >
>> > >
>> >
>>
>

Re: [ANNOUNCE] Apache Solr 8.6.0 released

[Ishan Chattopadhyaya]

Dear Solr Users,

In this release (Solr 8.6), we have deprecated the following:

  1. Data Import Handler

  2. HDFS support

  3. Cross Data Center Replication (CDCR)



All of these are scheduled to be removed in a future 9.x release.

It was decided that these components did not meet the standards of quality
and support that we wish to ensure for all components we ship. Some of
these also relied on design patterns that we no longer recommend for use in
critical production environments.

If you rely on these features, you are encouraged to try out community
supported versions of these, where available [0]. Where such community
support is not available, we encourage you to participate in the migration
of these components into community supported packages and help continue the
development. We envision that using packages for these components via
package manager will actually make it easier for users to use such features.

Regards,

Ishan Chattopadhyaya

(On behalf of the Apache Lucene/Solr PMC)

[0] -
https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr

On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant 
wrote:

> The Lucene PMC is pleased to announce the release of Apache Solr 8.6.0.
>
>
> Solr is the popular, blazing fast, open source NoSQL search platform from
> the Apache Lucene project. Its major features include powerful full-text
> search, hit highlighting, faceted search, dynamic clustering, database
> integration, rich document handling, and geospatial search. Solr is highly
> scalable, providing fault tolerant distributed search and indexing, and
> powers the search and navigation features of many of the world's largest
> internet sites.
>
>
> Solr 8.6.0 is available for immediate download at:
>
>
>   <https://lucene.apache.org/solr/downloads.html>
>
>
> ### Solr 8.6.0 Release Highlights:
>
>
>  * Cross-Collection Join Queries: Join queries can now work
> cross-collection, even when shared or when spanning nodes.
>
>  * Search: Performance improvement for some types of queries when exact
> hit count isn't needed by using BlockMax WAND algorithm.
>
>  * Streaming Expression: Percentiles and standard deviation aggregations
> added to stats, facet and time series.  Streaming expressions added to
> /export handler.  Drill Streaming Expression for efficient and accurate
> high cardinality aggregation.
>
>  * Package manager: Support for cluster (CoreContainer) level plugins.
>
>  * Health Check: HealthCheckHandler can now require that all cores are
> healthy before returning OK.
>
>  * Zookeeper read API: A read API at /api/cluster/zk/* to fetch raw ZK
> data and view contents of a ZK directory.
>
>  * Admin UI: New panel with security info in admin UI's dashboard.
>
>  * Query DSL: Support for {param:ref} and {bool: {excludeTags:""}}
>
>  * Ref Guide: Major redesign of Solr's documentation.
>
>
> Please read CHANGES.txt for a full list of new features and changes:
>
>
>   <https://lucene.apache.org/solr/8_6_0/changes/Changes.html>
>
>
> Solr 8.6.0 also includes features, optimizations  and bugfixes in the
> corresponding Apache Lucene release:
>
>
>   <https://lucene.apache.org/core/8_6_0/changes/Changes.html>
>
>
> Note: The Apache Software Foundation uses an extensive mirroring network
> for
>
> distributing releases. It is possible that the mirror you are using may
> not have
>
> replicated the release yet. If that is the case, please try another mirror.
>
> This also applies to Maven access.
>

Re: [ANNOUNCE] Apache Solr 8.6.0 released

[Ishan Chattopadhyaya]

On Wed, 15 Jul, 2020, 8:37 pm Bernd Fehling, 
wrote:

>
>
> Am 15.07.20 um 16:07 schrieb Ishan Chattopadhyaya:
> > Dear Solr Users,
> >
> > In this release (Solr 8.6), we have deprecated the following:
> >
> >   1. Data Import Handler
> >
> >   2. HDFS support
> >
> >   3. Cross Data Center Replication (CDCR)
> >
>
> Seriously? :-(
>

Please see SOLR-14022.


> So next steps will be kicking out Cloud and go back to single node or what?
>

Not something we've considered yet.


> Why don't you just freeze the whole Solr development and switch to Elastic?
>

Not something we've considered yet.


>
> >
> >
> > All of these are scheduled to be removed in a future 9.x release.
> >
> > It was decided that these components did not meet the standards of
> quality
> > and support that we wish to ensure for all components we ship. Some of
> > these also relied on design patterns that we no longer recommend for use
> in
> > critical production environments.
> >
> > If you rely on these features, you are encouraged to try out community
> > supported versions of these, where available [0]. Where such community
> > support is not available, we encourage you to participate in the
> migration
> > of these components into community supported packages and help continue
> the
> > development. We envision that using packages for these components via
> > package manager will actually make it easier for users to use such
> features.
> >
> > Regards,
> >
> > Ishan Chattopadhyaya
> >
> > (On behalf of the Apache Lucene/Solr PMC)
> >
> > [0] -
> >
> https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr
> >
> > On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant  >
> > wrote:
> >
> >> The Lucene PMC is pleased to announce the release of Apache Solr 8.6.0.
> >>
> >>
> >> Solr is the popular, blazing fast, open source NoSQL search platform
> from
> >> the Apache Lucene project. Its major features include powerful full-text
> >> search, hit highlighting, faceted search, dynamic clustering, database
> >> integration, rich document handling, and geospatial search. Solr is
> highly
> >> scalable, providing fault tolerant distributed search and indexing, and
> >> powers the search and navigation features of many of the world's largest
> >> internet sites.
> >>
> >>
> >> Solr 8.6.0 is available for immediate download at:
> >>
> >>
> >>   <https://lucene.apache.org/solr/downloads.html>
> >>
> >>
> >> ### Solr 8.6.0 Release Highlights:
> >>
> >>
> >>  * Cross-Collection Join Queries: Join queries can now work
> >> cross-collection, even when shared or when spanning nodes.
> >>
> >>  * Search: Performance improvement for some types of queries when exact
> >> hit count isn't needed by using BlockMax WAND algorithm.
> >>
> >>  * Streaming Expression: Percentiles and standard deviation aggregations
> >> added to stats, facet and time series.  Streaming expressions added to
> >> /export handler.  Drill Streaming Expression for efficient and accurate
> >> high cardinality aggregation.
> >>
> >>  * Package manager: Support for cluster (CoreContainer) level plugins.
> >>
> >>  * Health Check: HealthCheckHandler can now require that all cores are
> >> healthy before returning OK.
> >>
> >>  * Zookeeper read API: A read API at /api/cluster/zk/* to fetch raw ZK
> >> data and view contents of a ZK directory.
> >>
> >>  * Admin UI: New panel with security info in admin UI's dashboard.
> >>
> >>  * Query DSL: Support for {param:ref} and {bool: {excludeTags:""}}
> >>
> >>  * Ref Guide: Major redesign of Solr's documentation.
> >>
> >>
> >> Please read CHANGES.txt for a full list of new features and changes:
> >>
> >>
> >>   <https://lucene.apache.org/solr/8_6_0/changes/Changes.html>
> >>
> >>
> >> Solr 8.6.0 also includes features, optimizations  and bugfixes in the
> >> corresponding Apache Lucene release:
> >>
> >>
> >>   <https://lucene.apache.org/core/8_6_0/changes/Changes.html>
> >>
> >>
> >> Note: The Apache Software Foundation uses an extensive mirroring network
> >> for
> >>
> >> distributing releases. It is possible that the mirror you are using may
> >> not have
> >>
> >> replicated the release yet. If that is the case, please try another
> mirror.
> >>
> >> This also applies to Maven access.
> >>
> >
>

Re: [ANNOUNCE] Apache Solr 8.6.0 released

[Ishan Chattopadhyaya]

Thanks Aroop for your feedback. We shall try to ensure continuity of
functionality via packages. Your help in those efforts would be greatly
appreciated as well. Let us take this discussion to SOLR-14660.

> Is there a replacement for DIH?
DIH is available as a community supported package. However, it is an
anti-pattern for a search engine to be pulling data from outside. Instead,
please consider writing separate indexing programs that pull data from the
database systems and index into Solr. It is not only a good practice, but
also more efficient in terms of throughput. For more information on this,
please start another thread in solr-users@ list, and more people can
suggest best alternatives here.


On Fri, Jul 17, 2020 at 5:50 AM matthew sporleder 
wrote:

> Is there a replacement for DIH?
>
> On Wed, Jul 15, 2020 at 10:08 AM Ishan Chattopadhyaya
>  wrote:
> >
> > Dear Solr Users,
> >
> > In this release (Solr 8.6), we have deprecated the following:
> >
> >   1. Data Import Handler
> >
> >   2. HDFS support
> >
> >   3. Cross Data Center Replication (CDCR)
> >
> >
> >
> > All of these are scheduled to be removed in a future 9.x release.
> >
> > It was decided that these components did not meet the standards of
> quality
> > and support that we wish to ensure for all components we ship. Some of
> > these also relied on design patterns that we no longer recommend for use
> in
> > critical production environments.
> >
> > If you rely on these features, you are encouraged to try out community
> > supported versions of these, where available [0]. Where such community
> > support is not available, we encourage you to participate in the
> migration
> > of these components into community supported packages and help continue
> the
> > development. We envision that using packages for these components via
> > package manager will actually make it easier for users to use such
> features.
> >
> > Regards,
> >
> > Ishan Chattopadhyaya
> >
> > (On behalf of the Apache Lucene/Solr PMC)
> >
> > [0] -
> >
> https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr
> >
> > On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant  >
> > wrote:
> >
> > > The Lucene PMC is pleased to announce the release of Apache Solr 8.6.0.
> > >
> > >
> > > Solr is the popular, blazing fast, open source NoSQL search platform
> from
> > > the Apache Lucene project. Its major features include powerful
> full-text
> > > search, hit highlighting, faceted search, dynamic clustering, database
> > > integration, rich document handling, and geospatial search. Solr is
> highly
> > > scalable, providing fault tolerant distributed search and indexing, and
> > > powers the search and navigation features of many of the world's
> largest
> > > internet sites.
> > >
> > >
> > > Solr 8.6.0 is available for immediate download at:
> > >
> > >
> > >   <https://lucene.apache.org/solr/downloads.html>
> > >
> > >
> > > ### Solr 8.6.0 Release Highlights:
> > >
> > >
> > >  * Cross-Collection Join Queries: Join queries can now work
> > > cross-collection, even when shared or when spanning nodes.
> > >
> > >  * Search: Performance improvement for some types of queries when exact
> > > hit count isn't needed by using BlockMax WAND algorithm.
> > >
> > >  * Streaming Expression: Percentiles and standard deviation
> aggregations
> > > added to stats, facet and time series.  Streaming expressions added to
> > > /export handler.  Drill Streaming Expression for efficient and accurate
> > > high cardinality aggregation.
> > >
> > >  * Package manager: Support for cluster (CoreContainer) level plugins.
> > >
> > >  * Health Check: HealthCheckHandler can now require that all cores are
> > > healthy before returning OK.
> > >
> > >  * Zookeeper read API: A read API at /api/cluster/zk/* to fetch raw ZK
> > > data and view contents of a ZK directory.
> > >
> > >  * Admin UI: New panel with security info in admin UI's dashboard.
> > >
> > >  * Query DSL: Support for {param:ref} and {bool: {excludeTags:""}}
> > >
> > >  * Ref Guide: Major redesign of Solr's documentation.
> > >
> > >
> > > Please read CHANGES.txt for a full list of new features and changes:
> > >
> > >
> > >   <https://lucene.apache.org/solr/8_6_0/changes/Changes.html>
> > >
> > >
> > > Solr 8.6.0 also includes features, optimizations  and bugfixes in the
> > > corresponding Apache Lucene release:
> > >
> > >
> > >   <https://lucene.apache.org/core/8_6_0/changes/Changes.html>
> > >
> > >
> > > Note: The Apache Software Foundation uses an extensive mirroring
> network
> > > for
> > >
> > > distributing releases. It is possible that the mirror you are using may
> > > not have
> > >
> > > replicated the release yet. If that is the case, please try another
> mirror.
> > >
> > > This also applies to Maven access.
> > >
>

Re: CDCR stress-test issues

[Ishan Chattopadhyaya]

FYI, CDCR support, as it exists in Solr today, has been deprecated in 8.6.
It suffers from serious design flaws and it allows such things to happen
that you observe. While there may be workarounds, it is advisable to not
rely on CDCR in production.

Thanks,
Ishan

On Thu, 2 Jul, 2020, 1:12 am Oakley, Craig (NIH/NLM/NCBI) [C],
 wrote:

> For the record, it is not just Solr7.4 which has the problem. When I start
> afresh with Solr8.5.2, both symptoms persist.
>
> With Solr8.5.2, tlogs accumulate endlessly at the non-Leader nodes of the
> Source SolrCloud and are never released regardless of maxNumLogsToKeep
> setting
>
> And with Solr8.5.2, if four scripts run simultaneously for a few minutes,
> each script running a loop each iteration of which adds batches of 6
> records to the Source SolrCloud, a couple dozen records wind up on the
> Source without ever arriving at the Target SolrCloud (although the Target
> does have records which were added after the missing records).
>
> Does anyone yet have any suggestion how to get CDCR to work properly?
>
>
> -Original Message-
> From: Oakley, Craig (NIH/NLM/NCBI) [C] 
> Sent: Wednesday, June 24, 2020 9:46 AM
> To: solr-user@lucene.apache.org
> Subject: CDCR stress-test issues
>
> In attempting to stress-test CDCR (running Solr 7.4), I am running into a
> couple of issues.
>
> One is that the tlog files keep accumulating for some nodes in the CDCR
> system, particularly for the non-Leader nodes in the Source SolrCloud. No
> quantity of hard commits seem to cause any of these tlog files to be
> released. This can become a problem upon reboot if there are hundreds of
> thousands of tlog files, and Solr fails to start (complaining that there
> are too many open files).
>
> The tlogs had been accumulating on all the nodes of the CDCR set of
> SolrClouds until I added these two lines to the solrconfig.xml file (for
> testing purposes, using numbers much lower than in the examples):
> 5
> 2
> Since then, it is mostly the non-Leader nodes of the Source SolrCloud
> which accumulates tlog files (the Target SolrCloud does seem to have a
> tendency to clean up the tlog files, as does the Leader of the Source
> SolrCloud). If I use ADDREPLICAPROP and REBALANCELEADERS to change which
> node is the Leader, and if I then start adding more data, the tlogs on the
> new Leader sometimes will go away, but then the old Leader begins
> accumulating tlog files. I am dubious whether frequent reassignment of
> Leadership would be a practical solution.
>
> I also have several times attempted to simulate a production environment
> by running several loops simultaneously, each of which inserts multiple
> records on each iteration of the loop. Several times, I end up with a dozen
> records on (both replicas of) the Source which never make it to (either
> replica of) the Target. The Target has thousands of records which were
> inserted before the missing records, and thousands of records which were
> inserted after the missing records (and all these records, the replicated
> and the missing, were inserted by curl commands which only differed in
> sequential numbers incorporated into the values being inserted).
>
> I also have a question regarding SOLR-13141: the 11/Feb/19 comment says
> that the fix for Solr 7.3 had a problem; and the header says "Affects
> Version/s: 7.5, 7.6": does that indicate that Solr 7.4 is not affected?
>
> Are  there any suggestions?
>
> Thanks
>

Recent and upcoming deprecations

[Ishan Chattopadhyaya]

Hi Solr Users,
Here is a list of recent and upcoming deprecations in Solr 8.x.
https://cwiki.apache.org/confluence/display/SOLR/Deprecations

Please feel free to chime in if you have any questions. You can comment
here or in the specific JIRA issues.

Thanks and regards,
Ishan Chattopadhyaya

Re: [ANNOUNCE] Apache Solr 8.6.0 released

[Ishan Chattopadhyaya]

For any further discussion on the deprecations, please find a thread
"Recent and upcoming deprecations" [0] and we can discuss there.
Thanks,
Ishan

[0] -
https://www.mail-archive.com/solr-user@lucene.apache.org/msg151762.html

On Fri, Jul 17, 2020 at 8:50 AM matthew sporleder 
wrote:

> I hear all of that and agree, obviously, but "curl
> solr:8983/collection/dataimport?blah" in cron was *pretty freaking
> easy* ;)
>
> Not sure why "pull" is elevated to "anti-pattern"; data is data is data
>
> On Thu, Jul 16, 2020 at 8:49 PM Ishan Chattopadhyaya
>  wrote:
> >
> > Thanks Aroop for your feedback. We shall try to ensure continuity of
> > functionality via packages. Your help in those efforts would be greatly
> > appreciated as well. Let us take this discussion to SOLR-14660.
> >
> > > Is there a replacement for DIH?
> > DIH is available as a community supported package. However, it is an
> > anti-pattern for a search engine to be pulling data from outside.
> Instead,
> > please consider writing separate indexing programs that pull data from
> the
> > database systems and index into Solr. It is not only a good practice, but
> > also more efficient in terms of throughput. For more information on this,
> > please start another thread in solr-users@ list, and more people can
> > suggest best alternatives here.
> >
> >
> > On Fri, Jul 17, 2020 at 5:50 AM matthew sporleder 
> > wrote:
> >
> > > Is there a replacement for DIH?
> > >
> > > On Wed, Jul 15, 2020 at 10:08 AM Ishan Chattopadhyaya
> > >  wrote:
> > > >
> > > > Dear Solr Users,
> > > >
> > > > In this release (Solr 8.6), we have deprecated the following:
> > > >
> > > >   1. Data Import Handler
> > > >
> > > >   2. HDFS support
> > > >
> > > >   3. Cross Data Center Replication (CDCR)
> > > >
> > > >
> > > >
> > > > All of these are scheduled to be removed in a future 9.x release.
> > > >
> > > > It was decided that these components did not meet the standards of
> > > quality
> > > > and support that we wish to ensure for all components we ship. Some
> of
> > > > these also relied on design patterns that we no longer recommend for
> use
> > > in
> > > > critical production environments.
> > > >
> > > > If you rely on these features, you are encouraged to try out
> community
> > > > supported versions of these, where available [0]. Where such
> community
> > > > support is not available, we encourage you to participate in the
> > > migration
> > > > of these components into community supported packages and help
> continue
> > > the
> > > > development. We envision that using packages for these components via
> > > > package manager will actually make it easier for users to use such
> > > features.
> > > >
> > > > Regards,
> > > >
> > > > Ishan Chattopadhyaya
> > > >
> > > > (On behalf of the Apache Lucene/Solr PMC)
> > > >
> > > > [0] -
> > > >
> > >
> https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr
> > > >
> > > > On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant <
> bruno.roust...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > The Lucene PMC is pleased to announce the release of Apache Solr
> 8.6.0.
> > > > >
> > > > >
> > > > > Solr is the popular, blazing fast, open source NoSQL search
> platform
> > > from
> > > > > the Apache Lucene project. Its major features include powerful
> > > full-text
> > > > > search, hit highlighting, faceted search, dynamic clustering,
> database
> > > > > integration, rich document handling, and geospatial search. Solr is
> > > highly
> > > > > scalable, providing fault tolerant distributed search and
> indexing, and
> > > > > powers the search and navigation features of many of the world's
> > > largest
> > > > > internet sites.
> > > > >
> > > > >
> > > > > Solr 8.6.0 is available for immediate download at:
> > > > >
> > > > >
> > > > >   <https://lucene.apache.org/solr/downloads.html>
> > > > >
> >

Re: Solr fails to start with G1 GC

[Ishan Chattopadhyaya]

Try using haveged for your entropy problem.

On Thu, 16 Jul, 2020, 10:20 pm Walter Underwood, 
wrote:

> Instead of editing bin/solr, you should be able to set GC_TUNE in
> solr.in.sh, as I showed in my post below.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
> > On Jul 16, 2020, at 7:52 AM, krishan goyal 
> wrote:
> >
> > The issue was figured out by starting solr with the -f parameter which
> > starts solr in foreground and provides the errors if any
> >
> > Got an error - "Conflicting collector combinations in option list; please
> > refer to the release notes for the combinations allowed"
> >
> > Turns out bin/solr file starts with CMS by default and had to disable
> that
> > to resolve the conflict.
> >
> >
> > On Wed, Jul 15, 2020 at 10:20 PM Walter Underwood  >
> > wrote:
> >
> >> I don’t see a heap size specified, so it is probably trying to run with
> >> a 512 Megabyte heap. That might just not work with the 32M region
> >> size.
> >>
> >> Here are the options we have been using for 3+ years on about 150 hosts.
> >>
> >> SOLR_HEAP=8g
> >> # Use G1 GC  -- wunder 2017-01-23
> >> # Settings from https://wiki.apache.org/solr/ShawnHeisey
> >> GC_TUNE=" \
> >> -XX:+UseG1GC \
> >> -XX:+ParallelRefProcEnabled \
> >> -XX:G1HeapRegionSize=8m \
> >> -XX:MaxGCPauseMillis=200 \
> >> -XX:+UseLargePages \
> >> -XX:+AggressiveOpts \
> >> "
> >>
> >> wunder
> >> Walter Underwood
> >> wun...@wunderwood.org
> >> http://observer.wunderwood.org/  (my blog)
> >>
> >>> On Jul 15, 2020, at 4:24 AM, krishan goyal 
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am using Solr 7.7
> >>>
> >>> I am trying to start my solr server with G1 GC instead of the default
> CMS
> >>> but the solr service doesn't get up.
> >>>
> >>> The command I use to start solr is
> >>>
> >>> bin/solr start -p 25280 -a "-Dsolr.solr.home=
> >>> -Denable.slave=true -Denable.master=false -XX:+UseG1GC
> >>> -XX:MaxGCPauseMillis=500 -XX:+UnlockExperimentalVMOptions
> >>> -XX:G1MaxNewSizePercent=30 -XX:G1NewSizePercent=5
> >> -XX:G1HeapRegionSize=32M
> >>> -XX:InitiatingHeapOccupancyPercent=70"
> >>>
> >>> I have tried various permutations of the start command by dropping /
> >> adding
> >>> other parameters but it doesn't work. However starts up just fine with
> >>> just "-Dsolr.solr.home= -Denable.slave=true
> >>> -Denable.master=false" and starts up with the default CMS collector
> >>>
> >>> I don't get any useful error logs too. It waits for default 180 secs
> and
> >>> then prints
> >>>
> >>> Warning: Available entropy is low. As a result, use of the UUIDField,
> >> SSL,
> >>> or any other features that require
> >>> RNG might not work properly. To check for the amount of available
> >> entropy,
> >>> use 'cat /proc/sys/kernel/random/entropy_avail'.
> >>>
> >>> Waiting up to 180 seconds to see Solr running on port 25280 [|]  Still
> >> not
> >>> seeing Solr listening on 25280 after 180 seconds!
> >>> 2020-07-15 07:07:52.042 INFO  (coreCloseExecutor-60-thread-6) [
> >>> x:coreName] o.a.s.c.SolrCore [coreName]  CLOSING SolrCore
> >>> org.apache.solr.core.SolrCore@7cc638d8
> >>> 2020-07-15 07:07:52.099 INFO  (coreCloseExecutor-60-thread-6) [
> >>> x:coreName] o.a.s.m.SolrMetricManager Closing metric reporters for
> >>> registry=solr.core.coreName, tag=7cc638d8
> >>> 2020-07-15 07:07:52.100 INFO  (coreCloseExecutor-60-thread-6) [
> >>> x:coreName] o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metrics.reporters.SolrJmxReporter@5216981f: rootName
> =
> >>> null, domain = solr.core.coreName, service url = null, agent id = null]
> >> for
> >>> registry solr.core.coreName /
> >> com.codahale.metrics.MetricRegistry@32988ddf
> >>> 2020-07-15 07:07:52.173 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.SolrMetricManager Closing metric reporters for
> >> registry=solr.node,
> >>> tag=null
> >>> 2020-07-15 07:07:52.173 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metrics.reporters.SolrJmxReporter@28952dea: rootName
> =
> >>> null, domain = solr.node, service url = null, agent id = null] for
> >> registry
> >>> solr.node / com.codahale.metrics.MetricRegistry@655f4a3f
> >>> 2020-07-15 07:07:52.175 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.jvm,
> >>> tag=null
> >>> 2020-07-15 07:07:52.175 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metrics.reporters.SolrJmxReporter@69c6161d: rootName
> =
> >>> null, domain = solr.jvm, service url = null, agent id = null] for
> >> registry
> >>> solr.jvm / com.codahale.metrics.MetricRegistry@1252ce77
> >>> 2020-07-15 07:07:52.176 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.SolrMetricManager Closing metric reporters for
> >> registry=solr.jetty,
> >>> tag=null
> >>> 2020-07-15 07:07:52.176 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metr

Re: Cannot read ZK Kerberos conf when enabling java security manager on 8.6

[Ishan Chattopadhyaya]

I think this warrants a JIRA. To work around this issue for now, you can
use an environment variable SOLR_SECURITY_MANAGER_ENABLED=false before
starting Solr.

On Thu, Jul 16, 2020 at 11:58 PM Jörn Franke  wrote:

> The solution would be probably a policy file shipped with Solr that allows
> the ZK jar to create a logincontext. I suggest that Solr ships it otherwise
> one would need to adapt it for every Solr update manually to include the
> version of the ZK jar.
>
> On Thu, Jul 16, 2020 at 8:15 PM Jörn Franke  wrote:
>
> > I believe it is a bug in Solr because we need to create a policy to allow
> > creating a login context:
> > See here chapter "Running the Code with a Security Manager"
> >
> >
> http://www.informatik.hs-furtwangen.de/doku/java/j2sdk-1_4_1-doc/guide/security/jaas/tutorials/LoginConfigFile.html
> >
> > Please confirm and I will create a JIRA issue for Solr
> >
> > On Thu, Jul 16, 2020 at 8:06 PM Jörn Franke 
> wrote:
> >
> >> Hallo,
> >>
> >> I am using Solr 8.6.0.
> >> When activating the Java security manager then Solr cannot use anymore
> >> the jaas-client conf specified via java.security.auth.login.conf with
> >> Zookeeper. We have configured Kerberos authentication for Zookeeper.
> >> When disabling java security manager it works perfectly fine.
> >>
> >> The exact error message is : „No JAAS configuration section named
> >> 'Client' was found“. Somehow it seems that the Java security manager
> blocks
> >> access to that file .
> >> The directory for the file is in the -Dsolr.allowPaths
> >>  Could this be a bug or is it a misconfiguration?
> >>
> >>
> >> Thank you.
> >>
> >> Best regards
> >
> >
>

Re: CVE-2018-11802: Apache Solr authorization bug vulnerability disclosure

[Ishan Chattopadhyaya]

This fix has also been backported to Solr 6.6.6 for users who are
stuck with Solr 6.x.

(Sorry, I hadn't updated the issue and hence this was missed in the
original mail.)

On Wed, Apr 24, 2019 at 12:35 PM Noble Paul  wrote:
>
> CVE-2018-11802: Apache Solr authorization bug disclosure
> Severity: Important
> Vendor: The Apache Software Foundation
> Versions Affected: Apache Solr 7.6 or less
>
> Description:
> jira  ticket : https://issues.apache.org/jira/browse/SOLR-12514
> In apache Solr the cluster can be partitioned into multiple
> collections and only a subset of nodes actually host any given
> collection. However, if a node receives a request for a collection it
> does not host, it proxies the request to a relevant node and serves
> the request. Solr bypasses all authorization settings for such
> requests. This affects all Solr versions that uses the default
> authorization mechanism of Solr (RuleBasedAuthorizationPlugin)
>
> Mitigation:
> A fix is provided in Solr 7.7 version and upwards. If you use Solr's
> authorization mechanism, please upgrade to a version newer than Solr
> 7.7.
>
> Credit: This issue was discovered by Mahesh Kumar Vasanthu Somashekar.
>
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>

Re: Solr 8.1 issue with collection aliases

[Ishan Chattopadhyaya]

Thanks Jörn for reporting this. Sounds like some backward comparability
break with aliases. Can you please open a JIRA? I'll take a look at
reproducing it soon.

On Wed, 15 May, 2019, 7:18 PM Mikhail Khludnev,  wrote:

> It seems creating alias in Solr Admin UI is broken. It's a minor issue for
> 8.1.0
> I've alias via REST call
> http://localhost:8983/solr/admin/collections?action=CREATEALIAS&name=testalias&collections=gettingstarted
>  successfully.
> Jörn, thanks for reporting.
>
> On Tue, May 14, 2019 at 11:03 PM Jörn Franke  wrote:
>
>> Hi,
>>
>> I tried to upgrade from 8.0 to 8.1. I noticed that there is an issue with
>> collection aliases, but I am not 100% sure it is due to the upgrade.
>>
>> Situation:
>> I have a collection called c_testcollection.
>> I have an alias called testcollection.
>> Alias "testcollection" points to "c_testcollection".
>> On Solr 8.0 no issue
>>
>> After upgrade to Solr 8.1:
>> When I do a query on c_testcollection then there is no issue:
>> http://localhost:8983/solr/c_testcollection/select?q=test
>> When I do a query on testcollection then I receive the stacktrace below
>> http://localhost:8983/solr/testcollection/select?q=test
>>
>> Additionally I observe a strange behavior in the admin ui. When I try to
>> create an alias (e.g. new) for a new collection (e.g. c_new) then it
>> creates two aliases:
>> new => c_new
>> c_new => c_new
>> if i then do a query on the alias new it works without issues. If I remove
>> the alias from c_new to c_new then I get the same error. Is this desired
>> behaviour?
>> It is rather annoying to have unnecessary aliases, because I need to
>> filter
>> them out in my application when retrieving all aliases.
>> Is there a related issue.
>>
>> Here the stacktrace:
>> {
>>   "error":{
>> "trace":"java.lang.NullPointerException\n\tat
>>
>> java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:351)\n\tat
>>
>> org.apache.solr.common.cloud.Aliases.resolveAliasesGivenAliasMap(Aliases.java:258)\n\tat
>>
>> org.apache.solr.common.cloud.Aliases.resolveAliases(Aliases.java:181)\n\tat
>>
>> org.apache.solr.servlet.HttpSolrCall.resolveCollectionListOrAlias(HttpSolrCall.java:385)\n\tat
>> org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:273)\n\tat
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:486)\n\tat
>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)\n\tat
>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)\n\tat
>>
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
>>
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>>
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>>
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
>>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>>
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>>
>> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>> org.eclipse.jetty.server.Server.handle(Server.java:502)\n\tat
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)\n\tat
>> org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305)\n\tat
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)\n\tat
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.

Re: Solr 8.1 issue with collection aliases

[Ishan Chattopadhyaya]

Please open a JIRA.

On Thu, 16 May, 2019, 8:09 AM Jörn Franke,  wrote:

> Sorry autocorrection. It is not only a admin UI issue. I described in my
> previous email that access through the collection alias does not work. I
> cannot even do execute the select query handler if I use the collection
> alias instead of the collection name.
> So it is maybe more problematic.
>
> Am 16.05.2019 um 04:36 schrieb Jörn Franke :
>
> Note only an admin UI issue. Access collections via their alias does not
> work.
>
> Am 15.05.2019 um 15:47 schrieb Mikhail Khludnev :
>
> It seems creating alias in Solr Admin UI is broken. It's a minor issue for
> 8.1.0
> I've alias via REST call
> http://localhost:8983/solr/admin/collections?action=CREATEALIAS&name=testalias&collections=gettingstarted
>  successfully.
> Jörn, thanks for reporting.
>
> On Tue, May 14, 2019 at 11:03 PM Jörn Franke  wrote:
>
>> Hi,
>>
>> I tried to upgrade from 8.0 to 8.1. I noticed that there is an issue with
>> collection aliases, but I am not 100% sure it is due to the upgrade.
>>
>> Situation:
>> I have a collection called c_testcollection.
>> I have an alias called testcollection.
>> Alias "testcollection" points to "c_testcollection".
>> On Solr 8.0 no issue
>>
>> After upgrade to Solr 8.1:
>> When I do a query on c_testcollection then there is no issue:
>> http://localhost:8983/solr/c_testcollection/select?q=test
>> When I do a query on testcollection then I receive the stacktrace below
>> http://localhost:8983/solr/testcollection/select?q=test
>>
>> Additionally I observe a strange behavior in the admin ui. When I try to
>> create an alias (e.g. new) for a new collection (e.g. c_new) then it
>> creates two aliases:
>> new => c_new
>> c_new => c_new
>> if i then do a query on the alias new it works without issues. If I remove
>> the alias from c_new to c_new then I get the same error. Is this desired
>> behaviour?
>> It is rather annoying to have unnecessary aliases, because I need to
>> filter
>> them out in my application when retrieving all aliases.
>> Is there a related issue.
>>
>> Here the stacktrace:
>> {
>>   "error":{
>> "trace":"java.lang.NullPointerException\n\tat
>>
>> java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:351)\n\tat
>>
>> org.apache.solr.common.cloud.Aliases.resolveAliasesGivenAliasMap(Aliases.java:258)\n\tat
>>
>> org.apache.solr.common.cloud.Aliases.resolveAliases(Aliases.java:181)\n\tat
>>
>> org.apache.solr.servlet.HttpSolrCall.resolveCollectionListOrAlias(HttpSolrCall.java:385)\n\tat
>> org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:273)\n\tat
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:486)\n\tat
>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)\n\tat
>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)\n\tat
>>
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
>>
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>>
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>>
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat
>>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)\n\tat
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>>
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>>
>> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>> org.eclipse.jetty.server.Server.handle(Server.java:502)\n\tat
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)\n\tat
>> org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305)\n\tat
>>
>> org.eclipse.jetty.util.thread.strate

[ANNOUNCE] Apache Solr 8.1.0 released

[Ishan Chattopadhyaya]

= 16 March 2019, Apache Solr™ 8.1.0 available =

The Lucene PMC is pleased to announce the release of Apache Solr 8.1.0

Solr is the popular, blazing fast, open source NoSQL search platform
from the Apache Lucene project. Its major features include powerful
full-text search, hit highlighting, faceted search and analytics, rich
document parsing, geospatial search, extensive REST APIs as well as
parallel SQL. Solr is enterprise grade, secure and highly scalable,
providing fault tolerant distributed search and indexing, and powers
the search and navigation features of many of the world's largest
internet sites.

The release is available for immediate download at:

http://www.apache.org/dyn/closer.lua/lucene/solr/8.1.0

Please read CHANGES.txt for a detailed list of changes:

https://lucene.apache.org/solr/8_1_0/changes/Changes.html

== Solr 8.1.0 Release Highlights ==

 * Partial/Atomic Updates for nested documents. This enables atomic
updates for nested documents, without the need to supply the whole
nested hierarchy (which would be overwritten if absent).
 * Category Routed Aliases feature introduced for data driven
assignment of documents to collections based on values of a field
 * JWT Token authentication plugin with OpenID Connect implicit flow
login through Admin UI
 * REINDEXCOLLECTION command for re-indexing of existing collections
 * Collection RENAME command and support using aliases in most
collection admin commands
 * Read-only mode for SolrCloud collections

Solr 8.1.0 also includes many other new features as well as numerous
optimizations and bugfixes of the corresponding Apache Lucene release.

Please report any feedback to the mailing lists
(http://lucene.apache.org/solr/discussion.html)

Note: The Apache Software Foundation uses an extensive mirroring
network for distributing releases. It is possible that the mirror you
are using may not have replicated the release yet. If that is the
case, please try another mirror. This also goes for Maven access.

Re: Solr 8.1 issue with collection aliases

[Ishan Chattopadhyaya]

Does this warrant a 8.1.1 release? I think this is serious enough.

On Thu, May 16, 2019 at 12:03 PM Jörn Franke  wrote:
>
> SOLR-13475
>
> > Am 16.05.2019 um 05:24 schrieb Ishan Chattopadhyaya 
> > :
> >
> > Please open a JIRA.
> >
> >> On Thu, 16 May, 2019, 8:09 AM Jörn Franke,  wrote:
> >> Sorry autocorrection. It is not only a admin UI issue. I described in my 
> >> previous email that access through the collection alias does not work. I 
> >> cannot even do execute the select query handler if I use the collection 
> >> alias instead of the collection name.
> >> So it is maybe more problematic.
> >>
> >>> Am 16.05.2019 um 04:36 schrieb Jörn Franke :
> >>>
> >>> Note only an admin UI issue. Access collections via their alias does not 
> >>> work.
> >>>
> >>>> Am 15.05.2019 um 15:47 schrieb Mikhail Khludnev :
> >>>>
> >>>> It seems creating alias in Solr Admin UI is broken. It's a minor issue 
> >>>> for 8.1.0
> >>>> I've alias via REST call 
> >>>> http://localhost:8983/solr/admin/collections?action=CREATEALIAS&name=testalias&collections=gettingstarted
> >>>>   successfully.
> >>>> Jörn, thanks for reporting.
> >>>>
> >>>>> On Tue, May 14, 2019 at 11:03 PM Jörn Franke  
> >>>>> wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I tried to upgrade from 8.0 to 8.1. I noticed that there is an issue 
> >>>>> with
> >>>>> collection aliases, but I am not 100% sure it is due to the upgrade.
> >>>>>
> >>>>> Situation:
> >>>>> I have a collection called c_testcollection.
> >>>>> I have an alias called testcollection.
> >>>>> Alias "testcollection" points to "c_testcollection".
> >>>>> On Solr 8.0 no issue
> >>>>>
> >>>>> After upgrade to Solr 8.1:
> >>>>> When I do a query on c_testcollection then there is no issue:
> >>>>> http://localhost:8983/solr/c_testcollection/select?q=test
> >>>>> When I do a query on testcollection then I receive the stacktrace below
> >>>>> http://localhost:8983/solr/testcollection/select?q=test
> >>>>>
> >>>>> Additionally I observe a strange behavior in the admin ui. When I try to
> >>>>> create an alias (e.g. new) for a new collection (e.g. c_new) then it
> >>>>> creates two aliases:
> >>>>> new => c_new
> >>>>> c_new => c_new
> >>>>> if i then do a query on the alias new it works without issues. If I 
> >>>>> remove
> >>>>> the alias from c_new to c_new then I get the same error. Is this desired
> >>>>> behaviour?
> >>>>> It is rather annoying to have unnecessary aliases, because I need to 
> >>>>> filter
> >>>>> them out in my application when retrieving all aliases.
> >>>>> Is there a related issue.
> >>>>>
> >>>>> Here the stacktrace:
> >>>>> {
> >>>>>   "error":{
> >>>>> "trace":"java.lang.NullPointerException\n\tat
> >>>>> java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:351)\n\tat
> >>>>> org.apache.solr.common.cloud.Aliases.resolveAliasesGivenAliasMap(Aliases.java:258)\n\tat
> >>>>> org.apache.solr.common.cloud.Aliases.resolveAliases(Aliases.java:181)\n\tat
> >>>>> org.apache.solr.servlet.HttpSolrCall.resolveCollectionListOrAlias(HttpSolrCall.java:385)\n\tat
> >>>>> org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:273)\n\tat
> >>>>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:486)\n\tat
> >>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)\n\tat
> >>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)\n\tat
> >>>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat
> >>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat
> >>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
> >>>>> org.eclipse.jetty.security.SecurityHa

Re: Solr 8.1 issue with collection aliases

[Ishan Chattopadhyaya]

Absolutely. This is a critical feature.
Andrzej, would you have time to do a 8.1.1 release? We also need to
coordinate with Jan, since he's doing a 7.7.2 release right now.

On Thu, May 16, 2019 at 5:18 PM Jörn Franke  wrote:
>
> For the specific client the Solr 8.1 is not usable with this bug.
>
> Collection aliases are also a crucial feature for doing “zero-downtime” 
> reindexing or changing the Schema of a collection or for switching back to an 
> old Index if the new Index structure has bugs etc.
>
> However  I also understand that there are other considerations by other 
> people.
>
> > Am 16.05.2019 um 11:55 schrieb Ishan Chattopadhyaya 
> > :
> >
> > Does this warrant a 8.1.1 release? I think this is serious enough.
> >
> >> On Thu, May 16, 2019 at 12:03 PM Jörn Franke  wrote:
> >>
> >> SOLR-13475
> >>
> >>> Am 16.05.2019 um 05:24 schrieb Ishan Chattopadhyaya 
> >>> :
> >>>
> >>> Please open a JIRA.
> >>>
> >>>> On Thu, 16 May, 2019, 8:09 AM Jörn Franke,  wrote:
> >>>> Sorry autocorrection. It is not only a admin UI issue. I described in my 
> >>>> previous email that access through the collection alias does not work. I 
> >>>> cannot even do execute the select query handler if I use the collection 
> >>>> alias instead of the collection name.
> >>>> So it is maybe more problematic.
> >>>>
> >>>>> Am 16.05.2019 um 04:36 schrieb Jörn Franke :
> >>>>>
> >>>>> Note only an admin UI issue. Access collections via their alias does 
> >>>>> not work.
> >>>>>
> >>>>>> Am 15.05.2019 um 15:47 schrieb Mikhail Khludnev :
> >>>>>>
> >>>>>> It seems creating alias in Solr Admin UI is broken. It's a minor issue 
> >>>>>> for 8.1.0
> >>>>>> I've alias via REST call 
> >>>>>> http://localhost:8983/solr/admin/collections?action=CREATEALIAS&name=testalias&collections=gettingstarted
> >>>>>>   successfully.
> >>>>>> Jörn, thanks for reporting.
> >>>>>>
> >>>>>>> On Tue, May 14, 2019 at 11:03 PM Jörn Franke  
> >>>>>>> wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I tried to upgrade from 8.0 to 8.1. I noticed that there is an issue 
> >>>>>>> with
> >>>>>>> collection aliases, but I am not 100% sure it is due to the upgrade.
> >>>>>>>
> >>>>>>> Situation:
> >>>>>>> I have a collection called c_testcollection.
> >>>>>>> I have an alias called testcollection.
> >>>>>>> Alias "testcollection" points to "c_testcollection".
> >>>>>>> On Solr 8.0 no issue
> >>>>>>>
> >>>>>>> After upgrade to Solr 8.1:
> >>>>>>> When I do a query on c_testcollection then there is no issue:
> >>>>>>> http://localhost:8983/solr/c_testcollection/select?q=test
> >>>>>>> When I do a query on testcollection then I receive the stacktrace 
> >>>>>>> below
> >>>>>>> http://localhost:8983/solr/testcollection/select?q=test
> >>>>>>>
> >>>>>>> Additionally I observe a strange behavior in the admin ui. When I try 
> >>>>>>> to
> >>>>>>> create an alias (e.g. new) for a new collection (e.g. c_new) then it
> >>>>>>> creates two aliases:
> >>>>>>> new => c_new
> >>>>>>> c_new => c_new
> >>>>>>> if i then do a query on the alias new it works without issues. If I 
> >>>>>>> remove
> >>>>>>> the alias from c_new to c_new then I get the same error. Is this 
> >>>>>>> desired
> >>>>>>> behaviour?
> >>>>>>> It is rather annoying to have unnecessary aliases, because I need to 
> >>>>>>> filter
> >>>>>>> them out in my application when retrieving all aliases.
> >>>>>>> Is there a related issue.
> >>>>>>>
> >>>>>>> Here the stacktrace:
&

[ANNOUNCE] Apache Solr 8.3.0 released

[Ishan Chattopadhyaya]

## 2 November 2019, Apache Solr™ 8.3.0 available

The Lucene PMC is pleased to announce the release of Apache Solr 8.3.0.

Solr is the popular, blazing fast, open source NoSQL search platform
from the Apache Lucene project. Its major features include powerful
full-text search, hit highlighting, faceted search, dynamic
clustering, database integration, rich document handling, and
geospatial search. Solr is highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features of many of the world's largest internet sites.

Solr 8.3.0 is available for immediate download at:

  

### Solr 8.3.0 Release Highlights:

  *Two dimensional routed aliases are now available for organizing
collections based on the data values of two fields
  *SPLITSHARD implements a new splitByPrefix option that takes into
account the actual document distribution when using compositeIds
  *QueryElevationComponent can have query rules configured with
match="subset" wherein the words need only match a subset of the
query's words and in any order
  *Command line option to export documents to a file
  *Support deterministic replica routing preferences for better cache usage
  *Ability to query aliases in Solr Admin UI
  *JWTAuthPlugin supports multiple JWKS endpoints and multiple IdP issuers
  *JSON faceting now supports arbitrary ranges for range facets
  *Support integral plots, cosine distance and string truncation with
math expressions (Joel Bernstein)
  *New cat() stream source to create tuples from lines in local files
  *Add upper, lower, trim and split Stream Evaluators
  *Add CsvStream, TsvStream Streaming Expressions and supporting
Stream Evaluators
  *Add CaffeineCache, an efficient implementation of SolrCache
  *Live SPLITSHARD can lose updates due to cluster state change
between checking if the current shard is active and later checking if
there are any sub-shard leaders to forward the update to
  *Fix for SPLITSHARD (async) with failures in underlying
sub-operations can result in data loss
  *Allow dynamic resizing of SolrCache-s
  *Allow optional redaction of data saved by 'bin/solr autoscaling -save'
  *Optimized large managed schema modifications (internal O(n^2) problem)
  *Max idle time support for SolrCache implementations
  *Add Prometheus Exporter GC and Heap options
  *SSL: Adding Enabling/Disabling client's hostname verification config
  *Introducing SolrClient.ping(collection) in SolrJ
  *Fix for CDCR bootstrap not replicating index to the replicas of
target cluster
  *Fixed a race condition when initializing metrics for new security
plugins on security.json change
  *Fixed JWTAuthPlugin to update metrics prior to continuing w/other
filters or returning error
  *Fixed distributed grouping when multiple 'fl' params are specified
  *JMX MBeans are not exposed because of race condition between
creating platform mbean server and registering mbeans
  *Fix for class-cast issues during atomic-update 'removeregex' operations
  *Fix for multi-node race condition to create/remove nodeLost markers
  *Fix for too many cascading calls to remote servers, which can bring
down nodes
  *Fix for MOVEREPLICA ignoring replica type and always adding 'nrt' replicas
  *Fix: DistributedZkUpdateProcessor should propagate URP.finish()
lifecycle (regression since 8.1)


Please read CHANGES.txt for a full list of new features and changes:

  

Solr 8.3.0 also includes features, optimizations  and bugfixes in the
corresponding Apache Lucene release:

  


Note: The Apache Software Foundation uses an extensive mirroring network for
distributing releases. It is possible that the mirror you are using may not have
replicated the release yet. If that is the case, please try another mirror.
This also applies to Maven access.

[ANNOUNCE] Apache Solr 8.3.1 released

[Ishan Chattopadhyaya]

## 3 December 2019, Apache Solr™ 8.3.1 available

The Lucene PMC is pleased to announce the release of Apache Solr 8.3.1.

Solr is the popular, blazing fast, open source NoSQL search platform
from the Apache Lucene project. Its major features include powerful
full-text search, hit highlighting, faceted search, dynamic
clustering, database integration, rich document handling, and
geospatial search. Solr is highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features of many of the world's largest internet sites.

Solr 8.3.1 is available for immediate download at:

  

### Solr 8.3.1 Release Highlights:

  * JavaBinCodec has concurrent modification of CharArr resulting in
corrupt internode updates
  * findRequestType in AuditEvent is more robust
  * CoreContainer.auditloggerPlugin is volatile now
  * Velocity response writer's resource loading now possible only
through startup parameters


Please read CHANGES.txt for a full list of changes:

  

Solr 8.3.1 also includes  and bugfixes in the corresponding Apache
Lucene release:

  

Note: The Apache Software Foundation uses an extensive mirroring network for
distributing releases. It is possible that the mirror you are using may not have
replicated the release yet. If that is the case, please try another mirror.
This also applies to Maven access.

Re: [ANNOUNCE] Apache Solr 8.3.1 released

[Ishan Chattopadhyaya]

Thanks, I'll fix.

On Mon, Dec 9, 2019 at 4:15 PM Colvin Cowie  wrote:
>
> Oh, just looking at the way the announcement reads on
> http://lucene.apache.org/solr/news.html :
> Solr 8.3.1 Release Highlights:
>
>- JavaBinCodec has concurrent modification of CharArr resulting in
>corrupt internode updates
>
> That kind of sounds like the corrupt internode updates is something that
> has been *introduced* by the release rather than being fixed. Maybe that
> could just be changed to:
>
>- Fixed: JavaBinCodec has concurrent modification of CharArr resulting
>in corrupt internode updates
>
>
> Thanks
>
> On Fri, 6 Dec 2019 at 01:22, Paras Lehana 
> wrote:
>
> > Yup, now reflected. :)
> >
> > On Thu, 5 Dec, 2019, 19:43 Erick Erickson, 
> > wrote:
> >
> > > It’s there for me when I click on your link.
> > >
> > > > On Dec 5, 2019, at 1:08 AM, Paras Lehana 
> > > wrote:
> > > >
> > > > Hey Ishan,
> > > >
> > > > Cannot find 8.3.1 here: https://lucene.apache.org/solr/downloads.html
> > > (8.3.0
> > > > is listed here).
> > > >
> > > > Anyways, I'm downloading it from here:
> > > > https://archive.apache.org/dist/lucene/solr/8.3.1/
> > > >
> > > >
> > > >
> > > > On Wed, 4 Dec 2019 at 20:27, Rahul Goswami 
> > > wrote:
> > > >
> > > >> Thanks Ishan. I was just going through the list of fixes in 8.3.1
> > > >> (published in changes.txt) and couldn't see the below JIRA.
> > > >>
> > > >> SOLR-13971 <http://issues.apache.org/jira/browse/SOLR-13971>:
> > Velocity
> > > >> response writer's resource loading now possible only through startup
> > > >> parameters.
> > > >>
> > > >> Is it linked appropriately? Or is it some access rights issue for
> > > non-PMC
> > > >> members like me ?
> > > >>
> > > >> Thanks,
> > > >> Rahul
> > > >>
> > > >>
> > > >> On Wed, Dec 4, 2019 at 7:12 AM Noble Paul 
> > wrote:
> > > >>
> > > >>> Thanks ishan
> > > >>>
> > > >>> On Wed, Dec 4, 2019, 3:32 PM Ishan Chattopadhyaya <
> > > >>> ichattopadhy...@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>>> ## 3 December 2019, Apache Solr™ 8.3.1 available
> > > >>>>
> > > >>>> The Lucene PMC is pleased to announce the release of Apache Solr
> > > 8.3.1.
> > > >>>>
> > > >>>> Solr is the popular, blazing fast, open source NoSQL search platform
> > > >>>> from the Apache Lucene project. Its major features include powerful
> > > >>>> full-text search, hit highlighting, faceted search, dynamic
> > > >>>> clustering, database integration, rich document handling, and
> > > >>>> geospatial search. Solr is highly scalable, providing fault tolerant
> > > >>>> distributed search and indexing, and powers the search and
> > navigation
> > > >>>> features of many of the world's largest internet sites.
> > > >>>>
> > > >>>> Solr 8.3.1 is available for immediate download at:
> > > >>>>
> > > >>>>  <https://lucene.apache.org/solr/downloads.html>
> > > >>>>
> > > >>>> ### Solr 8.3.1 Release Highlights:
> > > >>>>
> > > >>>>  * JavaBinCodec has concurrent modification of CharArr resulting in
> > > >>>> corrupt internode updates
> > > >>>>  * findRequestType in AuditEvent is more robust
> > > >>>>  * CoreContainer.auditloggerPlugin is volatile now
> > > >>>>  * Velocity response writer's resource loading now possible only
> > > >>>> through startup parameters
> > > >>>>
> > > >>>>
> > > >>>> Please read CHANGES.txt for a full list of changes:
> > > >>>>
> > > >>>>  <https://lucene.apache.org/solr/8_3_1/changes/Changes.html>
> > > >>>>
> > > >>>> Solr 8.3.1 also includes  and bugfixes in the corresponding Apache
> > > >>>> Lucene release:
> > > >>>>
> > > >>>>  <https://lucene.apache.org/core/8_3_1/changes/Changes.html>
> > > >>>>
> > > >>>> Note: The Apache Software Foundation uses an extensive mirroring
> > > >> network
> > > >>>> for
> > > >>>> distributing releases. It is possible that the mirror you are using
> > > may
> > > >>>> not have
> > > >>>> replicated the release yet. If that is the case, please try another
> > > >>> mirror.
> > > >>>> This also applies to Maven access.
> > > >>>>
> > > >>>>
> > -
> > > >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > >>>> For additional commands, e-mail: dev-h...@lucene.apache.org
> > > >>>>
> > > >>>>
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > --
> > > > Regards,
> > > >
> > > > *Paras Lehana* [65871]
> > > > Development Engineer, Auto-Suggest,
> > > > IndiaMART Intermesh Ltd.
> > > >
> > > > 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
> > > > Noida, UP, IN - 201303
> > > >
> > > > Mob.: +91-9560911996
> > > > Work: 01203916600 | Extn:  *8173*
> > > >
> > > > --
> > > > *
> > > > *
> > > >
> > > > <https://www.facebook.com/IndiaMART/videos/578196442936091/>
> > >
> > >
> >
> > --
> > *
> > *
> >
> >  <https://www.facebook.com/IndiaMART/videos/578196442936091/>
> >

[ANNOUNCE] Apache Solr 8.4.1 released

[Ishan Chattopadhyaya]

## 13 January 2020, Apache Solr™ 8.4.1 available

The Lucene PMC is pleased to announce the release of Apache Solr 8.4.1.

Solr is the popular, blazing fast, open source NoSQL search platform
from the Apache Lucene project. Its major features include powerful
full-text search, hit highlighting, faceted search, dynamic
clustering, database integration, rich document handling, and
geospatial search. Solr is highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features of many of the world's largest internet sites.

Solr 8.4.1 is available for immediate download at:

  

### Solr 8.4.1 Release Highlights:

  * Fix for overseer serialization to support rolling upgrade (broken since 8.4)
  * Fix for SSL support with SOLR_SSL_NEED_CLIENT_AUTH (broken since 8.2)
  * Package manager to store public keys in a special "trusted"
location instead of in ZooKeeper

Please read CHANGES.txt for a full list of changes:

  

Solr 8.4.1 also includes  and bugfixes in the corresponding Apache
Lucene release:

  

Note: The Apache Software Foundation uses an extensive mirroring network for
distributing releases. It is possible that the mirror you are using may not have
replicated the release yet. If that is the case, please try another mirror.
This also applies to Maven access.

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

There is a Slack backed by official IRC support. Please see
https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
details on how to join it.

On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, 
wrote:

> Relevance Slack is open to anyone working on search & relevance - #solr is
> only one of the channels, there's lots more! Hope to see you there.
>
> Cheers
>
> Charlie
> https://opensourceconnections.com/slack
>
>
> On 16/01/2021 02:18, matthew sporleder wrote:
> > IRC has kind of died off,
> > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > I'm on https://opensourceconnections.com/slack after taking their solr
> > training class and assume it's mostly open to solr community.
> >
> > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> >  wrote:
> >> Hi all,
> >>
> >> I did some googling and didn't find anything, but is there a Slack
> >> workspace for Solr? I think this could be useful to expand interaction
> >> within the community of Solr users and connect people solving similar
> >> problems.
> >>
> >> I'd be happy to get this setup if it does not exist already.
> >>
> >> Justin
>
>
> --
> Charlie Hull - Managing Consultant at OpenSource Connections Limited
> 
> Founding member of The Search Network 
> and co-author of Searching the Enterprise
> 
> tel/fax: +44 (0)8700 118334
> mobile: +44 (0)7767 825828
>

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

We should update the official documentation for community page to include
this Slack org. If someone would like to volunteer to submit a patch
against lucene-site repo, would be great.

On Tue, 26 Jan, 2021, 4:02 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> There is a Slack backed by official IRC support. Please see
> https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
> details on how to join it.
>
> On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> ch...@opensourceconnections.com> wrote:
>
>> Relevance Slack is open to anyone working on search & relevance - #solr
>> is only one of the channels, there's lots more! Hope to see you there.
>>
>> Cheers
>>
>> Charlie
>> https://opensourceconnections.com/slack
>>
>>
>> On 16/01/2021 02:18, matthew sporleder wrote:
>> > IRC has kind of died off,
>> > https://lucene.apache.org/solr/community.html has a slack mentioned,
>> > I'm on https://opensourceconnections.com/slack after taking their solr
>> > training class and assume it's mostly open to solr community.
>> >
>> > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
>> >  wrote:
>> >> Hi all,
>> >>
>> >> I did some googling and didn't find anything, but is there a Slack
>> >> workspace for Solr? I think this could be useful to expand interaction
>> >> within the community of Solr users and connect people solving similar
>> >> problems.
>> >>
>> >> I'd be happy to get this setup if it does not exist already.
>> >>
>> >> Justin
>>
>>
>> --
>> Charlie Hull - Managing Consultant at OpenSource Connections Limited
>> 
>> Founding member of The Search Network <https://thesearchnetwork.com/>
>> and co-author of Searching the Enterprise
>> <https://opensourceconnections.com/about-us/books-resources/>
>> tel/fax: +44 (0)8700 118334
>> mobile: +44 (0)7767 825828
>>
>

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

Thanks ufuk. I'll take a look.

On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz, 
wrote:

> It’s asking for a searchscale.com email address?
>
> Sent from Mail for Windows 10
>
> From: Ishan Chattopadhyaya
> Sent: 26 January 2021 13:33
> To: solr-user
> Subject: Re: Solr Slack Workspace
>
> There is a Slack backed by official IRC support. Please see
> https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
> details on how to join it.
>
> On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> ch...@opensourceconnections.com>
> wrote:
>
> > Relevance Slack is open to anyone working on search & relevance - #solr
> is
> > only one of the channels, there's lots more! Hope to see you there.
> >
> > Cheers
> >
> > Charlie
> > https://opensourceconnections.com/slack
> >
> >
> > On 16/01/2021 02:18, matthew sporleder wrote:
> > > IRC has kind of died off,
> > > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > > I'm on https://opensourceconnections.com/slack after taking their solr
> > > training class and assume it's mostly open to solr community.
> > >
> > > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> > >  wrote:
> > >> Hi all,
> > >>
> > >> I did some googling and didn't find anything, but is there a Slack
> > >> workspace for Solr? I think this could be useful to expand interaction
> > >> within the community of Solr users and connect people solving similar
> > >> problems.
> > >>
> > >> I'd be happy to get this setup if it does not exist already.
> > >>
> > >> Justin
> >
> >
> > --
> > Charlie Hull - Managing Consultant at OpenSource Connections Limited
> > 
> > Founding member of The Search Network <https://thesearchnetwork.com/>
> > and co-author of Searching the Enterprise
> > <https://opensourceconnections.com/about-us/books-resources/>
> > tel/fax: +44 (0)8700 118334
> > mobile: +44 (0)7767 825828
> >
>
>

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

Hi all,
I've created an invite link for the Slack workspace:
https://s.apache.org/solr-slack.
Please test it out. I'll send a broader notification once this is tested
out to be working well.
Thanks and regards,
Ishan

On Thu, Jan 28, 2021 at 12:26 AM Justin Sweeney 
wrote:

> Thanks, I joined the Relevance Slack:
> https://opensourceconnections.com/slack, I definitely think a dedicated
> Solr workspace would also be good allowing for channels to get involved
> with development as well as user based questions.
>
> It does seem like slack has made it increasingly difficult to create open
> workspaces and not force someone to approve or only allow specific email
> domains. Has anyone tried to do that recently? I tried for an hour or so
> last weekend and it seemed to not be very straightforward anymore.
>
> On Tue, Jan 26, 2021 at 12:57 PM Houston Putman 
> wrote:
>
> > There is https://solr-dev.slack.com
> >
> > It's not really used, but it's there and we can open it up for people to
> > join and start using.
> >
> > On Tue, Jan 26, 2021 at 5:38 AM Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com> wrote:
> >
> > > Thanks ufuk. I'll take a look.
> > >
> > > On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz,  >
> > > wrote:
> > >
> > > > It’s asking for a searchscale.com email address?
> > > >
> > > > Sent from Mail for Windows 10
> > > >
> > > > From: Ishan Chattopadhyaya
> > > > Sent: 26 January 2021 13:33
> > > > To: solr-user
> > > > Subject: Re: Solr Slack Workspace
> > > >
> > > > There is a Slack backed by official IRC support. Please see
> > > > https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html
> > for
> > > > details on how to join it.
> > > >
> > > > On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> > > > ch...@opensourceconnections.com>
> > > > wrote:
> > > >
> > > > > Relevance Slack is open to anyone working on search & relevance -
> > #solr
> > > > is
> > > > > only one of the channels, there's lots more! Hope to see you there.
> > > > >
> > > > > Cheers
> > > > >
> > > > > Charlie
> > > > > https://opensourceconnections.com/slack
> > > > >
> > > > >
> > > > > On 16/01/2021 02:18, matthew sporleder wrote:
> > > > > > IRC has kind of died off,
> > > > > > https://lucene.apache.org/solr/community.html has a slack
> > mentioned,
> > > > > > I'm on https://opensourceconnections.com/slack after taking
> their
> > > solr
> > > > > > training class and assume it's mostly open to solr community.
> > > > > >
> > > > > > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> > > > > >  wrote:
> > > > > >> Hi all,
> > > > > >>
> > > > > >> I did some googling and didn't find anything, but is there a
> Slack
> > > > > >> workspace for Solr? I think this could be useful to expand
> > > interaction
> > > > > >> within the community of Solr users and connect people solving
> > > similar
> > > > > >> problems.
> > > > > >>
> > > > > >> I'd be happy to get this setup if it does not exist already.
> > > > > >>
> > > > > >> Justin
> > > > >
> > > > >
> > > > > --
> > > > > Charlie Hull - Managing Consultant at OpenSource Connections
> Limited
> > > > > 
> > > > > Founding member of The Search Network <
> https://thesearchnetwork.com/
> > >
> > > > > and co-author of Searching the Enterprise
> > > > > <https://opensourceconnections.com/about-us/books-resources/>
> > > > > tel/fax: +44 (0)8700 118334
> > > > > mobile: +44 (0)7767 825828
> > > > >
> > > >
> > > >
> > >
> >
>

Re: Excessive logging 8.8.0

[Ishan Chattopadhyaya]

I think we should release a 8.8.1 with that fixed.

On Fri, 5 Feb, 2021, 4:09 pm Markus Jelsma, 
wrote:

> Thanks!
>
> Op do 4 feb. 2021 om 20:04 schreef Chris Hostetter <
> hossman_luc...@fucit.org
> >:
>
> >
> > FWIW: that log message was added to branch_8x by 3c02c9197376 as part of
> > SOLR-15052 ... it's based on master commit 8505d4d416fd -- but that does
> > not add that same logging message ... so it definitely smells like a
> > mistake to me that 8x would add this INFO level log message that master
> > doesn't have.
> >
> > it's worth noting that 3c02c9197376 included many other "log.info(...)"
> > messages that had 'nocommit' comments to change them to debug later ...
> > making me more confident this is a mistake...
> >
> > https://issues.apache.org/jira/browse/SOLR-15136
> >
> >
> > : Date: Thu, 4 Feb 2021 12:45:16 +0100
> > : From: Markus Jelsma 
> > : Reply-To: solr-user@lucene.apache.org
> > : To: solr-user@lucene.apache.org
> > : Subject: Excessive logging 8.8.0
> > :
> > : Hello all,
> > :
> > : We upgraded some nodes to 8.8.0 and notice there is excessive logging
> on
> > : INFO when some traffic/indexing is going on:
> > :
> > : 2021-02-04 11:42:48.535 INFO  (qtp261748192-268) [c:data s:shard2
> > : r:core_node4 x:data_shard2_replica_t2] o.a.s.c.c.ZkStateReader already
> > : watching , added to s
> > : tateWatchers
> > :
> > : Is this to be expected?
> > :
> > : Thanks,
> > : Markus
> > :
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
>

Re: Solr Slack Workspace

[Ishan Chattopadhyaya]

No, it is not ASF slack. A separate slack org, just for Solr.

On Sat, 6 Feb, 2021, 6:35 am Anshum Gupta,  wrote:

> Hey Ishan,
>
> Thanks for doing this. Is this the ASF Slack space or something else?
>
>
> On Tue, Feb 2, 2021 at 2:04 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
> > Hi all,
> > I've created an invite link for the Slack workspace:
> > https://s.apache.org/solr-slack.
> > Please test it out. I'll send a broader notification once this is tested
> > out to be working well.
> > Thanks and regards,
> > Ishan
> >
> > On Thu, Jan 28, 2021 at 12:26 AM Justin Sweeney <
> > justin.sweene...@gmail.com>
> > wrote:
> >
> > > Thanks, I joined the Relevance Slack:
> > > https://opensourceconnections.com/slack, I definitely think a
> dedicated
> > > Solr workspace would also be good allowing for channels to get involved
> > > with development as well as user based questions.
> > >
> > > It does seem like slack has made it increasingly difficult to create
> open
> > > workspaces and not force someone to approve or only allow specific
> email
> > > domains. Has anyone tried to do that recently? I tried for an hour or
> so
> > > last weekend and it seemed to not be very straightforward anymore.
> > >
> > > On Tue, Jan 26, 2021 at 12:57 PM Houston Putman <
> houstonput...@gmail.com
> > >
> > > wrote:
> > >
> > > > There is https://solr-dev.slack.com
> > > >
> > > > It's not really used, but it's there and we can open it up for people
> > to
> > > > join and start using.
> > > >
> > > > On Tue, Jan 26, 2021 at 5:38 AM Ishan Chattopadhyaya <
> > > > ichattopadhy...@gmail.com> wrote:
> > > >
> > > > > Thanks ufuk. I'll take a look.
> > > > >
> > > > > On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz,
> >  > > >
> > > > > wrote:
> > > > >
> > > > > > It’s asking for a searchscale.com email address?
> > > > > >
> > > > > > Sent from Mail for Windows 10
> > > > > >
> > > > > > From: Ishan Chattopadhyaya
> > > > > > Sent: 26 January 2021 13:33
> > > > > > To: solr-user
> > > > > > Subject: Re: Solr Slack Workspace
> > > > > >
> > > > > > There is a Slack backed by official IRC support. Please see
> > > > > >
> > https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html
> > > > for
> > > > > > details on how to join it.
> > > > > >
> > > > > > On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> > > > > > ch...@opensourceconnections.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Relevance Slack is open to anyone working on search &
> relevance -
> > > > #solr
> > > > > > is
> > > > > > > only one of the channels, there's lots more! Hope to see you
> > there.
> > > > > > >
> > > > > > > Cheers
> > > > > > >
> > > > > > > Charlie
> > > > > > > https://opensourceconnections.com/slack
> > > > > > >
> > > > > > >
> > > > > > > On 16/01/2021 02:18, matthew sporleder wrote:
> > > > > > > > IRC has kind of died off,
> > > > > > > > https://lucene.apache.org/solr/community.html has a slack
> > > > mentioned,
> > > > > > > > I'm on https://opensourceconnections.com/slack after taking
> > > their
> > > > > solr
> > > > > > > > training class and assume it's mostly open to solr community.
> > > > > > > >
> > > > > > > > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> > > > > > > >  wrote:
> > > > > > > >> Hi all,
> > > > > > > >>
> > > > > > > >> I did some googling and didn't find anything, but is there a
> > > Slack
> > > > > > > >> workspace for Solr? I think this could be useful to expand
> > > > > interaction
> > > > > > > >> within the community of Solr users and connect people
> solving
> > > > > similar
> > > > > > > >> problems.
> > > > > > > >>
> > > > > > > >> I'd be happy to get this setup if it does not exist already.
> > > > > > > >>
> > > > > > > >> Justin
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Charlie Hull - Managing Consultant at OpenSource Connections
> > > Limited
> > > > > > > 
> > > > > > > Founding member of The Search Network <
> > > https://thesearchnetwork.com/
> > > > >
> > > > > > > and co-author of Searching the Enterprise
> > > > > > > <https://opensourceconnections.com/about-us/books-resources/>
> > > > > > > tel/fax: +44 (0)8700 118334
> > > > > > > mobile: +44 (0)7767 825828
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> Anshum Gupta
>

Re: Excessive logging 8.8.0

[Ishan Chattopadhyaya]

This should be fixed now in https://issues.apache.org/jira/browse/SOLR-15136.
Thanks Markus.

On Sat, Feb 6, 2021 at 7:33 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I think we should release a 8.8.1 with that fixed.
>
> On Fri, 5 Feb, 2021, 4:09 pm Markus Jelsma, 
> wrote:
>
>> Thanks!
>>
>> Op do 4 feb. 2021 om 20:04 schreef Chris Hostetter <
>> hossman_luc...@fucit.org
>> >:
>>
>> >
>> > FWIW: that log message was added to branch_8x by 3c02c9197376 as part of
>> > SOLR-15052 ... it's based on master commit 8505d4d416fd -- but that does
>> > not add that same logging message ... so it definitely smells like a
>> > mistake to me that 8x would add this INFO level log message that master
>> > doesn't have.
>> >
>> > it's worth noting that 3c02c9197376 included many other "log.info(...)"
>> > messages that had 'nocommit' comments to change them to debug later ...
>> > making me more confident this is a mistake...
>> >
>> > https://issues.apache.org/jira/browse/SOLR-15136
>> >
>> >
>> > : Date: Thu, 4 Feb 2021 12:45:16 +0100
>> > : From: Markus Jelsma 
>> > : Reply-To: solr-user@lucene.apache.org
>> > : To: solr-user@lucene.apache.org
>> > : Subject: Excessive logging 8.8.0
>> > :
>> > : Hello all,
>> > :
>> > : We upgraded some nodes to 8.8.0 and notice there is excessive logging
>> on
>> > : INFO when some traffic/indexing is going on:
>> > :
>> > : 2021-02-04 11:42:48.535 INFO  (qtp261748192-268) [c:data s:shard2
>> > : r:core_node4 x:data_shard2_replica_t2] o.a.s.c.c.ZkStateReader already
>> > : watching , added to s
>> > : tateWatchers
>> > :
>> > : Is this to be expected?
>> > :
>> > : Thanks,
>> > : Markus
>> > :
>> >
>> > -Hoss
>> > http://www.lucidworks.com/
>> >
>>
>

Re: Asymmetric Key Size not sufficient

[Ishan Chattopadhyaya]

Recent versions of Solr use 2048.
https://github.com/apache/lucene-solr/blob/branch_8_6/solr/core/src/java/org/apache/solr/util/CryptoKeys.java#L332

Thanks for your report.

On Fri, Feb 12, 2021 at 3:44 PM Mahir Kabir  wrote:

> Hello,
>
> I am a Ph.D. student at Virginia Tech, USA. While working on a security
> project-related work, we came across the following vulnerability in the
> source code -
>
> In file
>
> https://github.com/apache/lucene-solr/blob/branch_6_6/solr/core/src/java/org/apache/solr/util/CryptoKeys.java
> <
> https://github.com/apache/ranger/blob/71e1dd40366c8eb8e9c498b0b5158d85d603af02/kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java
> >
> (at
> Line 300) Key Size was set as 1024.
>
> *Security Impact*:
>
> < 2048 key size for RSA algorithm makes the system vulnerable to
> brute-force attack
>
> *Useful resource*:
> https://rules.sonarsource.com/java/type/Vulnerability/RSPEC-4426
> https://rules.sonarsource.com/java/type/Vulnerability/RSPEC-4426
>
> *Solution we suggest*:
>
> For RSA algorithm, the key size should be >= 2048
>
> *Please share with us your opinions/comments if there is any*:
>
> Is the bug report helpful?
>
> Please let us know what you think about the issue. Any feedback will be
> appreciated.
>
> Thank you,
> Md Mahir Asef Kabir
> Ph.D. Student
> Department of CS
> Virginia Tech
>

Re: SOLR upgrade

[Ishan Chattopadhyaya]

Just avoid 8.8.0 for the moment, until 8.8.1 is released. 8.7.x should be
fine.

On Fri, Feb 12, 2021 at 10:28 PM Alessandro Benedetti 
wrote:

> Hi,
> following up on Charlie's detailed response I would recommend carefully
> assess the code you are using to interact with Apache Solr (on top of the
> Solr changes themselves).
> Assuming you are using some sort of client, it's extremely important to
> fully understand both the syntax and semantic of each call.
> I saw a lot of "compiling ok" search-api migrations that were ok
> syntactically but doing a disaster from the semantic perspective (missing
> important parameters ect).
>
> In case you have plugins to maintain this would be even more complicated
> than just make them compile.
>
> Regards
> --
> Alessandro Benedetti
> Apache Lucene/Solr Committer
> Director, R&D Software Engineer, Search Consultant
> www.sease.io
>
>
> On Tue, 9 Feb 2021 at 11:01, Charlie Hull  >
> wrote:
>
> > Hi Lulu,
> >
> > I'm afraid you're going to have to recognise that Solr 5.2.1 is very
> > out-of-date and the changes between this version and the current 8.x
> > releases are significant. A direct jump is I think the only sensible
> > option.
> >
> > Although you could take the current configuration and attempt to upgrade
> > it to work with 8.x, I recommend that you should take the chance to look
> > at your whole infrastructure (from data ingestion through to query
> > construction) and consider what needs upgrading/redesigning for both
> > performance and future-proofing. You shouldn't just attempt a
> > lift-and-shift of the current setup - some things just won't work and
> > some may lock you into future issues. If you're running at large scale
> > (I've talked to some people at the BL before and I know you have some
> > huge indexes there!) then a redesign may be necessary for scalability
> > reasons (cost and feasibility). You should also consider your skills
> > base and how the team can stay up to date with Solr changes and modern
> > search practice.
> >
> > Hope this helps - this is a common situation which I've seen many times
> > before, you're certainly not the oldest version of Solr running I've
> > seen recently either!
> >
> > best
> >
> > Charlie
> >
> > On 09/02/2021 01:14, Paul, Lulu wrote:
> > > Hi SOLR team,
> > >
> > > Please may I ask for advice regarding upgrading the SOLR version (our
> > project currently running on solr-5.2.1) to the latest version?
> > > What are the steps, breaking changes and potential issues ? Could this
> > be done as an incremental version upgrade or a direct jump to the newest
> > version?
> > >
> > > Much appreciate the advice, Thank you!
> > >
> > > Best Wishes
> > > Lulu
> > >
> > >
> > >
> >
> **
> > > Experience the British Library online at www.bl.uk
> > > The British Library's latest Annual Report and Accounts :
> > www.bl.uk/aboutus/annrep/index.html<
> > http://www.bl.uk/aboutus/annrep/index.html>
> > > Help the British Library conserve the world's knowledge. Adopt a Book.
> > www.bl.uk/adoptabook
> > > The Library's St Pancras site is WiFi - enabled
> > >
> >
> *
> > > The information contained in this e-mail is confidential and may be
> > legally privileged. It is intended for the addressee(s) only. If you are
> > not the intended recipient, please delete this e-mail and notify the
> > postmas...@bl.uk : The contents of this e-mail
> > must not be disclosed or copied without the sender's consent.
> > > The statements and opinions expressed in this message are those of the
> > author and do not necessarily reflect those of the British Library. The
> > British Library does not take any responsibility for the views of the
> > author.
> > >
> >
> *
> > > Think before you print
> > >
> >
> > --
> > Charlie Hull - Managing Consultant at OpenSource Connections Limited
> > 
> > Founding member of The Search Network 
> > and co-author of Searching the Enterprise
> > 
> > tel/fax: +44 (0)8700 118334
> > mobile: +44 (0)7767 825828
> >
>

Re: [ANNOUNCE] Apache Solr 8.8.0 released

[Ishan Chattopadhyaya]

Hi all,
This release contains a critical bug, that should be fixed in 8.8.1
shortly. Please avoid upgrading to this release for the moment.
https://twitter.com/ichattopadhyaya/status/1360163382171586562
Apologies for the inconvenience.
Thanks,
Ishan

On Mon, Feb 1, 2021 at 6:01 PM Noble Paul  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> Solr is the popular, blazing fast, open source NoSQL search platform from
> the Apache Lucene project. Its major features include powerful full-text
> search, hit highlighting, faceted search and analytics, rich document
> parsing, geospatial search, extensive REST APIs as well as parallel SQL.
> Solr is enterprise grade, secure and highly scalable, providing fault
> tolerant distributed search and indexing, and powers the search and
> navigation features of many of the world's largest internet sites.
>
> The release is available for immediate download at:
>
>  https://lucene.apache.org/solr/downloads.html
>
> Please read CHANGES.txt for a detailed list of changes:
>
>  https://lucene.apache.org/solr/8_8_0/changes/Changes.html Solr 8.8.0
>
> Release Highlights:
>
> Reducing overseer bottlenecks using per-replica states. More stability and
> lesser load on large cluster that use this feature.
>
> Better restart and collection creation performance.
>
> Interleaving support in Learning To Rank
>
> A summary of important changes is published in the Solr Reference Guide at:
>
>  https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html.
>
> For the most exhaustive list, see the full release notes at
> https://lucene.apache.org/solr/8_8_0/changes/Changes.html
>
> or
>
> by viewing the CHANGES.txt file accompanying the distribution. Solr's
> release notes usually don't include Lucene layer changes. Lucene's release
> notes are at
>
> https://lucene.apache.org/core/8_8_0/changes/Changes.html
>
> - -
> Noble Paul
> -BEGIN PGP SIGNATURE-
> Version: FlowCrypt Email Encryption 8.0.0
> Comment: Seamlessly send and receive encrypted email
>
> wsFzBAEBCgAGBQJgF/RnACEJEMOP9ew/z9s+FiEEz85fu5IMPHRc7uCEw4/1
> 7D/P2z6fzRAAm4AKbeIGWfPK+0nsrZCAPaDucGZYVL0lPQr3eF4jnmhi60dF
> Sv9rD5Mq5ZSTTuJlpwoaxowxVp4M1tV1vmCdfBRkgoUD3dwS/snryr/AK69R
> zdjjV/BABtcMNA7cMYIrkolGl37g4kI1alLfU36Uf/3M0NfUcw0keW1XuMOr
> uV7AzXhZGw4eL4LJt7I7gXJs1kgE6/sPSmoKBVckKisrruiUSYmH9r/EhXXU
> YB8cxd5tenMrchbjcOquC9X2JJjB++/LyJw3mFNIO5W3UpjqwtI8IGDo1Sxl
> fM32FuAWVVDZsiBKXuRzsIO/iEPfgZFfTcoSJkD0Rt/Q6gJPZIuBmiUFaYfs
> 9fzufNDuXdPKFEndSHfwdPMJwvk3XA5+xYzhkcQH+3FKOPmYXkvLolOC3j+r
> ZtbgI421jDIahpVPbFtgUPB2dM3mw34B73wP5MIOHHxz22tVKe6PBOeihccK
> mOr0r1tZHR+11aijYf+Nlhv3hpbpRoDbQ7pRkRyu53Od47p6itZAi60TFFIJ
> bDw26wZRNRrEuYhriJUeM7ahvJNlcE6VaO0szUDL5g/x2Oa9jKMHPpsUF9pS
> 9HbJWcnflxq0iU+sfdv7Eoxzv6zkXMTUsbpT2XjKcZZN5jd2rWV3JfiU6FiZ
> jpqJBHzwGan9qKKswNKyDKhoa2jPdSYIbqQ=
> =NbSI
> -END PGP SIGNATURE-
>