Why do Solr nodes go into Recovery status

[suresh pendap]

Hi,

Why and in what scenarios do Solr nodes go into recovery status?

Given that Solr is a CP system it means that the writes for a Document
index are acknowledged only after they are propagated and acknowledged by
all the replicas of the Shard.

This means that technically the replica nodes should not fall behind and do
not have to go into recovery mode.

Is my above understanding correct?

Can a below scenario happen?

1. Assume that we have 3 replicas for Shard shard1 with the names
shard1_replica1, shard1_replica2 and shard1_replica3.

2. Due to some reason, network issue or something else, the shard1_replica2
is not reachable by the other replicas and it is marked as Down by the
other replicas (shard1_replica1 and shard1_replica3 in this case)

3. The network issue is restored and the shard1_replica2 is reachable
again. It goes into a recovery mode and tries to recover all the documents
from the leader of shard1.

Is this the only scenario in which a node can go into recovery status?

In other words, does the node has to go into a Down status before getting
back into a recovery status?


Regards

Re: Why do Solr nodes go into Recovery status

[suresh pendap]

GW,
Did you mean a separate transaction log on Solr or on Zookeeper?

-suresh

On Tue, Jun 6, 2017 at 5:23 AM, GW  wrote:

> I've heard of systems tanking like this on Windows during OS updates.
> Because of this, I run all my updates in attendance even though I'm Linux.
> My Nodes run as VM's, I shut down Solr gracefully, snap shot a backup of
> the VM, update and run. If things go screwy I can always roll back. To me
> it sounds like a lack of resources or a kink in your networking, assuming
> your set up is correct. Watch for home made network cables. I've seen soft
> crimp connectors put on solid wire which can wreck a switch port forever.
> Do you have a separate transaction log device on each Zookeeper? I made
> this mistake in the beginning and had similar problems under load.
>
>
> GW
>
> On 5 June 2017 at 22:32, Erick Erickson  wrote:
>
> > bq: This means that technically the replica nodes should not fall behind
> > and do
> > not have to go into recovery mode
> >
> > Well, true if nothing weird happens. By "weird" I mean anything that
> > interferes with the leader getting anything other than a success code
> > back from a follower it sends  document to.
> >
> > bq: Is this the only scenario in which a node can go into recovery
> status?
> >
> > No, there are others. One for-instance: Leader sends a doc to the
> > follower and the request times out (huge  GC pauses, the doc takes too
> > long to index for whatever reason etc). The leader then sends a
> > message to the follower to go directly into the recovery state since
> > the leader has no way of knowing whether the follower successfully
> > wrote the document to it's transaction log. You'll see messages about
> > "leader initiated recovery" in the follower's solr log in this case.
> >
> > two bits of pedantry:
> >
> > bq:  Down by the other replicas
> >
> > Almost. we're talking indexing here and IIUC only the leader can send
> > another node into recovery as all updates go through the leader.
> >
> > If I'm going to be nit-picky, Zookeeper can _also_ cause a node to be
> > marked as down if it's periodic ping of the node fails to return.
> > Actually I think this is done through another Solr node that ZK
> > notifies
> >
> > bq: It goes into a recovery mode and tries to recover all the
> > documents from the leader of shard1.
> >
> > Also nit-picky. But if the follower isn't "too far" behind it can be
> > brought back into sync from via "peer sync" where it gets the missed
> > docs sent to it from the tlog of a healthy replica. "Too far" is 100
> > docs by default, but can be set in solrconfig.xml if necessary. If
> > that limit is exceeded, then indeed the entire index is copied from
> > the leader.
> >
> > Best,
> > Erick
> >
> >
> >
> > On Mon, Jun 5, 2017 at 5:18 PM, suresh pendap 
> > wrote:
> > > Hi,
> > >
> > > Why and in what scenarios do Solr nodes go into recovery status?
> > >
> > > Given that Solr is a CP system it means that the writes for a Document
> > > index are acknowledged only after they are propagated and acknowledged
> by
> > > all the replicas of the Shard.
> > >
> > > This means that technically the replica nodes should not fall behind
> and
> > do
> > > not have to go into recovery mode.
> > >
> > > Is my above understanding correct?
> > >
> > > Can a below scenario happen?
> > >
> > > 1. Assume that we have 3 replicas for Shard shard1 with the names
> > > shard1_replica1, shard1_replica2 and shard1_replica3.
> > >
> > > 2. Due to some reason, network issue or something else, the
> > shard1_replica2
> > > is not reachable by the other replicas and it is marked as Down by the
> > > other replicas (shard1_replica1 and shard1_replica3 in this case)
> > >
> > > 3. The network issue is restored and the shard1_replica2 is reachable
> > > again. It goes into a recovery mode and tries to recover all the
> > documents
> > > from the leader of shard1.
> > >
> > > Is this the only scenario in which a node can go into recovery status?
> > >
> > > In other words, does the node has to go into a Down status before
> getting
> > > back into a recovery status?
> > >
> > >
> > > Regards
> >
>

Re: Why do Solr nodes go into Recovery status

[suresh pendap]

Thanks Erick for the reply.

When the leader asks the follower to go into recovery status,  does it stop
sending future updates to this replica until it becomes fully in sync with
the leader?

Regards
Suresh

On Mon, Jun 5, 2017 at 8:32 PM, Erick Erickson 
wrote:

> bq: This means that technically the replica nodes should not fall behind
> and do
> not have to go into recovery mode
>
> Well, true if nothing weird happens. By "weird" I mean anything that
> interferes with the leader getting anything other than a success code
> back from a follower it sends  document to.
>
> bq: Is this the only scenario in which a node can go into recovery status?
>
> No, there are others. One for-instance: Leader sends a doc to the
> follower and the request times out (huge  GC pauses, the doc takes too
> long to index for whatever reason etc). The leader then sends a
> message to the follower to go directly into the recovery state since
> the leader has no way of knowing whether the follower successfully
> wrote the document to it's transaction log. You'll see messages about
> "leader initiated recovery" in the follower's solr log in this case.
>
> two bits of pedantry:
>
> bq:  Down by the other replicas
>
> Almost. we're talking indexing here and IIUC only the leader can send
> another node into recovery as all updates go through the leader.
>
> If I'm going to be nit-picky, Zookeeper can _also_ cause a node to be
> marked as down if it's periodic ping of the node fails to return.
> Actually I think this is done through another Solr node that ZK
> notifies
>
> bq: It goes into a recovery mode and tries to recover all the
> documents from the leader of shard1.
>
> Also nit-picky. But if the follower isn't "too far" behind it can be
> brought back into sync from via "peer sync" where it gets the missed
> docs sent to it from the tlog of a healthy replica. "Too far" is 100
> docs by default, but can be set in solrconfig.xml if necessary. If
> that limit is exceeded, then indeed the entire index is copied from
> the leader.
>
> Best,
> Erick
>
>
>
> On Mon, Jun 5, 2017 at 5:18 PM, suresh pendap 
> wrote:
> > Hi,
> >
> > Why and in what scenarios do Solr nodes go into recovery status?
> >
> > Given that Solr is a CP system it means that the writes for a Document
> > index are acknowledged only after they are propagated and acknowledged by
> > all the replicas of the Shard.
> >
> > This means that technically the replica nodes should not fall behind and
> do
> > not have to go into recovery mode.
> >
> > Is my above understanding correct?
> >
> > Can a below scenario happen?
> >
> > 1. Assume that we have 3 replicas for Shard shard1 with the names
> > shard1_replica1, shard1_replica2 and shard1_replica3.
> >
> > 2. Due to some reason, network issue or something else, the
> shard1_replica2
> > is not reachable by the other replicas and it is marked as Down by the
> > other replicas (shard1_replica1 and shard1_replica3 in this case)
> >
> > 3. The network issue is restored and the shard1_replica2 is reachable
> > again. It goes into a recovery mode and tries to recover all the
> documents
> > from the leader of shard1.
> >
> > Is this the only scenario in which a node can go into recovery status?
> >
> > In other words, does the node has to go into a Down status before getting
> > back into a recovery status?
> >
> >
> > Regards
>

default values for numRecordsToKeep and maxNumLogsToKeep

[suresh pendap]

Hi,
After looking at the source code I see that the default values for
numRecordsToKeep is 100 and maxNumLogsToKeep is 10.

So it seems by default the replica can only have 1000 document updates lag
before the replica goes for a Full recovery from the leader.

I would like to know the rationale for keeping so low values for these
configurations.

If the idea for these configuration params is to avoid full recovery for
down replicas then shouldn't the default values for these config params be
high?

I understand that higher values would mean more disk space consumed by the
update logs, but the current default values seem to be very low.

Is my understanding of these configuration params correct?

Thanks
Suresh

regarding cursorMark feature for deep pagination

[suresh pendap]

Hi,

This question is more about the Implementation detail of the cursorMark
feature.

I was reading about using the cursorMark feature for deep pagination in
Solr mentioned in this blog http://yonik.com/solr/paging-and-deep-paging/

It is not clear to me as to how it is more efficient as compared to the
regular pagination.

The blog says that there is no state maintained on the server side.

If there is no state maintained then where does it get its efficiency from?

Assuming that it does maintain the state on the server side, does the next
page request has to go the same aggregator node which had served the first
page?


Thanks
Suresh

Re: regarding cursorMark feature for deep pagination

[suresh pendap]

Eric,
Thanks!! for the link.

-suresh

On Wed, Jul 19, 2017 at 8:11 AM, Erick Erickson 
wrote:

> Chris Hostetter has a writeup here that has a good explanation:
>
> https://lucidworks.com/2013/12/12/coming-soon-to-solr-
> efficient-cursor-based-iteration-of-large-result-sets/
>
> Best,
> Erick
>
> On Tue, Jul 18, 2017 at 10:00 PM, suresh pendap 
> wrote:
> > Hi,
> >
> > This question is more about the Implementation detail of the cursorMark
> > feature.
> >
> > I was reading about using the cursorMark feature for deep pagination in
> > Solr mentioned in this blog http://yonik.com/solr/paging-
> and-deep-paging/
> >
> > It is not clear to me as to how it is more efficient as compared to the
> > regular pagination.
> >
> > The blog says that there is no state maintained on the server side.
> >
> > If there is no state maintained then where does it get its efficiency
> from?
> >
> > Assuming that it does maintain the state on the server side, does the
> next
> > page request has to go the same aggregator node which had served the
> first
> > page?
> >
> >
> > Thanks
> > Suresh
>

precedence for configurations in solrconfig.xml file

[suresh pendap]

Hi,
If I have a configoverlay.json file with the below content

{"props":{"updateHandler":{"autoCommit":{
"maxTime":5,
"maxDocs":1


and I also have a JVM properties set on the Solr JVM instance as


-Dsolr.autoCommit.maxtime=2 -Dsolr.autoCommit.maxDocs=10



I would like to know the order of precedence in which the

configurations are applied.


Regards

Suresh

identifying source of queries

[suresh pendap]

Hi,
We have found that application teams often fire ad-hoc queries, some of
these are very expensive queries and can bring the solr cluster down.  Some
times they just build custom scripts which does some offline analytics by
firing expensive queries, the solr cluster was originally not sized for
executing such queries.

When an issue happens we identify from the solr logs the query which is
taking long time. But some times we do not even know who is firing these
queries and hence it takes a while to stop them.

We would like be able to identify the source of the solr queries.

Is there a way to tag the solr queries?
Can we associate some tags or query identifier with the query?
These tags should be made mandatory without which the solr query should
fail?

We would like to build a custom component which logs the query, the query
identifier (the tag which user provides) and the IP address of the client
machine which fired this query.


Thanks
Suresh

Re: identifying source of queries

[suresh pendap]

Thanks Shalin for the reply.
Do I need to also update the query parsers in order to handle the new query
param?
I can build a custom component but dabbling with query parsers would be way
too much for me to handle.

Thanks
Suresh




On Tue, Aug 8, 2017 at 9:49 PM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> There is no in-built way but if you are willing to create a custom
> query component then it should be easy to mandate that every query
> must have a tag parameter by throwing an exception otherwise. Any
> query param you pass to a distributed query request should be
> propagated to all query nodes as well.
>
> On Wed, Aug 9, 2017 at 8:58 AM, suresh pendap 
> wrote:
> > Hi,
> > We have found that application teams often fire ad-hoc queries, some of
> > these are very expensive queries and can bring the solr cluster down.
> Some
> > times they just build custom scripts which does some offline analytics by
> > firing expensive queries, the solr cluster was originally not sized for
> > executing such queries.
> >
> > When an issue happens we identify from the solr logs the query which is
> > taking long time. But some times we do not even know who is firing these
> > queries and hence it takes a while to stop them.
> >
> > We would like be able to identify the source of the solr queries.
> >
> > Is there a way to tag the solr queries?
> > Can we associate some tags or query identifier with the query?
> > These tags should be made mandatory without which the solr query should
> > fail?
> >
> > We would like to build a custom component which logs the query, the query
> > identifier (the tag which user provides) and the IP address of the client
> > machine which fired this query.
> >
> >
> > Thanks
> > Suresh
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

query with wild card with AND taking lot of time

[suresh pendap]

Hello everybody,

We are seeing that the below query is running very slow and taking almost 4
seconds to finish


[] webapp=/solr path=/select
params={df=_text_&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=modified_dtm+desc&shard.url=http://:8983/solr/flat_product_index_shard7_replica1/%7Chttp://:8983/solr/flat_product_index_shard7_replica2/%7Chttp://:8983/solr/flat_product_index_shard7_replica0/&rows=11&version=2&q=product_identifier_type:DOTCOM_OFFER+AND+abstract_or_primary_product_id:*+AND+(gtin:)+AND+-product_class_type:BUNDLE+AND+-hasProduct:N&NOW=1504196301534&isShard=true&timeAllowed=25000&wt=javabin}
hits=0 status=0 QTime=3663


It seems like the abstract_or_primary_product_id:* clause is contributing
to the overall response time. It seems that the
abstract_or_primary_product_id:* . clause is not adding any value in the
query criteria and can be safely removed.  Is my understanding correct?

I would like to know if the order of the clauses in the AND query would
affect the response time of the query?

For e.g . f1: 3 AND f2:10 AND f3:* vs . f3:* AND f1:3 AND f2:10

Doesn't Lucene/Solr pick up the optimal query execution plan?

Is there anyway to look at the query execution plan generated by Lucene?

Regards
Suresh

Re: query with wild card with AND taking lot of time

[suresh pendap]

Thanks Lincoln for your suggestions. It was very helpful.

I am still curious as to why is the original query taking long time.  It is
something that Lucene should have ideally optimized.
Is there any way to see the execution plan used by Lucene?

Thanks
Suresh


On Thu, Aug 31, 2017 at 11:11 AM, Josh Lincoln 
wrote:

> As I understand it, using a different fq for each clause makes the
> resultant caches more likely to be used in future requests.
>
> For the query
> fq=first:bob AND last:smith
> a subsequent query for
> fq=first:tim AND last:smith
> won't be able to use the fq cache from the first query.
>
> However, if the first query was
> fq=first:bob
> fq=last:smith
> and subsequently
> fq=first:tim
> fq=last:smith
> then the second query will at least benefit from the last:smith cache
>
> Because fq clauses are always ANDed, this does not work for ORed clauses.
>
> I suppose if some conditions are frequently used together it may be better
> to put them in the same fq so there's only one cache. E.g. if an ecommerce
> site reqularly queried for featured:Y AND instock:Y
>
> On Thu, Aug 31, 2017 at 1:48 PM David Hastings <
> hastings.recurs...@gmail.com>
> wrote:
>
> > >
> > > 2) Because all your clauses are more like filters and are ANDed
> together,
> > > you'll likely get better performance by putting them _each_ in an fq
> > > E.g.
> > > fq=product_identifier_type:DOTCOM_OFFER
> > > fq=abstract_or_primary_product_id:[* TO *]
> >
> >
> > why is this the case?  is it just better to have no logic operators in
> the
> > filter queries?
> >
> >
> >
> > On Thu, Aug 31, 2017 at 1:47 PM, Josh Lincoln 
> > wrote:
> >
> > > Suresh,
> > > Two things I noticed.
> > > 1) If your intent is to only match records where there's something,
> > > anything, in abstract_or_primary_product_id, you should use
> fieldname:[*
> > > TO
> > > *]  but that will exclude records where that field is empty/missing. If
> > you
> > > want to match records even if that field is empty/missing, then you
> > should
> > > remove that clause entirely
> > > 2) Because all your clauses are more like filters and are ANDed
> together,
> > > you'll likely get better performance by putting them _each_ in an fq
> > > E.g.
> > > fq=product_identifier_type:DOTCOM_OFFER
> > > fq=abstract_or_primary_product_id:[* TO *]
> > > fq=gtin:
> > > fq=product_class_type:BUNDLE
> > > fq=hasProduct:N
> > >
> > >
> > > On Thu, Aug 31, 2017 at 1:35 PM suresh pendap  >
> > > wrote:
> > >
> > > > Hello everybody,
> > > >
> > > > We are seeing that the below query is running very slow and taking
> > > almost 4
> > > > seconds to finish
> > > >
> > > >
> > > > [] webapp=/solr path=/select
> > > >
> > > > params={df=_text_&distrib=false&fl=id&shards.purpose=4&
> > > start=0&fsv=true&sort=modified_dtm+desc&shard.url=http://
> > > > :8983/solr/flat_product_index_shard7_replica1/
> > > %7Chttp://:8983/solr/flat_product_index_shard7_
> > > replica2/%7Chttp://:8983/solr/flat_product_index_
> > >
> > shard7_replica0/&rows=11&version=2&q=product_
> identifier_type:DOTCOM_OFFER+
> > > AND+abstract_or_primary_product_id:*+AND+(gtin:<
> > > numericValue>)+AND+-product_class_type:BUNDLE+AND+-hasProduct:N&NOW=
> > > 1504196301534&isShard=true&timeAllowed=25000&wt=javabin}
> > > > hits=0 status=0 QTime=3663
> > > >
> > > >
> > > > It seems like the abstract_or_primary_product_id:* clause is
> > > contributing
> > > > to the overall response time. It seems that the
> > > > abstract_or_primary_product_id:* . clause is not adding any value in
> > the
> > > > query criteria and can be safely removed.  Is my understanding
> correct?
> > > >
> > > > I would like to know if the order of the clauses in the AND query
> would
> > > > affect the response time of the query?
> > > >
> > > > For e.g . f1: 3 AND f2:10 AND f3:* vs . f3:* AND f1:3 AND f2:10
> > > >
> > > > Doesn't Lucene/Solr pick up the optimal query execution plan?
> > > >
> > > > Is there anyway to look at the query execution plan generated by
> > Lucene?
> > > >
> > > > Regards
> > > > Suresh
> > > >
> > >
> >
>

Solr merge related metrics not showing up in jconsole

[suresh pendap]

Hello,

I am using Solr version 6.6 and I am following the document to get the
Segment merge related metrics
https://lucene.apache.org/solr/guide/6_6/metrics-reporting.html#index-merge-metrics

I added the configuration to expose the merge related metrics to my
solrconfig.xml file as below



  524288
  true
  true

...
  

merge metrics not showing up in Jconsole

[suresh pendap]

Hi,
I am using Solr version 6.6.0 and am following the document
https://lucene.apache.org/solr/guide/6_6/metrics-reporting.html#index-merge-metrics
to enable Index merge metrics.

I have added the below config to my solrconfig.xml file


  

  524288
  true
  true

...
  



Then I restarted the solr servers.  I do see the INDEX merge metrics when I
use the REST API
http://localhost:8983/solr/admin/metrics?wt=json&&group=core

However I do not see the INDEX category and the merge metrics in the
Jconsole window.

Is this a known issue?

Do I need to configure something else to be able to expose the Index Merge
metrics as JMX mbeans?


Thanks
Suresh

Re: merge metrics not showing up in Jconsole

[suresh pendap]

I see only these metrics in my Jconsole window

[image: Inline image 1]

On Sun, Dec 3, 2017 at 4:19 PM, suresh pendap 
wrote:

> Hi,
> I am using Solr version 6.6.0 and am following the document
> https://lucene.apache.org/solr/guide/6_6/metrics-reporting.html#index-
> merge-metrics to enable Index merge metrics.
>
> I have added the below config to my solrconfig.xml file
>
>
>   
> 
>   524288
>   true
>   true
> 
> ...
>   
>
>
>
> Then I restarted the solr servers.  I do see the INDEX merge metrics when
> I use the REST API
> http://localhost:8983/solr/admin/metrics?wt=json&&group=core
>
> However I do not see the INDEX category and the merge metrics in the
> Jconsole window.
>
> Is this a known issue?
>
> Do I need to configure something else to be able to expose the Index Merge
> metrics as JMX mbeans?
>
>
> Thanks
> Suresh
>
>

Re: merge metrics not showing up in Jconsole

[suresh pendap]

Hi,
I wanted to check if it is a known issue that the merge metrics are not
exposed as JMX beans.
Any one else in the community ran into this issue?

Thanks
Suresh


On Sun, Dec 3, 2017 at 4:24 PM, suresh pendap 
wrote:

> I see only these metrics in my Jconsole window
>
> [image: Inline image 1]
>
> On Sun, Dec 3, 2017 at 4:19 PM, suresh pendap 
> wrote:
>
>> Hi,
>> I am using Solr version 6.6.0 and am following the document
>> https://lucene.apache.org/solr/guide/6_6/metrics-
>> reporting.html#index-merge-metrics to enable Index merge metrics.
>>
>> I have added the below config to my solrconfig.xml file
>>
>>
>>   
>> 
>>   524288
>>   true
>>   true
>> 
>> ...
>>   
>>
>>
>>
>> Then I restarted the solr servers.  I do see the INDEX merge metrics when
>> I use the REST API
>> http://localhost:8983/solr/admin/metrics?wt=json&&group=core
>>
>> However I do not see the INDEX category and the merge metrics in the
>> Jconsole window.
>>
>> Is this a known issue?
>>
>> Do I need to configure something else to be able to expose the Index
>> Merge metrics as JMX mbeans?
>>
>>
>> Thanks
>> Suresh
>>
>>
>

Re: Scaling issue with Solr

[Suresh Pendap]

What is the downside of configuring ramBufferSizeMB to be equal to 5GB ?
Is it only that the window of time for flush is larger, so recovery time will 
be higher in case of a crash?

Thanks
Suresh

On 12/27/17, 1:34 PM, "Erick Erickson"  wrote:

You are probably hitting more and more background merging which will
slow things down. Your system looks to be severely undersized for this
scale.

One thing you can try (and I emphasize I haven't prototyped this) is
to increase your RamBufferSizeMB solrcofnig.xml setting significantly.
By default, Solr won't merge segments to greater than 5G, so
theoretically you could just set your ramBufferSizeMB to that figure
and avoid merging all together. Or you could try configuring the
NoMergePolicy in solrconfig.xml (but beware that you're going to
create a lot of segments unless you set the rambuffersize higher).

How this will affect your indexing throughput I frankly have no data.
You can see that with numbers like this, though, a 4G heap is much too
small.

Best,
Erick

On Wed, Dec 27, 2017 at 2:18 AM, Prasad Tendulkar
 wrote:
> Hello All,
>
> We have been building a Solr based solution to hold a large amount of 
data (approx 4 TB/day or > 24 Billion documents per day). We are developing a 
prototype on a small scale just to evaluate Solr performance gradually. Here is 
our setup configuration.
>
> Solr cloud:
> node1: 16 GB RAM, 8 Core CPU, 1TB disk
> node2: 16 GB RAM, 8 Core CPU, 1TB disk
>
> Zookeeper is also installed on above 2 machines in cluster mode.
> Solr commit intervals: Soft commit 3 minutes, Hard commit 15 seconds
> Schema: Basic configuration. 5 fields indexed (out of one is 
text_general), 6 fields stored.
> Collection: 12 shards (6 per node)
> Heap memory: 4 GB per node
> Disk cache: 12 GB per node
> Document is a syslog message.
>
> Documents are being ingested into Solr from different nodes. 12 SolrJ 
clients ingest data into the Solr cloud.
>
> We are experiencing issues when we keep the setup running for long time 
and after processing around 100 GB of index size (I.e. Around 600 Million 
documents). Note that we are only indexing the data and not querying it. So 
there should not be any query overhead. From the VM analysis we figured out 
that over time the disk operations starts declining and so does the CPU, RAM 
and Network usage of the Solr nodes. We concluded that Solr is unable to handle 
one big collection due to index read/write overhead and most of the time it 
ends up doing only the commit (evident in Solr logs). And because of that 
indexing is getting hampered (?)
>
> So we thought of creating small sized collections instead of one big 
collection anticipating the commit performance might improve. But eventually 
the performance degrades even with that and we observe more or less similar 
charts for CPU, memory, disk and network.
>
> To put forth some stats here are the number of documents processed every 
hour
>
> 1St hour: 250 million
> 2nd hour: 250 million
> 3rd hour: 240 million
> 4th hour: 200 million
> .
> .
> 11th hour: 80 million
>
> Could you please help us identifying the root cause of degradation in the 
performance? Are we doing something wrong with the Solr configuration or the 
collections/sharding etc? Due to this performance degradation we are currently 
stuck with Solr.
>
> Thank you very much in advance.
>
> Prasad Tendulkar
>
>

backward compatibility of Solr 6.3 version with old Sol4j clients

[Suresh Pendap]

Hi,

Will Solrj client 4.10.3 version work with  Solr 6.3 version of the server?   I 
was trying to look up the documentation but no where the compatibility matrix 
between server and client is provided.
Has some one already used this combination?

Regards
Suresh

unable to get more throughput with more threads

[Suresh Pendap]

Hi,
I am new to SOLR search engine technology and I am trying to get some 
performance numbers to get maximum throughput from the SOLR cluster of a given 
size.
I am currently doing only query load testing in which I randomly fire a bunch 
of queries to the SOLR cluster to generate the query load.  I understand that 
it is not the ideal workload as the
ingestion and commits happening invalidate the Solr Caches, so it is advisable 
to perform query load along with some documents being ingested.

The SOLR cluster was made up of 2 shards and 2 replicas. So there were total 4 
replicas serving the queries. The SOLR nodes were running on an LXD container 
with 12 cores and 88GB RAM.
The heap size allocated was 8g min and 8g max. All the other SOLR 
configurations were default.

The client node was running on an 8 core VM.

I performed the test with 1 thread, 10 client threads and 50 client threads.  I 
noticed that as I increased the number of threads, the query latency kept 
increasing drastically which I was not expecting.

Since my initial test was randomly picking queries from a file, I decided to 
keep things constant and ran the program which fired the same query again and 
again. Since it is the same query, all the documents will
be in the Cache and the query response time should be very fast. I was also 
expecting that with 10 or 50 client threads, the query latency should not be 
increasing.

The throughput increased only up to 10 client threads but then it was same for 
50 threads, 100 threads and the latency of the query kept increasing as I 
increased the number of threads.
The query was returning 2 documents only.

The table below summarizes the numbers that I was saying with a single query.





#No of Client Nodes
#No of Threads  99 pct Latency  95 pct latency  throughput  CPU 
Utilization Server Configuration

1   1   9 ms7 ms180 reqs/sec8%

Heap size: ms=8g, mx=8g

default configuration


1   10  400 ms  360 ms  360 reqs/sec10%

Heap size: ms=8g, mx=8g

default configuration




I also ran the client program on the SOLR server node in order to rule our the 
network latency factor. On the server node also the response time was higher 
for 10 threads, but the amplification was smaller.

I am getting an impression that probably my query requests are getting queued 
up and limited due to probably some thread pool size on the server side.  
However I saw that the default jetty.xml does
have the thread pool of min size of 10 and  max of 1.

Is there any other internal SOLR thread pool configuration which might be 
limiting the query response time?

I wanted to check with the community if what I am seeing is abnormal behavior, 
what could be the issue?  Is there any configuration that I can tweak to get 
better query response times for more load?

Regards
Suresh

Re: unable to get more throughput with more threads

[Suresh Pendap]

I am using version 6.3 of Solr

On 3/23/17 7:56 PM, "Aman Deep Singh"  wrote:

>system

Re: unable to get more throughput with more threads

[Suresh Pendap]

Edwin,
The heap was not being used much, only 1GB of heap was being used out of
8GB.  I do have space to allocate more to the heap size.
I was reading in some SOLR performance blogs that it is better not to use
large heap size, instead it is better to provide lot of space to the
Operating system
Disk cache so that maximum documents stay in memory in the buffer cache.

Regards
Suresh


On 3/23/17 7:52 PM, "Zheng Lin Edwin Yeo"  wrote:

>I also did find that beyond 10 threads for 8GB heap size , there isn't
>much
>improvement with the performance. But you can increase your heap size a
>little if your system allows it.
>
>By the way, which Solr version are you using?
>
>Regards,
>Edwin
>
>
>On 24 March 2017 at 09:21, Matt Magnusson  wrote:
>
>> Out of curosity, what is your index size? I'm trying to do something
>> similar with maximizing output, I'm currently looking at streaming
>> expressions which I'm seeing some interesting results for, I'm also
>> finding that the direct mass query route seems to hit a wall for
>> performance. I'm also finding that about 10 threads seems to be an
>> optimum number.
>>
>> On Thu, Mar 23, 2017 at 8:10 PM, Suresh Pendap 
>> wrote:
>> > Hi,
>> > I am new to SOLR search engine technology and I am trying to get some
>> performance numbers to get maximum throughput from the SOLR cluster of a
>> given size.
>> > I am currently doing only query load testing in which I randomly fire
>>a
>> bunch of queries to the SOLR cluster to generate the query load.  I
>> understand that it is not the ideal workload as the
>> > ingestion and commits happening invalidate the Solr Caches, so it is
>> advisable to perform query load along with some documents being
>>ingested.
>> >
>> > The SOLR cluster was made up of 2 shards and 2 replicas. So there were
>> total 4 replicas serving the queries. The SOLR nodes were running on an
>>LXD
>> container with 12 cores and 88GB RAM.
>> > The heap size allocated was 8g min and 8g max. All the other SOLR
>> configurations were default.
>> >
>> > The client node was running on an 8 core VM.
>> >
>> > I performed the test with 1 thread, 10 client threads and 50 client
>> threads.  I noticed that as I increased the number of threads, the query
>> latency kept increasing drastically which I was not expecting.
>> >
>> > Since my initial test was randomly picking queries from a file, I
>> decided to keep things constant and ran the program which fired the same
>> query again and again. Since it is the same query, all the documents
>>will
>> > be in the Cache and the query response time should be very fast. I was
>> also expecting that with 10 or 50 client threads, the query latency
>>should
>> not be increasing.
>> >
>> > The throughput increased only up to 10 client threads but then it was
>> same for 50 threads, 100 threads and the latency of the query kept
>> increasing as I increased the number of threads.
>> > The query was returning 2 documents only.
>> >
>> > The table below summarizes the numbers that I was saying with a single
>> query.
>> >
>> >
>> >
>> >
>> >
>> > #No of Client Nodes
>> > #No of Threads  99 pct Latency  95 pct latency  throughput
>> CPU Utilization Server Configuration
>> >
>> > 1   1   9 ms7 ms180 reqs/sec8%
>> >
>> > Heap size: ms=8g, mx=8g
>> >
>> > default configuration
>> >
>> >
>> > 1   10  400 ms  360 ms  360 reqs/sec10%
>> >
>> > Heap size: ms=8g, mx=8g
>> >
>> > default configuration
>> >
>> >
>> >
>> >
>> > I also ran the client program on the SOLR server node in order to rule
>> our the network latency factor. On the server node also the response
>>time
>> was higher for 10 threads, but the amplification was smaller.
>> >
>> > I am getting an impression that probably my query requests are getting
>> queued up and limited due to probably some thread pool size on the
>>server
>> side.  However I saw that the default jetty.xml does
>> > have the thread pool of min size of 10 and  max of 1.
>> >
>> > Is there any other internal SOLR thread pool configuration which might
>> be limiting the query response time?
>> >
>> > I wanted to check with the community if what I am seeing is abnormal
>> behavior, what could be the issue?  Is there any configuration that I
>>can
>> tweak to get better query response times for more load?
>> >
>> > Regards
>> > Suresh
>> >
>>

Re: unable to get more throughput with more threads

[Suresh Pendap]

Erick,
- I think I checked that my QueryResultsCache and DocumentCache ratios
were close to 1. I will double check that by repeating my test.
- I think checking the Qtimes in the log is a very good suggestion, I will
also check that the next time I run my test
- It is not possible as the client is just a java client program which
just fires the queries using a REST Client API

Is there any way that SOLR publishes its thread pool statistics?

For e.g in cassandra you have a command like nodetool tpstats which
provides a nice table stats for all the thread pools involved,
how many are pending jobs etc?


Thanks
Suresh

On 3/23/17 9:33 PM, "Erick Erickson"  wrote:

>I'd check my I/O. Since you're firing the same query, I expect that
>you aren't I/O bound at all, since, as you say, the docs should
>already be in memory. This assumes that your document cache size is >
>0. You can check this. Go to the admin UI, select one of your cores
>(not collection) and go to plugins/stats. You should see the
>documentCache as one of the entries and you should be hitting an
>insane hit ratio close to 100% as your test runs.
>
>Also check your queryResultCache. That also should be near 100% in
>your test. Do note that these caches really never hit this "for real",
>but as you indicated this is a highly artificial test so such high hit
>ratios are what I'd expect.
>
>Assuming that those caches are being hit near 100%, Solr really isn't
>doing any work to speak of so there almost has to be some kind of
>queueing going on.
>
>The fact that your CPU is only running 8-10% is an indication that
>you're requests are queued up somewhere, but where I have no clue. The
>Jetty thread pool is quite large.  What are the QTimes reported in the
>responses? My guess is that the QTime stays pretty constant (and very
>low) even as your response time increases, another indication that
>you're queueing.
>
>Hmmm, is it possible that on the queueing is on the _client_ side?
>What aggregate throughput to you get if you fire up 10 _clients_ each
>with one thread rather than 1 client and 10 threads? That's a shot in
>the dark, but worth a try I suppose. And how does your client fire
>queries? SolrJ? Http? Jmeter or the like?
>
>But yeah, this is weird. Since you're firing the same query, Solr
>isn't really doing any work at all.
>
>Best,
>Erick
>
>On Thu, Mar 23, 2017 at 7:56 PM, Aman Deep Singh
> wrote:
>> You can play with the merge factor in the index config.
>> If their is no frequent updates then make it 2 ,it will give you High
>> throughput and less latency.
>>
>> On 24-Mar-2017 8:22 AM, "Zheng Lin Edwin Yeo" 
>>wrote:
>>
>>> I also did find that beyond 10 threads for 8GB heap size , there isn't
>>>much
>>> improvement with the performance. But you can increase your heap size a
>>> little if your system allows it.
>>>
>>> By the way, which Solr version are you using?
>>>
>>> Regards,
>>> Edwin
>>>
>>>
>>> On 24 March 2017 at 09:21, Matt Magnusson  wrote:
>>>
>>> > Out of curosity, what is your index size? I'm trying to do something
>>> > similar with maximizing output, I'm currently looking at streaming
>>> > expressions which I'm seeing some interesting results for, I'm also
>>> > finding that the direct mass query route seems to hit a wall for
>>> > performance. I'm also finding that about 10 threads seems to be an
>>> > optimum number.
>>> >
>>> > On Thu, Mar 23, 2017 at 8:10 PM, Suresh Pendap
>>>
>>> > wrote:
>>> > > Hi,
>>> > > I am new to SOLR search engine technology and I am trying to get
>>>some
>>> > performance numbers to get maximum throughput from the SOLR cluster
>>>of a
>>> > given size.
>>> > > I am currently doing only query load testing in which I randomly
>>>fire a
>>> > bunch of queries to the SOLR cluster to generate the query load.  I
>>> > understand that it is not the ideal workload as the
>>> > > ingestion and commits happening invalidate the Solr Caches, so it
>>>is
>>> > advisable to perform query load along with some documents being
>>>ingested.
>>> > >
>>> > > The SOLR cluster was made up of 2 shards and 2 replicas. So there
>>>were
>>> > total 4 replicas serving the queries. The SOLR nodes were running on
>>>an
>>> LXD
>>> > container with 12 cores and 88GB RAM.
>>> &g

Re: unable to get more throughput with more threads

[Suresh Pendap]

Hi Shawn,
It looks like you probably have pointed to the root cause of the issue.
I am using a java client and using HttpClient library directly to fire the
Http get queries. I am not using SolrJ client for firing the queries.

The following is my code

CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse response = null;
try {
   response = httpclient.execute(httpGet);
} finally {

}

Is a route per httpHost/port combination?

I will try to set the MaxConnPerRoute and maxTotalConn to a larger value
and see if that gives me some benefit.


Thanks
Suresh









On 3/24/17 6:05 AM, "Shawn Heisey"  wrote:

>On 3/23/2017 6:10 PM, Suresh Pendap wrote:
>> I performed the test with 1 thread, 10 client threads and 50 client
>> threads. I noticed that as I increased the number of threads, the
>> query latency kept increasing drastically which I was not expecting.
>
>What language and Solr library was the client using?  If it's Java and
>SolrJ, then the following will apply.  If the client is written in a
>language other than Java, you may find that there are similar default
>settings in the HTTP library:
>
>A dependency called HttpClient is used by SolrJ.  The default settings
>for HttpClient are only capable of making *two* simultaneous connections
>to a target server.  Further connections will wait until existing
>connections are complete.  Unless it is overridden, SolrJ creates the
>HttpClient object with default settings.
>
>If more threads are needed, the SolrClient object must be built with a
>custom HttpClient.  Here's some SolrJ code to create a
>multithread-capable client object (300 threads to a single server):
>
>  RequestConfig rc = RequestConfig.custom().setConnectTimeout(15000)
>  .setSocketTimeout(Const.SOCKET_TIMEOUT).build();
>  httpClient = HttpClients.custom().setDefaultRequestConfig(rc)
> 
>.setMaxConnPerRoute(300).setMaxConnTotal(5000).disableAutomaticRetries()
>  .build();
>  client = new HttpSolrClient(serverBaseUrl, httpClient);
>
>I have also placed this code at the following URL.  It will expire in
>one month:
>
>http://apaste.info/BpoWY
>
>A similar technique can be used with CloudSolrClient if needed.
>
>It's my opinion that SolrJ needs to create client objects by default
>that are capable of more threads, but I have not yet put any time into
>making it happen.
>
>Thanks,
>Shawn
>

Solr Query Performance benchmarking

[Suresh Pendap]

Hi,
I am trying to perform Solr Query performance benchmarking and trying to 
measure the maximum throughput and latency that I can get from.a given Solr 
cluster.

Following are my configurations

Number of Solr Nodes: 4
Number of shards: 2
replication-factor:  2
Index size: 55 GB
Shard/Core size: 27.7 GB
maxConnsPerHost: 1000

The Solr nodes are VM's with 16 core vCpu and 112GB RAM.  The CPU is 1-1 and it 
is not overcommitted.

I am generating query load using a Java client program which fires Solr queries 
read from a static file.  The client java program is using the Apache Http 
Client library to invoke the queries. I have already configured the client to 
create 300 max connections.

The type of queries are mostly of the below pattern
q=*:*&fl=orderNo,purchaseOrderNos,timestamp,eventName,eventID,_src_&fq=((orderNo:+AND+purchaseOrderNos:))&sort=eventTimestamp+desc&rows=20&wt=javabin&version=2

Max throughput that I get: 12000 to 12500 reqs/sec
95 percentile query latency: 30 to 40 msec

I am measuring the latency and throughput on the client side in my program.  
The max throughput that I am able to get (sum of each individual clients 
throughput) is 12000 reqs/sec.  I am running with 4 clients each with 50 
threads.  Even if I increase the number of clients, the throughput still seems 
to be the same. It seems like I am hitting the maximum capacity of the cluster 
or some other limit due to which I am not able to put more stress on the server.

My CPU is hitting 60% to 70%.  I have not been able to increase the CPU usage 
more than this even when increasing client threads or generating load with more 
client nodes.

The memory used is around 16% on all the nodes except on one node I am seeing 
the memory used is 41%.

There is hardly any IO happening as it is a read test.

I am wondering what is limiting my throughput, is there some internal thread 
pool limit that I am hitting due to which I am not able to increase my 
CPU/memory usage?

My JVM settings are provided below. I am using G1GC and


-DSTOP.KEY=solrrocks
-DSTOP.PORT=7983
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.port=13001
-Dcom.sun.management.jmxremote.rmi.port=13001
-Dcom.sun.management.jmxremote.ssl=false
-Djetty.home=/app/solr6/server
-Djetty.port=8983
-Dlog4j.configuration=file:
-Dsolr.autoSoftCommit.maxTime=5000
-Dsolr.autoSoftCommit.minTime=5000
-Dsolr.install.dir=/app/solr6
-Dsolr.log.dir=/app/solrdata6/logs
-Dsolr.log.muteconsole
-Dsolr.solr.home=
-Duser.timezone=UTC
-DzkClientTimeout=15000
-DzkHost=
-XX:+AlwaysPreTouch
-XX:+ResizeTLAB
-XX:+UseG1GC
-XX:+UseGCLogFileRotation
-XX:+UseLargePages
-XX:+UseTLAB
-XX:-UseBiasedLocking
-XX:GCLogFileSize=20M
-XX:MaxGCPauseMillis=50
-XX:NumberOfGCLogFiles=9
-XX:OnOutOfMemoryError=/app/solr6/bin/oom_solr.sh
-Xloggc:
-Xms11g
-Xmx11g
-Xss256k
-verbose:gc
















I have not customized the Solr Cache values.  The DocumentCache, 
QueryResultCache, FieldValueCache everything is using default values.  I read 
in one of the SolrPerformance documents that it is better to leave more memory 
to the Operating system and utilize the OS buffer cache.

Is it the best query throughput that I can extract from this sized cluster and 
index size combination?

Any ideas is highly appreciated.

Thanks
Suresh

Regarding rule based replica placement in Solr

[Suresh Pendap]

Hi,

I read the documentation about this feature over 
here.
  I did not get a very clear understanding of how this feature works and how I 
can configure it in Solr.

This is what I would like to achieve and am wondering if anyone has tried this 
feature and what their experience has been with it.


If I would like to configure a rule to make sure that for every shard at-least 
one replica is assigned to each rack before more than one replicas are assigned 
to the same rack, how would that rule look like?

For e.g

If I have a collection with 2 shards and replication-factor: 2 and I have 3 
racks say rack1, rack2, rack3, I want to make sure that at-least one replica is 
assigned to each rack.
So the replicas in this case should be assigned as shard1_replica1: rack1, 
shard1_replica2:rack2, shard2_replica1:rack3 and shard2_replica2:
It should not assign the replicas as shard1_replica1:rack1, 
shard1_replica2:rack1, shard2_replica1: rack2, shard2_replica2: rack2. In this 
case all the replicas of a shard are assigned to the same rack.

This is not ideal and not good for HA reasons.


Regards

Suresh

Re: EXT: Re: Solr Query Performance benchmarking

[Suresh Pendap]

Thanks everyone for taking time to respond to my email. I think you are
correct in that the query results might be coming from main memory as I
only had around 7k queries.
However it is still not clear to me, given that everything was being
served from main memory, why is that I am not able to push the CPU usage
further up by putting more load on the cluster?

Thanks
Suresh

On 4/28/17, 6:44 PM, "Shawn Heisey"  wrote:

>On 4/28/2017 12:43 PM, Toke Eskildsen wrote:
>> Shawn Heisey  wrote:
>>> Adding more shards as Toke suggested *might* help,[...]
>> I seem to have phrased my suggestion poorly. What I meant to suggest
>> was a switch to a single shard (with 4 replicas) setup, instead of the
>> current 2 shards (with 2 replicas).
>
>Reading it a second time, it's me who made the error here.  You did say
>1 shard and 4 replicas, I didn't read it correctly.
>
>Apologies!
>
>Thanks,
>Shawn
>
>

Query regarding Solr Caches

[Suresh Pendap]

Hi,
This question might have been asked on the solr user mailing list earlier. Solr 
has four different types of Cache DocumentCache, QueryResultCache, 
FieldValueCache and FilterQueryCache
I would like to know which of these Caches are off heap cache?  Which Caches 
have the maximum impact on the query latency and it is recommended to configure 
that Cache?

I also would like to know if there is any document which provides guidelines on 
performing Capacity planning for a Solr cluster.

Regards
Suresh

regarding Solr Caches

[suresh pendap]

Hi,
This question might have been asked on the solr user mailing list earlier.
Solr has four different types of Cache DocumentCache, QueryResultCache,
FieldValueCache and FilterQueryCache.

Are these Caches memory mapped or they reside in the JVM heap?
Which Caches have the maximum impact on the query latency?
Any recommendations around Cache size settings or any documentation would
be very helpful.

I also would like to know if there is any document which provides
guidelines on performing Capacity planning for a Solr cluster.

Regards
Suresh

Re: EXT: Re: Query regarding Solr Caches

[Suresh Pendap]

Hi Shawn,
Thanks for the reply, it is useful. The reason I asked about the Cache
sizes is I had read that configuring the Cache sizes of Solr does not
provide you enough benefits, instead it is better to provide a lot of
memory space to the Solr outside the JVM heap.

Is it true that in general the strategy of using the OS buffer cache works
better instead of using a large Cache inside the JVM heap?

I was looking for some numbers that people have used for configuring these
Caches in the past and the rational for choosing these values.

Thanks
Suresh


On 5/11/17, 11:13 PM, "Shawn Heisey"  wrote:

>On 5/11/2017 4:58 PM, Suresh Pendap wrote:
>> This question might have been asked on the solr user mailing list
>>earlier. Solr has four different types of Cache DocumentCache,
>>QueryResultCache, FieldValueCache and FilterQueryCache
>> I would like to know which of these Caches are off heap cache?
>
>None of them are off-heap.  For a while, there was a forked project
>called Heliosearch that did have a bunch of off-heap memory structures,
>but it is my understanding that Heliosearch is effectively dead now.
>
>There are at least three issues for bringing this capability to Solr,
>but none of them have been added yet.
>
>https://issues.apache.org/jira/browse/SOLR-6638
>https://issues.apache.org/jira/browse/SOLR-7211
>https://issues.apache.org/jira/browse/SOLR-7210
>
>> Which Caches have the maximum impact on the query latency and it is
>>recommended to configure that Cache?
>
>This is so dependent on your exact index and your exact queries that
>nobody can give you a reliable answer to that question.
>
>The filterCache is pretty good at speeding things up when you have
>filter queries that get reused frequently, but depending on how your
>setup works, it might not provide the most impact.
>
>> I also would like to know if there is any document which provides
>>guidelines on performing Capacity planning for a Solr cluster.
>
>No.  It's not possible to provide general information.  See this:
>
>https://lucidworks.com/sizing-hardware-in-the-abstract-why-we-dont-have-a-
>definitive-answer/
>
>Thanks,
>Shawn
>

regarding exposing merge metrics

[suresh pendap]

Hi,
I am following the instructions from
https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
 in order to expose the Index merge related metrics.

The document says that we have to add the below snippet in order to expose
the merge metrics


  ...
  

  524288
  true

...
  
...



I would like to know why is this metrics not exposed by default just like
all the other metrics?

Is there any performance overhead that we should be concerned about it?

If there was no particular reason, then can we expose it by default?



Regards
Suresh

Re: regarding exposing merge metrics

[suresh pendap]

Thanks Shalin for sharing the link. However if I follow the thread then it
seems like there was no conclusive evidence found that the performance
degradation was due to the merge or index related metrics.
If that is the case then can we just get rid of the config and publish
these metrics by default?


Regards
suresh



On Mon, Jan 8, 2018 at 10:25 PM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> The merge metrics were enabled by default in 6.4 but they were turned
> off in 6.4.2 because of large performance degradations. For more
> details, see https://issues.apache.org/jira/browse/SOLR-10130
>
> On Tue, Jan 9, 2018 at 9:11 AM, S G  wrote:
> > Yes, this is actually confusing and the documentation (
> > https://lucene.apache.org/solr/guide/7_2/metrics-reporting.html) does
> not
> > help either:
> >
> > *Index Merge Metrics* : These metrics are collected in respective
> > registries for each core (e.g., solr.core.collection1….), under the INDEX
> > category.
> > Basic metrics are always collected - collection of additional metrics can
> > be turned on using boolean parameters in the /config/indexConfig/metrics.
> >
> > However, we do not see the merge-metrics being collected if the above
> > config is absent. So what basic metrics are always collected for merge?
> > And why do the merge metrics require an additional config while most of
> the
> > others are reported directly?
> >
> > Thanks
> > SG
> >
> >
> >
> > On Mon, Jan 8, 2018 at 2:02 PM, suresh pendap 
> > wrote:
> >
> >> Hi,
> >> I am following the instructions from
> >> https://lucene.apache.org/solr/guide/7_1/metrics-reporting.html
> >>  in order to expose the Index merge related metrics.
> >>
> >> The document says that we have to add the below snippet in order to
> expose
> >> the merge metrics
> >>
> >> 
> >>   ...
> >>   
> >> 
> >>   524288
> >>   true
> >> 
> >> ...
> >>   
> >> ...
> >>
> >>
> >>
> >> I would like to know why is this metrics not exposed by default just
> like
> >> all the other metrics?
> >>
> >> Is there any performance overhead that we should be concerned about it?
> >>
> >> If there was no particular reason, then can we expose it by default?
> >>
> >>
> >>
> >> Regards
> >> Suresh
> >>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: regarding exposing merge metrics

[suresh pendap]

Hi Shawn,
Thanks for replying to my questions.
So is it correct to assume that exposing merge metrics is not known to
cause any performance degradation?

-suresh

On Wed, Jan 10, 2018 at 5:40 PM, Shawn Heisey  wrote:

> On 1/10/2018 11:08 AM, S G wrote:
>
>> Last comment by Shawn on SOLR-10130 is:
>> Metrics was just a theory, sounds like that's not it.
>>
>> It would be very interesting to know what really caused the slowdown and
>> do
>> we really need the config or not.
>>
>
> That comment wasn't actually about SOLR-10130 itself.I commented on that
> issue because it was semi-related to what I was looking into, so I figured
> the developer who fixed it might have some insight to the question I needed
> answered.
>
> I think that SOLR-10130 was probably handled correctly and that they did
> indeed find/fix the problem.
>
> My comments were speculating about completely different performance issues
> -- someone was seeing reduced performance upgrading from 4.x to a version
> that included the fix for SOLR-10130.  Because SOLR-10130 already indicated
> that metrics could cause performance issues, I was wondering whether the
> metrics that were added for keeping track of Jetty operation could possibly
> be the source of the user's problems.  The response I got indicated that it
> was unlikely that the Jetty metrics were the cause.
>
> Performance regressions between 4.x and later versions have been noted by
> many users.  I think some of them were caused by changes in Lucene, and
> it's possible that those who understand these issues have not yet found a
> solution to fix them.
>
> Thanks,
> Shawn
>
>

configuring zookeeper authentication and authorization

[suresh pendap]

Hi,
I am following the Solr documentation to configure ZK authentication and
ACLS from here
https://lucene.apache.org/solr/guide/6_6/zookeeper-access-control.html

I am planning to go with MD5 Digest authentication mechanism

I am assuming that you still have to enable authentication on the Zookeeper
server side.

Is that correct?


The zookeeper documentation for configuring authentication is not very easy
to follow.

I am trying to follow this document
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication

The above page seem to mix the instructions for SASL and DIGEST
authentication schemes and it is very hard to follow.

Is there a better resource or documentation that I can follow?



-suresh

regarding Extracting text from Images

[suresh pendap]

Hello,
I am reading the Solr documentation about integration with Tika and Solr
Cell framework over here
https://lucene.apache.org/solr/guide/6_6/uploading-data-with-solr-cell-using-apache-tika.html

I would like to know if the can Solr Cell framework also be used to extract
text from the image files?

Regards
Suresh

Re: regarding Extracting text from Images

[suresh pendap]

Hi Alex,
Thanks for your reply. How do we integrate tesseract with Solr?  Do we have
to implement Custom update processor or extend the
ExtractingRequestProcessor?

Regards
Suresh

On Wed, Oct 23, 2019 at 11:21 AM Alexandre Rafalovitch 
wrote:

> I believe Tika that powers this can do so with extra libraries (tesseract?)
> But Solr does not bundle those extras.
>
> In any case, you may want to run Tika externally to avoid the
> conversion/extraction process be a burden to Solr itself.
>
> Regards,
>  Alex
>
> On Wed, Oct 23, 2019, 1:58 PM suresh pendap, 
> wrote:
>
> > Hello,
> > I am reading the Solr documentation about integration with Tika and Solr
> > Cell framework over here
> >
> >
> https://lucene.apache.org/solr/guide/6_6/uploading-data-with-solr-cell-using-apache-tika.html
> >
> > I would like to know if the can Solr Cell framework also be used to
> extract
> > text from the image files?
> >
> > Regards
> > Suresh
> >
>