Add replica fails with following exception
I have a script that creates a collection whenever vm is up and then add the replicas. However ADDREPLICAS fails everytime with following exception. However when i manually hit the following curl it works all fine. The same line is in the script. " http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=rules&shard=shard1&node= $solrHost:8983_solr" 2020-05-08 08:20:55.621 ERROR (qtp2048537720-17) [c:rules ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: ADDREPLICA failed to create replica at org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:63) at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:281) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:253) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:839) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:805) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:558) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917) at java.base/java.lang.Thread.run(Unknown Source)
Re: Add replica fails with following exception
My guess is that "$solrHost” is being resolved differently when executed from the shell .vs. your script. Best, Erick > On May 8, 2020, at 4:41 AM, Vishal Vaibhav wrote: > > I have a script that creates a collection whenever vm is up and then add > the replicas. However ADDREPLICAS fails everytime with following > exception. However when i manually hit the following curl it works all > fine. The same line is in the script. > " > http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=rules&shard=shard1&node= > $solrHost:8983_solr" > > > 2020-05-08 08:20:55.621 ERROR (qtp2048537720-17) [c:rules ] > o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: ADDREPLICA > failed to create replica > at > org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:63) > at > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:281) > at > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:253) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) > at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:839) > at > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:805) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:558) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at org.eclipse.jetty.server.Server.handle(Server.java:505) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917) > at java.base/java.lang.Thread.run(Unknown Source)
Re: Add replica fails with following exception
Do addreplicas work only when leader is elected ? Is that a thing ? On Fri, 8 May 2020 at 2:11 PM, Vishal Vaibhav wrote: > I have a script that creates a collection whenever vm is up and then add > the replicas. However ADDREPLICAS fails everytime with following > exception. However when i manually hit the following curl it works all > fine. The same line is in the script. > " > http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=rules&shard=shard1&node= > $solrHost:8983_solr" > > > 2020-05-08 08:20:55.621 ERROR (qtp2048537720-17) [c:rules ] > o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: ADDREPLICA > failed to create replica > at > org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:63) > at > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:281) > at > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:253) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) > at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:839) > at > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:805) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:558) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at org.eclipse.jetty.server.Server.handle(Server.java:505) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917) > at java.base/java.lang.Thread.run(Unknown Source) >
Re: Add replica fails with following exception
Do addreplicas work only when leader is elected ? On Fri, 8 May 2020 at 5:43 PM, Erick Erickson wrote: > My guess is that "$solrHost” is being resolved differently when > executed from the shell .vs. your script. > > Best, > Erick > > > On May 8, 2020, at 4:41 AM, Vishal Vaibhav wrote: > > > > I have a script that creates a collection whenever vm is up and then add > > the replicas. However ADDREPLICAS fails everytime with following > > exception. However when i manually hit the following curl it works all > > fine. The same line is in the script. > > " > > > http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=rules&shard=shard1&node= > > $solrHost:8983_solr" > > > > > > 2020-05-08 08:20:55.621 ERROR (qtp2048537720-17) [c:rules ] > > o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: > ADDREPLICA > > failed to create replica > > at > > > org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:63) > > at > > > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:281) > > at > > > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:253) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) > > at > org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:839) > > at > > > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:805) > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:558) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) > > at > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > > at > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) > > at > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152) > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > at > > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > > at org.eclipse.jetty.server.Server.handle(Server.java:505) > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) > > at > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) > > at > > org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > at > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > at > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > > at > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > > at > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > > at > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781) > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917) > > at java.base/java.lang.Thread.run(Unknown Source) > >
Synonym Graph Filter - how to preserve original
In 8.3, There should be the way to preserve the original terms, but could not find it. Does anyone know? Thanks, Jae
Re: Synonym Graph Filter - how to preserve original
Hi Jae, Do try to explain your problem with an example. Also share how you are writing the synonyms file. Best Regards, Atin Janki On Fri, May 8, 2020 at 6:14 PM Jae Joo wrote: > In 8.3, There should be the way to preserve the original terms, but could > not find it. > > Does anyone know? > > Thanks, > > Jae >
Re: Synonym Graph Filter - how to preserve original
putting original term in the synonym list works. On Fri, May 8, 2020 at 1:05 PM atin janki wrote: > Hi Jae, > > Do try to explain your problem with an example. Also share how you are > writing the synonyms file. > Best Regards, > Atin Janki > > > On Fri, May 8, 2020 at 6:14 PM Jae Joo wrote: > > > In 8.3, There should be the way to preserve the original terms, but could > > not find it. > > > > Does anyone know? > > > > Thanks, > > > > Jae > > >
Re: Synonym Graph Filter - how to preserve original
Depends on how you define the synonym file. See: https://lucene.apache.org/solr/guide/6_6/filter-descriptions.html In particular: There are two ways to specify synonym mappings: • A comma-separated list of words. If the token matches any of the words, then all the words in the list are substituted, which will include the original token. • Two comma-separated lists of words with the symbol "⇒" between them. If the token matches any word on the left, then the list on the right is substituted. The original token will not be included unless it is also in the list on the right. > On May 8, 2020, at 1:10 PM, Jae Joo wrote: > > putting original term in the synonym list works. > > > On Fri, May 8, 2020 at 1:05 PM atin janki wrote: > >> Hi Jae, >> >> Do try to explain your problem with an example. Also share how you are >> writing the synonyms file. >> Best Regards, >> Atin Janki >> >> >> On Fri, May 8, 2020 at 6:14 PM Jae Joo wrote: >> >>> In 8.3, There should be the way to preserve the original terms, but could >>> not find it. >>> >>> Does anyone know? >>> >>> Thanks, >>> >>> Jae >>> >>
solr payloads performance
Hi everyone, Have a question regarding typical e-commerce scenario: each item may have different price in different store. suppose there are 10 million items and 1000 stores. Option 1: use solr payloads, each document have store_prices_payload:store1|price1 store2|price2 . store1000|price1000 Option 2: use dynamic fields and have 1000 fields in each document, i.e. field1: store1_price: price1 field2: store2_price: price2 ... field1000: store1000_price: price1000 Option 2 doesn't look elegant, but is there any performance benchmark on solr payloads? In terms of filtering, sorting or faceting, how would query performance compare between the two? Thanks, Wei
Re: Unbalanced shard requests
Update: after I remove the shards.preference parameter from solrconfig.xml, issue is gone and internal shard requests are now balanced. The same parameter works fine with solr 7.6. Still not sure of the root cause, but I observed a strange coincidence: the nodes that are most frequently picked for shard requests are the first node in each shard returned from the CLUSTERSTATUS api. Seems something wrong with shuffling equally compared nodes when shards.preference is set. Will report back if I find more. On Mon, Apr 27, 2020 at 5:59 PM Wei wrote: > Hi Eric, > > I am measuring the number of shard requests, and it's for query only, no > indexing requests. I have an external load balancer and see each node > received about the equal number of external queries. However for the > internal shard queries, the distribution is uneven:6 nodes (one in > each shard, some of them are leaders and some are non-leaders ) gets about > 80% of the shard requests, the other 54 nodes gets about 20% of the shard > requests. I checked a few other parameters set: > > -Dsolr.disable.shardsWhitelist=true > shards.preference=replica.location:local,replica.type:TLOG > > Nothing seems to cause the strange behavior. Any suggestions how to > debug this? > > -Wei > > > On Mon, Apr 27, 2020 at 5:42 PM Erick Erickson > wrote: > >> Wei: >> >> How are you measuring utilization here? The number of incoming requests >> or CPU? >> >> The leader for each shard are certainly handling all of the indexing >> requests since they’re TLOG replicas, so that’s one thing that might >> skewing your measurements. >> >> Best, >> Erick >> >> > On Apr 27, 2020, at 7:13 PM, Wei wrote: >> > >> > Hi everyone, >> > >> > I have a strange issue after upgrade from 7.6.0 to 8.4.1. My cloud has 6 >> > shards with 10 TLOG replicas each shard. After upgrade I noticed that >> one >> > of the replicas in each shard is handling most of the distributed shard >> > requests, so 6 nodes are heavily loaded while other nodes are idle. >> There >> > is no change in shard handler configuration: >> > >> > > > "HttpShardHandlerFactory"> >> > >> >3 >> > >> >3 >> > >> >500 >> > >> > >> > >> > >> > What could cause the unbalanced internal distributed request? >> > >> > >> > Thanks in advance. >> > >> > >> > >> > Wei >> >>
