Upgrade to v7.6 https://issues.apache.org/jira/browse/SOLR-12814 <https://issues.apache.org/jira/browse/SOLR-12814>
-- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 21. des. 2018 kl. 21:00 skrev Tannen, Lev (USAEO) [Contractor] > <lev.tan...@usdoj.gov>: > > Hello Solr community, > > My solrcloud system consists of 3 machines, each running a zookeeper and a > solr server. It manages about 200 collections with 1 shard each. It is up and > running, but about every minutes it produces messages in the message log on > each computer (messages are at the end). These messages do not relate to any > requests. It looks like they are produced by some internal mechanism. Using a > suggestion found on the Internet I have already increased the > requestHeaderSize and the responseHeaderSize to 81920, but this did not help. > Do anyone know what do these messages mean and how to get reed of them? > Thank you. > Lev Tannen > > Message on Computer1: > 2018-12-21 19:44:46.552 WARN (qtp817348612-21) [ ] o.e.j.h.HttpParser URI > is too large >81920 > 2018-12-21 19:44:46.560 INFO (qtp817348612-16) [ ] o.a.s.s.HttpSolrCall > [admin] webapp=null path=/admin/metrics > params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} > status=0 QTime=1 > > Message on computer 2 > 2018-12-21 19:44:46.530 WARN (qtp817348612-14) [ ] o.e.j.h.HttpParser URI > is too large >81920 > 2018-12-21 19:44:46.537 INFO (qtp817348612-14) [ ] o.a.s.s.HttpSolrCall > [admin] webapp=null path=/admin/metrics > params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} > status=0 QTime=2 > > Message on computer3 (a leader) > 2018-12-21 19:44:46.513 WARN (qtp817348612-14) [ ] o.e.j.h.HttpParser URI > is too large >81920 > 2018-12-21 19:44:46.514 WARN (MetricsHistoryHandler-12-thread-1) [ ] > o.a.s.c.s.i.SolrClientNodeStateProvider could not get tags from node > usahubslvcvw121.usa.doj.gov:8983_solr > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at http://usahubslvcvw121.usa.doj.gov:8983/solr: Expected mime > type application/octet-stream but got text/html. <h1>Bad Message > 414</h1><pre>reason: URI Too Long</pre> > at > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:292) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchMetrics(SolrClientNodeStateProvider.java:150) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getReplicaInfo(SolrClientNodeStateProvider.java:131) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:478) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:368) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:230) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [?:1.8.0_181] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [?:1.8.0_181] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [?:1.8.0_181] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [?:1.8.0_181] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_181] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_181] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181] > 2018-12-21 19:44:46.518 INFO (qtp817348612-178) [ ] o.a.s.s.HttpSolrCall > [admin] webapp=null path=/admin/metrics > params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} > status=0 QTime=1 > 2018-12-21 19:44:46.530 WARN (MetricsHistoryHandler-12-thread-1) [ ] > o.a.s.c.s.i.SolrClientNodeStateProvider could not get tags from node > usahubslvcvw122.usa.doj.gov:8983_solr > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at http://usahubslvcvw122.usa.doj.gov:8983/solr: Expected mime > type application/octet-stream but got text/html. <h1>Bad Message > 414</h1><pre>reason: URI Too Long</pre> > at > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:292) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchMetrics(SolrClientNodeStateProvider.java:150) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getReplicaInfo(SolrClientNodeStateProvider.java:131) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:478) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:368) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:230) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [?:1.8.0_181] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [?:1.8.0_181] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [?:1.8.0_181] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [?:1.8.0_181] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_181] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_181] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181] > 2018-12-21 19:44:46.553 WARN (MetricsHistoryHandler-12-thread-1) [ ] > o.a.s.c.s.i.SolrClientNodeStateProvider could not get tags from node > usahubslvcvw123.usa.doj.gov:8983_solr > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at http://usahubslvcvw123.usa.doj.gov:8983/solr: Expected mime > type application/octet-stream but got text/html. <h1>Bad Message > 414</h1><pre>reason: URI Too Long</pre> > at > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:292) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchMetrics(SolrClientNodeStateProvider.java:150) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getReplicaInfo(SolrClientNodeStateProvider.java:131) > ~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:14] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:478) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:368) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:230) > ~[solr-core-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc - > jpountz - 2018-06-18 16:55:13] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [?:1.8.0_181] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [?:1.8.0_181] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [?:1.8.0_181] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [?:1.8.0_181] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_181] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_181] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181] >