I set up Solr Cloud 7.7.2 and follow the tutorial
https://lucene.apache.org/solr/guide/8_2/getting-started-with-solrcloud.html#getting-started-with-solrcloud
I still see exactly the same behavior, even with 0 records and out of the
box. Why would it generates 40.000 Futex syscall in 30 seconds even if
there's no record on it ?
On Fri, Nov 1, 2019 at 9:15 AM Daniel de Oliveira Mantovani <
daniel.oliveira.mantov...@gmail.com> wrote:
> Hello Shalin,
>
> It's not an OS bug,
> Linux samba 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019
> x86_64 x86_64 x86_64 GNU/Linux
>
> I'm sending the full thread dump log:
> 2019-11-01 11:05:13
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.231-b11 mixed mode):
>
> "Attach Listener" #259 daemon prio=9 os_prio=0 tid=0x00007fcb78001000
> nid=0x2659 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-258" #258 prio=5 os_prio=0 tid=0x00007fcabc321000
> nid=0x25c7 waiting on condition [0x00007fcb356ca000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000005a626cdd8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-257" #257 prio=5 os_prio=0 tid=0x00007fcb24001000
> nid=0x25bc waiting on condition [0x00007fcb35b95000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f00242c0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-256" #256 prio=5 os_prio=0 tid=0x00007fca88003000
> nid=0x2594 runnable [0x00007fcb37441000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a34218> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a34200> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0931118> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-254" #254 prio=5 os_prio=0 tid=0x00007fca88005800
> nid=0x2570 runnable [0x00007fcb3580f000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a2fdd8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a2fdc0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0243070> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-252" #252 prio=5 os_prio=0 tid=0x00007fca90008000
> nid=0x2441 runnable [0x00007fc59c444000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a30180> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a30168> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0243220> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-250" #250 prio=5 os_prio=0 tid=0x00007fca90001000
> nid=0x243d runnable [0x00007fcb3570b000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a313c8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a313b0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f06da640> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-249" #249 prio=5 os_prio=0 tid=0x00007fca94001000
> nid=0x2435 runnable [0x00007fcb373bf000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a30528> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a30510> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895ec0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-247" #247 prio=5 os_prio=0 tid=0x00007fcb18006000
> nid=0x2428 runnable [0x00007fcb372bb000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a31770> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a31758> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f06da6d0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-245" #245 prio=5 os_prio=0 tid=0x00007fcb08004800
> nid=0x2417 runnable [0x00007fcb36060000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a31b18> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a31b00> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f06da5b0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-243" #243 prio=5 os_prio=0 tid=0x00007fcae001f000
> nid=0x240c runnable [0x00007fcb35607000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a31ec0> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a31ea8> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f06da490> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-241" #241 prio=5 os_prio=0 tid=0x00007fcb14001000
> nid=0x23ff runnable [0x00007fcb3727a000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f08aabe8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f08aabd0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0242a88> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-239" #239 prio=5 os_prio=0 tid=0x00007fcb0c001000
> nid=0x23f1 runnable [0x00007fcb35648000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f09c4028> (a sun.nio.ch.Util$3)
> - locked <0x00000004f09c4010> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895cc8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-237" #237 prio=5 os_prio=0 tid=0x00007fca80001000
> nid=0x23de runnable [0x00007fc59ca8d000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a34098> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a34080> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0930ed8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-235" #235 prio=5 os_prio=0 tid=0x00007fc588026000
> nid=0x23d3 runnable [0x00007fcb35e99000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f08aac60> (a sun.nio.ch.Util$3)
> - locked <0x00000004f08aac48> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f02429f8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-233" #233 prio=5 os_prio=0 tid=0x00007fcacc021000
> nid=0x23c6 runnable [0x00007fcb35585000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f08aacc0> (a sun.nio.ch.Util$3)
> - locked <0x00000004f08aaca8> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0242ba8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-231" #231 prio=5 os_prio=0 tid=0x00007fc554004800
> nid=0x23b9 runnable [0x00007fcb35891000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f08aade0> (a sun.nio.ch.Util$3)
> - locked <0x00000004f08aadc8> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0242ad0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-228" #228 prio=5 os_prio=0 tid=0x00007fcacc015800
> nid=0x23a4 runnable [0x00007fcb357ce000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a340f8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a340e0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0930f68> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-226" #226 prio=5 os_prio=0 tid=0x00007fca98005000
> nid=0x2394 runnable [0x00007fcb35440000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f09c40e8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f09c40d0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895ba8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-224" #224 prio=5 os_prio=0 tid=0x00007fca60016800
> nid=0x2388 runnable [0x00007fcb37482000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a34158> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a34140> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0930ff8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-222" #222 prio=5 os_prio=0 tid=0x00007fcb2c003000
> nid=0x2378 runnable [0x00007fcb355c6000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f09c4148> (a sun.nio.ch.Util$3)
> - locked <0x00000004f09c4130> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895b18> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-220" #220 prio=5 os_prio=0 tid=0x00007fca78003000
> nid=0x236a runnable [0x00007fcb374c3000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a341b8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a341a0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0931088> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-218" #218 prio=5 os_prio=0 tid=0x00007fcb28002000
> nid=0x235f runnable [0x00007fcb360a1000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f09c41a8> (a sun.nio.ch.Util$3)
> - locked <0x00000004f09c4190> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895a88> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-216" #216 prio=5 os_prio=0 tid=0x00007fcaec002000
> nid=0x2353 waiting on condition [0x00007fc59c84c000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000405560e38> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-214" #214 prio=5 os_prio=0 tid=0x00007fc58c001800
> nid=0x2341 waiting on condition [0x00007fcb37400000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f00242c0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-202" #202 prio=5 os_prio=0 tid=0x00007fca88001000
> nid=0x22f3 runnable [0x00007fcb358d2000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a31020> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a31008> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895d10> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-200" #200 prio=5 os_prio=0 tid=0x00007fcaa8002800
> nid=0x22e7 runnable [0x00007fcb3574c000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a308d0> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a308b8> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895e30> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-198" #198 prio=5 os_prio=0 tid=0x00007fca90007800
> nid=0x22d7 runnable [0x00007fcb35689000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0a30c78> (a sun.nio.ch.Util$3)
> - locked <0x00000004f0a30c60> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895da0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "Scheduler-842741472" #89 prio=5 os_prio=0 tid=0x00007fca60007000
> nid=0x1c62 waiting on condition [0x00007fc59cace000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f0941af0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "SolrRrdBackendFactory-11-thread-2" #87 prio=5 os_prio=0
> tid=0x00007fcac0001000 nid=0x1c60 waiting on condition [0x00007fcb35bd6000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f0996978> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "searcherExecutor-15-thread-1" #83 prio=5 os_prio=0 tid=0x00007fcab86b5800
> nid=0x18a7 waiting on condition [0x00007fcb34e21000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f45eb9f8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "searcherExecutor-14-thread-1" #84 prio=5 os_prio=0 tid=0x00007fcabc9c1000
> nid=0x18a6 waiting on condition [0x00007fcb34de0000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f45eb710> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-78-acceptor-3@6942c593-ServerConnector
> @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983}" #78 prio=5 os_prio=0
> tid=0x00007fca68001000 nid=0x1888 runnable [0x00007fcb35481000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> - locked <0x00000004f08aab70> (a java.lang.Object)
> at
> org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
> at
> org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-77-acceptor-2@26493f4e-ServerConnector
> @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983}" #77 prio=5 os_prio=0
> tid=0x00007fca74001000 nid=0x1886 waiting for monitor entry
> [0x00007fcb354c2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:234)
> - waiting to lock <0x00000004f08aab70> (a java.lang.Object)
> at
> org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
> at
> org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-76-acceptor-1@292b8c3e-ServerConnector
> @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983}" #76 prio=5 os_prio=0
> tid=0x00007fca70001000 nid=0x1885 waiting for monitor entry
> [0x00007fcb35503000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:234)
> - waiting to lock <0x00000004f08aab70> (a java.lang.Object)
> at
> org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
> at
> org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-75-acceptor-0@10c4f96f-ServerConnector
> @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983}" #75 prio=5 os_prio=0
> tid=0x00007fca7c001000 nid=0x1880 waiting for monitor entry
> [0x00007fcb35544000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:234)
> - waiting to lock <0x00000004f08aab70> (a java.lang.Object)
> at
> org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
> at
> org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "qtp1571967156-66" #66 prio=5 os_prio=0 tid=0x00007fca98001000 nid=0x186f
> runnable [0x00007fcb3578d000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f09c4088> (a sun.nio.ch.Util$3)
> - locked <0x00000004f09c4070> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0895c38> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
> at
> org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "DestroyJavaVM" #61 prio=5 os_prio=0 tid=0x00007fcbd800e800 nid=0x175c
> waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Thread-13" #58 prio=5 os_prio=0 tid=0x00007fcbd8f3f800 nid=0x1866 in
> Object.wait() [0x00007fcb35b54000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.solr.core.CloserThread.run(CoreContainer.java:1850)
> - locked <0x00000004f086a0f8> (a java.lang.Object)
>
> Locked ownable synchronizers:
> - None
>
> "MetricsHistoryHandler-12-thread-1" #55 prio=5 os_prio=0
> tid=0x00007fcbd8ed3000 nid=0x185e waiting on condition [0x00007fcb35c17000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f08abf80> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "SolrRrdBackendFactory-11-thread-1" #54 prio=5 os_prio=0
> tid=0x00007fcbd8ed2000 nid=0x185d waiting on condition [0x00007fcb35c58000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f0996978> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "OverseerAutoScalingTriggerThread-72060990031069185-192.168.1.40:8983_solr-n_0000001018"
> #52 prio=5 os_prio=0 tid=0x00007fcbd83d8000 nid=0x185b waiting on condition
> [0x00007fcb35eda000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f08ac8e0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "OverseerCollectionConfigSetProcessor-72060990031069185-192.168.1.40:8983_solr-n_0000001018"
> #51 daemon prio=5 os_prio=0 tid=0x00007fcbd83d6800 nid=0x185a runnable
> [0x00007fcb35f1b000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f09c7b38> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)
> at
> org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:243)
> at
> org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:233)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "OverseerStateUpdate-72060990031069185-192.168.1.40:8983_solr-n_0000001018"
> #47 daemon prio=5 os_prio=0 tid=0x00007fcbd83d5800 nid=0x1859 runnable
> [0x00007fcb35f5c000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f0a2ef20> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)
> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:207)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "zkConnectionManagerCallback-13-thread-1" #50 prio=5 os_prio=0
> tid=0x00007fcad8001000 nid=0x1858 waiting on condition [0x00007fcb35f9d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f08ac430> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "main-EventThread" #49 daemon prio=5 os_prio=0 tid=0x00007fcbd8941800
> nid=0x1857 waiting on condition [0x00007fcb35fde000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f0997bd8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
>
> Locked ownable synchronizers:
> - None
>
> "main-SendThread(localhost:2181)" #48 daemon prio=5 os_prio=0
> tid=0x00007fcbd8948000 nid=0x1856 runnable [0x00007fcb3601f000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f0997908> (a sun.nio.ch.Util$3)
> - locked <0x00000004f09978f0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f0242f08> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)
>
> Locked ownable synchronizers:
> - None
>
> "zkConnectionManagerCallback-6-thread-1" #44 prio=5 os_prio=0
> tid=0x00007fcaf4001000 nid=0x17a1 waiting on condition [0x00007fcb362e2000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f08a9178> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "main-EventThread" #43 daemon prio=5 os_prio=0 tid=0x00007fcbd8db8800
> nid=0x17a0 waiting on condition [0x00007fcb36323000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f08a95c8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
>
> Locked ownable synchronizers:
> - None
>
> "main-SendThread(samba:2181)" #42 daemon prio=5 os_prio=0
> tid=0x00007fcbd8db7000 nid=0x179f runnable [0x00007fcb36364000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0x00000004f08a9898> (a sun.nio.ch.Util$3)
> - locked <0x00000004f08a9880> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00000004f03ea3a8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)
>
> Locked ownable synchronizers:
> - None
>
> "Connection evictor" #41 daemon prio=5 os_prio=0 tid=0x00007fcbd8daf800
> nid=0x179e waiting on condition [0x00007fcb36c79000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "Connection evictor" #40 daemon prio=5 os_prio=0 tid=0x00007fcbd8dae000
> nid=0x179d waiting on condition [0x00007fcb36c38000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "Connection evictor" #39 daemon prio=5 os_prio=0 tid=0x00007fcbd8da5000
> nid=0x179c waiting on condition [0x00007fcb36cba000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "Scheduler-1456339771" #35 prio=5 os_prio=0 tid=0x00007fcbd8ae3800
> nid=0x178e waiting on condition [0x00007fcb36efb000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000004f02dc358> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "ShutdownMonitor" #24 daemon prio=5 os_prio=0 tid=0x00007fcbd86e4800
> nid=0x1783 runnable [0x00007fcb37704000]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
> at java.net.ServerSocket.implAccept(ServerSocket.java:545)
> at java.net.ServerSocket.accept(ServerSocket.java:513)
> at
> org.eclipse.jetty.server.ShutdownMonitor$ShutdownMonitorRunnable.run(ShutdownMonitor.java:335)
> at java.lang.Thread.run(Thread.java:748)
>
> Locked ownable synchronizers:
> - None
>
> "Service Thread" #21 daemon prio=9 os_prio=0 tid=0x00007fcbd8153800
> nid=0x177b runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C1 CompilerThread14" #20 daemon prio=9 os_prio=0 tid=0x00007fcbd8150800
> nid=0x177a waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C1 CompilerThread13" #19 daemon prio=9 os_prio=0 tid=0x00007fcbd814e000
> nid=0x1779 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C1 CompilerThread12" #18 daemon prio=9 os_prio=0 tid=0x00007fcbd814c800
> nid=0x1778 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C1 CompilerThread11" #17 daemon prio=9 os_prio=0 tid=0x00007fcbd814a000
> nid=0x1777 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C1 CompilerThread10" #16 daemon prio=9 os_prio=0 tid=0x00007fcbd8148000
> nid=0x1776 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread9" #15 daemon prio=9 os_prio=0 tid=0x00007fcbd8146800
> nid=0x1775 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread8" #14 daemon prio=9 os_prio=0 tid=0x00007fcbd8144000
> nid=0x1774 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread7" #13 daemon prio=9 os_prio=0 tid=0x00007fcbd8141800
> nid=0x1773 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread6" #12 daemon prio=9 os_prio=0 tid=0x00007fcbd813f800
> nid=0x1772 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread5" #11 daemon prio=9 os_prio=0 tid=0x00007fcbd813d800
> nid=0x1771 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread4" #10 daemon prio=9 os_prio=0 tid=0x00007fcbd813b000
> nid=0x1770 waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread3" #9 daemon prio=9 os_prio=0 tid=0x00007fcbd8131000
> nid=0x176f waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread2" #8 daemon prio=9 os_prio=0 tid=0x00007fcbd812f000
> nid=0x176e waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread1" #7 daemon prio=9 os_prio=0 tid=0x00007fcbd812c800
> nid=0x176d waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x00007fcbd812a800
> nid=0x176c runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Signal Dispatcher" #5 daemon prio=9 os_prio=0 tid=0x00007fcbd8128800
> nid=0x176b runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=0
> tid=0x00007fcbd8127000 nid=0x176a waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fcbd80f6800 nid=0x1769
> in Object.wait() [0x00007fcbb807d000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
> - locked <0x00000004f03573a0> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
>
> Locked ownable synchronizers:
> - None
>
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fcbd80f2000
> nid=0x1768 in Object.wait() [0x00007fcbdc072000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0x00000004f03574a8> (a java.lang.ref.Reference$Lock)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
>
> Locked ownable synchronizers:
> - None
>
> "VM Thread" os_prio=0 tid=0x00007fcbd80e8800 nid=0x1766 runnable
>
> "Gang worker#0 (Parallel GC Threads)" os_prio=0 tid=0x00007fcbd8015000
> nid=0x175d runnable
>
> "Gang worker#1 (Parallel GC Threads)" os_prio=0 tid=0x00007fcbd8021800
> nid=0x175e runnable
>
> "Gang worker#2 (Parallel GC Threads)" os_prio=0 tid=0x00007fcbd8023000
> nid=0x175f runnable
>
> "Gang worker#3 (Parallel GC Threads)" os_prio=0 tid=0x00007fcbd8024800
> nid=0x1760 runnable
>
> "Concurrent Mark-Sweep GC Thread" os_prio=0 tid=0x00007fcbd806e800
> nid=0x1765 runnable
>
> "Gang worker#0 (Parallel CMS Threads)" os_prio=0 tid=0x00007fcbd8067800
> nid=0x1761 runnable
>
> "Gang worker#1 (Parallel CMS Threads)" os_prio=0 tid=0x00007fcbd8069000
> nid=0x1762 runnable
>
> "Gang worker#2 (Parallel CMS Threads)" os_prio=0 tid=0x00007fcbd806b000
> nid=0x1763 runnable
>
> "Gang worker#3 (Parallel CMS Threads)" os_prio=0 tid=0x00007fcbd806c800
> nid=0x1764 runnable
>
> "VM Periodic Task Thread" os_prio=0 tid=0x00007fcbd8156000 nid=0x177c
> waiting on condition
>
> JNI global references: 1002
>
>
>
> Thank you
>
> On Fri, Nov 1, 2019 at 8:26 AM Shalin Shekhar Mangar <
> shalinman...@gmail.com> wrote:
>
>> Perhaps this bug in the kernel?
>> https://groups.google.com/forum/#!topic/mechanical-sympathy/QbmpZxp6C64
>>
>> Can you check if you are running an affected kernel?
>>
>> On Fri, Nov 1, 2019 at 7:09 AM Daniel de Oliveira Mantovani <
>> daniel.oliveira.mantov...@gmail.com> wrote:
>>
>> > I have a Solr 7.6.0 Cloud with 4 nodes and while indexing Solr hangs.
>> >
>> > $ java -version
>> > java version "1.8.0_231"
>> > Java(TM) SE Runtime Environment (build 1.8.0_231-b11)
>> > Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode
>> >
>> > $ sudo strace -f -c -p 5972
>> > % time seconds usecs/call calls errors syscall
>> > ------ ----------- ----------- --------- --------- ----------------
>> > 71.02 33.329124 662 50335 13317 futex
>> > 21.43 10.055086 335170 30 28 restart_syscall
>> > 7.49 3.512846 14760 238 epoll_wait
>> > 0.07 0.030526 2544 12 poll
>> > 0.00 0.001571 5 322 write
>> > 0.00 0.000439 4 117 times
>> > 0.00 0.000355 2 183 4 read
>> > 0.00 0.000236 3 72 mprotect
>> > 0.00 0.000161 1 118 epoll_ctl
>> > 0.00 0.000126 0 352 stat
>> > 0.00 0.000089 11 8 getdents
>> > 0.00 0.000077 10 8 sendto
>> > 0.00 0.000069 69 1 statfs
>> > 0.00 0.000043 22 2 writev
>> > 0.00 0.000029 4 7 openat
>> > 0.00 0.000021 4 6 fstat
>> > 0.00 0.000018 1 22 lstat
>> > 0.00 0.000018 9 2 dup
>> > 0.00 0.000010 1 9 close
>> > 0.00 0.000010 3 4 fcntl
>> > 0.00 0.000005 3 2 access
>> > 0.00 0.000002 2 1 mmap
>> > 0.00 0.000002 0 26 sched_yield
>> > 0.00 0.000001 0 8 recvfrom
>> > 0.00 0.000001 1 2 readlink
>> > ------ ----------- ----------- --------- --------- ----------------
>> > 100.00 46.930865 51887 13349 total
>> >
>> > $ sudo strace -f -p 5972
>> > strace: Process 5972 attached with 86 threads
>> > [pid 7524] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 7523] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 7511] epoll_wait(142, <unfinished ...>
>> > [pid 7471] epoll_wait(160, <unfinished ...>
>> > [pid 7266] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 7267] epoll_wait(151, <unfinished ...>
>> > [pid 7264] futex(0x7fcbd8f3eb98, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 7265] epoll_wait(157, <unfinished ...>
>> > [pid 6311] futex(0x7fcabc8c42f8, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6310] futex(0x7fcab843c998, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6281] epoll_wait(154, <unfinished ...>
>> > [pid 6280] futex(0x7fca68001d7c, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6278] futex(0x7fca74001d78, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6277] accept(136, <unfinished ...>
>> > [pid 6272] futex(0x7fca7c001d7c, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6271] epoll_wait(208, <unfinished ...>
>> > [pid 6270] epoll_wait(205, <unfinished ...>
>> > [pid 6262] epoll_wait(199, <unfinished ...>
>> > [pid 6261] epoll_wait(196, <unfinished ...>
>> > [pid 6258] epoll_wait(193, <unfinished ...>
>> > [pid 6269] epoll_wait(202, <unfinished ...>
>> > [pid 6257] epoll_wait(190, <unfinished ...>
>> > [pid 6256] epoll_wait(187, <unfinished ...>
>> > [pid 6255] epoll_wait(184, <unfinished ...>
>> > [pid 6253] epoll_wait(178, <unfinished ...>
>> > [pid 6250] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 6252] epoll_wait(175, <unfinished ...>
>> > [pid 6251] epoll_wait(172, <unfinished ...>
>> > [pid 6254] epoll_wait(181, <unfinished ...>
>> > [pid 6246] futex(0x7fcbd8f3f378, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6238] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 6237] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 6235] futex(0x7fcbd83d97b8, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6234] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 6233] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 6232] futex(0x7fcad8002608, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6231] futex(0x7fcbd89425ac, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6049] futex(0x7fcaf4002388, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6230] epoll_wait(134, <unfinished ...>
>> > [pid 6048] futex(0x7fcbd8db9ac8, FUTEX_WAIT_PRIVATE, 0, NULL
>> <unfinished
>> > ...>
>> > [pid 6047] epoll_wait(130, <unfinished ...>
>> > [pid 6046] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> > [pid 6045] restart_syscall(<... resuming interrupted futex ...>
>> > <unfinished ...>
>> >
>> > JVM args:
>> >
>> >
>> > -
>> >
>> -DSTOP.KEY=solrrocks-DSTOP.PORT=7983-Djetty.home=/opt/solr/server-Djetty.port=8983-Dlog4j.configurationFile=file:/opt/solr/server/resources/log4j2.xml-Dsolr.data.home=-Dsolr.default.confdir=/opt/solr/server/solr/configsets/_default/conf-Dsolr.install.dir=/opt/solr-Dsolr.jetty.https.port=8983-Dsolr.log.dir=/opt/solr/example/cloud/node1/solr/../logs-Dsolr.log.muteconsole-Dsolr.solr.home=/opt/solr/example/cloud/node1/solr-Duser.timezone=UTC-DzkClientTimeout=15000-DzkHost=localhost:2181-XX:+CMSParallelRemarkEnabled-XX:+CMSScavengeBeforeRemark-XX:+ParallelRefProcEnabled-XX:+PrintGCApplicationStoppedTime-XX:+PrintGCDateStamps-XX:+PrintGCDetails-XX:+PrintGCTimeStamps-XX:+PrintHeapAtGC-XX:+PrintTenuringDistribution-XX:+UseCMSInitiatingOccupancyOnly-XX:+UseConcMarkSweepGC-XX:+UseGCLogFileRotation-XX:-OmitStackTraceInFastThrow-XX:CMSInitiatingOccupancyFraction=50-XX:CMSMaxAbortablePrecleanTime=6000-XX:ConcGCThreads=4-XX:GCLogFileSize=20M-XX:MaxTenuringThreshold=8-XX:NewRatio=3-XX:NumberOfGCLogFiles=9-XX:OnOutOfMemoryError=/opt/solr/bin/oom_solr.sh
>> > 8983
>> >
>> >
>> /opt/solr/example/cloud/node1/solr/../logs-XX:ParallelGCThreads=4-XX:PretenureSizeThreshold=64m-XX:SurvivorRatio=4-XX:TargetSurvivorRatio=90-Xloggc:/opt/solr/example/cloud/node1/solr/../logs/solr_gc.log-Xms512m-Xmx15g-Xss256k-verbose:gc
>> >
>> > Java Thread Dumb:
>> > name
>> > cpuTime / userTime
>> > process reaper (135)
>> > java.util.concurrent.SynchronousQueue$TransferStack@70b8631c
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > • java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill
>> > (SynchronousQueue.java:460)
>> > • java.util.concurrent.SynchronousQueue$TransferStack.transfer
>> > (SynchronousQueue.java:362)
>> > • java.util.concurrent.SynchronousQueue.poll (SynchronousQueue.java:941)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1073)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 17.2219ms
>> > 0.0000ms
>> > qtp1571967156-129 (129)
>> > 36.0159ms
>> > 0.0000ms
>> > process reaper (117)
>> > java.util.concurrent.SynchronousQueue$TransferStack@70b8631c
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > • java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill
>> > (SynchronousQueue.java:460)
>> > • java.util.concurrent.SynchronousQueue$TransferStack.transfer
>> > (SynchronousQueue.java:362)
>> > • java.util.concurrent.SynchronousQueue.poll (SynchronousQueue.java:941)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1073)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 19.2751ms
>> > 0.0000ms
>> > qtp1571967156-116 (116)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 1.5766ms
>> > 0.0000ms
>> > qtp1571967156-115 (115)
>> > • sun.management.ThreadImpl.getThreadInfo1 (Native Method)
>> > • sun.management.ThreadImpl.getThreadInfo (ThreadImpl.java:176)
>> > • org.apache.solr.handler.admin.ThreadDumpHandler.handleRequestBody
>> > (ThreadDumpHandler.java:71)
>> > • org.apache.solr.handler.RequestHandlerBase.handleRequest
>> > (RequestHandlerBase.java:199)
>> > • org.apache.solr.handler.admin.InfoHandler.handle (InfoHandler.java:91)
>> > • org.apache.solr.handler.admin.InfoHandler.handleRequestBody
>> > (InfoHandler.java:81)
>> > • org.apache.solr.handler.RequestHandlerBase.handleRequest
>> > (RequestHandlerBase.java:199)
>> > • org.apache.solr.servlet.HttpSolrCall.handleAdmin
>> (HttpSolrCall.java:734)
>> > • org.apache.solr.servlet.HttpSolrCall.handleAdminRequest
>> > (HttpSolrCall.java:715)
>> > • org.apache.solr.servlet.HttpSolrCall.call (HttpSolrCall.java:496)
>> > • org.apache.solr.servlet.SolrDispatchFilter.doFilter
>> > (SolrDispatchFilter.java:377)
>> > • org.apache.solr.servlet.SolrDispatchFilter.doFilter
>> > (SolrDispatchFilter.java:323)
>> > • org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter
>> > (ServletHandler.java:1634)
>> > • org.eclipse.jetty.servlet.ServletHandler.doHandle
>> > (ServletHandler.java:533)
>> > • org.eclipse.jetty.server.handler.ScopedHandler.handle
>> > (ScopedHandler.java:146)
>> > • org.eclipse.jetty.security.SecurityHandler.handle
>> > (SecurityHandler.java:548)
>> > • org.eclipse.jetty.server.handler.HandlerWrapper.handle
>> > (HandlerWrapper.java:132)
>> > • org.eclipse.jetty.server.handler.ScopedHandler.nextHandle
>> > (ScopedHandler.java:257)
>> > • org.eclipse.jetty.server.session.SessionHandler.doHandle
>> > (SessionHandler.java:1595)
>> > • org.eclipse.jetty.server.handler.ScopedHandler.nextHandle
>> > (ScopedHandler.java:255)
>> > • org.eclipse.jetty.server.handler.ContextHandler.doHandle
>> > (ContextHandler.java:1317)
>> > • org.eclipse.jetty.server.handler.ScopedHandler.nextScope
>> > (ScopedHandler.java:203)
>> > • org.eclipse.jetty.servlet.ServletHandler.doScope
>> > (ServletHandler.java:473)
>> > • org.eclipse.jetty.server.session.SessionHandler.doScope
>> > (SessionHandler.java:1564)
>> > • org.eclipse.jetty.server.handler.ScopedHandler.nextScope
>> > (ScopedHandler.java:201)
>> > • org.eclipse.jetty.server.handler.ContextHandler.doScope
>> > (ContextHandler.java:1219)
>> > • org.eclipse.jetty.server.handler.ScopedHandler.handle
>> > (ScopedHandler.java:144)
>> > • org.eclipse.jetty.server.handler.ContextHandlerCollection.handle
>> > (ContextHandlerCollection.java:219)
>> > • org.eclipse.jetty.server.handler.HandlerCollection.handle
>> > (HandlerCollection.java:126)
>> > • org.eclipse.jetty.server.handler.HandlerWrapper.handle
>> > (HandlerWrapper.java:132)
>> > • org.eclipse.jetty.rewrite.handler.RewriteHandler.handle
>> > (RewriteHandler.java:335)
>> > • org.eclipse.jetty.server.handler.HandlerWrapper.handle
>> > (HandlerWrapper.java:132)
>> > • org.eclipse.jetty.server.Server.handle (Server.java:531)
>> > • org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:352)
>> > • org.eclipse.jetty.server.HttpConnection.onFillable
>> > (HttpConnection.java:260)
>> > • org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded
>> > (AbstractConnection.java:281)
>> > • org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:102)
>> > • org.eclipse.jetty.io.ChannelEndPoint$2.run (ChannelEndPoint.java:118)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask
>> > (EatWhatYouKill.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:310)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 81.0773ms
>> > 70.0000ms
>> > qtp1571967156-114 (114)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 14.7759ms
>> > 10.0000ms
>> > qtp1571967156-113 (113)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 1.7541ms
>> > 0.0000ms
>> > qtp1571967156-112 (112)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 28.2561ms
>> > 10.0000ms
>> > qtp1571967156-109 (109)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 20.5324ms
>> > 10.0000ms
>> > qtp1571967156-107 (107)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 13.4180ms
>> > 0.0000ms
>> > qtp1571967156-90 (90)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 108.5163ms
>> > 100.0000ms
>> > Scheduler-842741472 (89)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27022346
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:1093)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:809)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 5.4794ms
>> > 0.0000ms
>> > qtp1571967156-88 (88)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 38.6417ms
>> > 10.0000ms
>> > SolrRrdBackendFactory-11-thread-2 (87)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7813101a
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:1093)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:809)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 3.3288ms
>> > 0.0000ms
>> > searcherExecutor-15-thread-1 (83)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4004f6b2
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.LinkedBlockingQueue.take
>> > (LinkedBlockingQueue.java:442)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 8.9663ms
>> > 0.0000ms
>> > searcherExecutor-14-thread-1 (84)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7397cd2d
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.LinkedBlockingQueue.take
>> > (LinkedBlockingQueue.java:442)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 10.0677ms
>> > 0.0000ms
>> > qtp1571967156-79 (79)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 110.6471ms
>> > 90.0000ms
>> > qtp1571967156-78-acceptor-3@6942c593-ServerConnector
>> > @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983} (78)
>> > java.lang.Object@4539d80a
>> > • sun.nio.ch.ServerSocketChannelImpl.accept
>> > (ServerSocketChannelImpl.java:234)
>> > • org.eclipse.jetty.server.ServerConnector.accept
>> > (ServerConnector.java:369)
>> > • org.eclipse.jetty.server.AbstractConnector$Acceptor.run
>> > (AbstractConnector.java:639)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 4.9023ms
>> > 0.0000ms
>> > qtp1571967156-77-acceptor-2@26493f4e-ServerConnector
>> > @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983} (77)
>> > java.lang.Object@4539d80a
>> > • sun.nio.ch.ServerSocketChannelImpl.accept
>> > (ServerSocketChannelImpl.java:234)
>> > • org.eclipse.jetty.server.ServerConnector.accept
>> > (ServerConnector.java:369)
>> > • org.eclipse.jetty.server.AbstractConnector$Acceptor.run
>> > (AbstractConnector.java:639)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 5.0835ms
>> > 0.0000ms
>> > qtp1571967156-76-acceptor-1@292b8c3e-ServerConnector
>> > @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983} (76)
>> > • sun.nio.ch.ServerSocketChannelImpl.accept0 (Native Method)
>> > • sun.nio.ch.ServerSocketChannelImpl.accept
>> > (ServerSocketChannelImpl.java:422)
>> > • sun.nio.ch.ServerSocketChannelImpl.accept
>> > (ServerSocketChannelImpl.java:250)
>> > • org.eclipse.jetty.server.ServerConnector.accept
>> > (ServerConnector.java:369)
>> > • org.eclipse.jetty.server.AbstractConnector$Acceptor.run
>> > (AbstractConnector.java:639)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 5.6738ms
>> > 0.0000ms
>> > qtp1571967156-75-acceptor-0@10c4f96f-ServerConnector
>> > @362a019c{HTTP/1.1,[http/1.1]}{0.0.0.0:8983} (75)
>> > java.lang.Object@4539d80a
>> > • sun.nio.ch.ServerSocketChannelImpl.accept
>> > (ServerSocketChannelImpl.java:234)
>> > • org.eclipse.jetty.server.ServerConnector.accept
>> > (ServerConnector.java:369)
>> > • org.eclipse.jetty.server.AbstractConnector$Acceptor.run
>> > (AbstractConnector.java:639)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 4.9762ms
>> > 0.0000ms
>> > qtp1571967156-74 (74)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@25ca8377
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • org.eclipse.jetty.util.BlockingArrayQueue.poll
>> > (BlockingArrayQueue.java:392)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll
>> > (QueuedThreadPool.java:653)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.access$800
>> > (QueuedThreadPool.java:48)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:717)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 4.6754ms
>> > 0.0000ms
>> > qtp1571967156-73 (73)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@25ca8377
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • org.eclipse.jetty.util.BlockingArrayQueue.poll
>> > (BlockingArrayQueue.java:392)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll
>> > (QueuedThreadPool.java:653)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.access$800
>> > (QueuedThreadPool.java:48)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:717)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 6.0771ms
>> > 0.0000ms
>> > qtp1571967156-72 (72)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 28.1516ms
>> > 10.0000ms
>> > qtp1571967156-71 (71)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 31.7190ms
>> > 10.0000ms
>> > qtp1571967156-70 (70)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 22.8869ms
>> > 0.0000ms
>> > qtp1571967156-69 (69)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@25ca8377
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • org.eclipse.jetty.util.BlockingArrayQueue.poll
>> > (BlockingArrayQueue.java:392)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll
>> > (QueuedThreadPool.java:653)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.access$800
>> > (QueuedThreadPool.java:48)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:717)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 5.6719ms
>> > 0.0000ms
>> > qtp1571967156-68 (68)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 7.0843ms
>> > 0.0000ms
>> > qtp1571967156-67 (67)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 25.2077ms
>> > 10.0000ms
>> > qtp1571967156-66 (66)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 59.3490ms
>> > 40.0000ms
>> > qtp1571967156-65 (65)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 77.9195ms
>> > 60.0000ms
>> > qtp1571967156-64 (64)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 44.4205ms
>> > 30.0000ms
>> > qtp1571967156-63 (63)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:101)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select
>> > (ManagedSelector.java:396)
>> > • org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce
>> > (ManagedSelector.java:333)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask
>> > (EatWhatYouKill.java:357)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce
>> > (EatWhatYouKill.java:181)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce
>> > (EatWhatYouKill.java:168)
>> > • org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run
>> > (EatWhatYouKill.java:126)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:366)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 129.4624ms
>> > 90.0000ms
>> > qtp1571967156-62 (62)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@421090bf
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2163)
>> > •
>> >
>> >
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait
>> > (ReservedThreadExecutor.java:292)
>> > •
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run
>> > (ReservedThreadExecutor.java:357)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool.runJob
>> > (QueuedThreadPool.java:762)
>> > • org.eclipse.jetty.util.thread.QueuedThreadPool$2.run
>> > (QueuedThreadPool.java:680)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 51.2605ms
>> > 30.0000ms
>> > DestroyJavaVM (61)
>> > •
>> > 2002.8222ms
>> > 1850.0000ms
>> > Thread-13 (58)
>> > java.lang.Object@f2f13c1
>> > • java.lang.Object.wait (Native Method)
>> > • java.lang.Object.wait (Object.java:502)
>> > • org.apache.solr.core.CloserThread.run (CoreContainer.java:1850)
>> > 0.7894ms
>> > 0.0000ms
>> > MetricsHistoryHandler-12-thread-1 (55)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@45d73a6
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:1093)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:809)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 348.4029ms
>> > 310.0000ms
>> > SolrRrdBackendFactory-11-thread-1 (54)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7813101a
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:1088)
>> > • java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take
>> > (ScheduledThreadPoolExecutor.java:809)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 5.7281ms
>> > 0.0000ms
>> >
>> >
>> OverseerAutoScalingTriggerThread-72060990031069185-192.168.1.40:8983_solr-n_0000001018
>> > (52)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@75fd9961
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run
>> > (OverseerTriggerThread.java:189)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 58.1405ms
>> > 50.0000ms
>> >
>> >
>> OverseerCollectionConfigSetProcessor-72060990031069185-192.168.1.40:8983_solr-n_0000001018
>> > (51)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@55c10b44
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • org.apache.solr.cloud.ZkDistributedQueue.peekElements
>> > (ZkDistributedQueue.java:463)
>> > • org.apache.solr.cloud.OverseerTaskQueue.peekTopN
>> > (OverseerTaskQueue.java:243)
>> > • org.apache.solr.cloud.OverseerTaskProcessor.run
>> > (OverseerTaskProcessor.java:233)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 220.6397ms
>> > 170.0000ms
>> >
>> OverseerStateUpdate-72060990031069185-192.168.1.40:8983_solr-n_0000001018
>> > (47)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2287ab26
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.parkNanos
>> (LockSupport.java:215)
>> > •
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos
>> > (AbstractQueuedSynchronizer.java:2078)
>> > • org.apache.solr.cloud.ZkDistributedQueue.peekElements
>> > (ZkDistributedQueue.java:463)
>> > • org.apache.solr.cloud.Overseer$ClusterStateUpdater.run
>> > (Overseer.java:207)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 189.5649ms
>> > 140.0000ms
>> > zkConnectionManagerCallback-13-thread-1 (50)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4da83c93
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.LinkedBlockingQueue.take
>> > (LinkedBlockingQueue.java:442)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 1.1550ms
>> > 0.0000ms
>> > main-EventThread (49)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@540792d6
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.LinkedBlockingQueue.take
>> > (LinkedBlockingQueue.java:442)
>> > • org.apache.zookeeper.ClientCnxn$EventThread.run (ClientCnxn.java:502)
>> > 2.6375ms
>> > 0.0000ms
>> > main-SendThread(localhost:2181) (48)
>> > • sun.nio.ch.EPollArrayWrapper.epollWait (Native Method)
>> > • sun.nio.ch.EPollArrayWrapper.poll (EPollArrayWrapper.java:269)
>> > • sun.nio.ch.EPollSelectorImpl.doSelect (EPollSelectorImpl.java:93)
>> > • sun.nio.ch.SelectorImpl.lockAndDoSelect (SelectorImpl.java:86)
>> > • sun.nio.ch.SelectorImpl.select (SelectorImpl.java:97)
>> > • org.apache.zookeeper.ClientCnxnSocketNIO.doTransport
>> > (ClientCnxnSocketNIO.java:349)
>> > • org.apache.zookeeper.ClientCnxn$SendThread.run (ClientCnxn.java:1144)
>> > 52.9289ms
>> > 10.0000ms
>> > zkConnectionManagerCallback-6-thread-1 (44)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9c13839
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.LinkedBlockingQueue.take
>> > (LinkedBlockingQueue.java:442)
>> > • java.util.concurrent.ThreadPoolExecutor.getTask
>> > (ThreadPoolExecutor.java:1074)
>> > • java.util.concurrent.ThreadPoolExecutor.runWorker
>> > (ThreadPoolExecutor.java:1134)
>> > • java.util.concurrent.ThreadPoolExecutor$Worker.run
>> > (ThreadPoolExecutor.java:624)
>> > • java.lang.Thread.run (Thread.java:748)
>> > 1.1128ms
>> > 0.0000ms
>> > main-EventThread (43)
>> >
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16806cbc
>> > • sun.misc.Unsafe.park (Native Method)
>> > • java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
>> > •
>> >
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await
>> > (AbstractQueuedSynchronizer.java:2039)
>> > • java.util.concurrent.LinkedBlockingQueue.take
>> > (LinkedBlockingQueue.java:442)
>> > • org.apache.zookeeper.ClientCnxn$EventThread.run (ClientCnxn.java:502)
>> >
>> >
>> > --
>> >
>> > --
>> > Daniel Mantovani
>> >
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
>
> --
>
> --
> Daniel Mantovani
>
>
--
--
Daniel Mantovani
