*total thread is 25.6k when solr hang.
On Sunday, July 5, 2020, 2:55:26 AM GMT+7, Tran Van Hoan
wrote:
All server only run Solr, zookeeper, exporters (node-exporter,
process-exporter, solr-exporter, zoo-exporter).
- network: no package loss, TCP no issue before incident, TCP drop ar
All server only run Solr, zookeeper, exporters (node-exporter,
process-exporter, solr-exporter, zoo-exporter).
- network: no package loss, TCP no issue before incident, TCP drop around
100-200/s when incident and overflow ~100 in somaxcon.
- total mem Avalilable is greater 25G (Solr's XMX = 30G
Network it's ok? Between nodes? The use? Swap it's disabled? Swapiness rhe
value it's 0?
Em sáb, 4 de jul de 2020 15:19, Tran Van Hoan
escreveu:
> I used physical servers, and IO wait is small :(!!!I saw that iptables
> dropped all ACK message from clients (not only client solr, prometheus
> sc
I used physical servers, and IO wait is small :(!!!I saw that iptables dropped
all ACK message from clients (not only client solr, prometheus scape metric
from exporter was dropped too).all when i check netstat -anp|grep 8983, all
socket are TIME_WAIT state.Only restart solrs, the incident was
Hi,
I had this problem. In my case was the wait/io in vm. I migrate my
environment to another place and solved.
Actually it's problem wirh wait/io at host physical (until backup it's a
problem over veeam).
Regards
Em sáb, 4 de jul de 2020 12:30, Tran Van Hoan
escreveu:
> The problem reoccurs
The problem reoccurs repeatly in recent days.
To day i tried dump heap and thread. Only dumping thread, heap can not because
solr instance was hang.Almost thread was blocked.
On Tuesday, June 23, 2020, 10:42:36 PM GMT+7, Tran Van Hoan
wrote:
I checked node exporter metrics and saw n
I checked node exporter metrics and saw network no problem
On Tuesday, June 23, 2020, 8:37:41 PM GMT+7, Tran Van Hoan
wrote:
I check node exporter, no problem with OS, hardware and network.I attached
images about solr metrics 7 days and 12h.
On Tuesday, June 23, 2020, 2:23:05 PM
What about a network issue?
Il giorno mar 23 giu 2020 alle ore 01:37 Tran Van Hoan
ha scritto:
>
> dear all,
>
> I have a solr cloud 8.2.0 with 6 instance per 6 server (64G RAM), each
> instance has xmx = xms = 30G.
>
> Today almost nodes in the solrcloud were dead 2 times from 8:00AM (5/6
> no
