Double the heap. All that CPU is the GC trying to free up space.
wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Oct 28, 2020, at 6:29 AM, Jaan Arjasepp <jaan.arjas...@coop.ee> wrote: > > Hi all, > > Its me again. Anyway, I did a little research and we tried different things > and well, some questions I want to ask and some things that I found. > > Well after monitoring my system with VirtualVM, I found that GC jumping is > from 0.5GB to 2.5GB and it has 4GB of memory for now, so it should not be an > issue anymore or what? But will observe it a bit as it might rise I guess a > bit. > > Next thing we found or are thinking about is that writing on a disk might be > an issue, we turned off the indexing and some other stuff, but I would say, > it did not save much still. > I also did go through all the schema fields, not that much really. They are > all docValues=true. Also I must say they are all automatically generated, so > no manual working there except one field, but this also has docValue=true. > Just curious, if the field is not a string/text, can it be docValue=false or > still better to have true? And as for uninversion, then we are not using much > facets nor other specific things in query, just simple queries. > > Though I must say we are updating documents quite a bunch, but usage of CPU > for being so high, not sure about that. Older version seemed not using CPU so > much... > > I am a bit running out of ideas and hoping that this will continue to work, > but I dont like the CPU usage even over night, when nobody uses it. We will > try to figure out the issue here and I hope I can ask more questions when in > doubt or out of ideas. Also I must admit, solr is really new for me > personally. > > Jaan > > -----Original Message----- > From: Walter Underwood <wun...@wunderwood.org> > Sent: 27 October 2020 18:44 > To: solr-user@lucene.apache.org > Subject: Re: SOLR uses too much CPU and GC is also weird on Windows server > > That first graph shows a JVM that does not have enough heap for the program > it is running. Look at the bottom of the dips. That is the amount of memory > still in use after a full GC. > > You want those dips to drop to about half of the available heap, so I’d > immediately increase that heap to 4G. That might not be enough, so you’ll > need to to watch that graph after the increase. > > I’ve been using 8G heaps with Solr since version 1.2. We run this config with > Java 8 on over 100 machines. We do not do any faceting, which can take more > memory. > > SOLR_HEAP=8g > # Use G1 GC -- wunder 2017-01-23 > # Settings from https://wiki.apache.org/solr/ShawnHeisey > GC_TUNE=" \ > -XX:+UseG1GC \ > -XX:+ParallelRefProcEnabled \ > -XX:G1HeapRegionSize=8m \ > -XX:MaxGCPauseMillis=200 \ > -XX:+UseLargePages \ > -XX:+AggressiveOpts \ > " > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > >> On Oct 27, 2020, at 12:48 AM, Jaan Arjasepp <jaan.arjas...@coop.ee> wrote: >> >> Hello, >> >> We have been using SOLR for quite some time. We used 6.0 and now we did a >> little upgrade to our system and servers and we started to use 8.6.1. >> We use it on a Windows Server 2019. >> Java version is 11 >> Basically using it in a default setting, except giving SOLR 2G of heap. It >> used 512, but it ran out of memory and stopped responding. Not sure if it >> was the issue. When older version, it managed fine with 512MB. >> SOLR is not in a cloud mode, but in solo mode as we use it internally and it >> does not have too many request nor indexing actually. >> Document sizes are not big, I guess. We only use one core. >> Document stats are here: >> Num Docs: 3627341 >> Max Doc: 4981019 >> Heap Memory Usage: 434400 >> Deleted Docs: 1353678 >> Version: 15999036 >> Segment Count: 30 >> >> The size of index is 2.66GB >> >> While making upgrade we had to modify one field and a bit of code that uses >> it. Thats basically it. It works. >> If needed more information about background of the system, I am happy to >> help. >> >> >> But now to the issue I am having. >> If SOLR is started, at first 40-60 minutes it works just fine. CPU is not >> high, heap usage seem normal. All is good, but then suddenly, the heap usage >> goes crazy, going up and down, up and down and CPU rises to 50-60% of the >> usage. Also I noticed over the weekend, when there are no writing usage, the >> CPU remains low and decent. I can try it this weekend again to see if and >> how this works out. >> Also it seems to me, that after 4-5 days of working like this, it stops >> responding, but needs to be confirmed with more heap also. >> >> Heap memory usage via JMX and jconsole -> >> https://drive.google.com/file/d/1Zo3B_xFsrrt-WRaxW-0A0QMXDNscXYih/view >> ?usp=sharing As you can see, it starts of normal, but then goes crazy >> and it has been like this over night. >> >> This is overall monitoring graphs, as you can see CPU is working hard >> or hardly working. -> >> https://drive.google.com/file/d/1_Gtz-Bi7LUrj8UZvKfmNMr-8gF_lM2Ra/view >> ?usp=sharing VM summary can be found here -> >> https://drive.google.com/file/d/1FvdCz0N5pFG1fmX_5OQ2855MVkaL048w/view >> ?usp=sharing And finally to have better and quick overview of the SOLR >> executing parameters that I have -> >> https://drive.google.com/file/d/10VCtYDxflJcvb1aOoxt0u3Nb5JzTjrAI/view >> ?usp=sharing >> >> If you can point me what I have to do to make it work, then I appreciate it >> a lot. >> >> Thank you in advance. >> >> Best regards, >> Jaan >> >> >
