Hello Shawn, I did go thru your posts on swap usage http://lucene.472066.n3. nabble.com/Solr-4-3-1-memory-swapping-td4126641.html and my situation is also similar. Below is top output from our prod and performance test machine and as you can see the swap utilization on Prod machine is 44% while on test machines it is zero.
I haven't been able to figure out what/where exactly is the problem. Both test and prod machines, in terms of Java version are same 1.8.0_91. What could help to further understand and debug the issue? Thanks, Susheel *Performance test machine * top - 15:03:52 up 37 days, 5:49, 1 user, load average: 0.06, 0.09, 0.04 Tasks: 191 total, 1 running, 190 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.1 us, 0.0 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 65977076 total, 60079356 used, 5897720 free, 1080 buffers KiB Swap: 2097148 total, 0 used, 2097148 free. 56416304 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 433 solr 20 0 71.813g 3.484g 311548 S 1.667 5.538 105:37.51 java 1 root 20 0 37504 5672 4020 S 0.000 0.009 3:43.54 systemd 2 root 20 0 0 0 0 S 0.000 0.000 0:04.91 kthreadd ... ... *Prod machine* op - 15:10:20 up 116 days, 13:47, 3 users, load average: 0.16, 0.20, 0.18 Tasks: 200 total, 1 running, 199 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 0.0 sy, 0.0 ni, 98.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 65976996 total, 65460256 used, 516740 free, 84 buffers KiB Swap: 2097148 total, *942288* used, 1154860 free. 61361628 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 20327 solr 20 0 53.792g 0.021t 0.017t S 15.95 33.42 5051:07 java 481 root 20 0 267688 152256 9012 S 0.332 0.231 46:47.60 splunkd 1 root 20 0 37532 5360 3896 S 0.000 0.008 6:06.47 systemd ... ... On Fri, Feb 9, 2018 at 5:44 PM, Emir Arnautović < emir.arnauto...@sematext.com> wrote: > No worries, I don't mind being confused with Erick ;) > > Emir > > On Feb 9, 2018 9:16 PM, "Susheel Kumar" <susheel2...@gmail.com> wrote: > > > Sorry, I meant Emir. > > > > On Fri, Feb 9, 2018 at 3:15 PM, Susheel Kumar <susheel2...@gmail.com> > > wrote: > > > > > Thanks, Shawn, Eric. I see that same using swapon -s. Looks like > during > > > the OS setup, it was set as 2 GB (Solr 6.0) and other 16GB (Solr 6.6) > > > > > > Our 6.0 instance has been running since 1+ year but recently our monit > > > started reporting swap usage above 30% and Solr dashboard showing the > > > same. I haven't been able to find what causes / why Solr is using the > > swap > > > space. The index size is well within memory size to fit and we have > > double > > > the index size on our performance test machines but there is no usage > of > > > swap space. > > > > > > Thanks, > > > Susheel > > > > > > On Wed, Feb 7, 2018 at 3:09 PM, Shawn Heisey <apa...@elyograg.org> > > wrote: > > > > > >> On 2/7/2018 12:01 PM, Susheel Kumar wrote: > > >> > > >>> Just trying to find where do we set swap space available to Solr > > >>> process. I > > >>> see in our 6.0 instances it was set to 2GB on and on 6.6 instances > its > > >>> set > > >>> to 16GB. > > >>> > > >> > > >> Solr has absolutely no involvement or control over swap space. > Neither > > >> does Java. This is a function of your operating system's memory > > >> management, and is typically set up when you first install your OS. > > >> > > >> https://www.linux.com/news/all-about-linux-swap-space > > >> https://en.wikipedia.org/wiki/Paging#Windows_NT > > >> > > >> If your system is using swap space, it's a strong indication that you > > >> don't have enough memory installed. If any of the memory that Solr > > uses is > > >> swapped out to disk, Solr performance is going to be REALLY bad. > > >> > > >> Thanks, > > >> Shawn > > >> > > > > > > > > >