I do queries while indexing, have done so for a long time, without
difficulty nor memory usage spikes from dual use. The system has been
designed to support that.
Again, one may look at the numbers using "top" or similar. Try Solr
v8.0 and 8.1 to see the difference which I experience here. For
reference, the only memory adjustables set in my configuration is in the
Solr startup script solr.in.sh saying add "-Xss1024k" in the SOLR_OPTS
list and setting SOLR_HEAP="4024m".
Thanks,
Joe D.
On 26/05/2019 19:43, Jörn Franke wrote:
I think this is also a very risky memory strategy. What happens if you Index
and query at the same time etc. maybe it is more worth to provide as much
memory as for concurrent operations are needed. This includes JVM memory but
also the disk caches.
Am 26.05.2019 um 20:38 schrieb Joe Doupnik <j...@netlab1.net>:
On 26/05/2019 19:15, Joe Doupnik wrote:
On 26/05/2019 19:08, Shawn Heisey wrote:
On 5/25/2019 9:40 AM, Joe Doupnik wrote:
Comparing memory consumption (real, not virtual) of quiesent Solr v8.0
and prior with Solr v8.1.0 reveals the older versions use about 1.6GB on my
systems but v8.1.0 uses 4.5 to 5+GB. Systems used are SUSE Linux, with Oracle
JDK v1.8 and openjdk v10. This is a major memory consumption issue. I have seen
no mention of it in the docs nor forums.
If Solr is using 4 to 5 GB of memory on your system, it is only doing that
because you told it that it was allowed to.
If you run a Java program with a minimum heap that's smaller than the max heap,
which Solr does not do by default, then what you will find is that Java *might*
stay lower than the maximum for a while. But eventually it WILL allocate the
entire maximum heap from the OS, plus some extra for Java itself to work with.
Solr 8.0 and Solr 8.1 are not different from each other in this regard.
Thanks,
Shawn
--------
Not to be argumentative, prior to Solr v8.1 quiesent resident memory
remained at about the 1.6GB level, and during active indexing it could exceed
3.5GB. With the same configuration settings Solr v8.1 changes that to use _a
lot_ more memory. Thus something significant has changed with Solr v8.1 when
compared to its predecessors. The question is what, and what can we do about it.
I am not about to enter a guessing game with Solr and Java and its heap
usage. That is far to complex to hope to win.
Thus, something changed, for the worse here in the field, and I do not
know what.
Thanks,
Joe D.
---------------
If I were forced to guess about this situation it woud be to flag an item
mentioned vaguely in passing: the garbage collector. How to return it to status
quo ante is not known here. Presumably such a step would be covered in the yet
to appear documentation for Solr v8.1
To add a little more to the story. Memory remained at the 1.6GB level except when
doing heavy indexing. To "adjust" Solr so that it always consumes too much, as
at present, is not acceptable, nor is acceptable risking trouble by setting an upper
limit down to say 1.6GB and thence cause indexing to fail.
We see the dilemna. Expert assistance is needed to resolve this.
Thanks,
Joe D.
