On 11/13/2019 2:03 AM, Hongxu Ma wrote:
I have a solr-cloud cluster with a big collection, after startup (no any
search/index operations), its jvm memory usage is 9GB (via top: RES).
Cluster and collection info:
each host: total 64G mem, two solr nodes with -xmx=15G
collection: total 9B billion docs (but each doc is very small: only some
bytes), total size 3TB.
My question is:
Is the 9G mem usage after startup normal? If so, I am worried that the follow
up index/search operations will cause an OOM error.
And how can I reduce the memory usage? Maybe I should introduce more host with
nodes, but besides this, is there any other solution?
With the "-Xmx=15G" option, you've told Java that it can use up to 15GB
for heap. It's total resident memory usage is eventually going to reach
a little over 15GB and probably never go down. This is how Java works.
The amount of memory that Java allocates immediately on program startup
is related to the -Xms setting. Normally Solr uses the same number for
both -Xms and -Xmx, but that can be changed if you desire. We recommend
using the same number. If -Xms is smaller than -Xmx, Java may allocate
less memory as soon as it starts, then Solr is going to run through its
startup procedure. We will not know exactly how much memory allocation
is going to occur when that happens ... but with billions of documents,
it's not going to be small.
Thanks,
Shawn
