An interesting supplement to this discussion. The experiment this
time was use Solr v8.1, omit the GC_TUNE items, but instead adjust
SOLR_HEAP. I had set the heap to 4GB, based on good intentions, and as
we have seen Solr v8.1 gobbles it up and does not return a farthing.
Thus I tried index
An interesting note on the memory returning issue for the G1
collector.
https://openjdk.java.net/jeps/346
Entitled "JEP 346: Promptly Return Unused Committed Memory from G1"
with a summary saying "Enhance the G1 garbage collector to automatically
return Java heap memory to the operating
My comments are inserted in-line this time. Thanks for the
amplifications Shawn.
On 27/05/2019 17:39, Shawn Heisey wrote:
On 5/27/2019 9:49 AM, Joe Doupnik wrote:
A few more numbers to contemplate. An experiment here, adding 80
PDF and PPTX files into an empty index.
Solr v8.0 regul
On 5/27/2019 9:49 AM, Joe Doupnik wrote:
A few more numbers to contemplate. An experiment here, adding 80
PDF and PPTX files into an empty index.
Solr v8.0 regular settings, 1.7GB quiesent memory consumption, 1.9GB
while indexing, 2.92 minutes to do the job.
Solr v8.0, using GC_TUNE from
A few more numbers to contemplate. An experiment here, adding 80
PDF and PPTX files into an empty index.
Solr v8.0 regular settings, 1.7GB quiesent memory consumption, 1.9GB
while indexing, 2.92 minutes to do the job.
Solr v8.0, using GC_TUNE from v8.1 solr.in.sh, 1.1GB quiesent, 1.3GB
whi
Solr really should use a limited pool for handling external requests. We’ve
driven it into OOM a few times with too much traffic, just creating a useless
number of threads.
But that requires separate pools for external requests and cluster-internal
requests, which would probably require separat
You are certainly correct about using external load balancers when
appropriate. However, a basic problem with servers, that of accepting
more incoming items than can be handled gracefully is as we know an
age-old one and solved by back pressure methods (particularly hard
limits). My experie
I think it is not fair blaiming Solr not also having a load balancer.
It is up to you and your needs to set up the required infrastucture
including load balancing. The are many products available on the market.
If your current system can't handle all requests then install more replicas.
Regards
B
While on the topic of resource consumption and locks etc, there is
one other aspect to which Solr has been vulnerable. It is failing to
fend off too many requests at one time. The standard approach is, of
course, named back pressure, such as not replying to a query until
resources permit an
Generalizations tend to fail when confronted with conflicting
evidence. The simple evidence is asking how much real memory the Solr
owned process has been allocated (top, or ps aux or similar) and that
yields two very different values (the ~1.6GB of Solr v8.0 and 4.5+GB of
Solr v8.1). I ha
I'm not sure this issue applies in this situation but it's worth taking a
look at:
https://issues.apache.org/jira/browse/SOLR-12833?focusedCommentId=16807868&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16807868
Although the memory issue in the ticket involves d
On 5/26/2019 12:52 PM, Joe Doupnik wrote:
I do queries while indexing, have done so for a long time, without
difficulty nor memory usage spikes from dual use. The system has been
designed to support that.
Again, one may look at the numbers using "top" or similar. Try Solr
v8.0 and 8.1
I do queries while indexing, have done so for a long time, without
difficulty nor memory usage spikes from dual use. The system has been
designed to support that.
Again, one may look at the numbers using "top" or similar. Try Solr
v8.0 and 8.1 to see the difference which I experience her
On 26/05/2019 19:38, Jörn Franke wrote:
Different garbage collector configuration? It does not mean that Solr uses more
memory if it is occupied - it could also mean that the JVM just kept it
reserved for future memory needs.
Am 25.05.2019 um 17:40 schrieb Joe Doupnik :
Comparing memory
I think this is also a very risky memory strategy. What happens if you Index
and query at the same time etc. maybe it is more worth to provide as much
memory as for concurrent operations are needed. This includes JVM memory but
also the disk caches.
> Am 26.05.2019 um 20:38 schrieb Joe Doupnik
On 26/05/2019 19:15, Joe Doupnik wrote:
On 26/05/2019 19:08, Shawn Heisey wrote:
On 5/25/2019 9:40 AM, Joe Doupnik wrote:
Comparing memory consumption (real, not virtual) of quiesent
Solr v8.0 and prior with Solr v8.1.0 reveals the older versions use
about 1.6GB on my systems but v8.1.0 u
Different garbage collector configuration? It does not mean that Solr uses more
memory if it is occupied - it could also mean that the JVM just kept it
reserved for future memory needs.
> Am 25.05.2019 um 17:40 schrieb Joe Doupnik :
>
> Comparing memory consumption (real, not virtual) of q
On 26/05/2019 19:08, Shawn Heisey wrote:
On 5/25/2019 9:40 AM, Joe Doupnik wrote:
Comparing memory consumption (real, not virtual) of quiesent
Solr v8.0 and prior with Solr v8.1.0 reveals the older versions use
about 1.6GB on my systems but v8.1.0 uses 4.5 to 5+GB. Systems used
are SUSE L
On 5/25/2019 9:40 AM, Joe Doupnik wrote:
Comparing memory consumption (real, not virtual) of quiesent Solr
v8.0 and prior with Solr v8.1.0 reveals the older versions use about
1.6GB on my systems but v8.1.0 uses 4.5 to 5+GB. Systems used are SUSE
Linux, with Oracle JDK v1.8 and openjdk v10
Comparing memory consumption (real, not virtual) of quiesent Solr
v8.0 and prior with Solr v8.1.0 reveals the older versions use about
1.6GB on my systems but v8.1.0 uses 4.5 to 5+GB. Systems used are SUSE
Linux, with Oracle JDK v1.8 and openjdk v10. This is a major memory
consumption issue
20 matches
Mail list logo
