Kind of obvious/logical, but seen some people forgetting that it is per
core - if single node host multiple shards, each will take 100MB.
Emir
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
On 20.01.2016 07:02, Sha
ramBufferSizeMB is independent of the maxIndexingThreads. If you set
it to 100MB then any lucene segment (or part of a segment) exceeding
100MB will be flushed to disk.
On Wed, Jan 20, 2016 at 3:50 AM, Angel Todorov wrote:
> hi guys,
>
> quick question - is the ramBufferSizeMB the maximum value n
Hi,
And there is a wonderful report in SPM for Solr that shows how your index
changes over time in terms of size, index files, segments, indexed docs,
deleted docs... very useful for understanding what's going on at that
level.
Otis
--
Performance Monitoring - http://sematext.com/spm
On Sep 20, 2
> Is it correct that a segment file is ready for merging after a commit has
> been done (e.g. using the autoCommit property), so I will see merges of 100
> and up documents (and the index writer continues writing into a new segment
> file)?
Yes, merging won't happen until after a segment is closed
Hi
Thanks a lot for your answer, Erick!
I changed the value of the autoSoftCommit property and it had the
expected effect. It can be noted that this is per Core, so I get four
getReader calls when my Solr contains four cores per autoSoftCommit
interval.
Is it correct that a segment file is
I _think_ the getReader calls are being triggered by the autoSoftCommit being
at one second. If so, this is probably OK. But bumping that up would nail
whether that's the case...
About RamBufferSizeMB. This has nothing to do with the size of the segments!
It's just how much memory is consumed befo
On Thu, Dec 2, 2010 at 4:31 PM, Burton-West, Tom wrote:
> We turned on infostream. Is there documentation about how to interpret it,
> or should I just grep through the codebase?
There isn't any documentation... and it changes over time as we add
new diagnostics.
> Is the excerpt below what
On Wed, Dec 1, 2010 at 3:01 PM, Shawn Heisey wrote:
> I have seen this. In Solr 1.4.1, the .fdt, .fdx, and the .tv* files do not
> segment, but all the other files do. I can't remember whether it behaves
> the same under 3.1, or whether it also creates these files in each segment.
Yep, that's t
-user@lucene.apache.org
Subject: Re: ramBufferSizeMB not reflected in segment sizes in index
On Wed, Dec 1, 2010 at 3:16 PM, Burton-West, Tom wrote:
> Thanks Mike,
>
> Yes we have many unique terms due to dirty OCR and 400 languages and probably
> lots of low doc freq terms as well (altho
On Wed, Dec 1, 2010 at 3:16 PM, Burton-West, Tom wrote:
> Thanks Mike,
>
> Yes we have many unique terms due to dirty OCR and 400 languages and probably
> lots of low doc freq terms as well (although with the ICUTokenizer and
> ICUFoldingFilter we should get fewer terms due to bad tokenization a
n the production indexer. If it doesn't I'll turn it on and post
here.
Tom
-Original Message-
From: Michael McCandless [mailto:luc...@mikemccandless.com]
Sent: Wednesday, December 01, 2010 2:43 PM
To: solr-user@lucene.apache.org
Subject: Re: ramBufferSizeMB not reflected in s
On 12/1/2010 12:13 PM, Burton-West, Tom wrote:
We have set the ramBufferSizeMB to 320 in both the indexDefaults and the
mainIndex sections of our solrconfig.xml:
320
20
We expected that this would mean that the index would not write to disk until
it reached somewhere approximately over 300MB
The ram efficiency (= size of segment once flushed divided by size of
RAM buffer) can vary drastically.
Because the in-RAM data structures must be "growable" (to append new
docs to the postings as they are encountered), the efficiency is never
100%. I think 50% is actually a "good" ram efficiency
13 matches
Mail list logo
