On 12/27/2017 3:15 PM, Damien Kamerman wrote:
> You seem to have the soft and hard commits the wrong way around. Hard
> commit is more expensive.
That is only the case if the hard commit has openSearcher set to true.
It is strongly recommended for ALL users to have openSearcher set to
false on au
Thanks Eric and others for the replies.
In the prototype we are ingesting only 300 gb of data into the 2 node solr
cloud. I am well aware of the fact that for 4Tb we need much larger setup.
We came across some posts suggesting keeping the hard commit intervals as small
as possible in case if t
NOTE: you'll also have to change your hard commit options to be long
enough to make the ram buffer fill up.
Yes, recoveries will likely be longer in the event of a crash. Frankly
at your ingestion rate though you'll be having "fun" with this in the
event of a crash.
But as others have noted, your
You may find that buying some more memory will be your best bang for the buck
in your set up. 32-64 gb isn’t expensive,
> On Dec 27, 2017, at 6:57 PM, Suresh Pendap wrote:
>
> What is the downside of configuring ramBufferSizeMB to be equal to 5GB ?
> Is it only that the window of time for flu
What is the downside of configuring ramBufferSizeMB to be equal to 5GB ?
Is it only that the window of time for flush is larger, so recovery time will
be higher in case of a crash?
Thanks
Suresh
On 12/27/17, 1:34 PM, "Erick Erickson" wrote:
You are probably hitting more and more background
You seem to have the soft and hard commits the wrong way around. Hard
commit is more expensive.
On 28 December 2017 at 09:10, Walter Underwood
wrote:
> Why are you using Solr for log search? Elasticsearch is widely used for
> log search and has the best infrastructure for that.
>
> For the past
Why are you using Solr for log search? Elasticsearch is widely used for log
search and has the best infrastructure for that.
For the past few years, it looks like a natural market segmentation is
happening, with Solr used for product search and ES for log search. By now, I
would not expect Solr
You are probably hitting more and more background merging which will
slow things down. Your system looks to be severely undersized for this
scale.
One thing you can try (and I emphasize I haven't prototyped this) is
to increase your RamBufferSizeMB solrcofnig.xml setting significantly.
By default,
Hello All,
We have been building a Solr based solution to hold a large amount of data
(approx 4 TB/day or > 24 Billion documents per day). We are developing a
prototype on a small scale just to evaluate Solr performance gradually. Here is
our setup configuration.
Solr cloud:
node1: 16 GB RAM,
