: We optimize less frequently than we used to. Down to twice a month from
once a day.
:
: Without optimizing the search speed stays the same, however the index size
increases to 70+ GB.
:
: Perhaps there is a different way to restrict disk usage.
Consider using the "maxSegments" option on o
Thanks Erick,
Will take a look at this article.
Cheers,
Jason
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Tuesday, November 01, 2011 8:05 AM
To: solr-user@lucene.apache.org
Subject: Re: Replicating Large Indexes
Yes, that's expected behavior.
Do you do a lot of deletes (or 'updates' of existing documents)?
Do you store lots of large fields? Maybe you can use compressed fields in that
case (we never have tried it so I cannot confirm how well it works or performs).
You can also turn off things like norms and vectors, etc. if you aren't
Thanks Robert.
We optimize less frequently than we used to. Down to twice a month from once
a day.
Without optimizing the search speed stays the same, however the index size
increases to 70+ GB.
Perhaps there is a different way to restrict disk usage.
Thanks,
Jason
Robert Stewart wrote:
Optimization merges index to a single segment (one huge file), so entire index
will be copied on replication. So you really do need 2x disk in some cases
then.
Do you really need to optimize? We have a pretty big total index (about 200
million docs) and we never optimize. But we do have a sh
Yes, that's expected behavior. When you optimize, all segments are
copied over to new
segments(s). Since all changed/new segments are replicated to the slave,
you'll (temporarily) have twice the data on your disk.
You can stop optimizing, it's often not really very useful despite its name.
That s
Hi Jason,
I'm very curious about how you build( rebuild ) such a big index efficiently?
Sorry that hijack this topic.
Floyd
2011/11/1 Jason Biggin :
> Wondering if anyone has experience with replicating large indexes. We have a
> Solr deployment with 1 master, 1 master/slave and 5 slaves. Our
