The index size of 1 shard is about 125GB and we are running 11 shards
with replication factor 2 so it's a lot of data. The deletions
percentage at the bottom of the segment page is around 25%. So it's
quite some space which we could recover. That's why I was looking for an
optimize.
Do you have any idea why the merge policy does not merge away the
deletions? Should I tweak some parameters somehow? It's a default
installation using the default settings and parameters. If you need more
info, just let me know...
Thx!
On 27-10-16 17:40, Erick Erickson wrote:
Why do you think you need to get rid of the deleted data? During normal
indexing, these will be "merged away". Optimizing has some downsides
for continually changing indexes, in particular since the default
tieredmergepolicy tries to merge "like size" segments, deletions will
accumulate in your one large segment and the percentage of
deleted documents may get even higher.
Unless there's some measurable performance gain that the users
will notice, I'd just leave this alone.
The exception here is if you have, say, an index that changes rarely
in which case optimizing then makes more sense.
Best,
Erick
On Thu, Oct 27, 2016 at 6:56 AM, Arkadi Colson <ark...@smartbit.be
<mailto:ark...@smartbit.be>> wrote:
Thanks for the answer!
Do you know if there is a way to trigger an optimize for only 1
shard and not the whole collection at once?
On 27-10-16 15:30, Pushkar Raste wrote:
Try commit with expungeDeletes="true"
I am not sure if it will merge old segments that have deleted
documents.
In the worst case you can 'optimize' your index which should take
care of removing deleted document
On Oct 27, 2016 4:20 AM, "Arkadi Colson" <ark...@smartbit.be
<mailto:ark...@smartbit.be>> wrote:
Hi
As you can see in the screenshot above in the oldest segments
there are a lot of deletions. In total the shard has about
26% deletions. How can I get rid of them so the index will be
smaller again?
Can this only be done with an optimize or does it also depend
on the merge policy? If it also depends also on the merge
policy which one should I choose then?
Thanks!
BR,
Arkadi
