Thanks for the responses Mark and Ramkumar.
The question I had was, why does Solr need 2 copies at any given time, leading
to 2x disk space usage.
Not sure if this information is not published anywhere, and makes HW
estimation almost impossible for large scale deployment. Even if the copies are
temporary, this becomes really expensive, especially when using SSD in
production, when the complex size is over 400TB indexes, running 1000's of solr
cloud shards.
If a solr follower has decided that it needs to do replication from leader and
capture full copy snapshot. Why can't it delete the old information and
replicate from scratch, not requiring more disk space.
Is the concern data loss (a case when both leader and follower lose data)?.
Thanks,
Rishi.
-----Original Message-----
From: Mark Miller <markrmil...@gmail.com>
To: solr-user <solr-user@lucene.apache.org>
Sent: Tue, Apr 28, 2015 10:52 am
Subject: Re: Multiple index.timestamp directories using up disk space
If copies of the index are not eventually cleaned up, I'd fill a JIRA
to
address the issue. Those directories should be removed over time. At
times
there will have to be a couple around at the same time and others may
take
a while to clean up.
- Mark
On Tue, Apr 28, 2015 at 3:27 AM Ramkumar
R. Aiyengar <
andyetitmo...@gmail.com> wrote:
> SolrCloud does need up to
twice the amount of disk space as your usual
> index size during replication.
Amongst other things, this ensures you have
> a full copy of the index at any
point. There's no way around this, I would
> suggest you provision the
additional disk space needed.
> On 20 Apr 2015 23:21, "Rishi Easwaran"
<rishi.easwa...@aol.com> wrote:
>
> > Hi All,
> >
> > We are seeing this
problem with solr 4.6 and solr 4.10.3.
> > For some reason, solr cloud tries to
recover and creates a new index
> > directory - (ex:index.20150420181214550),
while keeping the older index
> as
> > is. This creates an issues where the
disk space fills up and the shard
> > never ends up recovering.
> > Usually
this requires a manual intervention of bouncing the instance and
> > wiping
the disk clean to allow for a clean recovery.
> >
> > Any ideas on how to
prevent solr from creating multiple copies of index
> > directory.
> >
> >
Thanks,
> > Rishi.
> >
>
