There are several possibilities:
1> you have some process holding open your indexes, probably
other searchers. You *probably* are OK just committing
new changes if there is exactly *one* searcher keeping
your index open. If you have some process whereby
you periodically open a new search but you fail to close
the old one, then you'll use up an extra 40G for every
version of your index held open by your processes. That's
confusing... I'm saying that if you open any number of IRs,
you'll have 40G consumed. Then if you add
some more documents and open *another* IR, you'll have
another 40G consumed. They'll stay around until you close
your readers.
2> If you optimize, there can be up to 3X the index size being
consumed if you also have a previous reader opened.
So I suspect that sometime recently you've opened another
IR.....
HTH
Erick
On Tue, Jan 12, 2010 at 8:03 AM, Frederico Azeiteiro <
frederico.azeite...@cision.com> wrote:
> Hi all,
>
> I started working with solr about 1 month ago, and everything was
> running well both indexing as searching documents.
>
> I have a 40GB index with about 10 000 000 documents available. I index
> 3k docs for each 10m and commit after each insert.
>
> Since yesterday, I can't commit no articles to index. I manage to search
> ok, and index documents without commiting. But when I start the commit
> is takes a long time and eats all of the available disk space
> left(60GB). The commit eventually stops with full disk and I have to
> restart SOLR and get the 60GB returned to system.
>
> Before this, the commit was taking a few seconds to complete.
>
> Can someone help to debug the problem? Where should I start? Should I
> try to copy the index to other machine with more free space and try to
> commit? Should I try an optimize?
>
> Log for the last commit I tried:
>
> INFO: start
> commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=f
> alse)
> (Then, after a long time...)
> Exception in thread "Lucene Merge Thread #0"
> org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException:
> No space left on device
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(Co
> ncurrentMergeScheduler.java:351)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(Concurr
> entMergeScheduler.java:315)
> Caused by: java.io.IOException: No space left on device
>
> I'm using Ubuntu 9.04 and Solr 1.4.0.
>
> Thanks in advance,
>
> Frederico
>
