Hi Salman,
Sounds like somehow you are triggering merges or optimizes. What is your
mergeFactor?
Have you turned on the IndexWriter log?
In solrconfig.xml
<infoStream file="${solr.indexwriter.log.dir}">true</infoStream>
In our case we feed the directory name as a Java property in our java startup
script , but you can also hard code where you want the log written like in the
current example Solr config:
<infoStream file="INFOSTREAM.txt">false</infoStream>
That should provide some clues. For example you can see how many segments of
each level there are just before you do the commit that triggers the problem.
My first guess is that you have enough segments so that adding the documents
and committing triggers a cascading merge. (But this is a WAG without seeing
what's in your indexwriter log)
Can you also send your solrconfig so we can see your mergeFactor and
ramBufferSizeMB settings?
Tom
> > All,
> >
> > We have created index with CommonGrams and the final size is around
> 370GB.
> > Everything is working fine but now when we add more documents into index
> it
> > takes forever (almost 12 hours)...seems to change all the segments file
> in a
> > commit.
> >
> > The same commit used to take few mins with normal index.
> >
> > Any idea whats going on?
> >
> > --
> > Regards,
> >
> > Salman Akram
> > Principal Software Engineer - Tech Lead
> > NorthBay Solutions
> > 410-G4 Johar Town, Lahore
> > Off: +92-42-35290152
> >
> > Cell: +92-321-4391210 -- +92-300-4009941
> >
>
--
Regards,
Salman Akram
