if the DIH status does not say that it optimized, it is lucene mergeing the segments
On Mon, Mar 23, 2009 at 8:15 PM, sunnyfr <johanna...@gmail.com> wrote: > > I checked this out but It doesn't say nothing about optimizing. > I'm sure it's lucene part about merging or I don't know ...?? > > > Noble Paul നോബിള് नोब्ळ् wrote: >> >> the easiest way to find out what DIH did is to hit it's status >> command. It will give you a brief description of what all it did >> during last import >> >> On Mon, Mar 23, 2009 at 12:59 AM, Shalin Shekhar Mangar >> <shalinman...@gmail.com> wrote: >>> Lucene will automatically merge segments when they exceed the >>> mergeFactor. >>> This may be one reason but I'm not sure. >>> >>> I checked DataImportHandler's code again. It won't optimize if >>> optimize=false is specified. >>> >>> On Mon, Mar 23, 2009 at 12:43 AM, sunnyfr <johanna...@gmail.com> wrote: >>> >>>> >>>> Do you have any idea ??? >>>> :( >>>> >>>> cheer, >>>> >>>> >>>> sunnyfr wrote: >>>> > >>>> > Hi everybody ... still me :) >>>> > hoo happy day :) >>>> > >>>> > Just, I dont get where I miss something, I will try to be clear. >>>> > >>>> > this is my index folder (and we can notice the evolution according to >>>> the >>>> > delta import every 30mn) : >>>> > >>>> > r...@search-01:/data/solr# ls video/data/index/ >>>> > _2bel.fdt _2bel.fnm _2bel.nrm _2bel.tii _2bel.tvd _2bel.tvx >>>> > _2bem.fdt _2bem.fnm _2bem.nrm _2bem.tii _2bem.tvd _2bem.tvx >>>> > _2ben.frq _2ben.prx _2ben.tis _2beo.fdx segments.gen >>>> > _2bel.fdx _2bel.frq _2bel.prx _2bel.tis _2bel.tvf _2bel_1.del >>>> > _2bem.fdx _2bem.frq _2bem.prx _2bem.tis _2bem.tvf _2ben.fnm >>>> > _2ben.nrm _2ben.tii _2beo.fdt _2beo.fnm segments_230x >>>> > r...@search-01:/data/solr# ls video/data/index/ >>>> > _2bel.fdt _2bel.frq _2bel.tii _2bel.tvf _2bem.fdt _2bem.frq >>>> > _2bem.tii _2bem.tvf _2ben.frq _2ben.tii _2beo.fdx _2beo.nrm >>>> > _2beo.tis _2beo.tvx >>>> > _2bel.fdx _2bel.nrm _2bel.tis _2bel.tvx _2bem.fdx _2bem.nrm >>>> > _2bem.tis _2bem.tvx _2ben.nrm _2ben.tis _2beo.fnm _2beo.prx >>>> > _2beo.tvd segments.gen >>>> > _2bel.fnm _2bel.prx _2bel.tvd _2bel_1.del _2bem.fnm _2bem.prx >>>> > _2bem.tvd _2ben.fnm _2ben.prx _2beo.fdt _2beo.frq _2beo.tii >>>> > _2beo.tvf segments_230x >>>> > r...@search-01:/data/solr# ls video/data/index/ >>>> > _2beo.fdt _2beo.fdx _2beo.fnm _2beo.frq _2beo.nrm _2beo.prx >>>> > _2beo.tii _2beo.tis _2beo.tvd _2beo.tvf _2beo.tvx segments.gen >>>> > segments_230y >>>> > >>>> > So as you can notice my segments increased which is perfect for my >>>> > replication (faster to get JUST last segments) >>>> > But like you can see in my last ls my segment has been optimized. >>>> > >>>> > Like I can notice in my log : >>>> > Mar 19 15:42:37 search-01 jsvc.exec[23255]: Mar 19, 2009 3:42:37 PM >>>> > org.apache.solr.handler.dataimport.DocBuilder commit INFO: Full Import >>>> > completed successfully without optimization >>>> > Mar 19 15:42:37 search-01 jsvc.exec[23255]: Mar 19, 2009 3:42:37 PM >>>> > org.apache.solr.update.DirectUpdateHandler2 commit INFO: start >>>> > commit(optimize=true,waitFlush=false,waitSearcher=true) >>>> > >>>> > But I didn't fire any optimize, my delta-import is fired like : >>>> > /solr/video/dataimport?command=delta-import&optimize=false >>>> > >>>> > Solrconfig.xml : autocommit turnd off >>>> > <!--<luceneAutoCommit>false</luceneAutoCommit>--> >>>> > <!-- <commitLockTimeout>18500</commitLockTimeout>--> >>>> > <!-- autocommit pending docs if certain criteria are met >>>> > <autoCommit> >>>> > <maxDocs>10000</maxDocs> >>>> > </autoCommit> >>>> > --> >>>> > >>>> > Maybe it comes from lucene parameters? >>>> > <!-- options specific to the main on-disk lucene index --> >>>> > <useCompoundFile>false</useCompoundFile> >>>> > <ramBufferSizeMB>50</ramBufferSizeMB> >>>> > <mergeFactor>50</mergeFactor> >>>> > <!-- Deprecated --> >>>> > <!--<maxBufferedDocs>1000</maxBufferedDocs>--> >>>> > <maxMergeDocs>2147483647</maxMergeDocs> >>>> > <maxFieldLength>10000</maxFieldLength> >>>> > >>>> > Thanks a lot for your help, >>>> > Sunny >>>> > >>>> > >>>> > >>>> > >>>> >>>> -- >>>> View this message in context: >>>> http://www.nabble.com/Problem-for-replication-%3A-segment-optimized-automaticly-tp22601442p22649412.html >>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>> >>>> >>> >>> >>> -- >>> Regards, >>> Shalin Shekhar Mangar. >>> >> >> >> >> -- >> --Noble Paul >> >> > > -- > View this message in context: > http://www.nabble.com/Problem-for-replication-%3A-segment-optimized-automaticly-tp22601442p22661545.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
