Sorry :) it is not 500 doc per sec. ( It is what i wish I think) It is 500 doc per MINUTE..
On Tue, Sep 27, 2011 at 7:14 AM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > Hello, > > > PS: solr streamindex is not option because we need to submit javabin... > > > If you are referring to StreamingUpdateSolrServer, then the above statement > makes no sense and you should give SUSS a try. > > Are you sure your 16 reducers produce more than 500 docs/second? > I think somebody already suggested increasing the number of reducers to > ~32. > What happens to your CPU load and indexing speed then? > > > Otis > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > >________________________________ > >From: Lord Khan Han <khanuniver...@gmail.com> > >To: solr-user@lucene.apache.org > >Sent: Monday, September 26, 2011 7:09 AM > >Subject: SOLR Index Speed > > > >Hi, > > > >We have 500K web document and usind solr (trunk) to index it. We have > >special anaylizer which little bit heavy cpu . > >Our machine config: > > > >32 x cpu > >32 gig ram > >SAS HD > > > >We are sending document with 16 reduce client (from hadoop) to the stand > >alone solr server. the problem is we couldnt get speedier than the 500 doc > / > >per sec. 500K document tooks 7-8 hours to index :( > > > >While indexin the the solr server cpu load is around : 5-6 (32 max) it > >means %20 of the cpu total power. We have plenty ram ... > > > >I turned of auto commit and give 8198 rambuffer .. there is no io wait .. > > > >How can I make it faster ? > > > >PS: solr streamindex is not option because we need to submit javabin... > > > >thanks.. > > > > > > >
