I am using Apache ManifoldCF framework which connects to my local system and passes all the documents in C drive to Solr.
I am not doing any searches while indexing. There is total 362GB of data needs to be indexed. I am not performing any complex analysis. Thanks, Ameya On Tue, Jul 29, 2014 at 2:49 PM, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > Ameya Aware [ameya.aw...@gmail.com] wrote: > > [Solr -Xmx5120m] > > > I need to index around 300000 documents but with above parameters > > performance is coming very poor around 15000-20000 documents per hour. > > 4-5 documents/second is a lot less than the numbers people normally cite, > but we need to know more about what you are doing in order to help. > > One common reason for unexpected slow indexing is slow data extraction. > Where does your data come from and is is possible to perform a run where > you do not index but just extract, and measure how long that takes? > > Is your index being used for searches while indexing? If so, how many > searches/second? > > How large are the documents you index? How large is your total index? Do > you perform any complex analysis as part of the indexing? > > - Toke Eskildsen >
