Hi,
Please find the configurations below.
Machine configurations(Solr running here):
RAM - 4 GB
HardDisk - 180GB
Os - Red Hat linux version 5
Processor-2x Intel Core 2 Duo CPU @2.66GHz
Machine configurations(Mysql server is running here):
RAM - 4 GB
HardDisk - 180GB
Os - Red Hat linux version 5
Processor-2x Intel Core 2 Duo CPU @2.66GHz
My sql Server deatils:
My sql version - Mysql 5.0.22
Solr configuration details:
<indexDefaults>
<useCompoundFile>false</useCompoundFile>
<mergeFactor>20</mergeFactor>
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
<ramBufferSizeMB>100</ramBufferSizeMB>
<maxMergeDocs>2147483647</maxMergeDocs>
<maxFieldLength>10000</maxFieldLength>
<writeLockTimeout>1000</writeLockTimeout>
<commitLockTimeout>10000</commitLockTimeout>
<!--<luceneAutoCommit>false</luceneAutoCommit>-->
<!--<mergePolicy>org.apache.lucene.index.LogByteSizeMergePolicy</mergePolicy>-->
<!--<mergeScheduler>org.apache.lucene.index.ConcurrentMergeScheduler</mergeScheduler>-->
<lockType>single</lockType>
</indexDefaults>
<mainIndex>
<useCompoundFile>false</useCompoundFile>
<ramBufferSizeMB>100</ramBufferSizeMB>
<mergeFactor>20</mergeFactor>
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
<maxMergeDocs>2147483647</maxMergeDocs>
<maxFieldLength>10000</maxFieldLength>
<unlockOnStartup>false</unlockOnStartup>
</mainIndex>
<!-- the default high-performance update handler -->
<updateHandler class="solr.DirectUpdateHandler2">
<maxPendingDeletes>100000</maxPendingDeletes>
<autoCommit>
<maxDocs>10000</maxDocs>
<maxTime>60000</maxTime>
</autoCommit>
<!-- A postCommit event is fired after every commit or optimize command
<listener event="postCommit" class="solr.RunExecutableListener">
<str name="exe">solr/bin/snapshooter</str>
<str name="dir">.</str>
<bool name="wait">true</bool>
<arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
<arr name="env"> <str>MYVAR=val1</str> </arr>
</listener>
-->
<!-- A postOptimize event is fired only after every optimize command,
useful
in conjunction with index distribution to only distribute optimized
indicies
<listener event="postOptimize" class="solr.RunExecutableListener">
<str name="exe">snapshooter</str>
<str name="dir">solr/bin</str>
<bool name="wait">true</bool>
</listener>
-->
</updateHandler>
Solr document details:
21 fields are indexed and stored
3 fileds are indexed only.
3 fileds are stored only.
3 fileds are indexed,stored and multi valued
2 fileds indexed and multi valued
And i am copying some of the indexed fileds.In this 2 fileds are multivalued
and has thousands of values.
In db-config-file the main table contains 0.6 million records.
When i tested for the same records, the index has taken 1hr 30 min.In this
case one of the multivalued filed table doesn't have records.After putting
data into this table,for each main table record , this table has thousands
of records and this filed is indexed and stored.It is taking more than 24
hrs .
Solr is running on tomcat 6.0.26, jdk1.6.0_17 and solr 1.4.1
I am using JVM's default settings.
Why this is taking this much time?Any body has suggestions, where i am going
wrong.
Thanks,
JS
--
View this message in context:
http://lucene.472066.n3.nabble.com/Speeding-up-solr-indexing-tp1667054p1670737.html
Sent from the Solr - User mailing list archive at Nabble.com.
