Hi, Batch/bulk indexing is the way to go for speed.
* Disable autoSoftCommit feature for the bulk indexing. * Disable transaction log for the bulk indexing. Ater you finish bulk indexing, you can enable above. Again you are too generous with 1 second refresh rate (autoSoftCommit maxTime). Here is an excellent write up : http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ By the way, for auto hard commit openSearcher=false is advised. It is used to flush tlogs. On Wednesday, March 5, 2014 4:48 PM, sweety <sweetyshind...@yahoo.com> wrote: Before indexing , this was the memory layout, System Memory : 63.2% ,2.21 gb JVM Memory : 8.3% , 81.60mb of 981.38mb I have indexed 700 documents of total size 12MB. Following are the results i get : Qtime: 8122, System time : 00:00:12.7318648 System Memory : 65.4% ,2.29 gb JVM Memory : 15.3% , 148.32mb of 981.38mb After indexing 7,000 documents, Qtime: 51817, System time : 00:01:12.6028320 System Memory : 69.4% 2.43Gb JVM Memoery : *26.5%* , 266.60mb After indexing 70,000 documents of 1200mb size, this are the results : Qtime: 511447, System time : 00:11:14.0398768 System memory : 82.7% , 2.89Gb JVM memory :* 11.8%* , 118.46mb Here the JVM usage decreases as compared to 7000 doc, why is it so?? This is* solrconfig.xml *; <updateHandler class="solr.DirectUpdateHandler2"> <updateLog> <str name="dir">${solr.document.log.dir:}</str> </updateLog> <autoSoftCommit> <maxTime>1000</maxTime> </autoSoftCommit> <autoCommit> <maxTime>600000</maxTime> <openSearcher>true</openSearcher> </autoCommit> </updateHandler> I am indexing through solrnet, indexing each document, var res = solr.Add(doc). // Doc doc = new Doc(); How do i reduce the time for indexing, as the size of data indexed is quite less?? Will batch indexing reduce the indexing time?? But then, do i need to make changes in solrconfig.xml Also, i want the documents to be searched in 1sec of indexing. Is it true that, if softcommit is done, then faceting cannot be done on the data?? -- View this message in context: http://lucene.472066.n3.nabble.com/to-reduce-indexing-time-tp4121391.html Sent from the Solr - User mailing list archive at Nabble.com.
