On 6/18/2013 11:06 AM, Learner wrote:
My issue is that, I see that the documents are getting adding to server even before it reaches the queue size. Am I doing anything wrong? Or is queuesize not implemented yet?Also I dont see a very big performance improvements when I increase / decrease the number of threads. Can someone let me know the best way to improve indexing performance when using ConcurrentUpdateSolrserver FYI... I am running this program on 4 core machine.. Sample snippet: ConcurrentUpdateSolrServer server = new ConcurrentUpdateSolrServer( solrServer, 30000, 4); try { while ((line = bReader.readLine()) != null) { inputDocument = line.split("\t"); Do some processing.... server.add(doc); }}
It looks like the Javadocs for this class are misleading. The queuesize value doesn't cause documents to buffer until that many are present, it is used to limit the size of the internal queue. The object will always begin indexing immediately.
I notice that you have your add() call within a try block. You should know that CUSS will *never* throw an exception for things like your server being down or the update being badly formed. It always returns immediately with a success. I did put information about this in the javadocs for its predecessor StreamingUpdateSolrServer, but it looks like we need an update for CUSS.
If you aren't seeing performance increases from increasing threads, then it is very likely that Solr is keeping up with your application with no problems and that your application, or the place your application gets its data, is the bottleneck.
Thanks, Shawn
