Looks like the javadoc on this parameter could use a little tweaking. >From looking at the 4.3 source code (hoping I get this right :-), it appears >the ConcurrentUpdateSolrServer will begin sending documents (on a single >thread) as soon as the first document is added. New threads (up to threadCount) are created only when a document is added and the queue is more than half full. Kind of makes sense... why wait until the queue is full to send documents. And if one thread can keep up with your ETL (adds), there's really no need to create new threads.
You might want to create your own buffer (e.g. ArrayList) of the SolrInputDocument objects and then use the "add" API that accepts the collection. Calling "add" after creating 30,000 SolrInputDocument objects seems a bit much. Something smaller (like 1,000) might work better. You'll have to experiment to see what works best for your environment. -- James -----Original Message----- From: Learner [mailto:bbar...@gmail.com] Sent: Tuesday, June 18, 2013 1:07 PM To: solr-user@lucene.apache.org Subject: ConcurrentUpdateSolrserver - Queue size not working I am using ConcurrentUpdateSolrserver to create 4 threads (threadCount=4) with queueSize of 30000. Indexing works fine as expected. My issue is that, I see that the documents are getting adding to server even before it reaches the queue size. Am I doing anything wrong? Or is queuesize not implemented yet? Also I dont see a very big performance improvements when I increase / decrease the number of threads. Can someone let me know the best way to improve indexing performance when using ConcurrentUpdateSolrserver FYI... I am running this program on 4 core machine.. Sample snippet: ConcurrentUpdateSolrServer server = new ConcurrentUpdateSolrServer( solrServer, 30000, 4); try { while ((line = bReader.readLine()) != null) { inputDocument = line.split("\t"); Do some processing.... server.add(doc); }} -- View this message in context: http://lucene.472066.n3.nabble.com/ConcurrentUpdateSolrserver-Queue-size-not-working-tp4071408.html Sent from the Solr - User mailing list archive at Nabble.com.
