how many documents are you inserting ? may be you can create multiple instances of CommonshttpSolrServer and upload in parallel
On Thu, Apr 9, 2009 at 11:58 AM, vivek sar <vivex...@gmail.com> wrote: > Thanks Shalin and Paul. > > I'm not using MultipartRequest. I do share the same SolrServer between > two threads. I'm not using MultiThreadedHttpConnectionManager. I'm > simply using CommonsHttpSolrServer to create the SolrServer. I've also > tried StreamingUpdateSolrServer, which works much faster, but does > throws "connection reset" exception once in a while. > > Do I need to use MultiThreadedHttpConnectionManager? I couldn't find > anything on it on Wiki. > > I was also thinking of using EmbeddedSolrServer - in what case would I > be able to use it? Does my application and the Solr web app need to > run into the same JVM for this to work? How would I use the > EmbeddedSolrServer? > > Thanks, > -vivek > > > On Wed, Apr 8, 2009 at 10:46 PM, Shalin Shekhar Mangar > <shalinman...@gmail.com> wrote: >> Vivek, do you share the same SolrServer instance between your two threads? >> If so, are you using the MultiThreadedHttpConnectionManager when creating >> the HttpClient instance? >> >> On Wed, Apr 8, 2009 at 10:13 PM, vivek sar <vivex...@gmail.com> wrote: >> >>> single thread everything works fine. Two threads are fine too for a >>> while and all the sudden problem starts happening. >>> >>> I tried indexing using REST services as well (instead of Solrj), but >>> with that too I get following error after a while, >>> >>> 2009-04-08 10:04:08,126 ERROR [indexerThreadPool-5] Indexer - >>> indexData()-> Failed to index >>> java.net.SocketException: Broken pipe >>> at java.net.SocketOutputStream.socketWrite0(Native Method) >>> at >>> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) >>> at java.net.SocketOutputStream.write(SocketOutputStream.java:136) >>> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105) >>> at java.io.FilterOutputStream.write(FilterOutputStream.java:80) >>> at >>> org.apache.commons.httpclient.methods.StringRequestEntity.writeRequest(StringRequestEntity.java:145) >>> at >>> org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:499) >>> at >>> org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114) >>> at >>> org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096) >>> at >>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) >>> at >>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) >>> at >>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) >>> at >>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) >>> >>> >>> Note, I'm using "simple" lock type. I'd tried "single" type before >>> that once caused index corruption so I switched to "simple". >>> >>> Thanks, >>> -vivek >>> >>> 2009/4/8 Noble Paul നോബിള് नोब्ळ् <noble.p...@gmail.com>: >>> > do you see the same problem when you use a single thread? >>> > >>> > what is the version of SolrJ that you use? >>> > >>> > >>> > >>> > On Wed, Apr 8, 2009 at 1:19 PM, vivek sar <vivex...@gmail.com> wrote: >>> >> Hi, >>> >> >>> >> Any ideas on this issue? I ran into this again - once it starts >>> >> happening it keeps happening. One of the thread keeps failing. Here >>> >> are my SolrServer settings, >>> >> >>> >> int socketTO = 0; >>> >> int connectionTO = 100; >>> >> int maxConnectionPerHost = 10; >>> >> int maxTotalConnection = 50; >>> >> boolean followRedirects = false; >>> >> boolean allowCompression = true; >>> >> int maxRetries = 1; >>> >> >>> >> Note, I'm using two threads to simultaneously write to the same index. >>> >> >>> >> org.apache.solr.client.solrj.SolrServerException: >>> >> org.apache.commons.httpclient.ProtocolException: Unbuffered entity >>> >> enclosing request can not be repeated. >>> >> at >>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:470) >>> >> at >>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:242) >>> >> at >>> org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:259) >>> >> at >>> org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:48) >>> >> at >>> org.apache.solr.client.solrj.SolrServer.addBeans(SolrServer.java:57) >>> >> >>> >> Thanks, >>> >> -vivek >>> >> >>> >> On Sat, Apr 4, 2009 at 1:07 AM, vivek sar <vivex...@gmail.com> wrote: >>> >>> Hi, >>> >>> >>> >>> I'm sending 15K records at once using Solrj (server.addBeans(...)) >>> >>> and have two threads writing to same index. One thread goes fine, but >>> >>> the second thread always fails with, >>> >>> >>> >>> >>> >>> org.apache.solr.client.solrj.SolrServerException: >>> >>> org.apache.commons.httpclient.ProtocolException: Unbuffered entity >>> >>> enclosing request can not be repeated. >>> >>> at >>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:470) >>> >>> at >>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:242) >>> >>> at >>> org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:259) >>> >>> at >>> org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:48) >>> >>> at >>> org.apache.solr.client.solrj.SolrServer.addBeans(SolrServer.java:57) >>> >>> at >>> com.apple.afterchat.indexer.solr.handler.BeanIndexHandler.indexData(BeanIndexHandler.java:44) >>> >>> at >>> com.apple.afterchat.indexer.Indexer.indexData(Indexer.java:77) >>> >>> at com.apple.afterchat.indexer.Indexer.run(Indexer.java:39) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) >>> >>> at java.lang.Thread.run(Thread.java:637) >>> >>> Caused by: org.apache.commons.httpclient.ProtocolException: Unbuffered >>> >>> entity enclosing request can not be repeated. >>> >>> at >>> org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487) >>> >>> at >>> org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114) >>> >>> at >>> org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096) >>> >>> at >>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) >>> >>> at >>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) >>> >>> at >>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) >>> >>> at >>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) >>> >>> at >>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:417) >>> >>> >>> >>> Does anyone know what could be the problem? >>> >>> >>> >>> Thanks, >>> >>> -vivek >>> >>> >>> >> >>> > >>> > >>> > >>> > -- >>> > --Noble Paul >>> > >>> >> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> > -- --Noble Paul
