You either have to go to near real time (NRT), which is under development, but not committed to trunk yet or just stop warming up searchers and let the first user to open a searcher pay the penalty for warmup, (useColdSearchers as I remember).
Although I'd also ask whether this is a reasonable requirement, that the messages be searchable within milliseconds. Is 1 minute really too much time? 5 minutes? You can estimate the minimum time you can get away with by looking at the warmup times on the admin/stats page. Best Erick On Sat, Aug 13, 2011 at 9:47 PM, Naveen Gupta <nkgiit...@gmail.com> wrote: > Hi, > > Most of the settings are default. > > We have single node (Memory 1 GB, Index Size 4GB) > > We have a requirement where we are doing very fast commit. This is kind of > real time requirement where we are polling many threads from third party and > indexes into our system. > > We want these results to be available soon. > > We are committing for each user (may have 10k threads and inside that 1 > thread may have 10 messages). So overall documents per user will be having > around .1 million (100000) > > Earlier we were using commit Within as 10 milliseconds inside the document, > but that was slowing the indexing and we were not getting any error. > > As we removed the commit Within, indexing became very fast. But after that > we started experiencing in the system > > As i read many forums, everybody told that this is happening because of very > fast commit rate, but what is the solution for our problem? > > We are using CURL to post the data and commit > > Also till now we are using default solrconfig. > > Aug 14, 2011 12:12:04 AM org.apache.solr.common.SolrException log > SEVERE: org.apache.solr.common.SolrException: Error opening new searcher. > exceeded limit of maxWarmingSearchers=2, try again later. > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1052) > at > org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:424) > at > org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85) > at > org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:177) > at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77) > at > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:55) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > at java.lang.Thread.run(Thread.java:662) >
