Hi Erick;
I wanted to get a quick answer that's why I asked my question as that way.
Error is as follows:
INFO - 2013-08-21 22:01:30.978;
org.apache.solr.update.processor.LogUpdateProcessor; [collection1]
webapp=/solr path=/update params={wt=javabin&version=2}
{add=[com.deviantart.reachmeh
ere:http/gallery/, com.deviantart.reachstereo:http/,
com.deviantart.reachstereo:http/art/SE-mods-313298903,
com.deviantart.reachtheclouds:http/, com.deviantart.reachthegoddess:http/,
co
m.deviantart.reachthegoddess:http/art/retouched-160219962,
com.deviantart.reachthegoddess:http/badges/,
com.deviantart.reachthegoddess:http/favourites/,
com.deviantart.reachthetop:http/
art/Blue-Jean-Baby-82204657 (1444006227844530177),
com.deviantart.reachurdreams:http/, ... (163 adds)]} 0 38790
ERROR - 2013-08-21 22:01:30.979; org.apache.solr.common.SolrException;
java.lang.RuntimeException: [was class org.eclipse.jetty.io.EofException]
early EOF
at
com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:393)
at
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:245)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1812)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:365)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.eclipse.jetty.io.EofException: early EOF
at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:65)
at java.io.InputStream.read(InputStream.java:101)
at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:365)
at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:110)
at com.ctc.wstx.io.MergedReader.read(MergedReader.java:101)
at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:84)
at
com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:992)
at
com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4628)
at
com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126)
at
com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701)
at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649)
... 36 more
ERROR - 2013-08-21 22:01:30.980; org.apache.solr.common.SolrException;
null:java.lang.RuntimeException: [was class
org.eclipse.jetty.io.EofException] early EOF
at
com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18)
at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731)
at
com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657)
at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809)
at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:393)
at
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:245)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1812)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
I use Nutch (that uses Hadoop) to send documents from Hbase to Solr. I am
not indexing documents at Hadoop. I just send documents via Map/Reduce jobs
into my SolrCloud. Nutch sends documents as like that:
...
SolrServer solr = new CommonsHttpSolrServer(solrUrl);
...
private final List<SolrInputDocument> inputDocs = new
ArrayList<SolrInputDocument>();
...
solr.add(inputDocs);
...
inputDocs holds maximum of 1000 documents. After I add inputdocs into Solr
Server I truncate inputdocs list. Then I add new 1000 documents into that
list until every documents send to SolrCloud. When all documents send to
SolrCloud I call commit command.
My Hadoop job could not send documents into SolrCloud and stops to send
documents into Solr (Hadoop job fails) When I open my Solr Adming Page I
see that:
Physical Memory 98.1%
Swap Space NaN%
File Descriptor Count 2.5%
JVM-Memory 1.6%
All in all I think that problem is Physical Memory. I stopped indexing and
Physical Memory is usage is still same (it does not goes down). My machine
uses CentOS 6.4. Should I drop caches when percentage goes up or what do
you do for such kind of situations?
2013/8/24 Erick Erickson <erickerick...@gmail.com>
> This is sounding like an XY problem. What are you measuring
> when you say RAM usage is 99%? is this virtual memory? See:
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> What errors are you seeing when you say: "my node stops to receiving
> documents"?
>
> How are you sending 10M documents? All at once in a huge packet
> or some smaller number at a time? From where? How?
>
> And what does Hadoop have to do with anything? Are you putting
> the Solr index on Hadoop? How? The recent contrib?
>
> In short, you haven't provided very many details. You've been around
> long enough that I'm surprised you're saying "it doesn't work, how can
> I fix it?" without providing much in the way of details to help us help
> you.
>
> Best
> Erick
>
>
>
> On Sat, Aug 24, 2013 at 1:52 PM, Furkan KAMACI <furkankam...@gmail.com
> >wrote:
>
> > I make a test at my SolrCloud. I try to send 100 millions documents into
> my
> > node which has no replica via Hadoop. When document count send to that
> node
> > is around 30 millions, RAM usage of my machine becomes 99% (Solr Heap
> Usage
> > is not 99%, it uses just 3GB - 4GB of RAM). After a time later my node
> > stops to receiving documents to index and the Indexer Job fails as well.
> >
> > How can I force to clean OS cache (if it is OS cache that blocks) me or
> > what should I do (maybe sending 10 million documents and waiting a little
> > etc.) What fellows do at heavy indexing situations?
> >
>
