Hi Joel - thank you for your reply. Yes, the machine (Vesta) is up, and
I can access it. I don't see anything specific in the log, apart from
the same error, but this time to a different server. We have constant
indexing happening on this cluster, so if one went down, the indexing
would stop, and I've not seen that happen.
Interestingly, despite the error, the model is still built at least up
to some number of iterations. In other words, many iterations complete OK.
-Joe
On 4/2/2018 6:54 PM, Joel Bernstein wrote:
It looks like it accessing a replica that's down. Are the logs from
http://vesta:9100/solr/MODEL1024_1522696624083_shard20_replica_n75 reporting
any issues? When you go to that url is it back up and running?
Joel Bernstein
http://joelsolr.blogspot.com/
On Mon, Apr 2, 2018 at 3:55 PM, Joe Obernberger <
joseph.obernber...@gmail.com> wrote:
Hi All - when building machine learning models using information gain, I
sometimes get this error when the number of iterations is high. I'm using
about 20k news articles in my training set (about 10k positive, and 10k
negative), and (for this particular run) am using 500 terms and 25,000
iterations. I have gotten the error with a much lower number of iterations
(1,000) as well.
The specific stream command was:
update(models, batchSize="50",train(MODEL1024_1522696624083,features(
MODEL1024_1522696624083,q="*:*",featureSet="FSet_MODEL1024_1
522696624083",field="Text",outcome="out_i",positiveLabel=1,
numTerms=500),q="*:*",name="MODEL1024",field="Text",outcome=
"out_i",maxIterations="25000"))
The training data was split across 20 shards - specifically created with:
http://icarus.querymasters.com:9100/solr/admin/collections?
action=CREATE&name=MODEL1024_1522696624083&numShards=20&rep
licationFactor=2&maxShardsPerNode=5&collection.configName=TRAINING
Any ideas? The complete error is:
java.io.IOException: java.util.concurrent.ExecutionException:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://vesta:9100/solr/MODEL10
24_1522696624083_shard20_replica_n75: Expected mime type
application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/MODEL1024_1522696624083_shard20_replica_n75/select.
Reason:
<pre> Not Found</pre></p>
</body>
</html>
at org.apache.solr.client.solrj.io.stream.TextLogitStream.read(
TextLogitStream.java:498)
at org.apache.solr.client.solrj.io.stream.PushBackStream.read(P
ushBackStream.java:87)
at org.apache.solr.client.solrj.io.stream.UpdateStream.read(Upd
ateStream.java:109)
at org.apache.solr.client.solrj.io.stream.ExceptionStream.read(
ExceptionStream.java:68)
at org.apache.solr.handler.StreamHandler$TimerStream.read(
StreamHandler.java:627)
at org.apache.solr.client.solrj.io.stream.TupleStream.lambda$wr
iteMap$0(TupleStream.java:87)
at org.apache.solr.response.JSONWriter.writeIterator(JSONRespon
seWriter.java:523)
at org.apache.solr.response.TextResponseWriter.writeVal(TextRes
ponseWriter.java:180)
at org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter
.java:559)
at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(
TupleStream.java:84)
at org.apache.solr.response.JSONWriter.writeMap(JSONResponseWri
ter.java:547)
at org.apache.solr.response.TextResponseWriter.writeVal(TextRes
ponseWriter.java:198)
at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithD
ups(JSONResponseWriter.java:209)
at org.apache.solr.response.JSONWriter.writeNamedList(JSONRespo
nseWriter.java:325)
at org.apache.solr.response.JSONWriter.writeResponse(JSONRespon
seWriter.java:120)
at org.apache.solr.response.JSONResponseWriter.write(JSONRespon
seWriter.java:71)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryR
esponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrC
all.java:806)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:535)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDisp
atchFilter.java:382)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDisp
atchFilter.java:326)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilte
r(ServletHandler.java:1751)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan
dler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(Scoped
Handler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHa
ndler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(
SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(
ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand
ler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(
SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(
ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(Scoped
Handler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.ha
ndle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(
HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(Handl
erWrapper.java:134)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(Rewr
iteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(Handl
erWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConne
ction.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.
succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChann
elEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume
.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume
.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume
.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Queued
ThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedT
hreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://vesta:9100/solr/MODEL10
24_1522696624083_shard20_replica_n75: Expected mime type
application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/MODEL1024_1522696624083_shard20_replica_n75/select.
Reason:
<pre> Not Found</pre></p>
</body>
</html>
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.client.solrj.io.stream.TextLogitStream.read(
TextLogitStream.java:459)
... 47 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://vesta:9100/solr/MODEL10
24_1522696624083_shard20_replica_n75: Expected mime type
application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/MODEL1024_1522696624083_shard20_replica_n75/select.
Reason:
<pre> Not Found</pre></p>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMeth
od(HttpSolrClient.java:590)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
pSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
pSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest
.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest
.java:195)
at org.apache.solr.client.solrj.io.stream.TextLogitStream$Logit
Call.call(TextLogitStream.java:640)
at org.apache.solr.client.solrj.io.stream.TextLogitStream$Logit
Call.call(TextLogitStream.java:582)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
xecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
lExecutor.java:624)
... 1 more
Last Check: 4/2/2018, 3:47:15 PM
Thank you!
-Joe Obernberger
---
This email has been checked for viruses by AVG.
http://www.avg.com
