SOLR 3.5 sometimes throws java.lang.NumberFormatException: For input string: "java.math.BigDecimal:1848.66"
Welcome all, We have a very strange problem with SOLR 3.5. It SOMETIMES throws exceptions: 2012-10-31 10:20:06,408 SEVERE [org.apache.solr.core.SolrCore:185] (http-10.205.49.74-8080-155) org.apache.solr.common.SolrException: ERROR: [doc=MyDoc # d3mo1351674222122-1 # 2012-10-31 08:03:42.122] Error adding field 'AMOUNT'='java.math.BigDecimal:1848.66' at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:324) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:53) at my.package.solr.dihutils.DateCheckUpdateProcessorFactory$DateCheckUpdateProcessor.processAdd(DateCheckUpdateProcessorFactory.java:91) at org.apache.solr.handler.BinaryUpdateRequestHandler$2.document(BinaryUpdateRequestHandler.java:79) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:139) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readIterator(JavaBinUpdateRequestCodec.java:129) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:211) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readNamedList(JavaBinUpdateRequestCodec.java:114) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:176) at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:102) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:144) at org.apache.solr.handler.BinaryUpdateRequestHandler.parseAndLoadDocs(BinaryUpdateRequestHandler.java:69) at org.apache.solr.handler.BinaryUpdateRequestHandler.access$000(BinaryUpdateRequestHandler.java:45) at org.apache.solr.handler.BinaryUpdateRequestHandler$1.load(BinaryUpdateRequestHandler.java:56) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1372) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190) at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NumberFormatException: For input string: "java.math.BigDecimal:1848.66" at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1222) at java.lang.Double.parseDouble(Double.java:510) at org.apache.solr.schema.TrieField.createField(TrieField.java:418) at org.apache.solr.schema.SchemaField.createField(SchemaField.java:104) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:203) at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:281) ... 36 more Exceptions are thrown always for different BigDecimal values (so the problem is not related to BigDecimal value). We have no idea what's going on. Any ideas? Greetings -- Marcin P
Re: SOLR 3.5 sometimes throws java.lang.NumberFormatException: For input string: "java.math.BigDecimal:1848.66"
First we were adding BigDecimal object to SolrInputDocument directly as field value. Now we are adding BigDecimal.toPlainString() as field value. SOLR relies on JavaBinCodec class which does de/serialization in it's own way - some kind of bug in there? I don't know what is the proper way to handle BigDecimal values in SOLR 3.5 after all? 2012/10/31 Erick Erickson : > It _looks_ like somehow the string you're sending as a BigDecimal is, > literally, "java.math.BigDecimal:1848.66" rather than "1848.66". How are > you generating the field value? I'm guessing that your (SolrJ?) program is > somehow messing this up... > > Best > Erick > > > On Wed, Oct 31, 2012 at 7:28 AM, Marcin Pilaczynski > wrote: > >> Welcome all, >> >> We have a very strange problem with SOLR 3.5. It SOMETIMES throws >> exceptions: >> >> 2012-10-31 10:20:06,408 SEVERE [org.apache.solr.core.SolrCore:185] >> (http-10.205.49.74-8080-155) org.apache.solr.common.SolrException: >> ERROR: [doc=MyDoc # d3mo1351674222122-1 # 2012-10-31 08:03:42.122] >> Error adding field 'AMOUNT'='java.math.BigDecimal:1848.66' >> at >> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:324) >> at >> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:53) >> at >> my.package.solr.dihutils.DateCheckUpdateProcessorFactory$DateCheckUpdateProcessor.processAdd(DateCheckUpdateProcessorFactory.java:91) >> at >> org.apache.solr.handler.BinaryUpdateRequestHandler$2.document(BinaryUpdateRequestHandler.java:79) >> at >> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:139) >> at >> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readIterator(JavaBinUpdateRequestCodec.java:129) >> at >> org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:211) >> at >> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$2.readNamedList(JavaBinUpdateRequestCodec.java:114) >> at >> org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:176) >> at >> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:102) >> at >> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:144) >> at >> org.apache.solr.handler.BinaryUpdateRequestHandler.parseAndLoadDocs(BinaryUpdateRequestHandler.java:69) >> at >> org.apache.solr.handler.BinaryUpdateRequestHandler.access$000(BinaryUpdateRequestHandler.java:45) >> at >> org.apache.solr.handler.BinaryUpdateRequestHandler$1.load(BinaryUpdateRequestHandler.java:56) >> at >> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1372) >> at >> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >> at >> org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190) >> at >> org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92) >> at >> org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126) >> at >> org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> at >> org.jboss.web.tomcat.service.jca.CachedConnectionValve.invo
Re: SOLR 3.5 sometimes throws java.lang.NumberFormatException: For input string: "java.math.BigDecimal:1848.66"
Thanks for adding this issue to JIRA. I tried to find the exact problem using debugger step by step analysis, but do to lack of SOLR internals knowledge and time I didn't find anything fishy. I only found that when we were feeding SOLR directly with BigDecimal objects, class JavaBinCodec at line 157 was reading this strange String "java.math.BigDecimal:1848.66" from some kind of InputStream. But why this string was in this stream? 2012/10/31 Chris Hostetter : > > : SOLR relies on JavaBinCodec class which does de/serialization in it's > : own way - some kind of bug in there? > : > : I don't know what is the proper way to handle BigDecimal values in > : SOLR 3.5 after all? > > The safe thing to do is only add "primitive" java objects that Solr > understands natively - String, Long, Integer, Float, Double, Boolean, > and Date. the JavaBinCodec has some logic for trying to deal with other > types of objects -- but i *thought* it's fall back was to just rely on > "toString" for any class it doesn't recognize -- so it does seem like > there is a bug in there somewhere... > > https://issues.apache.org/jira/browse/SOLR-4021 > > > -Hoss -- Marcin P
