also forgot to update before that I have enabled basicauthentication and provided the details in security.json and uploaded it via zookeeper.
Thanks, On Tue, Jun 2, 2020 at 6:42 PM yaswanth kumar <yaswanth...@gmail.com> wrote: > also I am seeing the below error as a parent one from solr.log > > at org.apache.solr.util.CryptoKeys$RSAKeyPair.encrypt(CryptoKeys.java:366) > org.apache.solr.common.SolrException: javax.crypto.BadPaddingException: > RSA private key operation failed > at > org.apache.solr.util.CryptoKeys$RSAKeyPair.encrypt(CryptoKeys.java:366) > ~[solr-core-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d013 - > ishan - 2020-01-10 1$ > at > org.apache.solr.security.PKIAuthenticationPlugin.generateToken(PKIAuthenticationPlugin.java:305) > ~[solr-core-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d0$ > at > org.apache.solr.security.PKIAuthenticationPlugin.access$200(PKIAuthenticationPlugin.java:61) > ~[solr-core-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d013 -$ > at > org.apache.solr.security.PKIAuthenticationPlugin$2.onQueued(PKIAuthenticationPlugin.java:239) > ~[solr-core-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d013 $ > at > org.apache.solr.client.solrj.impl.Http2SolrClient.decorateRequest(Http2SolrClient.java:469) > ~[solr-solrj-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d013 -$ > at > org.apache.solr.client.solrj.impl.Http2SolrClient.initOutStream(Http2SolrClient.java:324) > ~[solr-solrj-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d013 - i$ > at > org.apache.solr.client.solrj.impl.ConcurrentUpdateHttp2SolrClient$Runner.sendUpdateStream(ConcurrentUpdateHttp2SolrClient.java:227) > ~[solr-solrj-8.4.1.jar:8.4.1 83$ > at > org.apache.solr.client.solrj.impl.ConcurrentUpdateHttp2SolrClient$Runner.run(ConcurrentUpdateHttp2SolrClient.java:181) > ~[solr-solrj-8.4.1.jar:8.4.1 832bf13dd918709$ > at > com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:181) > ~[metrics-core-4.0.5.jar:4.0.5] > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) > ~[solr-solrj-8.4.1.jar:8.4.1 832bf13dd9187095831caf6978$ > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$142/0000000000000000.run(Unknown > Source) ~[?:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > ~[?:?] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > ~[?:?] > at java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: javax.crypto.BadPaddingException: RSA private key operation > failed > at > sun.security.rsa.NativeRSACore.crtCrypt_Native(NativeRSACore.java:149) > ~[?:?] > at sun.security.rsa.NativeRSACore.rsa(NativeRSACore.java:91) ~[?:?] > at sun.security.rsa.RSACore.rsa(RSACore.java:149) ~[?:?] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:355) > ~[?:?] > at > com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:392) ~[?:?] > at javax.crypto.Cipher.doFinal(Cipher.java:2260) ~[?:?] > at > org.apache.solr.util.CryptoKeys$RSAKeyPair.encrypt(CryptoKeys.java:364) > ~[solr-core-8.4.1.jar:8.4.1 832bf13dd9187095831caf69783179d41059d013 - > ishan - 2020-01-10 1$ > ... 13 more > > On Tue, Jun 2, 2020 at 6:37 PM yaswanth kumar <yaswanth...@gmail.com> > wrote: > >> Thanks Franke, but yes for all these questions I did configured it >> properly, I made sure to include >> >> <Set name="KeyStoreType"><Property name="solr.jetty.keystore.type" >> default="JKS"/></Set> >> <Set name="TrustStoreType"><Property name="solr.jetty.truststore.type" >> default="JKS"/></Set> >> in the jetty-ssl.xml along with the path keystore and truststore. >> >> Also I have made sure that trusstore exists on all nodes and also I am >> using the same file for both keystore and truststore as below >> <Set name="KeyStorePath"><Property name="solr.jetty.keystore" >> default="./etc/solr-keystore.jks"/></Set> >> <Set name="KeyStorePassword"><Property >> name="solr.jetty.keystore.password" default="xxxx"/></Set> >> <Set name="TrustStorePath"><Property name="solr.jetty.truststore" >> default="./etc/solr-keystore.jks"/></Set> >> <Set name="TrustStorePassword"><Property >> name="solr.jetty.truststore.password" default="xxxx"/></Set> >> >> also urlScheme for ZK is set to https >> >> >> Also the main error that I posted is the one that I am seeing as a return >> response where as the below one is what I see from solr logs >> >> 2020-06-02 22:32:04.472 ERROR (qtp984876512-93) [c:default s:shard1 >> r:core_node3 x:default_shard1_replica_n1] o.a.s.s.HttpSolrCall >> null:org.apache.solr.update.processor.Distr$ >> at >> org.apache.solr.update.processor.DistributedZkUpdateProcessor.doDistribFinish(DistributedZkUpdateProcessor.java:1189) >> at >> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1096) >> at >> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:182) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) >> at >> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:78) >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211) >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2596) >> at >> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:799) >> at >> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:578) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351) >> at >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) >> >> >> One strange observation is that when I hit update api on the leader node >> its working without any error, and now immediately if I hit non-leader its >> working fine (only once or twice), but if I keep on trying to hit this node >> again and again its then throwing the above error and once the error >> started happening , its consistent again. >> >> Please let me know if you need more information or if I am missing >> something else >> >> Thanks, >> >> On Tue, Jun 2, 2020 at 4:59 PM Jörn Franke <jornfra...@gmail.com> wrote: >> >>> Have you looked in the logfiles? >>> >>> Keystore Type correctly defined on all nodes? >>> >>> Have you configured the truststore on all nodes correctly? >>> >>> Have you set clusterprop urlScheme to htttps in ZK? >>> >>> >>> https://lucene.apache.org/solr/guide/7_5/enabling-ssl.html#configure-zookeeper >>> >>> >>> >>> > Am 02.06.2020 um 18:57 schrieb yaswanth kumar <yaswanth...@gmail.com>: >>> > >>> > team, can someone help me on the above topic? >>> > >>> >> On Mon, Jun 1, 2020 at 10:00 PM yaswanth kumar <yaswanth...@gmail.com >>> > >>> >> wrote: >>> >> >>> >> Trying to setup solr 8.4.1 + open jdk 11 on centos , enabled the ssl >>> >> configurations with all the certs in place, but the issue what I am >>> seeing >>> >> is when trying to hit /update api on non-leader solr node , its >>> throwing an >>> >> error >>> >> >>> >> configured 2 solr nodes with 1 zookeeper. >>> >> >>> >> metadata":[ >>> >> >>> >> >>> "error-class","org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException", >>> >> >>> >> >>> "root-error-class","org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException"], >>> >> "msg":"Async exception during distributed update: >>> >> javax.crypto.BadPaddingException: RSA private key operation failed", >>> >> >>> "trace":"org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException: >>> >> Async exception during distributed update: >>> >> javax.crypto.BadPaddingException: RSA private key operation >>> failed\n\tat >>> >> >>> org.apache.solr.update.processor.DistributedZkUpdateProcessor.doDistribFinish(DistributedZkUpdateProcessor.java:1189)\n\tat >>> >> >>> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1096)\n\tat >>> >> >>> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:182)\n\tat >>> >> >>> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)\n\tat >>> >> org.apache.solr.update.processor.UpdateRequestProcessor.finish........ >>> >> >>> >> Strangely this is happening when we try to hit a non-leader node, >>> hitting >>> >> leader node its working fine without any issue and getting the data >>> indexed. >>> >> >>> >> Not able to track down where the exact issue is happening. >>> >> >>> >> Thanks, >>> >> >>> >> -- >>> >> Thanks & Regards, >>> >> Yaswanth Kumar Konathala. >>> >> yaswanth...@gmail.com >>> >> >>> > >>> > >>> > -- >>> > Thanks & Regards, >>> > Yaswanth Kumar Konathala. >>> > yaswanth...@gmail.com >>> >> >> >> -- >> Thanks & Regards, >> Yaswanth Kumar Konathala. >> yaswanth...@gmail.com >> > > > -- > Thanks & Regards, > Yaswanth Kumar Konathala. > yaswanth...@gmail.com > -- Thanks & Regards, Yaswanth Kumar Konathala. yaswanth...@gmail.com
