mvolikas commented on PR #1833:
URL: https://github.com/apache/stormcrawler/pull/1833#issuecomment-4150778202
I tried to run from this branch using a local Solr cluster in Linux.
With Storm 2.8.2, I was getting the following exception:
```
18:50:11.431 [Thread-37-status-executor[5, 5]] ERROR o.a.s.u.Utils - Async
loop died!
java.lang.NoSuchMethodError: 'boolean
org.eclipse.jetty.http2.client.HTTP2Client.installBean(java.lang.Object)'
at
org.eclipse.jetty.http2.client.HTTP2Client.<init>(HTTP2Client.java:133)
at
org.apache.solr.client.solrj.jetty.HttpJettySolrClient.createHttpClient(HttpJettySolrClient.java:292)
at
org.apache.solr.client.solrj.jetty.HttpJettySolrClient.<init>(HttpJettySolrClient.java:152)
at
org.apache.solr.client.solrj.jetty.HttpJettySolrClient$Builder.build(HttpJettySolrClient.java:987)
at
org.apache.stormcrawler.solr.SolrConnection.getConnection(SolrConnection.java:364)
at
org.apache.stormcrawler.solr.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:89)
at
org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128)
at
org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138)
at
org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54)
at org.apache.storm.utils.Utils$1.run(Utils.java:393)
at java.base/java.lang.Thread.run(Thread.java:840)
```
Switched to Storm 2.8.5 and then this was gone but was getting the following
instead:
```
19:03:37.318 [SLOT_1027] ERROR o.a.s.d.s.Slot - Error when processing event
java.lang.NoClassDefFoundError:
org/apache/storm/shade/org/apache/commons/lang/StringUtils
at
org.apache.stormcrawler.util.URLStreamGrouping.prepare(URLStreamGrouping.java:74)
at
org.apache.storm.daemon.GrouperFactory.mkGrouper(GrouperFactory.java:113)
at
org.apache.storm.executor.Executor.outboundComponents(Executor.java:572)
at org.apache.storm.executor.Executor.<init>(Executor.java:164)
at
org.apache.storm.executor.bolt.BoltExecutor.<init>(BoltExecutor.java:66)
at org.apache.storm.executor.Executor.mkExecutor(Executor.java:205)
at
org.apache.storm.executor.LocalExecutor.mkExecutor(LocalExecutor.java:29)
at org.apache.storm.daemon.worker.Worker.loadWorker(Worker.java:257)
at
org.apache.storm.daemon.worker.Worker.lambda$start$1(Worker.java:206)
at
java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
at org.apache.storm.daemon.worker.Worker.start(Worker.java:205)
at
org.apache.storm.daemon.supervisor.LocalContainer.launch(LocalContainer.java:61)
at
org.apache.storm.daemon.supervisor.LocalContainerLauncher.launchContainer(LocalContainerLauncher.java:52)
at
org.apache.storm.daemon.supervisor.Slot.handleWaitingForBlobLocalization(Slot.java:459)
at
org.apache.storm.daemon.supervisor.Slot.stateMachineStep(Slot.java:194)
at org.apache.storm.daemon.supervisor.Slot.run(Slot.java:1051)
Caused by: java.lang.ClassNotFoundException:
org.apache.storm.shade.org.apache.commons.lang.StringUtils
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 17 more
```
This was because of imports `import
org.apache.storm.shade.org.apache.commons.lang.StringUtils;`.
The issue is resolved if you switch to `import
org.apache.commons.lang3.StringUtils;`.
Then another Jetty related exception:
```
19:30:57.629 [Thread-42] ERROR o.a.s.m.MetricsConsumerBolt - Exception
occurred during handle metrics
java.lang.NoSuchMethodError: 'org.eclipse.jetty.io.Transport
org.eclipse.jetty.io.ClientConnector.newTransport()'
at
org.eclipse.jetty.client.HttpClient.createOrigin(HttpClient.java:478)
at
org.eclipse.jetty.http2.client.transport.HttpClientTransportOverHTTP2.newOrigin(HttpClientTransportOverHTTP2.java:114)
at
org.eclipse.jetty.client.HttpClient.resolveDestination(HttpClient.java:455)
at
org.eclipse.jetty.client.transport.HttpRequest.send(HttpRequest.java:750)
at
org.apache.solr.client.solrj.jetty.HttpJettySolrClient.requestAsync(HttpJettySolrClient.java:407)
at
org.apache.solr.client.solrj.impl.HttpSolrClientBase.requestAsync(HttpSolrClientBase.java:359)
at
org.apache.solr.client.solrj.jetty.LBJettySolrClient.lambda$requestAsyncWithUrl$0(LBJettySolrClient.java:51)
at
org.apache.solr.client.solrj.jetty.HttpJettySolrClient.requestWithBaseUrl(HttpJettySolrClient.java:537)
at
org.apache.solr.client.solrj.jetty.LBJettySolrClient.requestAsyncWithUrl(LBJettySolrClient.java:51)
at
org.apache.solr.client.solrj.impl.LBAsyncSolrClient.doAsyncRequest(LBAsyncSolrClient.java:143)
at
org.apache.solr.client.solrj.impl.LBAsyncSolrClient.requestAsync(LBAsyncSolrClient.java:104)
at
org.apache.stormcrawler.solr.SolrConnection.flushUpdates(SolrConnection.java:235)
at
org.apache.stormcrawler.solr.SolrConnection.updateAsync(SolrConnection.java:177)
at
org.apache.stormcrawler.solr.SolrConnection.addAsync(SolrConnection.java:254)
at
org.apache.stormcrawler.solr.metrics.MetricsConsumer.indexDataPoint(MetricsConsumer.java:114)
at
org.apache.stormcrawler.solr.metrics.MetricsConsumer.handleDataPoints(MetricsConsumer.java:81)
at
org.apache.stormcrawler.solr.metrics.MetricsConsumer.handleDataPoints(MetricsConsumer.java:85)
at
org.apache.stormcrawler.solr.metrics.MetricsConsumer.handleDataPoints(MetricsConsumer.java:74)
at
org.apache.storm.metric.MetricsConsumerBolt$MetricsHandlerRunnable.run(MetricsConsumerBolt.java:131)
at java.base/java.lang.Thread.run(Thread.java:840)
```
I found this was because Storm 2.8.5 brings Jetty 12.1.6 and Solr depends on
Jetty 12.0.27.
I resolved it by pinning specific versions (12.1.6) in the Solr archetype
POM, but I'm not sure whether this is a good idea or there are better ways
around this.
After all these changes I managed to run and didn't notice any unexpected
behavior while stopping and restarting one of the two Solr nodes :-).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
