dacort commented on PR #392: URL: https://github.com/apache/incubator-livy/pull/392#issuecomment-1503829916
Fixed the `java.lang.NoClassDefFoundError` error with additional exclusions. Unit tests are failing, however I think it's due to a flaky test based on behavior observed in CI where unit tests pass on the PR but fail once merged into master and vice-cersa. There's an old [mailing list thread](https://www.mail-archive.com/[email protected]/msg00794.html) that also discusses the issue, but with no resolution. ``` The second error, TestSparkClient.testJobSubmission, is random. I see the failure ~25% of the times. As it turned out, the Echo job was returning, sometimes, even before we added the listener on the job handle. So the listener.onJobStarted method is not invoked. This can be fixed in multiple ways but I updated the Echo job, which used to simply return its attribute value, to do what ScalaEcho job is doing. So the Echo job now returns "jc.sc().parallelize(list, 1).collect().get(0);". Even this could still be fast enough to cause the same error. However I couldn't reproduce it again. Also the updated code is taking ~1 sec or more on spark's side. So hopefully this should be ok. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
