Hit an Authenticated Elasticsearch 5.x problem on the current develop branch.
I just tested the HEAD of develop by performing:
./make-distribution.sh \
-Dscala.version=2.11.8 \
-Dspark.version=2.1.0 \
-Dhadoop.version=2.7.3 \
-Delasticsearch.version=5.1.1
Then, tried build/train/deploy of our Universal Recommender template.
Locally, it makes it through train to the point when it saves to Elasticsearch,
failing with:
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.elasticsearch.client.RestClient.performRequest(Ljava/lang/String;Ljava/lang/String;Ljava/util/Map;[Lorg/apache/http/Header;)Lorg/elasticsearch/client/Response;
> at org.template.EsClient$.createIndex(EsClient.scala:132)
> at org.template.EsClient$.hotSwap(EsClient.scala:218)
> at org.template.URModel.save(URModel.scala:86)
I tried deploying it to Heroku as well, and it fails much earlier when simply
connecting to Elasticsearch:
> remote: Exception in thread "main"
> org.elasticsearch.client.ResponseException: HEAD
> https://xxxxxx.us-east-1.bonsaisearch.net:443/pio_meta: HTTP/1.1 401
> Unauthorized
> remote: at
> org.elasticsearch.client.RestClient$1.completed(RestClient.java:311)
> remote: at
> org.elasticsearch.client.RestClient$1.completed(RestClient.java:300)
> remote: at
> shadeio.data.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
> remote: at
> shadeio.data.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
> remote: at
> shadeio.data.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
> remote: at
> shadeio.data.http.nio.protocol.HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:309)
> remote: at
> shadeio.data.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:255)
These issues were previously found to be caused by this shade rule:
https://github.com/apache/incubator-predictionio/blob/develop/storage/elasticsearch/build.sbt#L42
It looks like the shaded package does not actually use the new authentication
code.
Chan Lee mentioned to me that he was only able to make the TravisCI build pass
by adding this shade rule, but it is clearly breaking the authenicated
Elasticsearch functionality.
Any ideas how to solve this?
*Mars
( <> .. <> )
> On Aug 3, 2017, at 11:02, Donald Szeto <[email protected]> wrote:
>
> On Thu, Aug 3, 2017 at 10:07 AM, Mars Hall <[email protected]> wrote:
>
>> I just opened a PR to add docs for batch predict.
>>
>> Moving forward with the 0.12.0 release sounds great. Today, I will pull
>> develop and see how it's working with the Heroku buildpack.
>>
>
> Awesome. Thanks!
>
>
>>> On Aug 3, 2017, at 00:37, takako shimamoto <[email protected]> wrote:
>>>
>>> I think it's almost ready, and now we just have to update the current
>>> documentation.
>>> The deadline of several unresolved issues for Target Version/s:
>>> 0.12.0-incubating is extended, right?
>>
>
> Yes. Let's extend those that have not started working if there's no
> objection.
