[
https://issues.apache.org/jira/browse/PIO-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169650#comment-16169650
]
dongjie.shi commented on PIO-122:
---------------------------------
how you built PIO 0.11
predictionio-0.11.0-incubating-src/make-distribution.sh -Dscala.version=2.11.8
-Dspark.version=${SPARK_VERSION}
-Delasticsearch.version=${ELASTICSEARCH_VERSION}
the template you are using
https://github.com/apache/incubator-predictionio-template-text-classifier.git
the conf/pio-env.sh of your PIO 0.11 build
Please check it in the Description
Elasticsearch version
1.4.4 and 1.7.3
logs when run pio train
[INFO] [Pio$] Your engine is ready for training.
root@caterpillar:/opt/work/incubator-predictionio-template-text-classifier# pio
train
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/work/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/work/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Runner$] Submission command:
/opt/work/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit
--class org.apache.predictionio.workflow.CreateWorkflow --jars
file:/opt/work/incubator-predictionio-template-text-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/opt/work/incubator-predictionio-template-text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/opt/work/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/opt/work/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/opt/work/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/opt/work/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar,file:/opt/work/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar
--files
file:/opt/work/PredictionIO-0.11.0-incubating/conf/log4j.properties,file:/opt/work/PredictionIO-0.11.0-incubating/vendors/hbase-1.3.1/conf/hbase-site.xml
--driver-class-path
/opt/work/PredictionIO-0.11.0-incubating/conf:/opt/work/PredictionIO-0.11.0-incubating/vendors/hbase-1.3.1/conf
--driver-java-options -Dpio.log.dir=/root
file:/opt/work/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar
--engine-id org.example.textclassification.TextClassificationEngine
--engine-version dc0803987a872c91b3c92d32466cf641f5f8a00a --engine-variant
file:/opt/work/incubator-predictionio-template-text-classifier/engine.json
--verbosity 0 --json-extractor Both --env
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_VERSION=0.11.0,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/opt/work/PredictionIO-0.11.0-incubating/vendors/hbase-1.3.1,PIO_HOME=/opt/work/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=predictionio,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/opt/work/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.3,PIO_FS_TMPDIR=/root/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/opt/work/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
[INFO] [Engine] Extracting datasource params...
[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be used.
[INFO] [Engine] Datasource params: (,DataSourceParams(MyTextApp,None))
[INFO] [Engine] Extracting preparator params...
[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be used.
[INFO] [Engine] Preparator params: (,PreparatorParams(1,500))
[INFO] [Engine] Extracting serving params...
[INFO] [Engine] Serving params: (,Empty)
[ERROR] [Storage$] Error initializing storage client for source ELASTICSEARCH
Exception in thread "main"
org.apache.predictionio.data.storage.StorageClientException: Data source
ELASTICSEARCH was not properly initialized.
at
org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)
at
org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)
at scala.Option.getOrElse(Option.scala:120)
at
org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)
at
org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)
at
org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)
at
org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)
at
org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> always get Error initializing storage client for source ELASTICSEARCH in 0.11
> when pio train
> --------------------------------------------------------------------------------------------
>
> Key: PIO-122
> URL: https://issues.apache.org/jira/browse/PIO-122
> Project: PredictionIO
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.11.0-incubating
> Reporter: dongjie.shi
>
> Always get this error when I try to run PIO train to test a local PIO with
> HBase and Elasticsearch.
> [ERROR] [Storage$] Error initializing storage client for source ELASTICSEARCH
> Exception in thread "main"
> org.apache.predictionio.data.storage.StorageClientException: Data source
> ELASTICSEARCH was not properly initialized.
> at
> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)
> at
> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)
> at scala.Option.getOrElse(Option.scala:120)
> at
> org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)
> at
> org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)
> at
> org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)
> at
> org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)
> at
> org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> It seems that every component is ok in pio staus:
> [INFO] [Management$] Your system is all ready to go.
> And about the configurations in pio-env.conf, I have checked them for many
> times.
> PIO_FS_BASEDIR=${HOME}/.pio_store
> PIO_FS_ENGINESDIR=${PIO_FS_BASEDIR}/engines
> PIO_FS_TMPDIR=${PIO_FS_BASEDIR}/tmp
> SPARK_HOME=${SPARK_HOME}
> HBASE_CONF_DIR=${HBASE_HOME}/conf
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=predictionio
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=${ELASTICSEARCH_HOME}
> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
> PIO_STORAGE_SOURCES_LOCALFS_PATH=${PIO_FS_BASEDIR}/models
> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
> PIO_STORAGE_SOURCES_HBASE_HOME=${HBASE_HOME}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
