[
https://issues.apache.org/jira/browse/PIO-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893682#comment-15893682
]
ASF GitHub Bot commented on PIO-30:
-----------------------------------
Github user shimamoto commented on the issue:
https://github.com/apache/incubator-predictionio/pull/345
Hi @chanlee514
I am interested in Spark 2.x support. But this PR bothers me a little. Do
you mean that the true nature of this PR is that we can choose either Spark 1.6
or Spark 2.x when to run `make-distribution.sh`? I mean we cannot choose which
Scala version to use. If we choose 1.6, Scala version is determined 2.10
because Spark 1.x is built with Scala 2.10 by default, otherwise 2.11.
But #295 comment says that we can configure the following:
```
-Dbuild.profile=scala-2.11 -Dspark.version=1.6.0
```
If we configure this, PredictionIO should download the Spark source package
and build with Scala 2.11
support(http://spark.apache.org/docs/1.6.3/building-spark.html#building-for-scala-211).
Is this the same as what you imagined? Could you inform me of the future
policy of Spark and Scala support?
> Cross build for different versions of scala and spark
> -----------------------------------------------------
>
> Key: PIO-30
> URL: https://issues.apache.org/jira/browse/PIO-30
> Project: PredictionIO
> Issue Type: Improvement
> Reporter: Marcin Ziemiński
> Assignee: Chan
> Fix For: 0.11.0
>
>
> The present version of Scala is 2.10 and Spark is 1.4, which is quite old.
> With Spark 2.0.0 come many performance improvements and features, that people
> will definitely like to add to their templates. I am also aware that past
> cannot be ignored and simply dumping 1.x might not be an option for other
> users.
> I propose setting up a crossbuild in sbt to build with scala 2.10 and Spark
> 1.6 and a separate one for Scala 2.11 and Spark 2.0. Most of the files will
> be consistent between versions including API. The problematic ones will be
> divided between additional source directories: src/main/scala-2.10/ and
> src/main/scala-2.11/. The dockerized tests should also take the two versions
> into consideration
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
