This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e1315cd [SPARK-31701][R][SQL] Bump up the minimum Arrow version as
0.15.1 in SparkR
e1315cd is described below
commit e1315cd65631823123af0d14771b0f699809251b
Author: HyukjinKwon <[email protected]>
AuthorDate: Wed May 13 10:03:12 2020 -0700
[SPARK-31701][R][SQL] Bump up the minimum Arrow version as 0.15.1 in SparkR
### What changes were proposed in this pull request?
This PR proposes to set the minimum Arrow version as 0.15.1 to be
consistent with PySpark side at.
### Why are the changes needed?
It will reduce the maintenance overhead to match the Arrow versions, and
minimize the supported range. SparkR Arrow optimization is experimental yet.
### Does this PR introduce _any_ user-facing change?
No, it's the change in unreleased branches only.
### How was this patch tested?
0.15.x was already tested at SPARK-29378, and we're testing the latest
version of SparkR currently in AppVeyor. I already manually tested too.
Closes #28520 from HyukjinKwon/SPARK-31701.
Authored-by: HyukjinKwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
R/pkg/DESCRIPTION | 2 +-
docs/sparkr.md | 13 ++++---------
2 files changed, 5 insertions(+), 10 deletions(-)
diff --git a/R/pkg/DESCRIPTION b/R/pkg/DESCRIPTION
index aa51b98..52d7e1f 100644
--- a/R/pkg/DESCRIPTION
+++ b/R/pkg/DESCRIPTION
@@ -23,7 +23,7 @@ Suggests:
testthat,
e1071,
survival,
- arrow
+ arrow (>= 0.15.1)
Collate:
'schema.R'
'generics.R'
diff --git a/docs/sparkr.md b/docs/sparkr.md
index d937bc9..d86fa86 100644
--- a/docs/sparkr.md
+++ b/docs/sparkr.md
@@ -666,20 +666,15 @@ Apache Arrow is an in-memory columnar data format that is
used in Spark to effic
## Ensure Arrow Installed
-Arrow R library is available on CRAN as of
[ARROW-3204](https://issues.apache.org/jira/browse/ARROW-3204). It can be
installed as below.
+Arrow R library is available on CRAN and it can be installed as below.
```bash
Rscript -e 'install.packages("arrow", repos="https://cloud.r-project.org/";)'
```
+Please refer [the official documentation of Apache
Arrow](https://arrow.apache.org/docs/r/) for more detials.
-If you need to install old versions, it should be installed directly from
Github. You can use `remotes::install_github` as below.
-
-```bash
-Rscript -e 'remotes::install_github("apache/[email protected]", subdir
= "r")'
-```
-
-`apache-arrow-0.12.1` is a version tag that can be checked in [Arrow at
Github](https://github.com/apache/arrow/releases). You must ensure that Arrow R
package is installed and available on all cluster nodes.
-The current supported minimum version is 0.12.1; however, this might change
between the minor releases since Arrow optimization in SparkR is experimental.
+Note that you must ensure that Arrow R package is installed and available on
all cluster nodes.
+The current supported minimum version is 0.15.1; however, this might change
between the minor releases since Arrow optimization in SparkR is experimental.
## Enabling for Conversion to/from R DataFrame, `dapply` and `gapply`
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
