[
https://issues.apache.org/jira/browse/SPARK-44157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930648#comment-17930648
]
Sakthi commented on SPARK-44157:
--------------------------------
This is mostly because probably Spark historically depended on the older
version ({{{}protobuf-java-2.5.0{}}}), which was likely retained for
compatibility reasons or due to Python-side dependencies.
> Outdated JARs in PySpark package
> --------------------------------
>
> Key: SPARK-44157
> URL: https://issues.apache.org/jira/browse/SPARK-44157
> Project: Spark
> Issue Type: Bug
> Components: Build, PySpark
> Affects Versions: 3.4.1
> Reporter: Adrian Gonzalez-Martin
> Priority: Minor
> Labels: pyspark
>
> The JARs which ship embedded within PySpark's package in PyPi don't seem
> aligned with the deps specified in Spark's own `pom.xml`.
> For example, in Spark's `pom.xml`, `protobuf-java` is set to `3.21.12`:
> [https://github.com/apache/spark/blob/6b1ff22dde1ead51cbf370be6e48a802daae58b6/pom.xml#L127]
> However, if we look at the JARs embedded within PySpark tarball, the version
> of `protobuf-java` is `2.5.0` (i.e.
> `..../site-packages/pyspark/jars/protobuf-java-2.5.0.jar`). Same seems to
> apply to all other dependencies.
> This introduces a set of CVEs which are fixed on upstream Spark, but are
> still present in PySpark (e.g. `CVE-2022-3509`, `CVE-2021-22569`, `
> CVE-2015-5237` and a few others). As well as potentially introduce a source
> of conflict whenever there's a breaking change on these deps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
