pan3793 opened a new pull request, #7886:
URL: https://github.com/apache/hadoop/pull/7886
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
https://docs.oracle.com/en/java/javase/24/security/migrating-deprecated-removal-subject-getsubject-and-subject-doas-methods-subject-current-and-subje.html
> In JDK 24, the Security Manager has been permanently disabled. See [JEP
486](https://openjdk.org/jeps/486) for more information.
This PR is extracted from @stoty's PR
https://github.com/apache/hadoop/pull/7434, with some tweaks.
The main goal is to make minimal changes to make the Hadoop client
compatible with Java 25, which unlocks downstream projects that rely on the
Hadoop client, e.g. Spark, to support Java 25.
### How was this patch tested?
Integrated with Spark 4.1.0-SNAPSHOT (master branch) with Java 25.
Note: Spark uses the Hadoop Shaded client. Currently, the Hadoop trunk
branch's hadoop-`client-minicluster` is broken, it requires some fixes (will
send dedicated PRs to fix) ahead.
Before
```
java.lang.UnsupportedOperationException: getSubject is not supported
at java.base/javax.security.auth.Subject.getSubject(Subject.java:277)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3888)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3878)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3666)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:510)
at
org.apache.spark.deploy.yarn.Client$.org$apache$spark$deploy$yarn$Client$$getQualifiedLocalPath(Client.scala:1751)
at
org.apache.spark.deploy.yarn.Client$.addFileToClasspath(Client.scala:1660)
at
org.apache.spark.deploy.yarn.Client$.$anonfun$populateClasspath$3(Client.scala:1549)
at
org.apache.spark.deploy.yarn.Client$.$anonfun$populateClasspath$3$adapted(Client.scala:1548)
...
```
After this patch, Spark passes all UT of YARN mode with Java 25.
@cxzl25 also helps test this patch using a Spark job on a Kerberized cluster
with Java 25, basic SQL, including reading and writing Hive tables, works as
expected.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
