[
https://issues.apache.org/jira/browse/HADOOP-19821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Pan updated HADOOP-19821:
-------------------------------
Description:
The issue was identified on Hadoop 3.4.3 RC1 validation - use OpenJDK 25 with
Kerberos enabled, SparkPi fails immediately and consistently.
The root cause is that JDK 24 introduces a behavior change on ServiceLoader's
returned iterator, `iterator.hasNext()` might throw NoClassDefFoundError, which
previously only happened on `iterator.next()`
See [https://bugs.openjdk.org/browse/JDK-8350481]
{code:java}
2026-02-20 19:41:32,117 INFO resourcemanager.ClientRMService: Application with
id 1 submitted by user spark
2026-02-20 19:41:32,126 WARN security.DelegationTokenRenewer: Unable to add the
application to the delegation token renewer.
java.lang.NoClassDefFoundError: com/google/protobuf/Message$Builder
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at
java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:2985)
at java.base/java.lang.Class.getConstructor0(Class.java:3180)
at java.base/java.lang.Class.getConstructor(Class.java:2199)
at
java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:623)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1111)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1142)
at java.base/java.util.ServiceLoader$1.hasNext(ServiceLoader.java:1164)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1246)
at
org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:136)
at
org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:166)
at
org.apache.hadoop.security.token.Token.identifierToString(Token.java:425)
at org.apache.hadoop.security.token.Token.toString(Token.java:445)
at java.base/java.lang.String.valueOf(String.java:4530)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:496)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$1800(DelegationTokenRenewer.java:83)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:1078)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:1055)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:496)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$1800(DelegationTokenRenewer.java:83)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:1078)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:1055)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545)
2026-02-20 19:41:32,139 INFO rmapp.RMAppImpl: application_1771616432238_0001
State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED
2026-02-20 19:41:32,139 WARN capacity.CapacityScheduler: Couldn't find
application application_1771616432238_0001 {code}
was:
JDK 24 introduces a behavior change on ServiceLoader's returned iterator,
`iterator.hasNext()` might throw NoClassDefFoundError, which previously only
happened on `iterator.next()`
See [https://bugs.openjdk.org/browse/JDK-8350481]
The issue was identified on Hadoop 3.4.3 RC1 validation - use OpenJDK 25 with
Kerberos
{code:java}
2026-02-20 19:41:32,117 INFO resourcemanager.ClientRMService: Application with
id 1 submitted by user spark
2026-02-20 19:41:32,126 WARN security.DelegationTokenRenewer: Unable to add the
application to the delegation token renewer.
java.lang.NoClassDefFoundError: com/google/protobuf/Message$Builder
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at
java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:2985)
at java.base/java.lang.Class.getConstructor0(Class.java:3180)
at java.base/java.lang.Class.getConstructor(Class.java:2199)
at
java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:623)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1111)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1142)
at java.base/java.util.ServiceLoader$1.hasNext(ServiceLoader.java:1164)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1246)
at
org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:136)
at
org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:166)
at
org.apache.hadoop.security.token.Token.identifierToString(Token.java:425)
at org.apache.hadoop.security.token.Token.toString(Token.java:445)
at java.base/java.lang.String.valueOf(String.java:4530)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:496)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$1800(DelegationTokenRenewer.java:83)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:1078)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:1055)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:496)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$1800(DelegationTokenRenewer.java:83)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:1078)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:1055)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545)
2026-02-20 19:41:32,139 INFO rmapp.RMAppImpl: application_1771616432238_0001
State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED
2026-02-20 19:41:32,139 WARN capacity.CapacityScheduler: Couldn't find
application application_1771616432238_0001 {code}
> [JDK25] Token initialization fails with NoClassDefFoundError
> ------------------------------------------------------------
>
> Key: HADOOP-19821
> URL: https://issues.apache.org/jira/browse/HADOOP-19821
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.4.3
> Environment: Hadoop 3.4.3 RC1
> OpenJDK 25
> Kerberos enabled
> Reporter: Cheng Pan
> Priority: Major
>
> The issue was identified on Hadoop 3.4.3 RC1 validation - use OpenJDK 25 with
> Kerberos enabled, SparkPi fails immediately and consistently.
> The root cause is that JDK 24 introduces a behavior change on ServiceLoader's
> returned iterator, `iterator.hasNext()` might throw NoClassDefFoundError,
> which previously only happened on `iterator.next()`
> See [https://bugs.openjdk.org/browse/JDK-8350481]
> {code:java}
> 2026-02-20 19:41:32,117 INFO resourcemanager.ClientRMService: Application
> with id 1 submitted by user spark
> 2026-02-20 19:41:32,126 WARN security.DelegationTokenRenewer: Unable to add
> the application to the delegation token renewer.
> java.lang.NoClassDefFoundError: com/google/protobuf/Message$Builder
> at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
> at
> java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:2985)
> at java.base/java.lang.Class.getConstructor0(Class.java:3180)
> at java.base/java.lang.Class.getConstructor(Class.java:2199)
> at
> java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:623)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1111)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1142)
> at
> java.base/java.util.ServiceLoader$1.hasNext(ServiceLoader.java:1164)
> at
> java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1246)
> at
> org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:136)
> at
> org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:166)
> at
> org.apache.hadoop.security.token.Token.identifierToString(Token.java:425)
> at org.apache.hadoop.security.token.Token.toString(Token.java:445)
> at java.base/java.lang.String.valueOf(String.java:4530)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:496)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$1800(DelegationTokenRenewer.java:83)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:1078)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:1055)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:496)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$1800(DelegationTokenRenewer.java:83)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:1078)
> at
> org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:1055)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545)
> 2026-02-20 19:41:32,139 INFO rmapp.RMAppImpl: application_1771616432238_0001
> State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED
> 2026-02-20 19:41:32,139 WARN capacity.CapacityScheduler: Couldn't find
> application application_1771616432238_0001 {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
