[
https://issues.apache.org/jira/browse/HADOOP-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18067482#comment-18067482
]
ASF GitHub Bot commented on HADOOP-19848:
-----------------------------------------
surendralilhore opened a new pull request, #8365:
URL: https://github.com/apache/hadoop/pull/8365
### Description of PR
When using hadoop-azure with Apache Spark 4.0, AzureNativeFileSystemStore
fails to load with:
_`Caused by: java.lang.ExceptionInInitializerError: Exception
java.lang.NoSuchMethodError: 'java.util.Properties
org.eclipse.jetty.util.log.Log.getProperties()' [in thread "Thread-31"]`_
**Root cause:**
**AzureNativeFileSystemStore.createPermissionJsonSerializer()** calls
org.eclipse.jetty.util.log.Log.getProperties() to suppress Jetty's log
announcement. While this method exists in **jetty-util** 9.4.57 (as shipped
with Hadoop 3.4.x), Spark 4.0 bundles a different version of Jetty on the
runtime classpath where this method has been removed. This causes a
compile-vs-runtime classpath mismatch — the code compiles fine against Hadoop's
Jetty version but fails at runtime under Spark's classloader.
This is a fragile dependency on a Jetty internal API
(org.eclipse.jetty.util.log.Log) that is unnecessary. Jetty reads the
org.eclipse.jetty.util.log.announce property from system properties, so
**System.setProperty()** achieves the same effect without coupling to Jetty
internals.
### How was this patch tested?
Tested with Spark 4.0 Job. After this change Azure filesytem initialization
is successful
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
> AzureNativeFileSystemStore fails with NoSuchMethodError on Jetty 9.4.57+
> ------------------------------------------------------------------------
>
> Key: HADOOP-19848
> URL: https://issues.apache.org/jira/browse/HADOOP-19848
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.4.0
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
> Priority: Major
>
> When using hadoop-azure with Apache Spark 4.0, {{AzureNativeFileSystemStore}}
> fails to load with:
>
> {noformat}
> Caused by: java.lang.ExceptionInInitializerError: Exception
> java.lang.NoSuchMethodError: 'java.util.Properties
> org.eclipse.jetty.util.log.Log.getProperties()' [in thread "Thread-31"]
> at
> org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createPermissionJsonSerializer(AzureNativeFileSystemStore.java:429)
> at
> org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.<clinit>(AzureNativeFileSystemStore.java:331)
> at
> org.apache.hadoop.fs.azure.NativeAzureFileSystem.createDefaultStore(NativeAzureFileSystem.java:1485)
> at
> org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1410)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3615){noformat}
> *Root cause:*
> {{AzureNativeFileSystemStore.createPermissionJsonSerializer()}} calls
> {{org.eclipse.jetty.util.log.Log.getProperties()}} to suppress Jetty's log
> announcement. While this method exists in {{jetty-util}} 9.4.57 (as shipped
> with
> Hadoop 3.4.x), Spark 4.0 bundles a different version of Jetty on the
> runtime classpath where this method has been removed. This causes a
> compile-vs-runtime classpath mismatch — the code compiles fine against
> Hadoop's Jetty version but fails at runtime under Spark's classloader.
> This is a fragile dependency on a Jetty internal API
> ({{{}org.eclipse.jetty.util.log.Log{}}}) that is unnecessary. Jetty reads the
> {{org.eclipse.jetty.util.log.announce}} property from system properties, so
> {{System.setProperty() }}achieves the same effect without coupling to Jetty
> internals.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
