Jose Luis López created HADOOP-19972:
----------------------------------------
Summary: Upgrade to Jetty 12 without changing the servlet namespace
Key: HADOOP-19972
URL: https://issues.apache.org/jira/browse/HADOOP-19972
Project: Hadoop Common
Issue Type: Task
Components: build, common
Reporter: Jose Luis López
WHY
===
Hadoop's web server is Jetty 9.4, which is end of life and no longer receives
security fixes. Getting off it is normally described as one large change,
because Jetty 10 and everything after it require the jakarta.servlet namespace.
That framing makes a security fix wait on an API break: the projects that embed
Hadoop's web stack - HBase, Hive, Spark, Ozone, Knox - would all have to move on
the same day.
Two facts sharpen the timing. Jetty 9.4 is unsupported now, and branch-3.4 and
branch-3.3 target Java 8 bytecode, so they can never take Jetty 12 - it needs
Java 17. For users, "off end-of-life Jetty" already means "on 3.5 or later".
Jetty 12 makes the two changes separable. It ships the servlet container as a
pluggable environment, and one of those environments, ee8, still serves
javax.servlet. This takes that route: a supported Jetty now, in an ordinary
minor release, with nothing for downstream projects to do.
WHAT CHANGES FOR ANYONE USING HADOOP
====================================
Hadoop runs on a supported web server. That is the whole of it.
No published method changes shape or disappears. The five public classes that
expose servlet types in their signatures expose exactly the same ones as before.
Projects that embed Hadoop's web stack need change nothing and need not rebuild.
No configuration setting changes.
Jetty 12 does behave differently from 9.4 in a few places, and those are handled
rather than passed on: a static file directory that would have started listing
its contents, a web socket upgrade that would have failed, and a listener that
could not be restarted once stopped. A deployment should see 9.4's behaviour
throughout.
WHAT CHANGES INSIDE
===================
Jetty moves from 9.4.58 to 12.0.37 on the ee8 environment. The servlet container
artifacts move under org.eclipse.jetty.ee8, and the servlet API now arrives as
org.eclipse.jetty.toolchain:jetty-servlet-api, which publishes the same
javax.servlet packages as the one it replaces.
Hadoop's HTTP server and its metrics, the YARN web application builder, the web
socket code and the remaining embedded servers - KMS, HttpFS, the scheduler
simulator and the YARN services API - are ported to the new API. The shaded
client artifacts and LICENSE-binary follow.
A new test states the contract this change is promising: the servlet types
Hadoop offers to the projects that embed it stay javax, and none of them may
become jakarta. It fails the day that stops being true.
Across the tree, 126 files still name javax.servlet and none names
jakarta.servlet - unchanged from today. 85 files changed in total.
WHAT THIS DOES NOT DO
=====================
It does not move Hadoop to jakarta.servlet, does not upgrade Jersey, and does
not touch the ee9, ee10 or ee11 environments.
ee8 is a staging post, not a destination. It preserves the namespace, but the
Jetty API port still had to happen, and ee8 carries a compatibility layer that
adapts every request. Unlike Jetty 10 and 11 it has a supported vendor behind
it, which is what makes it a reasonable place to stand for a release or two.
This is not an alternative to HADOOP-19912 and does not replace it.
HADOOP-19912 still lands the namespace change, in a major release, judged on its
own merits and with downstream projects warned. What this does is stop that
decision from being a precondition for fixing the security problem.
DEPENDS ON
==========
HADOOP-19970 and HADOOP-19971, in that order. The branch is stacked on both.
HADOOP-19970 matters functionally: without it, Jersey's test container drags a
Jetty 9 artifact onto around twenty test classpaths, and moving to Jetty 12
before that is fixed leaves those modules holding two Jetty versions at once -
which compiles and then fails at run time.
HOW IT WAS TESTED
=================
Beyond the unit suites, the areas where this kind of change goes wrong need
exercising against a running server rather than assumed: TLS configuration,
SPNEGO, WebHDFS, the YARN UI, KMS and HttpFS. HADOOP-19876 was an SSL
configuration regression on the current line and HADOOP-19848 a
NoSuchMethodError
from a patch-level bump, so that is where the risk sits.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
