morningman opened a new pull request, #69283:
URL: https://github.com/apache/doris/pull/69283
Cherry-picked from #68862 (master commit 677fc65e0fe) to branch-4.2.
### How this pick differs from #68862
branch-4.2 differs from master in a few places this change touches, so the
pick is adapted. Everything else applies as on master.
| Area | #68862 on master | This pick | Why |
|---|---|---|---|
| AWS SDK v1 | Removes `aws-java-sdk-s3/glue/sts` and `aws-java-sdk.version`
| Keeps every AWS SDK v1 dependency | fe-core on this branch still imports
`com.amazonaws` in 35 files: the bundled AWS Glue Data Catalog client,
`S3EventSourceOffsetProvider` and others |
| `jersey-apache-connector` | Not on the classpath | Managed at 3.1.11, the
version this branch already ships | Only this branch's fe-core has
`bce-java-sdk`, which depends on it. Spring Boot 4.1 imports the Jersey 4 BOM,
which no longer lists it, so it would fall back to the SDK's own 2.9.1 and
bring `javax.ws.rs-api` 2.0 next to Ranger's `jsr311-api` |
| `WebConfigurer` | Already imports `Ordered` | Adds the `Ordered` import |
The trailing-slash filter uses it; master added the import earlier for
`WebUiAvailabilityInterceptor` |
| `HttpServerCompatibilityTest` | `UiApiExceptionHandler` and
`Config.enable_web_ui`; `/rest/v1/ui/...` answers HTTP 401 |
`RestApiExceptionHandler`; `/rest/v1/...` answers HTTP 200 with `code` 401 in
the body | This branch has neither the UI API package nor the web UI switch,
and its REST error contract reports an authentication failure in the body |
| Java level of the new test | `java.net.http.HttpClient` and a record |
OkHttp, as the other HTTP tests on this branch, and a class | FE sources on
this branch still compile at the Java 8 language level
(`<release>8</release>`); the requests and assertions are the same |
| `ForwardToMasterTest` | Unchanged by #68862 | Expects the JSON a direct
request returns | See "Forwarded responses" below |
The remaining conflicts were textual: lines that exist only on this branch,
a shorter trino-main exclusion comment, and the fe-core `LdapClientTest` using
JUnit 4 `Assert`. The cglib removal, the `hadoop-yarn-client` exclusion,
SnakeYAML 2.6 and the Tomcat EL override removal are carried over unchanged.
#69232, which fixed master's compilation after #68862, is not needed here: AWS
SDK v1 stays, and `DorisHttpTestCase` on this branch does not import
`ServletWebServerApplicationContext`.
### Forwarded responses
`RestBaseController.forwardToMaster` sends a follower's request to the
master with a default `RestTemplate` and copies the caller's headers. When the
caller sends no `Accept` header, the `RestTemplate` picks it, and the order of
its converters decides the format of the master's answer:
| Spring Framework | Jackson converters in `new RestTemplate()` | Follower's
answer to a caller without `Accept` |
|---|---|---|
| 6.2.19 (before) | XML, JSON, Smile, CBOR, YAML | The master answers in XML
and the follower re-reads it, so lists come back wrapped in objects and nested
lists flattened: `"columnNames": {"columnNames": [...]}` |
| 7.0.9 (after) | JSON, Smile, CBOR, YAML, XML | The same JSON as a direct
request: `"columnNames": [...]`, `"rows": [[...]]` |
A caller that sends its own `Accept` header, as curl and browsers do, is not
affected: the header is copied to the master, which answers it as before.
master has the same change since #68862; its `ForwardToMasterTest` already
expects the direct-request shape, while this branch's version encoded the XML
round trip.
### What changes in the packaged FE
`mvn dependency:tree` over all 28 FE modules before and after this pick
shows 80 version changes, 21 removed and 30 added artifacts. Every compile or
runtime version this pick introduces is one that master ships with #68862,
apart from three `bce-java-sdk` Jersey modules, which follow the Jersey core to
4.0.2.
| | Before | After |
|---|---|---|
| Spring Boot / Framework / LDAP | 3.5.16 / 6.2.19 / 3.3.8 | 4.1.1 / 7.0.9 /
4.1.1 |
| Jetty | 12.0.38 (EE10) | 12.1.14 (EE11) |
| SnakeYAML | 2.0 | 2.6 |
| AWS SDK v1 | 1.12.669 | 1.12.669 (kept) |
| Removed from `fe/lib` | cglib 2.2; hadoop-yarn-client 3.4.2 with Jetty 9
WebSocket 9.4.58 (api, client, common) and jline 3.30.14; LatencyUtils 2.0.3,
which Micrometer 1.17 no longer uses | |
| Moved by the Spring Boot 4.1 BOM, as on master | Jersey 3.1.11, HK2 3.0.6,
Hibernate Validator 8.0.3, Tomcat EL 9.0.104, Reactor 3.7 / Reactor Netty 1.2,
Micrometer 1.15, Kotlin stdlib 1.9.25 | Jersey 4.0.2, HK2 4.0.0-M3, Hibernate
Validator 9.1.3, Tomcat EL 11.0.24, Reactor 3.8 / Reactor Netty 1.3, Micrometer
1.17, Kotlin stdlib 2.3.21 |
The YARN client classes that Hadoop's MapReduce code refers to
(`YarnClientUtils`, `ClientRMProxy` and others) still resolve from
`hadoop-client-api`; the only users of the removed jline are YARN's
`ContainerShellWebSocket` and Hadoop's own shaded copy. In the packaged jars
only commons-pool2's optional `CglibProxyHandler` and `CglibProxySource` refer
to cglib, and nothing uses those two classes.
### What problem does this PR solve?
Issue Number: None
Related PR: #68862
Problem Summary:
FE pins Spring Boot 3.5.16 and Spring Framework 6.2.19 after their OSS
support has ended. For
[CVE-2026-47883](https://spring.io/security/cve-2026-47883/),
[CVE-2026-47889](https://spring.io/security/cve-2026-47889/) and
[CVE-2026-47890](https://spring.io/security/cve-2026-47890/), the Framework 6.2
fixes are available only in enterprise-supported 6.2.20, while the OSS fixes
are in 7.0.9. Keeping the final OSS 6.2 release leaves this dependency without
an OSS security update path.
This PR upgrades Boot to 4.1.1, Framework to 7.0.9, Spring LDAP to 4.1.1 and
Jetty to 12.1.14 with Servlet 6.1 / EE11, retaining Java 17. It adapts the Boot
API packages and replaces the removed optional trailing-slash matching with
request wrapping before MVC routing and authentication. Boot's Jackson 2
compatibility module preserves existing HTTP JSON responses; SnakeYAML is
aligned with the Boot generation and the obsolete Tomcat EL override is removed.
It also removes the unused direct cglib dependency (AWS SDK v1 stays on this
branch, see above) and excludes the Hadoop YARN client that brings Jetty 9
WebSocket jars into FE. The Trino connector's existing Jetty 11 HTTP/2 client
dependencies remain pinned and require a separate upstream-client migration.
### Release note
Update the FE Spring and Jetty dependencies to supported OSS releases and
remove unused end-of-support libraries while preserving HTTP authentication,
JSON responses, cookies and trailing-slash URLs.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [x] Manual test (artifact dependency inspection, steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
Validation on branch-4.2:
- Build with Checkstyle enabled and the Maven build cache disabled; all 28
modules report 0 Checkstyle violations:
```bash
EXTRA_FE_MODULES='ldap=fe-authentication/fe-authentication-plugins/fe-authentication-plugin-ldap'
\
DISABLE_BUILD_UI=ON MVN_OPT='-Dmaven.build.cache.enabled=false' bash
build.sh --fe
```
- Run 31 test classes on the FE HTTP server and REST controllers (each
`DorisHttpTestCase` subclass starts the real `HttpServer`), fe-core LDAP, the
LDAP authentication plugin and the Trino connector: **206 tests, 205 passed, 1
skipped** (`CopyIntoTest` skips one case itself). The nested
`LdapAuthenticationPluginTest` suite runs separately, since a `*Test` filter
does not match its inner classes:
```bash
TZ=Asia/Shanghai FE_UT_PARALLEL=4 mvn test -Dcheckstyle.skip=true
-Dmaven.build.cache.enabled=false \
-DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -am \
-pl
fe-common,fe-core,fe-authentication/fe-authentication-plugins/fe-authentication-plugin-ldap,be-java-extensions/trino-connector-scanner
\
-Dtest='org.apache.doris.http.*Test,org/apache/doris/httpv2/**/*Test,org.apache.doris.datasource.doris.RemoteDoris*Test,org.apache.doris.common.util.DebugPointUtilTest,org.apache.doris.statistics.FollowerColumnSenderTest,org.apache.doris.mysql.authenticate.ldap.*Test,org.apache.doris.persist.LdapInfoTest,org.apache.doris.datasource.trinoconnector.*Test,org.apache.doris.authentication.plugin.ldap.*Test,org.apache.doris.trinoconnector.*Test'
# nested suite
TZ=Asia/Shanghai mvn test ... -pl
fe-authentication/fe-authentication-plugins/fe-authentication-plugin-ldap -am \
-Dtest='org/apache/doris/authentication/plugin/ldap/LdapAuthenticationPluginTest*'
```
- Compare `mvn dependency:tree` of every FE module with the branch head
before the pick and with master before and after #68862 (summary above), and
scan the class files in the packaged `fe/lib` for references to the removed
cglib, YARN client and jline classes.
- SQL regression suites were not run locally; this change targets web
dependencies and HTTP compatibility.
- Behavior changed:
- [ ] No.
- [x] Yes. Upgrade the web stack and normalize trailing slashes without
HTTP redirects; retain existing HTTP contracts. A follower answering a
forwarded request from a caller without an `Accept` header now returns the same
JSON as the master instead of the XML-round-tripped shape (see "Forwarded
responses").
- Does this need documentation?
- [x] No. No configuration or Java baseline changes are required.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
