https://bz.apache.org/bugzilla/show_bug.cgi?id=70091
--- Comment #3 from Ivaylo Zhelev <[email protected]> --- Thanks for picking this up. ANSWERS TO YOUR TWO QUESTIONS ============================= Q1: Can the reverse proxy be configured to set :scheme to http rather than https? No. The proxy in our deployment generates :scheme from the original request's target URI, which is https because the public-facing URL is https. We are not aware of a config knob that overrides it. (Also, per RFC 9113 8.3.1, :scheme reflects the request target's scheme, not the wire's TLS state -- so configuring the proxy to send :scheme: http when the original request was https would arguably misrepresent the request to the application.) In addition, the upstream-most hop in front of the application container is configured as an L4 TCP forwarder: it terminates TLS and then passes the decrypted byte stream through verbatim. In that mode it cannot rewrite HTTP/2 frames even in principle, since it doesn't parse them. Q2: Can the reverse proxy be configured to proxy over h2 rather than h2c? Not without significant changes to the backend stack that aren't justifiable just to satisfy this check. Our deployment terminates TLS at the proxy and forwards over plain TCP to the application container, where Tomcat is configured with a plain (non-TLS) Connector. Switching to h2-over-TLS upstream would require: - terminating TLS on the Tomcat Connector itself (SSLHostConfig, key material, etc.) - coordinating cert provisioning and rotation at the connector level - per-request TLS overhead on every backend request - corresponding changes to the upstream proxy's mode For a fleet of stateless web frontends, that's a much larger change than the failure mode warrants. So in practice we cannot switch to h2 upstream. WHAT WE'RE DOING IN THE MEANTIME ================================ The practical workaround we found is to configure the proxy to forward HTTP/1.1 to the backend instead of h2c. The H2 codec never sees the request, so the new validation never triggers. We lose h2c multiplexing on the proxy->backend hop, but for HTML/REST workloads that's a non-issue. The workaround does NOT help any deployment that genuinely needs end-to-end HTTP/2 (e.g. gRPC backends) -- those are stuck on 10.1.54. ON THE DESIGN QUESTION (explicit configuration before accepting a mismatch) =========================================================================== Fully agree that some explicit operator opt-in makes sense -- silently accepting any :scheme mismatch would defeat the purpose of the check. RemoteIpValve is, as you note, heavy-handed for a pure TLS-termination case where the operator just wants to say "yes, this connector is behind a TLS terminator, treat the connection as secure". An alternative that might be the cleanest fit: have the new check also honor the existing secure="true" Connector attribute, since that attribute has been the documented Tomcat way of declaring exactly this for many years (it already causes request.isSecure() to return true, sets the secure-cookie flag, etc.). Concretely: // org.apache.coyote.http2.Stream, emitHeader(), case ":scheme" boolean secureConnection = handler.getProtocol().getHttp11Protocol().isSSLEnabled() || handler.getProtocol().getHttp11Protocol().getSecure(); if ("https".equals(value) != secureConnection) { headerException = new StreamException(...); } Rationale: a Connector with secure="true" is already an explicit operator declaration that "this connection should be treated as TLS-secured even though SSL is not enabled at this connector" -- which is exactly the TLS-offload reverse-proxy case. Honoring it in the new check would mean: - the operator opt-in is preserved (secure="true" is not the default) - no new attribute, no new docs - existing TLS-offload deployments that already have secure="true" (the canonical pattern) recover automatically once the patch ships The relaxedSchemeValidation attribute proposed in the original description would still be a fine fallback if there's a reason secure="true" shouldn't be trusted for this check, but in our case the two are semantically equivalent. ADDITIONAL REPRO SIGNALS ======================== In case useful for unit-testing the fix: - The OS-level socket table (e.g. /proc/net/tcp6) on the affected backend showed many TIME_WAIT connections cycling rapidly. Tomcat was accepting the TCP connection, reading the HTTP/2 preface and HEADERS frame, then sending GOAWAY + FIN immediately (server-side close). The proxy retried, producing the cycle. - Sending oversized headers over the same connection returned ENHANCE_YOUR_CALM (HTTP/2 error code 11), confirming the connection was reaching the H2 codec and being closed there rather than failing at TCP/TLS. - curl --http2-prior-knowledge http://<backend-ip>:<port>/<path> returned 200 (curl sends :scheme: http, which matches isSSLEnabled() == false). Only requests with :scheme: https arriving over the plaintext socket failed. -- You are receiving this mail because: You are the assignee for the bug. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
