Re: [PR] Simplify usage of custom ssl configuration [tomcat]
Hakky54 commented on PR #805: URL: https://github.com/apache/tomcat/pull/805#issuecomment-2671109381 Hi @markt-asf any thoughts regarding this pr? Would love to get your opinion/feedback as we have worked together on the other pr in the past -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PR] BZ69446 - add parameter maxPutFileSize in DefaultServlet [tomcat]
Chenjp commented on PR #823: URL: https://github.com/apache/tomcat/pull/823#issuecomment-2673452793 > > Rate limiter filter, or WAF can detect and block those requests. > > Either of those two can also provide the limits you are requesting, here. Adding a feature to Tomcat which requires additional protection from e.g. WAF is kinda silly when the WAF could provide the new feature as well. As said, when partial put is disabled, malicious user can send many separate requests, in this case, server side available storage is reduced slowly and can be observed by server side disk space monitoring, tomcat traffic rate limiter, and WAF blocker etc.. When partial put enabled, tomcat user have to face a fatal storage exhaustion in an instant if WAF block policy in Content-Range header semantic layer. In this circumstance, disk space monitoring is ineffective, traffic rate limiter cannot detect it. I think that WAF is not a controllable assert for most Tomcat users. Partial put is predefined in RFC spec, consider that there may be multiple implementations not just DefaultServlet and WebDavServlet, then enforce file size limit only in DefaultServlet is not good enough. How about ***an extra Filter*** to limit PUT destination size? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Further improvements to the CVE-2024-56337 protection
On Thu, Feb 20, 2025 at 1:06 PM Mark Thomas wrote: > > All, > > The recent releases have improved things for users of embedded Tomcat > but there are still some issues. I am seeing reports via $work related > to Spring Boot. > > The problem is on Windows and Mac. The file systems are case insensitive > and DirResourceSet instances are read/write by default so the reflection > code gets called. The impact varies by Java version: > > Java 21 - no issue as the code doesn't get called > > Java 17 - exception is thrown and the web application fails to start due >to missing --add-opens Where does the exception come from this time ? Rémy > Java 11 - warning is logged but otherwise web application starts >normally > > Java 8 - web application starts normally > > > Having chatted at length with the Spring Boot team, they are going to > make the DirResourceSet instances read only by default which works > around the Java 17 issue but not the Java 11 issue as reflection also > occurs in the init of JreCompat. > > I have performed some testing and confirmed that the java.io.FileSystem > class is loaded by the JRE before any application code gets a chance to > execute. That means we can use checking the command line arguments to > determine whether the cache is enabled/disabled. > > To reduce the impact on anyone using Tomcat in embedded mode - including > but not limited to Spring Boot - I intend to make the following > improvements: > > Java 17 > Check the command line arguments to determine if the canonical resource > name cache has been enabled > Switch to lazy initialisation for JreCompat.useCanonCachesField so > reflection is not used unless Tomcat needs to try and disable the cache > > Java 16 and below > Check the command line arguments to determine if the canonical resource > name cache has been disabled > Switch to lazy initialisation for JreCompat.useCanonCachesField so > reflection is not used unless Tomcat needs to try and disable the cache > > > I don't think there is a need to rush out these changes but I'm going to > start work on them now so they are ready if we need them in a hurry. > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Further improvements to the CVE-2024-56337 protection
On 20/02/2025 13:36, Rémy Maucherat wrote: On Thu, Feb 20, 2025 at 1:06 PM Mark Thomas wrote: All, The recent releases have improved things for users of embedded Tomcat but there are still some issues. I am seeing reports via $work related to Spring Boot. The problem is on Windows and Mac. The file systems are case insensitive and DirResourceSet instances are read/write by default so the reflection code gets called. The impact varies by Java version: Java 21 - no issue as the code doesn't get called Java 17 - exception is thrown and the web application fails to start due to missing --add-opens Where does the exception come from this time ? JreCompat.disableCanonCaches() Mark Rémy Java 11 - warning is logged but otherwise web application starts normally Java 8 - web application starts normally Having chatted at length with the Spring Boot team, they are going to make the DirResourceSet instances read only by default which works around the Java 17 issue but not the Java 11 issue as reflection also occurs in the init of JreCompat. I have performed some testing and confirmed that the java.io.FileSystem class is loaded by the JRE before any application code gets a chance to execute. That means we can use checking the command line arguments to determine whether the cache is enabled/disabled. To reduce the impact on anyone using Tomcat in embedded mode - including but not limited to Spring Boot - I intend to make the following improvements: Java 17 Check the command line arguments to determine if the canonical resource name cache has been enabled Switch to lazy initialisation for JreCompat.useCanonCachesField so reflection is not used unless Tomcat needs to try and disable the cache Java 16 and below Check the command line arguments to determine if the canonical resource name cache has been disabled Switch to lazy initialisation for JreCompat.useCanonCachesField so reflection is not used unless Tomcat needs to try and disable the cache I don't think there is a need to rush out these changes but I'm going to start work on them now so they are ready if we need them in a hurry. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PR] BZ69446 - add parameter maxPutFileSize in DefaultServlet [tomcat]
ChristopherSchultz commented on PR #823: URL: https://github.com/apache/tomcat/pull/823#issuecomment-2671713171 > > The feedback was that this serves no purpose. I believe it is correct. > > For processing of POST and some others like that, processing takes memory. Also we do not know what the app will do with the data. Hence limiting could be important and it justifies having some setting. > > But PUT in the DefaultServlet is not the same. Let's say a security conscious user such as you already disabled partial PUT. A malicious user can then write the same amount of data on the server storage using a trivial increase of the amount of input even if your setting is in place. The main difference would be using separate HTTP requests. So then you would be expected to catch that pattern using some kind of extra protection (where it would also be possible to check the request length) ? > > As a result, adding this setting still does not make sense to me. > > Rate limiter filter, or WAF can detect and block those requests. Either of those two can also provide the limits you are requesting, here. Adding a feature to Tomcat which requires additional protection from e.g. WAF is kinda silly when the WAF could provide the new feature as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Further improvements to the CVE-2024-56337 protection
On 20/02/2025 13:52, Rémy Maucherat wrote: On Thu, Feb 20, 2025 at 2:42 PM Mark Thomas wrote: On 20/02/2025 13:36, Rémy Maucherat wrote: On Thu, Feb 20, 2025 at 1:06 PM Mark Thomas wrote: All, The recent releases have improved things for users of embedded Tomcat but there are still some issues. I am seeing reports via $work related to Spring Boot. The problem is on Windows and Mac. The file systems are case insensitive and DirResourceSet instances are read/write by default so the reflection code gets called. The impact varies by Java version: Java 21 - no issue as the code doesn't get called Java 17 - exception is thrown and the web application fails to start due to missing --add-opens Where does the exception come from this time ? JreCompat.disableCanonCaches() So in that case useCanonCachesField is not null, but using it fails ? That's unexpected. 20-Feb-2025 15:18:54.326 WARNING [main] org.apache.tomcat.util.compat.JreCompat.disableCanonCaches Failed to set the java.io.FileSystem.useCanonCaches static field java.lang.IllegalAccessException: class org.apache.tomcat.util.compat.JreCompat cannot access a member of class java.io.FileSystem (in module java.base) with modifiers "static final" at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392) at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674) at java.base/java.lang.reflect.Field.checkAccess(Field.java:1102) at java.base/java.lang.reflect.Field.set(Field.java:797) at org.apache.tomcat.util.compat.JreCompat.disableCanonCaches(JreCompat.java:274) at org.apache.catalina.webresources.DirResourceSet.initInternal(DirResourceSet.java:353) OTOH, some users were randomly saying that things were ok on Java 17, and others not. So there's something. Likely OS related. I'm making progress with the updates for Tomcat 11. Should have something to commit soon. Mark Rémy Mark Rémy Java 11 - warning is logged but otherwise web application starts normally Java 8 - web application starts normally Having chatted at length with the Spring Boot team, they are going to make the DirResourceSet instances read only by default which works around the Java 17 issue but not the Java 11 issue as reflection also occurs in the init of JreCompat. I have performed some testing and confirmed that the java.io.FileSystem class is loaded by the JRE before any application code gets a chance to execute. That means we can use checking the command line arguments to determine whether the cache is enabled/disabled. To reduce the impact on anyone using Tomcat in embedded mode - including but not limited to Spring Boot - I intend to make the following improvements: Java 17 Check the command line arguments to determine if the canonical resource name cache has been enabled Switch to lazy initialisation for JreCompat.useCanonCachesField so reflection is not used unless Tomcat needs to try and disable the cache Java 16 and below Check the command line arguments to determine if the canonical resource name cache has been disabled Switch to lazy initialisation for JreCompat.useCanonCachesField so reflection is not used unless Tomcat needs to try and disable the cache I don't think there is a need to rush out these changes but I'm going to start work on them now so they are ready if we need them in a hurry. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 69575] org.apache.coyote.CompressionConfig doesn't recognize "zstd" compression
https://bz.apache.org/bugzilla/show_bug.cgi?id=69575 --- Comment #5 from Remy Maucherat --- It seems to me it's a good time to (finally) add transfer-encoding support since it's the right way to do this. Nobody will use it though ;) It would be used if the client submits a "TE: gzip" header, and Tomcat will then respond with a "Transfer-Encoding: gzip, chunked" if on the fly compression was used for the response. It's also compatible with use of strong etags (which Tomcat supports now, another one of the Do The Right Thing (TM) item list). -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 01/02: Remove the case sensitivity check
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 11.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 4ea9c0043f569fba3e63c7982b081539ad72c080
Author: Mark Thomas
AuthorDate: Thu Feb 20 12:49:53 2025 +
Remove the case sensitivity check
The performance impact is minimal and getting the check right in all
cases is difficult due to various edge cases
---
.../catalina/webresources/DirResourceSet.java | 74 +++---
1 file changed, 9 insertions(+), 65 deletions(-)
diff --git a/java/org/apache/catalina/webresources/DirResourceSet.java
b/java/org/apache/catalina/webresources/DirResourceSet.java
index dbbc126c41..3e9676bdd3 100644
--- a/java/org/apache/catalina/webresources/DirResourceSet.java
+++ b/java/org/apache/catalina/webresources/DirResourceSet.java
@@ -46,8 +46,6 @@ public class DirResourceSet extends AbstractFileResourceSet
implements WebResour
private static final Log log = LogFactory.getLog(DirResourceSet.class);
-private boolean caseSensitive = true;
-
private Map resourceLocksByPath = new HashMap<>();
private Object resourceLocksByPathLock = new Object();
@@ -323,7 +321,6 @@ public class DirResourceSet extends AbstractFileResourceSet
implements WebResour
@Override
protected void initInternal() throws LifecycleException {
super.initInternal();
-caseSensitive = isCaseSensitive();
// Is this an exploded web application?
if (getWebAppMount().equals("")) {
// Look for a manifest
@@ -337,10 +334,6 @@ public class DirResourceSet extends
AbstractFileResourceSet implements WebResour
}
}
// Check for exposure to CVE-2024-56337
-if (caseSensitive) {
-// CVE-2024-56337 (nor CVE-2024-50379) is not exploitable on a
case sensitive file system
-return;
-}
if (isReadOnly()) {
// CVE-2024-56337 (nor CVE-2024-50379) is not exploitable on a
read-only ResourceSet
return;
@@ -367,65 +360,16 @@ public class DirResourceSet extends
AbstractFileResourceSet implements WebResour
}
-/*
- * Determines if this ResourceSet is based on a case sensitive file system
or not.
- *
- * File systems are usually case sensitive or not. Windows, via the
command 'fsutil.exe file setCaseSensitiveInfo
- * enable', may be case sensitive in some directories and case
insensitive in others.
- *
- * If this method incorrectly determines that the DirResourceSet is case
sensitive, the file locking mechanism that
- * ensures write operations are performed atomically will not operate
correctly. If this method incorrectly
- * determines that the DirResourceSet is case insensitive, there is a
small performance penalty for writes.
- *
- * Given the above, this method only reports the file system as case
sensitive if no indication of case
- * insensitivity is detected. This does mean that Windows based
DirResourceSet instances will be reported as case
- * insensitive even all of the directories in the DirResourceSet have been
configured as case sensitive.
- */
-private boolean isCaseSensitive() {
-try {
-String canonicalPath = getFileBase().getCanonicalPath();
-/*
- * If any lower case characters are found in the canonical file
name formed by converting the test file name
- * to upper case, the underlying file system must be, at least in
part, case insensitive.
- */
-File upper = new File(canonicalPath.toUpperCase(Locale.ENGLISH));
-String upperCanonicalPath = upper.getCanonicalPath();
-char[] upperCharacters = upperCanonicalPath.toCharArray();
-for (char c : upperCharacters) {
-if (Character.isLowerCase(c)) {
-return false;
-}
-}
-
-/*
- * If any upper case characters are found in the canonical file
name formed by converting the test file name
- * to lower case, the underlying file system must be, at least in
part, case insensitive.
- */
-File lower = new File(canonicalPath.toLowerCase(Locale.ENGLISH));
-String lowerCanonicalPath = lower.getCanonicalPath();
-char[] lowerCharacters = lowerCanonicalPath.toCharArray();
-for (char c : lowerCharacters) {
-if (Character.isUpperCase(c)) {
-return false;
-}
-}
-
-return true;
-} catch (IOException ioe) {
-log.warn(sm.getString("dirResourceSet.isCaseSensitive.fail",
getFileBase().getAbsolutePath()), ioe);
-}
-
-return false;
-}
-
-
private String getLockKey(String path) {
-// Normalize path to ensure that the same key is used fo
(tomcat) branch 11.0.x updated (d93b96e085 -> 413d82fab6)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git from d93b96e085 Remove unnecessary configuration new 4ea9c0043f Remove the case sensitivity check new 413d82fab6 Improve CVE-2024-56337 protection The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: bin/catalina.bat | 1 + bin/catalina.sh| 1 + .../catalina/webresources/DirResourceSet.java | 74 ++--- java/org/apache/tomcat/util/compat/JreCompat.java | 119 +++-- webapps/docs/changelog.xml | 6 +- 5 files changed, 104 insertions(+), 97 deletions(-) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 02/02: Improve CVE-2024-56337 protection
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 11.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 413d82fab675d43653bda5513ae7cfc777c80c2c
Author: Mark Thomas
AuthorDate: Thu Feb 20 16:45:02 2025 +
Improve CVE-2024-56337 protection
---
bin/catalina.bat | 1 +
bin/catalina.sh | 1 +
java/org/apache/tomcat/util/compat/JreCompat.java | 119 --
webapps/docs/changelog.xml| 6 +-
4 files changed, 95 insertions(+), 32 deletions(-)
diff --git a/bin/catalina.bat b/bin/catalina.bat
index 78a7eb9d9f..6c7f1baced 100755
--- a/bin/catalina.bat
+++ b/bin/catalina.bat
@@ -215,6 +215,7 @@ set
LOGGING_MANAGER=-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogMa
rem Configure module start-up parameters
set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.base/java.lang=ALL-UNNAMED"
+set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.base/java.lang.reflect=ALL-UNNAMED"
set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.base/java.io=ALL-UNNAMED"
set "JAVA_OPTS=%JAVA_OPTS% --add-opens=java.base/java.util=ALL-UNNAMED"
set "JAVA_OPTS=%JAVA_OPTS%
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
diff --git a/bin/catalina.sh b/bin/catalina.sh
index ee679ad0c6..885aaf721d 100755
--- a/bin/catalina.sh
+++ b/bin/catalina.sh
@@ -288,6 +288,7 @@ fi
# Add the module start-up parameters required by Tomcat
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.lang=ALL-UNNAMED"
+JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.lang.reflect=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.io=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
diff --git a/java/org/apache/tomcat/util/compat/JreCompat.java
b/java/org/apache/tomcat/util/compat/JreCompat.java
index adbd6b6d92..d40a6413c9 100644
--- a/java/org/apache/tomcat/util/compat/JreCompat.java
+++ b/java/org/apache/tomcat/util/compat/JreCompat.java
@@ -16,9 +16,16 @@
*/
package org.apache.tomcat.util.compat;
+import java.lang.invoke.MethodHandles;
+import java.lang.invoke.MethodHandles.Lookup;
+import java.lang.invoke.VarHandle;
+import java.lang.management.ManagementFactory;
import java.lang.reflect.Field;
import java.lang.reflect.InaccessibleObjectException;
+import java.lang.reflect.Modifier;
import java.security.PrivilegedExceptionAction;
+import java.util.List;
+import java.util.Optional;
import java.util.concurrent.Callable;
import java.util.concurrent.CompletionException;
@@ -43,7 +50,11 @@ public class JreCompat {
private static final boolean jre21Available;
private static final boolean jre22Available;
-private static final Field useCanonCachesField;
+private static final String USE_CANON_CACHES_CMD_ARG =
"-Dsun.io.useCanonCaches=";
+private static volatile Boolean canonCachesDisabled;
+private static final Object canonCachesDisabledLock = new Object();
+private static volatile Optional useCanonCachesField;
+private static final Object useCanonCachesFieldLock = new Object();
static {
boolean result = false;
@@ -80,21 +91,6 @@ public class JreCompat {
jre21Available = false;
jre19Available = false;
}
-
-Field f1 = null;
-try {
-Class clazz = Class.forName("java.io.FileSystem");
-f1 = clazz.getDeclaredField("useCanonCaches");
-f1.setAccessible(true);
-} catch (InaccessibleObjectException | ReflectiveOperationException |
IllegalArgumentException e) {
-/*
- * Log at debug level as this will only be an issue if the field
needs to be accessed and most
- * configurations will not need to do so. Appropriate warnings
will be logged if an attempt is made to use
- * the field when it could not be found/accessed.
- */
-log.debug(sm.getString("jreCompat.useCanonCaches.init"), e);
-}
-useCanonCachesField = f1;
}
@@ -234,28 +230,43 @@ public class JreCompat {
/*
- * The behaviour of the canonical file cache varies by Java version.
+ * The behaviour of the canonical file name cache varies by Java version.
*
* The cache was removed in Java 21 so these methods and the associated
code can be removed once the minimum Java
* version is 21.
*
- * For 12 <= Java <= 20, the cache was present but disabled by default.
Since the user may have changed the default
- * Tomcat has to assume the cache is enabled unless proven otherwise.
+ * For 12 <= Java <= 20, the cache was present but disabled by default.
*
- * For Java < 12, the cache was enabled by default. Tomcat assumes the
cache is enabled unless proven otherwise.
+ * Tomcat 11 has a minimum Java version of
(tomcat) branch main updated: Update with transfer-encoding support
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/main by this push:
new 2eb8e19552 Update with transfer-encoding support
2eb8e19552 is described below
commit 2eb8e19552ec78cc922cba2f1cbd05940e8b6951
Author: remm
AuthorDate: Thu Feb 20 18:57:39 2025 +0100
Update with transfer-encoding support
Although there is no client browser support, and there may never be any,
Transfer-Encoding is the right way to do compression the way Tomcat does
it in GzipOutputFilter.
This will be used if the client sends a TE: gzip header.
---
java/org/apache/coyote/CompressionConfig.java | 75 ++--
java/org/apache/tomcat/util/http/parser/TE.java | 84 +++
test/org/apache/coyote/TestCompressionConfig.java | 59
webapps/docs/changelog.xml| 5 ++
4 files changed, 189 insertions(+), 34 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index d4b27f3a5d..3545db1a52 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -32,6 +32,7 @@ import org.apache.tomcat.util.buf.MessageBytes;
import org.apache.tomcat.util.http.MimeHeaders;
import org.apache.tomcat.util.http.ResponseUtil;
import org.apache.tomcat.util.http.parser.AcceptEncoding;
+import org.apache.tomcat.util.http.parser.TE;
import org.apache.tomcat.util.http.parser.TokenList;
import org.apache.tomcat.util.res.StringManager;
@@ -192,6 +193,8 @@ public class CompressionConfig {
return false;
}
+boolean useTE = false;
+
MimeHeaders responseHeaders = response.getMimeHeaders();
// Check if content is not already compressed
@@ -230,40 +233,63 @@ public class CompressionConfig {
}
}
-// Check if the resource has a strong ETag
-String eTag = responseHeaders.getHeader("ETag");
-if (eTag != null && !eTag.trim().startsWith("W/")) {
-// Has an ETag that doesn't start with "W/..." so it must be a
-// strong ETag
-return false;
-}
-
-// If processing reaches this far, the response might be compressed.
-// Therefore, set the Vary header to keep proxies happy
-ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
-
-// Check if user-agent supports gzip encoding
-// Only interested in whether gzip encoding is supported. Other
-// encodings and weights can be ignored.
-Enumeration headerValues =
request.getMimeHeaders().values("accept-encoding");
+Enumeration headerValues =
request.getMimeHeaders().values("TE");
boolean foundGzip = false;
+// TE and accept-encoding seem to have equivalent syntax
while (!foundGzip && headerValues.hasMoreElements()) {
-List acceptEncodings = null;
+List tes = null;
try {
-acceptEncodings = AcceptEncoding.parse(new
StringReader(headerValues.nextElement()));
+tes = TE.parse(new StringReader(headerValues.nextElement()));
} catch (IOException ioe) {
// If there is a problem reading the header, disable
compression
return false;
}
-for (AcceptEncoding acceptEncoding : acceptEncodings) {
-if ("gzip".equalsIgnoreCase(acceptEncoding.getEncoding())) {
+for (TE te : tes) {
+if ("gzip".equalsIgnoreCase(te.getEncoding())) {
+useTE = true;
foundGzip = true;
break;
}
}
}
+// Check if the resource has a strong ETag
+String eTag = responseHeaders.getHeader("ETag");
+if (!useTE && eTag != null && !eTag.trim().startsWith("W/")) {
+// Has an ETag that doesn't start with "W/..." so it must be a
+// strong ETag
+return false;
+}
+
+if (!useTE) {
+// If processing reaches this far, the response might be
compressed.
+// Therefore, set the Vary header to keep proxies happy
+ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
+
+// Check if user-agent supports gzip encoding
+// Only interested in whether gzip encoding is supported. Other
+// encodings and weights can be ignored.
+headerValues = request.getMimeHeaders().values("accept-encoding");
+foundGzip = false;
+while (!foundGzip && headerValues.hasMoreElements()) {
+List acceptEncodings = null;
+try {
+acceptEncodings = AcceptEncoding.parse(new
(tomcat) branch 11.0.x updated: Update with transfer-encoding support
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 11.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/11.0.x by this push:
new ae67bb4bb6 Update with transfer-encoding support
ae67bb4bb6 is described below
commit ae67bb4bb6eaf2f5aa2d3eecccea8406878e501b
Author: remm
AuthorDate: Thu Feb 20 18:57:39 2025 +0100
Update with transfer-encoding support
Although there is no client browser support, and there may never be any,
Transfer-Encoding is the right way to do compression the way Tomcat does
it in GzipOutputFilter.
This will be used if the client sends a TE: gzip header.
---
java/org/apache/coyote/CompressionConfig.java | 75 ++--
java/org/apache/tomcat/util/http/parser/TE.java | 84 +++
test/org/apache/coyote/TestCompressionConfig.java | 59
webapps/docs/changelog.xml| 5 ++
4 files changed, 189 insertions(+), 34 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index d4b27f3a5d..3545db1a52 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -32,6 +32,7 @@ import org.apache.tomcat.util.buf.MessageBytes;
import org.apache.tomcat.util.http.MimeHeaders;
import org.apache.tomcat.util.http.ResponseUtil;
import org.apache.tomcat.util.http.parser.AcceptEncoding;
+import org.apache.tomcat.util.http.parser.TE;
import org.apache.tomcat.util.http.parser.TokenList;
import org.apache.tomcat.util.res.StringManager;
@@ -192,6 +193,8 @@ public class CompressionConfig {
return false;
}
+boolean useTE = false;
+
MimeHeaders responseHeaders = response.getMimeHeaders();
// Check if content is not already compressed
@@ -230,40 +233,63 @@ public class CompressionConfig {
}
}
-// Check if the resource has a strong ETag
-String eTag = responseHeaders.getHeader("ETag");
-if (eTag != null && !eTag.trim().startsWith("W/")) {
-// Has an ETag that doesn't start with "W/..." so it must be a
-// strong ETag
-return false;
-}
-
-// If processing reaches this far, the response might be compressed.
-// Therefore, set the Vary header to keep proxies happy
-ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
-
-// Check if user-agent supports gzip encoding
-// Only interested in whether gzip encoding is supported. Other
-// encodings and weights can be ignored.
-Enumeration headerValues =
request.getMimeHeaders().values("accept-encoding");
+Enumeration headerValues =
request.getMimeHeaders().values("TE");
boolean foundGzip = false;
+// TE and accept-encoding seem to have equivalent syntax
while (!foundGzip && headerValues.hasMoreElements()) {
-List acceptEncodings = null;
+List tes = null;
try {
-acceptEncodings = AcceptEncoding.parse(new
StringReader(headerValues.nextElement()));
+tes = TE.parse(new StringReader(headerValues.nextElement()));
} catch (IOException ioe) {
// If there is a problem reading the header, disable
compression
return false;
}
-for (AcceptEncoding acceptEncoding : acceptEncodings) {
-if ("gzip".equalsIgnoreCase(acceptEncoding.getEncoding())) {
+for (TE te : tes) {
+if ("gzip".equalsIgnoreCase(te.getEncoding())) {
+useTE = true;
foundGzip = true;
break;
}
}
}
+// Check if the resource has a strong ETag
+String eTag = responseHeaders.getHeader("ETag");
+if (!useTE && eTag != null && !eTag.trim().startsWith("W/")) {
+// Has an ETag that doesn't start with "W/..." so it must be a
+// strong ETag
+return false;
+}
+
+if (!useTE) {
+// If processing reaches this far, the response might be
compressed.
+// Therefore, set the Vary header to keep proxies happy
+ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
+
+// Check if user-agent supports gzip encoding
+// Only interested in whether gzip encoding is supported. Other
+// encodings and weights can be ignored.
+headerValues = request.getMimeHeaders().values("accept-encoding");
+foundGzip = false;
+while (!foundGzip && headerValues.hasMoreElements()) {
+List acceptEncodings = null;
+try {
+acceptEncodings = AcceptEncoding.parse(
(tomcat) branch 9.0.x updated: Update with transfer-encoding support
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new ac14646dba Update with transfer-encoding support
ac14646dba is described below
commit ac14646dba3221e8cd99c920ddcf0cec809c1fbe
Author: remm
AuthorDate: Thu Feb 20 18:57:39 2025 +0100
Update with transfer-encoding support
Although there is no client browser support, and there may never be any,
Transfer-Encoding is the right way to do compression the way Tomcat does
it in GzipOutputFilter.
This will be used if the client sends a TE: gzip header.
---
java/org/apache/coyote/CompressionConfig.java | 59
java/org/apache/tomcat/util/http/parser/TE.java | 84 +++
test/org/apache/coyote/TestCompressionConfig.java | 69 ++-
webapps/docs/changelog.xml| 5 ++
4 files changed, 187 insertions(+), 30 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index c7fe685f29..7bd2c186db 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -32,6 +32,7 @@ import org.apache.tomcat.util.buf.MessageBytes;
import org.apache.tomcat.util.http.MimeHeaders;
import org.apache.tomcat.util.http.ResponseUtil;
import org.apache.tomcat.util.http.parser.AcceptEncoding;
+import org.apache.tomcat.util.http.parser.TE;
import org.apache.tomcat.util.http.parser.TokenList;
import org.apache.tomcat.util.res.StringManager;
@@ -219,6 +220,8 @@ public class CompressionConfig {
return false;
}
+boolean useTE = false;
+
MimeHeaders responseHeaders = response.getMimeHeaders();
// Check if content is not already compressed
@@ -267,32 +270,55 @@ public class CompressionConfig {
}
}
-// If processing reaches this far, the response might be compressed.
-// Therefore, set the Vary header to keep proxies happy
-ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
-
-// Check if user-agent supports gzip encoding
-// Only interested in whether gzip encoding is supported. Other
-// encodings and weights can be ignored.
-Enumeration headerValues =
request.getMimeHeaders().values("accept-encoding");
+Enumeration headerValues =
request.getMimeHeaders().values("TE");
boolean foundGzip = false;
+// TE and accept-encoding seem to have equivalent syntax
while (!foundGzip && headerValues.hasMoreElements()) {
-List acceptEncodings = null;
+List tes = null;
try {
-acceptEncodings = AcceptEncoding.parse(new
StringReader(headerValues.nextElement()));
+tes = TE.parse(new StringReader(headerValues.nextElement()));
} catch (IOException ioe) {
// If there is a problem reading the header, disable
compression
return false;
}
-for (AcceptEncoding acceptEncoding : acceptEncodings) {
-if ("gzip".equalsIgnoreCase(acceptEncoding.getEncoding())) {
+for (TE te : tes) {
+if ("gzip".equalsIgnoreCase(te.getEncoding())) {
+useTE = true;
foundGzip = true;
break;
}
}
}
+if (!useTE) {
+// If processing reaches this far, the response might be
compressed.
+// Therefore, set the Vary header to keep proxies happy
+ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
+
+// Check if user-agent supports gzip encoding
+// Only interested in whether gzip encoding is supported. Other
+// encodings and weights can be ignored.
+headerValues = request.getMimeHeaders().values("accept-encoding");
+foundGzip = false;
+while (!foundGzip && headerValues.hasMoreElements()) {
+List acceptEncodings = null;
+try {
+acceptEncodings = AcceptEncoding.parse(new
StringReader(headerValues.nextElement()));
+} catch (IOException ioe) {
+// If there is a problem reading the header, disable
compression
+return false;
+}
+
+for (AcceptEncoding acceptEncoding : acceptEncodings) {
+if ("gzip".equalsIgnoreCase(acceptEncoding.getEncoding()))
{
+foundGzip = true;
+break;
+}
+}
+}
+}
+
if (!foundGzip) {
return false;
}
@@ -316,8 +342,13 @@ public class Compressi
(tomcat) branch 10.1.x updated: Update with transfer-encoding support
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 10.1.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/10.1.x by this push:
new 47adb1053f Update with transfer-encoding support
47adb1053f is described below
commit 47adb1053f137926bc62744021830978d6aa5bc7
Author: remm
AuthorDate: Thu Feb 20 18:57:39 2025 +0100
Update with transfer-encoding support
Although there is no client browser support, and there may never be any,
Transfer-Encoding is the right way to do compression the way Tomcat does
it in GzipOutputFilter.
This will be used if the client sends a TE: gzip header.
---
java/org/apache/coyote/CompressionConfig.java | 75 ++--
java/org/apache/tomcat/util/http/parser/TE.java | 84 +++
test/org/apache/coyote/TestCompressionConfig.java | 59
webapps/docs/changelog.xml| 5 ++
4 files changed, 189 insertions(+), 34 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index d4b27f3a5d..3545db1a52 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -32,6 +32,7 @@ import org.apache.tomcat.util.buf.MessageBytes;
import org.apache.tomcat.util.http.MimeHeaders;
import org.apache.tomcat.util.http.ResponseUtil;
import org.apache.tomcat.util.http.parser.AcceptEncoding;
+import org.apache.tomcat.util.http.parser.TE;
import org.apache.tomcat.util.http.parser.TokenList;
import org.apache.tomcat.util.res.StringManager;
@@ -192,6 +193,8 @@ public class CompressionConfig {
return false;
}
+boolean useTE = false;
+
MimeHeaders responseHeaders = response.getMimeHeaders();
// Check if content is not already compressed
@@ -230,40 +233,63 @@ public class CompressionConfig {
}
}
-// Check if the resource has a strong ETag
-String eTag = responseHeaders.getHeader("ETag");
-if (eTag != null && !eTag.trim().startsWith("W/")) {
-// Has an ETag that doesn't start with "W/..." so it must be a
-// strong ETag
-return false;
-}
-
-// If processing reaches this far, the response might be compressed.
-// Therefore, set the Vary header to keep proxies happy
-ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
-
-// Check if user-agent supports gzip encoding
-// Only interested in whether gzip encoding is supported. Other
-// encodings and weights can be ignored.
-Enumeration headerValues =
request.getMimeHeaders().values("accept-encoding");
+Enumeration headerValues =
request.getMimeHeaders().values("TE");
boolean foundGzip = false;
+// TE and accept-encoding seem to have equivalent syntax
while (!foundGzip && headerValues.hasMoreElements()) {
-List acceptEncodings = null;
+List tes = null;
try {
-acceptEncodings = AcceptEncoding.parse(new
StringReader(headerValues.nextElement()));
+tes = TE.parse(new StringReader(headerValues.nextElement()));
} catch (IOException ioe) {
// If there is a problem reading the header, disable
compression
return false;
}
-for (AcceptEncoding acceptEncoding : acceptEncodings) {
-if ("gzip".equalsIgnoreCase(acceptEncoding.getEncoding())) {
+for (TE te : tes) {
+if ("gzip".equalsIgnoreCase(te.getEncoding())) {
+useTE = true;
foundGzip = true;
break;
}
}
}
+// Check if the resource has a strong ETag
+String eTag = responseHeaders.getHeader("ETag");
+if (!useTE && eTag != null && !eTag.trim().startsWith("W/")) {
+// Has an ETag that doesn't start with "W/..." so it must be a
+// strong ETag
+return false;
+}
+
+if (!useTE) {
+// If processing reaches this far, the response might be
compressed.
+// Therefore, set the Vary header to keep proxies happy
+ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
+
+// Check if user-agent supports gzip encoding
+// Only interested in whether gzip encoding is supported. Other
+// encodings and weights can be ignored.
+headerValues = request.getMimeHeaders().values("accept-encoding");
+foundGzip = false;
+while (!foundGzip && headerValues.hasMoreElements()) {
+List acceptEncodings = null;
+try {
+acceptEncodings = AcceptEncoding.parse(
(tomcat) branch main updated: Further improve with feedback from 69575
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/main by this push:
new 080fa6cf7e Further improve with feedback from 69575
080fa6cf7e is described below
commit 080fa6cf7e4b4e74fab110eb10a95539ce027179
Author: remm
AuthorDate: Thu Feb 20 21:41:26 2025 +0100
Further improve with feedback from 69575
Add to existing content-encoding header values when compressing.
Skip compression with content-encoding with "identity".
Add more content-encoding values based on HTTP Content Coding Registry
(skip deprecated values).
---
java/org/apache/coyote/CompressionConfig.java | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index 3545db1a52..b32c18ef1c 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -194,6 +194,7 @@ public class CompressionConfig {
}
boolean useTE = false;
+boolean useCE = true;
MimeHeaders responseHeaders = response.getMimeHeaders();
@@ -211,8 +212,13 @@ public class CompressionConfig {
log.warn(sm.getString("compressionConfig.ContentEncodingParseFail"), e);
return false;
}
-if (tokens.contains("gzip") || tokens.contains("compress") ||
tokens.contains("deflate")
-|| tokens.contains("br") || tokens.contains("zstd")) {
+if (tokens.contains("identity")) {
+// If identity, do not do content modifications
+useCE = false;
+} else if (tokens.contains("br") || tokens.contains("compress") ||
tokens.contains("dcb")
+|| tokens.contains("dcz") || tokens.contains("deflate") ||
tokens.contains("gzip")
+|| tokens.contains("pack200-gzip") ||
tokens.contains("zstd")) {
+// Content should not be compressed twice
return false;
}
}
@@ -262,7 +268,7 @@ public class CompressionConfig {
return false;
}
-if (!useTE) {
+if (useCE && !useTE) {
// If processing reaches this far, the response might be
compressed.
// Therefore, set the Vary header to keep proxies happy
ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
@@ -318,7 +324,7 @@ public class CompressionConfig {
responseHeaders.addValue("Transfer-Encoding").setString("gzip");
} else {
// Configure the content encoding for compressed content
-responseHeaders.setValue("Content-Encoding").setString("gzip");
+responseHeaders.addValue("Content-Encoding").setString("gzip");
}
return true;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch 11.0.x updated: Further improve with feedback from 69575
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 11.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/11.0.x by this push:
new 6c9817e102 Further improve with feedback from 69575
6c9817e102 is described below
commit 6c9817e102ffbb8b3a896ef5b55e82b9fe8b5eec
Author: remm
AuthorDate: Thu Feb 20 21:41:26 2025 +0100
Further improve with feedback from 69575
Add to existing content-encoding header values when compressing.
Skip compression with content-encoding with "identity".
Add more content-encoding values based on HTTP Content Coding Registry
(skip deprecated values).
---
java/org/apache/coyote/CompressionConfig.java | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index 3545db1a52..b32c18ef1c 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -194,6 +194,7 @@ public class CompressionConfig {
}
boolean useTE = false;
+boolean useCE = true;
MimeHeaders responseHeaders = response.getMimeHeaders();
@@ -211,8 +212,13 @@ public class CompressionConfig {
log.warn(sm.getString("compressionConfig.ContentEncodingParseFail"), e);
return false;
}
-if (tokens.contains("gzip") || tokens.contains("compress") ||
tokens.contains("deflate")
-|| tokens.contains("br") || tokens.contains("zstd")) {
+if (tokens.contains("identity")) {
+// If identity, do not do content modifications
+useCE = false;
+} else if (tokens.contains("br") || tokens.contains("compress") ||
tokens.contains("dcb")
+|| tokens.contains("dcz") || tokens.contains("deflate") ||
tokens.contains("gzip")
+|| tokens.contains("pack200-gzip") ||
tokens.contains("zstd")) {
+// Content should not be compressed twice
return false;
}
}
@@ -262,7 +268,7 @@ public class CompressionConfig {
return false;
}
-if (!useTE) {
+if (useCE && !useTE) {
// If processing reaches this far, the response might be
compressed.
// Therefore, set the Vary header to keep proxies happy
ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
@@ -318,7 +324,7 @@ public class CompressionConfig {
responseHeaders.addValue("Transfer-Encoding").setString("gzip");
} else {
// Configure the content encoding for compressed content
-responseHeaders.setValue("Content-Encoding").setString("gzip");
+responseHeaders.addValue("Content-Encoding").setString("gzip");
}
return true;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch 10.1.x updated: Further improve with feedback from 69575
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 10.1.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/10.1.x by this push:
new e62b04efe1 Further improve with feedback from 69575
e62b04efe1 is described below
commit e62b04efe16f32cd10499550bcbd8b7c6a453818
Author: remm
AuthorDate: Thu Feb 20 21:41:26 2025 +0100
Further improve with feedback from 69575
Add to existing content-encoding header values when compressing.
Skip compression with content-encoding with "identity".
Add more content-encoding values based on HTTP Content Coding Registry
(skip deprecated values).
---
java/org/apache/coyote/CompressionConfig.java | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index 3545db1a52..b32c18ef1c 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -194,6 +194,7 @@ public class CompressionConfig {
}
boolean useTE = false;
+boolean useCE = true;
MimeHeaders responseHeaders = response.getMimeHeaders();
@@ -211,8 +212,13 @@ public class CompressionConfig {
log.warn(sm.getString("compressionConfig.ContentEncodingParseFail"), e);
return false;
}
-if (tokens.contains("gzip") || tokens.contains("compress") ||
tokens.contains("deflate")
-|| tokens.contains("br") || tokens.contains("zstd")) {
+if (tokens.contains("identity")) {
+// If identity, do not do content modifications
+useCE = false;
+} else if (tokens.contains("br") || tokens.contains("compress") ||
tokens.contains("dcb")
+|| tokens.contains("dcz") || tokens.contains("deflate") ||
tokens.contains("gzip")
+|| tokens.contains("pack200-gzip") ||
tokens.contains("zstd")) {
+// Content should not be compressed twice
return false;
}
}
@@ -262,7 +268,7 @@ public class CompressionConfig {
return false;
}
-if (!useTE) {
+if (useCE && !useTE) {
// If processing reaches this far, the response might be
compressed.
// Therefore, set the Vary header to keep proxies happy
ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
@@ -318,7 +324,7 @@ public class CompressionConfig {
responseHeaders.addValue("Transfer-Encoding").setString("gzip");
} else {
// Configure the content encoding for compressed content
-responseHeaders.setValue("Content-Encoding").setString("gzip");
+responseHeaders.addValue("Content-Encoding").setString("gzip");
}
return true;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 69575] org.apache.coyote.CompressionConfig doesn't recognize "zstd" compression
https://bz.apache.org/bugzilla/show_bug.cgi?id=69575 Remy Maucherat changed: What|Removed |Added Resolution|--- |FIXED Status|REOPENED|RESOLVED --- Comment #6 from Remy Maucherat --- Ok, second attempt, including the feedback. If "identity" is explicit, I prefer not compressing. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch 9.0.x updated: Further improve with feedback from 69575
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/9.0.x by this push:
new 2f285843e6 Further improve with feedback from 69575
2f285843e6 is described below
commit 2f285843e6153249ae148cb2a47c6e6f9a40dab6
Author: remm
AuthorDate: Thu Feb 20 21:41:26 2025 +0100
Further improve with feedback from 69575
Add to existing content-encoding header values when compressing.
Skip compression with content-encoding with "identity".
Add more content-encoding values based on HTTP Content Coding Registry
(skip deprecated values).
---
java/org/apache/coyote/CompressionConfig.java | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/java/org/apache/coyote/CompressionConfig.java
b/java/org/apache/coyote/CompressionConfig.java
index 7bd2c186db..1ef26d93b5 100644
--- a/java/org/apache/coyote/CompressionConfig.java
+++ b/java/org/apache/coyote/CompressionConfig.java
@@ -221,6 +221,7 @@ public class CompressionConfig {
}
boolean useTE = false;
+boolean useCE = true;
MimeHeaders responseHeaders = response.getMimeHeaders();
@@ -238,8 +239,13 @@ public class CompressionConfig {
log.warn(sm.getString("compressionConfig.ContentEncodingParseFail"), e);
return false;
}
-if (tokens.contains("gzip") || tokens.contains("compress") ||
tokens.contains("deflate")
-|| tokens.contains("br") || tokens.contains("zstd")) {
+if (tokens.contains("identity")) {
+// If identity, do not do content modifications
+useCE = false;
+} else if (tokens.contains("br") || tokens.contains("compress") ||
tokens.contains("dcb")
+|| tokens.contains("dcz") || tokens.contains("deflate") ||
tokens.contains("gzip")
+|| tokens.contains("pack200-gzip") ||
tokens.contains("zstd")) {
+// Content should not be compressed twice
return false;
}
}
@@ -291,7 +297,7 @@ public class CompressionConfig {
}
}
-if (!useTE) {
+if (useCE && !useTE) {
// If processing reaches this far, the response might be
compressed.
// Therefore, set the Vary header to keep proxies happy
ResponseUtil.addVaryFieldName(responseHeaders, "accept-encoding");
@@ -347,7 +353,7 @@ public class CompressionConfig {
responseHeaders.addValue("Transfer-Encoding").setString("gzip");
} else {
// Configure the content encoding for compressed content
-responseHeaders.setValue("Content-Encoding").setString("gzip");
+responseHeaders.addValue("Content-Encoding").setString("gzip");
}
return true;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Buildbot failure in on tomcat-11.0.x
Build status: BUILD FAILED: failed compile (failure) Worker used: bb_worker2_ubuntu URL: https://ci2.apache.org/#builders/112/builds/1530 Blamelist: remm Build Text: failed compile (failure) Status Detected: new failure Build Source Stamp: [branch 11.0.x] ae67bb4bb6eaf2f5aa2d3eecccea8406878e501b Steps: worker_preparation: 0 git: 0 shell: 0 shell_1: 0 shell_2: 0 shell_3: 0 shell_4: 0 shell_5: 0 shell_6: 0 compile: 1 shell_7: 0 shell_8: 0 shell_9: 0 shell_10: 0 Rsync docs to nightlies.apache.org: 0 shell_11: 0 Rsync RAT to nightlies.apache.org: 0 compile_1: 2 shell_12: 0 Rsync Logs to nightlies.apache.org: 0 -- ASF Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch main updated: Remove the case sensitivity check
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/main by this push:
new 82367b3891 Remove the case sensitivity check
82367b3891 is described below
commit 82367b3891b9ffd501ee7046b71ee2461303562c
Author: Mark Thomas
AuthorDate: Thu Feb 20 12:49:53 2025 +
Remove the case sensitivity check
The performance impact is minimal and getting the check right in all
cases is difficult due to various edge cases
---
.../catalina/webresources/DirResourceSet.java | 70 +++---
1 file changed, 9 insertions(+), 61 deletions(-)
diff --git a/java/org/apache/catalina/webresources/DirResourceSet.java
b/java/org/apache/catalina/webresources/DirResourceSet.java
index b6da202b5c..074839312b 100644
--- a/java/org/apache/catalina/webresources/DirResourceSet.java
+++ b/java/org/apache/catalina/webresources/DirResourceSet.java
@@ -45,8 +45,6 @@ public class DirResourceSet extends AbstractFileResourceSet
implements WebResour
private static final Log log = LogFactory.getLog(DirResourceSet.class);
-private boolean caseSensitive = true;
-
private Map resourceLocksByPath = new HashMap<>();
private Object resourceLocksByPathLock = new Object();
@@ -322,7 +320,6 @@ public class DirResourceSet extends AbstractFileResourceSet
implements WebResour
@Override
protected void initInternal() throws LifecycleException {
super.initInternal();
-caseSensitive = isCaseSensitive();
// Is this an exploded web application?
if (getWebAppMount().equals("")) {
// Look for a manifest
@@ -338,65 +335,16 @@ public class DirResourceSet extends
AbstractFileResourceSet implements WebResour
}
-/*
- * Determines if this ResourceSet is based on a case sensitive file system
or not.
- *
- * File systems are usually case sensitive or not. Windows, via the
command 'fsutil.exe file setCaseSensitiveInfo
- * enable', may be case sensitive in some directories and case
insensitive in others.
- *
- * If this method incorrectly determines that the DirResourceSet is case
sensitive, the file locking mechanism that
- * ensures write operations are performed atomically will not operate
correctly. If this method incorrectly
- * determines that the DirResourceSet is case insensitive, there is a
small performance penalty for writes.
- *
- * Given the above, this method only reports the file system as case
sensitive if no indication of case
- * insensitivity is detected. This does mean that Windows based
DirResourceSet instances will be reported as case
- * insensitive even all of the directories in the DirResourceSet have been
configured as case sensitive.
- */
-private boolean isCaseSensitive() {
-try {
-String canonicalPath = getFileBase().getCanonicalPath();
-/*
- * If any lower case characters are found in the canonical file
name formed by converting the test file name
- * to upper case, the underlying file system must be, at least in
part, case insensitive.
- */
-File upper = new File(canonicalPath.toUpperCase(Locale.ENGLISH));
-String upperCanonicalPath = upper.getCanonicalPath();
-char[] upperCharacters = upperCanonicalPath.toCharArray();
-for (char c : upperCharacters) {
-if (Character.isLowerCase(c)) {
-return false;
-}
-}
-
-/*
- * If any upper case characters are found in the canonical file
name formed by converting the test file name
- * to lower case, the underlying file system must be, at least in
part, case insensitive.
- */
-File lower = new File(canonicalPath.toLowerCase(Locale.ENGLISH));
-String lowerCanonicalPath = lower.getCanonicalPath();
-char[] lowerCharacters = lowerCanonicalPath.toCharArray();
-for (char c : lowerCharacters) {
-if (Character.isUpperCase(c)) {
-return false;
-}
-}
-
-return true;
-} catch (IOException ioe) {
-log.warn(sm.getString("dirResourceSet.isCaseSensitive.fail",
getFileBase().getAbsolutePath()), ioe);
-}
-
-return false;
-}
-
-
private String getLockKey(String path) {
-// Normalize path to ensure that the same key is used for the same
path.
-String normalisedPath = RequestUtil.normalize(path);
-if (caseSensitive) {
-return normalisedPath;
-}
-return normalisedPath.toLowerCase(Locale.ENGLISH);
+/*
+ * Normalize path to ensure that the same key is used for the same
path. Always convert path to lower
Re: [PR] WebResource read/write lock key changed to in lowercase arbitrarily [tomcat]
Chenjp closed pull request #829: WebResource read/write lock key changed to in lowercase arbitrarily URL: https://github.com/apache/tomcat/pull/829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Further improvements to the CVE-2024-56337 protection
All, The recent releases have improved things for users of embedded Tomcat but there are still some issues. I am seeing reports via $work related to Spring Boot. The problem is on Windows and Mac. The file systems are case insensitive and DirResourceSet instances are read/write by default so the reflection code gets called. The impact varies by Java version: Java 21 - no issue as the code doesn't get called Java 17 - exception is thrown and the web application fails to start due to missing --add-opens Java 11 - warning is logged but otherwise web application starts normally Java 8 - web application starts normally Having chatted at length with the Spring Boot team, they are going to make the DirResourceSet instances read only by default which works around the Java 17 issue but not the Java 11 issue as reflection also occurs in the init of JreCompat. I have performed some testing and confirmed that the java.io.FileSystem class is loaded by the JRE before any application code gets a chance to execute. That means we can use checking the command line arguments to determine whether the cache is enabled/disabled. To reduce the impact on anyone using Tomcat in embedded mode - including but not limited to Spring Boot - I intend to make the following improvements: Java 17 Check the command line arguments to determine if the canonical resource name cache has been enabled Switch to lazy initialisation for JreCompat.useCanonCachesField so reflection is not used unless Tomcat needs to try and disable the cache Java 16 and below Check the command line arguments to determine if the canonical resource name cache has been disabled Switch to lazy initialisation for JreCompat.useCanonCachesField so reflection is not used unless Tomcat needs to try and disable the cache I don't think there is a need to rush out these changes but I'm going to start work on them now so they are ready if we need them in a hurry. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch 11.0.x updated: Remove unnecessary configuration
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/11.0.x by this push: new d93b96e085 Remove unnecessary configuration d93b96e085 is described below commit d93b96e085958ef9e9d48e3491f9dc83bbc24bf3 Author: Mark Thomas AuthorDate: Thu Feb 20 12:12:47 2025 + Remove unnecessary configuration Tomcat 11 requires Java 17 where this cache is disabled by default. Disabling here doesn't change anything. --- bin/catalina.bat | 7 --- bin/catalina.sh | 7 --- 2 files changed, 14 deletions(-) diff --git a/bin/catalina.bat b/bin/catalina.bat index 88bf844c14..78a7eb9d9f 100755 --- a/bin/catalina.bat +++ b/bin/catalina.bat @@ -203,13 +203,6 @@ set "JSSE_OPTS=-Djdk.tls.ephemeralDHKeySize=2048" :gotJsseOpts set "JAVA_OPTS=%JAVA_OPTS% %JSSE_OPTS%" -rem Disable the global canonical file name cache to protect against CVE-2024-56337 -rem Note: The cache is disabled by default in Java 12 to 20 inclusive -rem The cache is removed in Java 21 onwards -rem Need to set this here as java.io.FileSystem caches this in a static field during class -rem initialisation so it needs to be set before any file system access -set "JAVA_OPTS=%JAVA_OPTS% -Dsun.io.useCanonCaches=false" - if not "%CATALINA_LOGGING_CONFIG%" == "" goto noJuliConfig set CATALINA_LOGGING_CONFIG=-Dnop if not exist "%CATALINA_BASE%\conf\logging.properties" goto noJuliConfig diff --git a/bin/catalina.sh b/bin/catalina.sh index c5a996cb65..ee679ad0c6 100755 --- a/bin/catalina.sh +++ b/bin/catalina.sh @@ -251,13 +251,6 @@ if [ -z "$JSSE_OPTS" ] ; then fi JAVA_OPTS="$JAVA_OPTS $JSSE_OPTS" -# Disable the global canonical file name cache to protect against CVE-2024-56337 -# Note: The cache is disabled by default in Java 12 to 20 inclusive -# The cache is removed in Java 21 onwards -# Need to set this here as java.io.FileSystem caches this in a static field during class -# initialisation so it needs to be set before any file system access -JAVA_OPTS="$JAVA_OPTS -Dsun.io.useCanonCaches=false" - # Set juli LogManager config file if it is present and an override has not been issued if [ -z "$CATALINA_LOGGING_CONFIG" ]; then if [ -r "$CATALINA_BASE"/conf/logging.properties ]; then - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Further improvements to the CVE-2024-56337 protection
On Thu, Feb 20, 2025 at 2:42 PM Mark Thomas wrote: > > On 20/02/2025 13:36, Rémy Maucherat wrote: > > On Thu, Feb 20, 2025 at 1:06 PM Mark Thomas wrote: > >> > >> All, > >> > >> The recent releases have improved things for users of embedded Tomcat > >> but there are still some issues. I am seeing reports via $work related > >> to Spring Boot. > >> > >> The problem is on Windows and Mac. The file systems are case insensitive > >> and DirResourceSet instances are read/write by default so the reflection > >> code gets called. The impact varies by Java version: > >> > >> Java 21 - no issue as the code doesn't get called > >> > >> Java 17 - exception is thrown and the web application fails to start due > >> to missing --add-opens > > > > Where does the exception come from this time ? > > JreCompat.disableCanonCaches() So in that case useCanonCachesField is not null, but using it fails ? That's unexpected. OTOH, some users were randomly saying that things were ok on Java 17, and others not. So there's something. Rémy > Mark > > > > > > Rémy > > > >> Java 11 - warning is logged but otherwise web application starts > >> normally > >> > >> Java 8 - web application starts normally > >> > >> > >> Having chatted at length with the Spring Boot team, they are going to > >> make the DirResourceSet instances read only by default which works > >> around the Java 17 issue but not the Java 11 issue as reflection also > >> occurs in the init of JreCompat. > >> > >> I have performed some testing and confirmed that the java.io.FileSystem > >> class is loaded by the JRE before any application code gets a chance to > >> execute. That means we can use checking the command line arguments to > >> determine whether the cache is enabled/disabled. > >> > >> To reduce the impact on anyone using Tomcat in embedded mode - including > >> but not limited to Spring Boot - I intend to make the following > >> improvements: > >> > >> Java 17 > >> Check the command line arguments to determine if the canonical resource > >> name cache has been enabled > >> Switch to lazy initialisation for JreCompat.useCanonCachesField so > >> reflection is not used unless Tomcat needs to try and disable the cache > >> > >> Java 16 and below > >> Check the command line arguments to determine if the canonical resource > >> name cache has been disabled > >> Switch to lazy initialisation for JreCompat.useCanonCachesField so > >> reflection is not used unless Tomcat needs to try and disable the cache > >> > >> > >> I don't think there is a need to rush out these changes but I'm going to > >> start work on them now so they are ready if we need them in a hurry. > >> > >> Mark > >> > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: dev-h...@tomcat.apache.org > >> > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Buildbot success in on tomcat-11.0.x
Build status: Build succeeded! Worker used: bb_worker2_ubuntu URL: https://ci2.apache.org/#builders/112/builds/1531 Blamelist: remm Build Text: build successful Status Detected: restored build Build Source Stamp: [branch 11.0.x] 6c9817e102ffbb8b3a896ef5b55e82b9fe8b5eec Steps: worker_preparation: 0 git: 0 shell: 0 shell_1: 0 shell_2: 0 shell_3: 0 shell_4: 0 shell_5: 0 shell_6: 0 compile: 1 shell_7: 0 shell_8: 0 shell_9: 0 shell_10: 0 Rsync docs to nightlies.apache.org: 0 shell_11: 0 Rsync RAT to nightlies.apache.org: 0 compile_1: 1 shell_12: 0 Rsync Logs to nightlies.apache.org: 0 -- ASF Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
