[Bug 69605] Possibility to clear the notFoundClassResource cache from within a running webapp
https://bz.apache.org/bugzilla/show_bug.cgi?id=69605 --- Comment #3 from Mark Thomas --- That is all considered Tomcat internals (the definition is in RELEASE-NOTES) but we do try not to change even the internal API unless we need to. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 01/03: Refactor TimeBucketCounter to support alternative implementations
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 6f65e9025b3d8aaa6c3c06f140d1962967ad1c8a Author: Mark Thomas AuthorDate: Fri Mar 7 13:27:58 2025 + Refactor TimeBucketCounter to support alternative implementations This is preparatory work for PR #794 --- .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 177 ++-- 2 files changed, 145 insertions(+), 217 deletions(-) diff --git a/java/org/apache/catalina/util/TimeBucketCounter.java b/java/org/apache/catalina/util/TimeBucketCounter.java index 3b4726f7ff..78951623cc 100644 --- a/java/org/apache/catalina/util/TimeBucketCounter.java +++ b/java/org/apache/catalina/util/TimeBucketCounter.java @@ -14,122 +14,91 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ExecutionException; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.ScheduledFuture; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -import org.apache.juli.logging.Log; -import org.apache.juli.logging.LogFactory; -import org.apache.tomcat.util.res.StringManager; /** - * This class maintains a thread safe hash map that has timestamp-based buckets followed by a string for a key, and a - * counter for a value. each time the increment() method is called it adds the key if it does not exist, increments its - * value and returns it. a maintenance thread cleans up keys that are prefixed by previous timestamp buckets. + * A fast counter that optimizes efficiency at the expense of approximate bucket indexing. */ -public class TimeBucketCounter { - -private static final Log log = LogFactory.getLog(TimeBucketCounter.class); -private static final StringManager sm = StringManager.getManager(TimeBucketCounter.class); - -/** - * Map to hold the buckets - */ -private final ConcurrentHashMap map = new ConcurrentHashMap<>(); +public class TimeBucketCounter extends TimeBucketCounterBase { -/** - * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute - */ +// Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute. private final int numBits; -/** - * Ratio of actual duration to config duration - */ +// Ratio of actual duration to config duration private final double ratio; -/** - * The future allowing control of the background processor. - */ -private ScheduledFuture maintenanceFuture; -private ScheduledFuture monitorFuture; -private final ScheduledExecutorService executorService; -private final long sleeptime; -/** - * Creates a new TimeBucketCounter with the specified lifetime. - * - * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 - * @param executorService the executor service which will be used to run the maintenance - */ public TimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(getActualDuration(bucketDuration), executorService); +this.numBits = determineShiftBitsOfDuration(bucketDuration); +this.ratio = ratioToPowerOf2(bucketDuration * 1000); +} -this.executorService = executorService; - -int durationMillis = bucketDuration * 1000; - -int bits = 0; -int pof2 = nextPowerOf2(durationMillis); -int bitCheck = pof2; -while (bitCheck > 1) { -bitCheck = pof2 >> ++bits; -} - -this.numBits = bits; -this.ratio = ratioToPowerOf2(durationMillis); +/** + * {@inheritDoc} + * + * Calculates the current time bucket index by shifting bits for fast division, e.g. shift 16 bits is the same as + * dividing by 65,536 which is about 1:05m. + */ +@Override +public long getBucketIndex(long timestamp) { +return timestamp >> this.numBits; +} -int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; -sleeptime = durationMillis / cleanupsPerBucketDuration; -// Start our thread -if (sleeptime > 0) { -monitorFuture = executorService.scheduleWithFixedDelay(new MaintenanceMonitor(), 0, 60, TimeUnit.SECONDS); -} +public int getNumBits() { +return numBits; } + /** - * Increments the counter for the passed identifier in the current time bucket and returns the new value. - * - * @param identifier an identifier for which we want to maintain count, e.g. IP Address + * The actual dur
(tomcat) 03/03: Add an exact rate limit filter.
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit f60933283eb9268a3203102654ebabc717581505 Author: Mark Thomas AuthorDate: Fri Mar 7 14:55:22 2025 + Add an exact rate limit filter. Based on PR #794 by Chenjp --- .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 +++ java/org/apache/catalina/util/FastRateLimiter.java | 1 + java/org/apache/catalina/util/RateLimiterBase.java | 10 + .../catalina/util/TimeBucketCounterBase.java | 3 - .../TestRateLimitFilterWithExactRateLimiter.java | 211 + webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 ++- 8 files changed, 323 insertions(+), 14 deletions(-) diff --git a/java/org/apache/catalina/filters/RateLimitFilter.java b/java/org/apache/catalina/filters/RateLimitFilter.java index d362aca766..89a48cb70f 100644 --- a/java/org/apache/catalina/filters/RateLimitFilter.java +++ b/java/org/apache/catalina/filters/RateLimitFilter.java @@ -47,8 +47,11 @@ import org.apache.tomcat.util.res.StringManager; * so it converts some configured values to more efficient values. For example, a configuration of a 60 seconds time * bucket is converted to 65.536 seconds. That allows for very fast bucket calculation using bit shift arithmetic. In * order to remain true to the user intent, the configured number of requests is then multiplied by the same ratio, so a - * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. You can specify a - * different class as long as it implements the org.apache.catalina.util.RateLimiter interface. + * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. An alternative + * implementation, org.apache.catalina.util.ExactRateLimiter, is intended to provide a less efficient but + * more accurate control, whose effective duration in seconds and number of requests configuration are consist with the + * user declared. You can specify a different class as long as it implements the + * org.apache.catalina.util.RateLimiter interface. * * * It is common to set up different restrictions for different URIs. For example, a login page or authentication script diff --git a/java/org/apache/catalina/util/ExactRateLimiter.java b/java/org/apache/catalina/util/ExactRateLimiter.java new file mode 100644 index 00..a1b8c9190f --- /dev/null +++ b/java/org/apache/catalina/util/ExactRateLimiter.java @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.util; + +import java.util.concurrent.ScheduledExecutorService; + +/** + * A RateLimiter that compromises efficiency for accuracy in order to provide exact rate limiting. + */ +public class ExactRateLimiter extends RateLimiterBase { + +@Override +protected String getDefaultPolicyName() { +return "exact"; +} + + +@Override +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new ExactTimeBucketCounter(duration, executorService); +} + + +/** + * An accurate counter with exact bucket index, but slightly less efficient than another fast counter provided with + * the {@link FastRateLimiter}. + */ +class ExactTimeBucketCounter extends TimeBucketCounterBase { + +ExactTimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(bucketDuration, executorService); +} + +@Override +public long getBucketIndex(long timestamp) { +return (timestamp / 1000) / getBucketDuration(); +} + +@Override +public double getRatio() { +// Actual value is exactly the same as declared +return 1.0d; +} + +@Override +public long getMillisUntilNextBucket() { +long millis = System.currentTimeMillis(); + +long nextTimeBucketMillis = (getB
(tomcat) branch 9.0.x updated (d7b0d6236e -> f60933283e)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git from d7b0d6236e More skip.installer, remove unless in depends-only target new 6f65e9025b Refactor TimeBucketCounter to support alternative implementations new cfe887822e Refactor RateLimiter FastRateLimiter for extension new f60933283e Add an exact rate limit filter. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 java/org/apache/catalina/util/FastRateLimiter.java | 75 + java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 155 + .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 174 ++- ...> TestRateLimitFilterWithExactRateLimiter.java} | 88 +- webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 +++- 10 files changed, 450 insertions(+), 346 deletions(-) create mode 100644 java/org/apache/catalina/util/ExactRateLimiter.java create mode 100644 java/org/apache/catalina/util/RateLimiterBase.java copy java/org/apache/catalina/util/{TimeBucketCounter.java => TimeBucketCounterBase.java} (54%) copy test/org/apache/catalina/filters/{TestRateLimitFilter.java => TestRateLimitFilterWithExactRateLimiter.java} (75%) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 02/03: Refactor RateLimiter FastRateLimiter for extension
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit cfe887822e1ef9fdcf4a00a05b97c3a65cd9d77d Author: Mark Thomas AuthorDate: Fri Mar 7 13:51:00 2025 + Refactor RateLimiter FastRateLimiter for extension Preparation for #794 --- java/org/apache/catalina/util/FastRateLimiter.java | 76 +-- java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 145 + 3 files changed, 154 insertions(+), 74 deletions(-) diff --git a/java/org/apache/catalina/util/FastRateLimiter.java b/java/org/apache/catalina/util/FastRateLimiter.java index 54f1f1e74e..17544c5d28 100644 --- a/java/org/apache/catalina/util/FastRateLimiter.java +++ b/java/org/apache/catalina/util/FastRateLimiter.java @@ -14,92 +14,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.atomic.AtomicInteger; - -import javax.servlet.FilterConfig; - -import org.apache.tomcat.util.threads.ScheduledThreadPoolExecutor; /** * A RateLimiter that compromises accuracy for speed in order to provide maximum throughput. */ -public class FastRateLimiter implements RateLimiter { - -private static AtomicInteger index = new AtomicInteger(); - -TimeBucketCounter bucketCounter; - -int duration; - -int requests; - -int actualRequests; - -int actualDuration; - -// Initial policy name can be rewritten by setPolicyName() -private String policyName = "fast-" + index.incrementAndGet(); - -@Override -public String getPolicyName() { -return policyName; -} - -@Override -public void setPolicyName(String name) { -this.policyName = name; -} +public class FastRateLimiter extends RateLimiterBase { @Override -public int getDuration() { -return actualDuration; +protected String getDefaultPolicyName() { +return "fast"; } -@Override -public void setDuration(int duration) { -this.duration = duration; -} - -@Override -public int getRequests() { -return actualRequests; -} - -@Override -public void setRequests(int requests) { -this.requests = requests; -} @Override -public int increment(String ipAddress) { -return bucketCounter.increment(ipAddress); +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new TimeBucketCounter(duration, executorService); } -@Override -public void destroy() { -bucketCounter.destroy(); -} - -@Override -public void setFilterConfig(FilterConfig filterConfig) { - -ScheduledExecutorService executorService = (ScheduledExecutorService) filterConfig.getServletContext() -.getAttribute(ScheduledThreadPoolExecutor.class.getName()); - -if (executorService == null) { -executorService = new java.util.concurrent.ScheduledThreadPoolExecutor(1); -} - -bucketCounter = new TimeBucketCounter(duration, executorService); -actualRequests = (int) Math.round(bucketCounter.getRatio() * requests); -actualDuration = bucketCounter.getActualDuration() / 1000; -} public TimeBucketCounter getBucketCounter() { -return bucketCounter; +return (TimeBucketCounter) bucketCounter; } } diff --git a/java/org/apache/catalina/util/RateLimiter.java b/java/org/apache/catalina/util/RateLimiter.java index 4b261d5073..84a5ef9a47 100644 --- a/java/org/apache/catalina/util/RateLimiter.java +++ b/java/org/apache/catalina/util/RateLimiter.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import javax.servlet.FilterConfig; @@ -46,13 +45,13 @@ public interface RateLimiter { void setRequests(int requests); /** - * Increments the number of requests by the given ipAddress in the current time window. + * Increments the number of requests by the given identifier in the current time window. * - * @param ipAddress the ip address + * @param identifier the identifier for which the number of associated requests should be incremented * * @return the new value after incrementing */ -int increment(String ipAddress); +int increment(String identifier); /** * Cleanup no longer needed resources. diff --git a/java/org/apache/catalina/util/RateLimiterBase.java b/java/org/apache/catalina/util/RateLimiterBase.java new file mode 100644 index 00..7c34bd56c4 --- /dev/null +++ b/java/org/apache/catalina/util/RateLimiterBa
(tomcat) branch 9.0.x updated: Fix processing of the time-taken token
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/9.0.x by this push: new 6047f4d947 Fix processing of the time-taken token 6047f4d947 is described below commit 6047f4d9470111688003f5a0aab40459dda354da Author: remm AuthorDate: Fri Mar 7 17:02:32 2025 +0100 Fix processing of the time-taken token Simplify. Add test case. --- .../catalina/valves/ExtendedAccessLogValve.java| 18 +-- .../valves/TestExtendedAccessLogValve.java | 171 + ...ve.java => TestExtendedAccessLogValveWrap.java} | 2 +- webapps/docs/changelog.xml | 4 + 4 files changed, 149 insertions(+), 46 deletions(-) diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index aebc853846..745ba56889 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -134,23 +134,7 @@ public class ExtendedAccessLogValve extends AccessLogValve { } /* Wrap all values in double quotes. */ -StringBuilder buffer = new StringBuilder(svalue.length() + 2); -buffer.append('\"'); -int i = 0; -while (i < svalue.length()) { -int j = svalue.indexOf('\"', i); -if (j == -1) { -buffer.append(svalue.substring(i)); -i = svalue.length(); -} else { -buffer.append(svalue.substring(i, j + 1)); -buffer.append('"'); -i = j + 1; -} -} - -buffer.append('\"'); -return buffer.toString(); +return "\"" + svalue.replace("\"", "\"\"") + "\""; } @Override diff --git a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java index fb3377ef3b..348807617d 100644 --- a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java +++ b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java @@ -16,48 +16,163 @@ */ package org.apache.catalina.valves; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; + +import javax.servlet.http.HttpServlet; +import javax.servlet.http.HttpServletRequest; +import javax.servlet.http.HttpServletResponse; + import org.junit.Assert; import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.junit.runners.Parameterized.Parameter; -public class TestExtendedAccessLogValve { +import org.apache.catalina.Context; +import org.apache.catalina.Host; +import org.apache.catalina.startup.Tomcat; +import org.apache.catalina.startup.TomcatBaseTest; +import org.apache.tomcat.util.buf.ByteChunk; -@Test -public void alpha() { -Assert.assertEquals("\"foo\"", ExtendedAccessLogValve.wrap("foo")); -} +@RunWith(Parameterized.class) +public class TestExtendedAccessLogValve extends TomcatBaseTest { -@Test -public void testNull() { -Assert.assertEquals("-", ExtendedAccessLogValve.wrap(null)); +@Parameterized.Parameters(name = "{index}: pattern=[{0}]") +public static Collection data() { +List patterns = new ArrayList<>(); +patterns.add(new Object[]{"basic", "time cs-method cs-uri-stem cs-uri-query"}); +patterns.add(new Object[]{"ip", "time cs-method sc-status c-ip s-ip s-dns c-dns"}); +patterns.add(new Object[]{"headers", "time cs-method cs(Referer) cs(Cookie) sc(Content-Type)"}); +patterns.add(new Object[]{"bytes", "date time cs-method cs-uri-stem bytes time-taken cached"}); +return patterns; } -@Test -public void empty() { -Assert.assertEquals("\"\"", ExtendedAccessLogValve.wrap("")); -} +@Parameter(0) +public String name; -@Test -public void singleQuoteMiddle() { -Assert.assertEquals("\"foo'bar\"", ExtendedAccessLogValve.wrap("foo'bar")); -} +@Parameter(1) +public String logPattern; @Test -public void doubleQuoteMiddle() { -Assert.assertEquals("\"foo\"\"bar\"", ExtendedAccessLogValve.wrap("foo\"bar")); +public void testLogFormat() throws Exception { + +Tomcat tomcat = getTomcatInstance(); +Host host = tomcat.getHost(); + +// Create temporary directory for logs +File logDir = getTemporaryDirectory(); + +ExtendedAccessLogValve valve = new ExtendedAccessLogValve(); +valve.setPattern(logPattern); +valve.setDirectory(logDir.getAbsolutePath()); +
[Bug 69607] New: MD5 algorithm insecure usage in tomcat-util
https://bz.apache.org/bugzilla/show_bug.cgi?id=69607 Bug ID: 69607 Summary: MD5 algorithm insecure usage in tomcat-util Product: Tomcat 11 Version: 11.0.4 Hardware: All OS: All Status: NEW Severity: normal Priority: P2 Component: Util Assignee: dev@tomcat.apache.org Reporter: sve...@redseal.net Target Milestone: --- We have identified that tomcat-util is using MD5 algorithm which is not considered secure in FIPS-140.3 mode. In FIPS mode server startup fails because of MD5 usage in tomcat-util. The issue arises because tomcat-util uses the MD5 algorithm for initialization in the ConcurrentMessageDigest class, located in the tomcat/util/security folder. This is located in static block during initialization and cannot be circumvented. Can we submit a patch to remove initialization of MD5 algorithm from static block of tomcat-util? SHA-1 can be used which is more secure algorithm and is compatible with FIPS-140.3 mode. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Buildbot success in on tomcat-11.0.x
Build status: Build succeeded! Worker used: bb_worker2_ubuntu URL: https://ci2.apache.org/#builders/112/builds/1562 Blamelist: Mark Thomas , remm Build Text: build successful Status Detected: restored build Build Source Stamp: [branch 11.0.x] abf9c58911690306dbb2be93dd9d2b219f4ff346 Steps: worker_preparation: 0 git: 0 shell: 0 shell_1: 0 shell_2: 0 shell_3: 0 shell_4: 0 shell_5: 0 shell_6: 0 compile: 1 shell_7: 0 shell_8: 0 shell_9: 0 shell_10: 0 Rsync docs to nightlies.apache.org: 0 shell_11: 0 Rsync RAT to nightlies.apache.org: 0 compile_1: 1 shell_12: 0 Rsync Logs to nightlies.apache.org: 0 -- ASF Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 69607] MD5 algorithm insecure usage in tomcat-util
https://bz.apache.org/bugzilla/show_bug.cgi?id=69607 --- Comment #3 from ywei...@redseal.net --- Yes, that makes sense, It is really the static init block that is causing us issues. We were actually just thinking to lazy init the MD5 cache and move it out of the static block. Would that be OK? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[VOTE][RESULT] Release Apache Tomcat 10.1.39
All, The following votes were cast: +1: remm, markt, csutherl, schultz +0: dsoumis, kkolinko No other votes were cast, therefore the vote passes. Thanks to everyone who contributed toward this release. -chris On 3/4/25 2:13 PM, Christopher Schultz wrote: The proposed Apache Tomcat 10.1.39 release is now available for voting. All committers and PMC members are kindly requested to provide a vote if possible. ANY TOMCAT USER MAY VOTE, though only PMC members votes are binding. We welcome non-committer votes or comments on release builds. Note that 10.1.37 and 10.1.38 were not released due to packaging errors. The notable changes compared to 10.1.36 are: - Improve the checks for exposure to and protection against CVE-2024-56337 so that reflection is not used unless required. The checks for whether the file system is case sensitive or not have been removed. - Use Transfer-Encoding for compression rather than Content-Encoding if the client submits a TE header containing gzip. - Add makensis as an option for building the Installer for Windows on non-Windows platforms. For full details, see the change log: https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 without changes. Java EE applications designed for Tomcat 9 and earlier may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will automatically convert them to Jakarta EE and copy them to the webapps directory. It can be obtained from: https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.38/ The Maven staging repo is: https://repository.apache.org/content/repositories/orgapachetomcat-1539 The tag is: https://github.com/apache/tomcat/tree/10.1.39 https://github.com/apache/tomcat/ commit/1e17b1d03591faa17a6eedeaeb3a9f9f7aac12fa Please reply with a +1 for release or +0/-0/-1 with an explanation. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r75357 - /dev/tomcat/tomcat-10/v10.1.39/ /release/tomcat/tomcat-10/v10.1.39/
Author: schultz Date: Fri Mar 7 19:31:40 2025 New Revision: 75357 Log: Promote release Added: release/tomcat/tomcat-10/v10.1.39/ - copied from r75356, dev/tomcat/tomcat-10/v10.1.39/ Removed: dev/tomcat/tomcat-10/v10.1.39/ - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch main updated: Fix processing of the time-taken token
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new 947802bb12 Fix processing of the time-taken token 947802bb12 is described below commit 947802bb12e42b6f98ef1552cfb4c46490bf76d4 Author: remm AuthorDate: Fri Mar 7 17:02:32 2025 +0100 Fix processing of the time-taken token Simplify. Add test case. --- .../catalina/valves/ExtendedAccessLogValve.java| 41 ++--- .../valves/TestExtendedAccessLogValve.java | 171 + ...ve.java => TestExtendedAccessLogValveWrap.java} | 2 +- webapps/docs/changelog.xml | 4 + 4 files changed, 162 insertions(+), 56 deletions(-) diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index 819096df0e..ce4de03805 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -135,23 +135,7 @@ public class ExtendedAccessLogValve extends AccessLogValve { } /* Wrap all values in double quotes. */ -StringBuilder buffer = new StringBuilder(svalue.length() + 2); -buffer.append('\"'); -int i = 0; -while (i < svalue.length()) { -int j = svalue.indexOf('\"', i); -if (j == -1) { -buffer.append(svalue.substring(i)); -i = svalue.length(); -} else { -buffer.append(svalue.substring(i, j + 1)); -buffer.append('"'); -i = j + 1; -} -} - -buffer.append('\"'); -return buffer.toString(); +return "\"" + svalue.replace("\"", "\"\"") + "\""; } @Override @@ -539,16 +523,19 @@ public class ExtendedAccessLogValve extends AccessLogValve { if (tokenizer.hasSubToken()) { String nextToken = tokenizer.getToken(); if ("taken".equals(nextToken)) { -nextToken = tokenizer.getToken(); - -if ("ns".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.NANOSECONDS); -} else if ("us".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.MICROSECONDS); -} else if ("ms".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.MILLISECONDS); -} else if ("fracsec".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS_FRACTIONAL); +if (tokenizer.hasSubToken()) { +nextToken = tokenizer.getToken(); +if ("ns".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.NANOSECONDS); +} else if ("us".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.MICROSECONDS); +} else if ("ms".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.MILLISECONDS); +} else if ("fracsec".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS_FRACTIONAL); +} else { +return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS); +} } else { return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS); } diff --git a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java index fb3377ef3b..58955fb73e 100644 --- a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java +++ b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java @@ -16,48 +16,163 @@ */ package org.apache.catalina.valves; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; + +import jakarta.servlet.http.HttpServlet; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; + import org.junit.Assert; import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.junit.runners.Parameterized.Parameter; -public class TestExtendedAccessLogValve { +import org.apache.catalina.Context; +import org.ap
(tomcat) branch 10.1.x updated: Fix processing of the time-taken token
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/10.1.x by this push: new e28959dc0b Fix processing of the time-taken token e28959dc0b is described below commit e28959dc0b05be7150fb1b7543031ded1dc25759 Author: remm AuthorDate: Fri Mar 7 17:02:32 2025 +0100 Fix processing of the time-taken token Simplify. Add test case. --- .../catalina/valves/ExtendedAccessLogValve.java| 41 ++--- .../valves/TestExtendedAccessLogValve.java | 171 + ...ve.java => TestExtendedAccessLogValveWrap.java} | 2 +- webapps/docs/changelog.xml | 4 + 4 files changed, 162 insertions(+), 56 deletions(-) diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index 17143c6f98..672c0c4fb8 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -135,23 +135,7 @@ public class ExtendedAccessLogValve extends AccessLogValve { } /* Wrap all values in double quotes. */ -StringBuilder buffer = new StringBuilder(svalue.length() + 2); -buffer.append('\"'); -int i = 0; -while (i < svalue.length()) { -int j = svalue.indexOf('\"', i); -if (j == -1) { -buffer.append(svalue.substring(i)); -i = svalue.length(); -} else { -buffer.append(svalue.substring(i, j + 1)); -buffer.append('"'); -i = j + 1; -} -} - -buffer.append('\"'); -return buffer.toString(); +return "\"" + svalue.replace("\"", "\"\"") + "\""; } @Override @@ -533,16 +517,19 @@ public class ExtendedAccessLogValve extends AccessLogValve { if (tokenizer.hasSubToken()) { String nextToken = tokenizer.getToken(); if ("taken".equals(nextToken)) { -nextToken = tokenizer.getToken(); - -if ("ns".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.NANOSECONDS); -} else if ("us".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.MICROSECONDS); -} else if ("ms".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.MILLISECONDS); -} else if ("fracsec".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS_FRACTIONAL); +if (tokenizer.hasSubToken()) { +nextToken = tokenizer.getToken(); +if ("ns".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.NANOSECONDS); +} else if ("us".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.MICROSECONDS); +} else if ("ms".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.MILLISECONDS); +} else if ("fracsec".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS_FRACTIONAL); +} else { +return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS); +} } else { return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS); } diff --git a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java index fb3377ef3b..58955fb73e 100644 --- a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java +++ b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java @@ -16,48 +16,163 @@ */ package org.apache.catalina.valves; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; + +import jakarta.servlet.http.HttpServlet; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; + import org.junit.Assert; import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.junit.runners.Parameterized.Parameter; -public class TestExtendedAccessLogValve { +import org.apache.catalina.Context; +import or
(tomcat) branch 11.0.x updated: Fix processing of the time-taken token
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/11.0.x by this push: new 550c47eb3e Fix processing of the time-taken token 550c47eb3e is described below commit 550c47eb3e8b0c43a93d0bf8f6221f9f13b67ed2 Author: remm AuthorDate: Fri Mar 7 17:02:32 2025 +0100 Fix processing of the time-taken token Simplify. Add test case. --- .../catalina/valves/ExtendedAccessLogValve.java| 41 ++--- .../valves/TestExtendedAccessLogValve.java | 171 + ...ve.java => TestExtendedAccessLogValveWrap.java} | 2 +- webapps/docs/changelog.xml | 4 + 4 files changed, 162 insertions(+), 56 deletions(-) diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index 819096df0e..ce4de03805 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -135,23 +135,7 @@ public class ExtendedAccessLogValve extends AccessLogValve { } /* Wrap all values in double quotes. */ -StringBuilder buffer = new StringBuilder(svalue.length() + 2); -buffer.append('\"'); -int i = 0; -while (i < svalue.length()) { -int j = svalue.indexOf('\"', i); -if (j == -1) { -buffer.append(svalue.substring(i)); -i = svalue.length(); -} else { -buffer.append(svalue.substring(i, j + 1)); -buffer.append('"'); -i = j + 1; -} -} - -buffer.append('\"'); -return buffer.toString(); +return "\"" + svalue.replace("\"", "\"\"") + "\""; } @Override @@ -539,16 +523,19 @@ public class ExtendedAccessLogValve extends AccessLogValve { if (tokenizer.hasSubToken()) { String nextToken = tokenizer.getToken(); if ("taken".equals(nextToken)) { -nextToken = tokenizer.getToken(); - -if ("ns".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.NANOSECONDS); -} else if ("us".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.MICROSECONDS); -} else if ("ms".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.MILLISECONDS); -} else if ("fracsec".equals(nextToken)) { -return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS_FRACTIONAL); +if (tokenizer.hasSubToken()) { +nextToken = tokenizer.getToken(); +if ("ns".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.NANOSECONDS); +} else if ("us".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.MICROSECONDS); +} else if ("ms".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.MILLISECONDS); +} else if ("fracsec".equals(nextToken)) { +return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS_FRACTIONAL); +} else { +return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS); +} } else { return new ElapsedTimeElement(ElapsedTimeElement.Style.SECONDS); } diff --git a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java index fb3377ef3b..58955fb73e 100644 --- a/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java +++ b/test/org/apache/catalina/valves/TestExtendedAccessLogValve.java @@ -16,48 +16,163 @@ */ package org.apache.catalina.valves; +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Pattern; + +import jakarta.servlet.http.HttpServlet; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; + import org.junit.Assert; import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.junit.runners.Parameterized.Parameter; -public class TestExtendedAccessLogValve { +import org.apache.catalina.Context; +import or
Re: [PR] enhancement: RateLimitFilter - Provides an exact rate limiting mechanism [tomcat]
markt-asf commented on code in PR #794: URL: https://github.com/apache/tomcat/pull/794#discussion_r1984984885 ## java/org/apache/catalina/util/TimeBucketCounterBase.java: ## @@ -0,0 +1,214 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.util; + +import java.util.Objects; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.juli.logging.Log; +import org.apache.juli.logging.LogFactory; +import org.apache.tomcat.util.res.StringManager; + +/** + * This class maintains a thread safe hash map that has timestamp-based buckets followed by a string for a key, and a + * counter for a integer value. each time the increment() method is called it adds the key if it does not exist, + * increments its value and returns it. + */ +public abstract class TimeBucketCounterBase { + +private static final Log log = LogFactory.getLog(TimeBucketCounterBase.class); +private static final StringManager sm = StringManager.getManager(TimeBucketCounterBase.class); + +private static final String BUCKET_KEY_DELIMITER = "^"; +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * /** The future allowing control of the background processor. + */ +private ScheduledFuture maintenanceFuture; +private ScheduledFuture monitorFuture; +private final ScheduledExecutorService executorService; +private final long sleeptime; +private int bucketDuration; + +/** + * Creates a new TimeBucketCounter with the specified lifetime. + * + * @param utilityExecutor the executor that should be used to handle maintenance task. + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + * + * @throws NullPointerException if executorService is null. + */ +public TimeBucketCounterBase(ScheduledExecutorService utilityExecutor, int bucketDuration) { +Objects.requireNonNull(utilityExecutor); +this.executorService = utilityExecutor; +this.bucketDuration = bucketDuration; + +int cleanupsPerBucketDuration = (bucketDuration >= 60) ? 6 : 3; +sleeptime = bucketDuration * 1000 / cleanupsPerBucketDuration; + +// Start our thread +if (sleeptime > 0) { +monitorFuture = executorService.scheduleWithFixedDelay(new MaintenanceMonitor(), 0, 60, TimeUnit.SECONDS); +} +} + +/** + * @return bucketDuration in seconds + */ +public int getBucketDuration() { +return bucketDuration; +} + +/** + * Returns the ratio between the configured duration param and the actual duration. + * @return the ratio between the configured duration param and the actual duration. + */ +public abstract double getRatio(); + +/** + * Increments the counter for the passed identifier in the current time bucket and returns the new value. + * + * @param identifier an identifier for which we want to maintain count, e.g. IP Address + * + * @return the count within the current time bucket + * + * @see TimeBucketCounterBase#genKey(String) + */ +public final int increment(String identifier) { +String key = genKey(identifier); +AtomicInteger ai = map.computeIfAbsent(key, v -> new AtomicInteger()); +return ai.incrementAndGet(); +} + +/** + * Generates the key of timeBucket counter maps with the specific identifier, and the timestamp is implicitly + * equivalent to "now". + * + * @param identifier an identifier for which we want to maintain count + * + * @return key of timeBucket counter maps + */ +protected final String genKey(String identifier) { +return genKey(identifier, System.currentTimeMillis()); +} + +/** + * Generates the key of timeBucket counter maps with the specific identifier and timestamp. + * + * @param identifier of
[Bug 69607] MD5 algorithm insecure usage in tomcat-util
https://bz.apache.org/bugzilla/show_bug.cgi?id=69607 ywei...@redseal.net changed: What|Removed |Added CC||ywei...@redseal.net -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 69607] MD5 algorithm insecure usage in tomcat-util
https://bz.apache.org/bugzilla/show_bug.cgi?id=69607 --- Comment #1 from Mark Thomas --- Remove completely, no. Handle gracefully if not available, yes. That might include deprecating digestMD5() in Tomcat 11 and removing for Tomcat 12 onwards. Note that MD5 is a required algorithm for Java <= 11. That requirement was dropped somewhere between Java 11 and Java 17. There are a couple of tests that use MD5 for digest auth. Those can probably be switch to SHA256 apart from the ones that are testing the range of digests available. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PR] enhancement: RateLimitFilter - Provides an exact rate limiting mechanism [tomcat]
markt-asf commented on PR #794: URL: https://github.com/apache/tomcat/pull/794#issuecomment-2706759471 It turns out the `TimeBucketCounter ` was by far the most complex. Once that was reviewed, the rest followed quite quickly. I'm leaving this PR open as there are some changes - particularly the change to eviction - that might need to be added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch main updated: Add an exact rate limit filter.
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new c1e50fb357 Add an exact rate limit filter. c1e50fb357 is described below commit c1e50fb3574ab6aeb02d1346fc0c5ff7b1f9ca08 Author: Mark Thomas AuthorDate: Fri Mar 7 14:55:22 2025 + Add an exact rate limit filter. Based on PR #794 by Chenjp --- .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 +++ java/org/apache/catalina/util/FastRateLimiter.java | 1 + java/org/apache/catalina/util/RateLimiterBase.java | 10 + .../catalina/util/TimeBucketCounterBase.java | 3 - .../TestRateLimitFilterWithExactRateLimiter.java | 211 + webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 ++- 8 files changed, 323 insertions(+), 14 deletions(-) diff --git a/java/org/apache/catalina/filters/RateLimitFilter.java b/java/org/apache/catalina/filters/RateLimitFilter.java index 8e07ca8182..80315948bb 100644 --- a/java/org/apache/catalina/filters/RateLimitFilter.java +++ b/java/org/apache/catalina/filters/RateLimitFilter.java @@ -47,8 +47,11 @@ import org.apache.tomcat.util.res.StringManager; * so it converts some configured values to more efficient values. For example, a configuration of a 60 seconds time * bucket is converted to 65.536 seconds. That allows for very fast bucket calculation using bit shift arithmetic. In * order to remain true to the user intent, the configured number of requests is then multiplied by the same ratio, so a - * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. You can specify a - * different class as long as it implements the org.apache.catalina.util.RateLimiter interface. + * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. An alternative + * implementation, org.apache.catalina.util.ExactRateLimiter, is intended to provide a less efficient but + * more accurate control, whose effective duration in seconds and number of requests configuration are consist with the + * user declared. You can specify a different class as long as it implements the + * org.apache.catalina.util.RateLimiter interface. * * * It is common to set up different restrictions for different URIs. For example, a login page or authentication script diff --git a/java/org/apache/catalina/util/ExactRateLimiter.java b/java/org/apache/catalina/util/ExactRateLimiter.java new file mode 100644 index 00..a1b8c9190f --- /dev/null +++ b/java/org/apache/catalina/util/ExactRateLimiter.java @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.util; + +import java.util.concurrent.ScheduledExecutorService; + +/** + * A RateLimiter that compromises efficiency for accuracy in order to provide exact rate limiting. + */ +public class ExactRateLimiter extends RateLimiterBase { + +@Override +protected String getDefaultPolicyName() { +return "exact"; +} + + +@Override +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new ExactTimeBucketCounter(duration, executorService); +} + + +/** + * An accurate counter with exact bucket index, but slightly less efficient than another fast counter provided with + * the {@link FastRateLimiter}. + */ +class ExactTimeBucketCounter extends TimeBucketCounterBase { + +ExactTimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(bucketDuration, executorService); +} + +@Override +public long getBucketIndex(long timestamp) { +return (timestamp / 1000) / getBucketDuration(); +} + +@Override +public double getRatio() { +// Actual value is exactly the same as declared +return 1.0d; +} + +@Override +
(tomcat) 03/03: Add an exact rate limit filter.
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit d16bdb35b62e0321e5120c15f580b4bd0893fa90 Author: Mark Thomas AuthorDate: Fri Mar 7 14:55:22 2025 + Add an exact rate limit filter. Based on PR #794 by Chenjp --- .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 +++ java/org/apache/catalina/util/FastRateLimiter.java | 1 + java/org/apache/catalina/util/RateLimiterBase.java | 10 + .../catalina/util/TimeBucketCounterBase.java | 3 - .../TestRateLimitFilterWithExactRateLimiter.java | 211 + webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 ++- 8 files changed, 323 insertions(+), 14 deletions(-) diff --git a/java/org/apache/catalina/filters/RateLimitFilter.java b/java/org/apache/catalina/filters/RateLimitFilter.java index 8e07ca8182..80315948bb 100644 --- a/java/org/apache/catalina/filters/RateLimitFilter.java +++ b/java/org/apache/catalina/filters/RateLimitFilter.java @@ -47,8 +47,11 @@ import org.apache.tomcat.util.res.StringManager; * so it converts some configured values to more efficient values. For example, a configuration of a 60 seconds time * bucket is converted to 65.536 seconds. That allows for very fast bucket calculation using bit shift arithmetic. In * order to remain true to the user intent, the configured number of requests is then multiplied by the same ratio, so a - * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. You can specify a - * different class as long as it implements the org.apache.catalina.util.RateLimiter interface. + * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. An alternative + * implementation, org.apache.catalina.util.ExactRateLimiter, is intended to provide a less efficient but + * more accurate control, whose effective duration in seconds and number of requests configuration are consist with the + * user declared. You can specify a different class as long as it implements the + * org.apache.catalina.util.RateLimiter interface. * * * It is common to set up different restrictions for different URIs. For example, a login page or authentication script diff --git a/java/org/apache/catalina/util/ExactRateLimiter.java b/java/org/apache/catalina/util/ExactRateLimiter.java new file mode 100644 index 00..a1b8c9190f --- /dev/null +++ b/java/org/apache/catalina/util/ExactRateLimiter.java @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.util; + +import java.util.concurrent.ScheduledExecutorService; + +/** + * A RateLimiter that compromises efficiency for accuracy in order to provide exact rate limiting. + */ +public class ExactRateLimiter extends RateLimiterBase { + +@Override +protected String getDefaultPolicyName() { +return "exact"; +} + + +@Override +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new ExactTimeBucketCounter(duration, executorService); +} + + +/** + * An accurate counter with exact bucket index, but slightly less efficient than another fast counter provided with + * the {@link FastRateLimiter}. + */ +class ExactTimeBucketCounter extends TimeBucketCounterBase { + +ExactTimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(bucketDuration, executorService); +} + +@Override +public long getBucketIndex(long timestamp) { +return (timestamp / 1000) / getBucketDuration(); +} + +@Override +public double getRatio() { +// Actual value is exactly the same as declared +return 1.0d; +} + +@Override +public long getMillisUntilNextBucket() { +long millis = System.currentTimeMillis(); + +long nextTimeBucketMillis = (get
(tomcat) 01/03: Refactor TimeBucketCounter to support alternative implementations
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 5ae9abb066953c46b2414b5f54f52e56b3f9bc3d Author: Mark Thomas AuthorDate: Fri Mar 7 13:27:58 2025 + Refactor TimeBucketCounter to support alternative implementations This is preparatory work for PR #794 --- .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 177 ++-- 2 files changed, 145 insertions(+), 217 deletions(-) diff --git a/java/org/apache/catalina/util/TimeBucketCounter.java b/java/org/apache/catalina/util/TimeBucketCounter.java index 3b4726f7ff..78951623cc 100644 --- a/java/org/apache/catalina/util/TimeBucketCounter.java +++ b/java/org/apache/catalina/util/TimeBucketCounter.java @@ -14,122 +14,91 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ExecutionException; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.ScheduledFuture; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -import org.apache.juli.logging.Log; -import org.apache.juli.logging.LogFactory; -import org.apache.tomcat.util.res.StringManager; /** - * This class maintains a thread safe hash map that has timestamp-based buckets followed by a string for a key, and a - * counter for a value. each time the increment() method is called it adds the key if it does not exist, increments its - * value and returns it. a maintenance thread cleans up keys that are prefixed by previous timestamp buckets. + * A fast counter that optimizes efficiency at the expense of approximate bucket indexing. */ -public class TimeBucketCounter { - -private static final Log log = LogFactory.getLog(TimeBucketCounter.class); -private static final StringManager sm = StringManager.getManager(TimeBucketCounter.class); - -/** - * Map to hold the buckets - */ -private final ConcurrentHashMap map = new ConcurrentHashMap<>(); +public class TimeBucketCounter extends TimeBucketCounterBase { -/** - * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute - */ +// Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute. private final int numBits; -/** - * Ratio of actual duration to config duration - */ +// Ratio of actual duration to config duration private final double ratio; -/** - * The future allowing control of the background processor. - */ -private ScheduledFuture maintenanceFuture; -private ScheduledFuture monitorFuture; -private final ScheduledExecutorService executorService; -private final long sleeptime; -/** - * Creates a new TimeBucketCounter with the specified lifetime. - * - * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 - * @param executorService the executor service which will be used to run the maintenance - */ public TimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(getActualDuration(bucketDuration), executorService); +this.numBits = determineShiftBitsOfDuration(bucketDuration); +this.ratio = ratioToPowerOf2(bucketDuration * 1000); +} -this.executorService = executorService; - -int durationMillis = bucketDuration * 1000; - -int bits = 0; -int pof2 = nextPowerOf2(durationMillis); -int bitCheck = pof2; -while (bitCheck > 1) { -bitCheck = pof2 >> ++bits; -} - -this.numBits = bits; -this.ratio = ratioToPowerOf2(durationMillis); +/** + * {@inheritDoc} + * + * Calculates the current time bucket index by shifting bits for fast division, e.g. shift 16 bits is the same as + * dividing by 65,536 which is about 1:05m. + */ +@Override +public long getBucketIndex(long timestamp) { +return timestamp >> this.numBits; +} -int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; -sleeptime = durationMillis / cleanupsPerBucketDuration; -// Start our thread -if (sleeptime > 0) { -monitorFuture = executorService.scheduleWithFixedDelay(new MaintenanceMonitor(), 0, 60, TimeUnit.SECONDS); -} +public int getNumBits() { +return numBits; } + /** - * Increments the counter for the passed identifier in the current time bucket and returns the new value. - * - * @param identifier an identifier for which we want to maintain count, e.g. IP Address + * The actual du
(tomcat) branch main updated: Refactor TimeBucketCounter to support alternative implementations
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new 8e51eae97d Refactor TimeBucketCounter to support alternative implementations 8e51eae97d is described below commit 8e51eae97ded75641318ea18768a47a14165c6ad Author: Mark Thomas AuthorDate: Fri Mar 7 13:27:58 2025 + Refactor TimeBucketCounter to support alternative implementations This is preparatory work for PR #794 --- .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 177 ++-- 2 files changed, 145 insertions(+), 217 deletions(-) diff --git a/java/org/apache/catalina/util/TimeBucketCounter.java b/java/org/apache/catalina/util/TimeBucketCounter.java index 3b4726f7ff..78951623cc 100644 --- a/java/org/apache/catalina/util/TimeBucketCounter.java +++ b/java/org/apache/catalina/util/TimeBucketCounter.java @@ -14,122 +14,91 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ExecutionException; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.ScheduledFuture; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -import org.apache.juli.logging.Log; -import org.apache.juli.logging.LogFactory; -import org.apache.tomcat.util.res.StringManager; /** - * This class maintains a thread safe hash map that has timestamp-based buckets followed by a string for a key, and a - * counter for a value. each time the increment() method is called it adds the key if it does not exist, increments its - * value and returns it. a maintenance thread cleans up keys that are prefixed by previous timestamp buckets. + * A fast counter that optimizes efficiency at the expense of approximate bucket indexing. */ -public class TimeBucketCounter { - -private static final Log log = LogFactory.getLog(TimeBucketCounter.class); -private static final StringManager sm = StringManager.getManager(TimeBucketCounter.class); - -/** - * Map to hold the buckets - */ -private final ConcurrentHashMap map = new ConcurrentHashMap<>(); +public class TimeBucketCounter extends TimeBucketCounterBase { -/** - * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute - */ +// Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute. private final int numBits; -/** - * Ratio of actual duration to config duration - */ +// Ratio of actual duration to config duration private final double ratio; -/** - * The future allowing control of the background processor. - */ -private ScheduledFuture maintenanceFuture; -private ScheduledFuture monitorFuture; -private final ScheduledExecutorService executorService; -private final long sleeptime; -/** - * Creates a new TimeBucketCounter with the specified lifetime. - * - * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 - * @param executorService the executor service which will be used to run the maintenance - */ public TimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(getActualDuration(bucketDuration), executorService); +this.numBits = determineShiftBitsOfDuration(bucketDuration); +this.ratio = ratioToPowerOf2(bucketDuration * 1000); +} -this.executorService = executorService; - -int durationMillis = bucketDuration * 1000; - -int bits = 0; -int pof2 = nextPowerOf2(durationMillis); -int bitCheck = pof2; -while (bitCheck > 1) { -bitCheck = pof2 >> ++bits; -} - -this.numBits = bits; -this.ratio = ratioToPowerOf2(durationMillis); +/** + * {@inheritDoc} + * + * Calculates the current time bucket index by shifting bits for fast division, e.g. shift 16 bits is the same as + * dividing by 65,536 which is about 1:05m. + */ +@Override +public long getBucketIndex(long timestamp) { +return timestamp >> this.numBits; +} -int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; -sleeptime = durationMillis / cleanupsPerBucketDuration; -// Start our thread -if (sleeptime > 0) { -monitorFuture = executorService.scheduleWithFixedDelay(new MaintenanceMonitor(), 0, 60, TimeUnit.SECONDS); -} +public int getNumBits() { +return numBits; } + /** - * Increments the counter for the passed iden
[Bug 69605] Possibility to clear the notFoundClassResource cache from within a running webapp
https://bz.apache.org/bugzilla/show_bug.cgi?id=69605 --- Comment #4 from so...@his.de --- Thanks :) -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 03/03: Add an exact rate limit filter.
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit f2c550898c456cbfc9da780bbfb85ab9065a26ba Author: Mark Thomas AuthorDate: Fri Mar 7 14:55:22 2025 + Add an exact rate limit filter. Based on PR #794 by Chenjp --- .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 +++ java/org/apache/catalina/util/FastRateLimiter.java | 1 + java/org/apache/catalina/util/RateLimiterBase.java | 10 + .../catalina/util/TimeBucketCounterBase.java | 3 - .../TestRateLimitFilterWithExactRateLimiter.java | 211 + webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 ++- 8 files changed, 323 insertions(+), 14 deletions(-) diff --git a/java/org/apache/catalina/filters/RateLimitFilter.java b/java/org/apache/catalina/filters/RateLimitFilter.java index 8e07ca8182..80315948bb 100644 --- a/java/org/apache/catalina/filters/RateLimitFilter.java +++ b/java/org/apache/catalina/filters/RateLimitFilter.java @@ -47,8 +47,11 @@ import org.apache.tomcat.util.res.StringManager; * so it converts some configured values to more efficient values. For example, a configuration of a 60 seconds time * bucket is converted to 65.536 seconds. That allows for very fast bucket calculation using bit shift arithmetic. In * order to remain true to the user intent, the configured number of requests is then multiplied by the same ratio, so a - * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. You can specify a - * different class as long as it implements the org.apache.catalina.util.RateLimiter interface. + * configuration of 100 Requests per 60 seconds, has the real values of 109 Requests per 65 seconds. An alternative + * implementation, org.apache.catalina.util.ExactRateLimiter, is intended to provide a less efficient but + * more accurate control, whose effective duration in seconds and number of requests configuration are consist with the + * user declared. You can specify a different class as long as it implements the + * org.apache.catalina.util.RateLimiter interface. * * * It is common to set up different restrictions for different URIs. For example, a login page or authentication script diff --git a/java/org/apache/catalina/util/ExactRateLimiter.java b/java/org/apache/catalina/util/ExactRateLimiter.java new file mode 100644 index 00..a1b8c9190f --- /dev/null +++ b/java/org/apache/catalina/util/ExactRateLimiter.java @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.catalina.util; + +import java.util.concurrent.ScheduledExecutorService; + +/** + * A RateLimiter that compromises efficiency for accuracy in order to provide exact rate limiting. + */ +public class ExactRateLimiter extends RateLimiterBase { + +@Override +protected String getDefaultPolicyName() { +return "exact"; +} + + +@Override +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new ExactTimeBucketCounter(duration, executorService); +} + + +/** + * An accurate counter with exact bucket index, but slightly less efficient than another fast counter provided with + * the {@link FastRateLimiter}. + */ +class ExactTimeBucketCounter extends TimeBucketCounterBase { + +ExactTimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(bucketDuration, executorService); +} + +@Override +public long getBucketIndex(long timestamp) { +return (timestamp / 1000) / getBucketDuration(); +} + +@Override +public double getRatio() { +// Actual value is exactly the same as declared +return 1.0d; +} + +@Override +public long getMillisUntilNextBucket() { +long millis = System.currentTimeMillis(); + +long nextTimeBucketMillis = (get
(tomcat) branch 10.1.x updated (6b9f84aff6 -> f2c550898c)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/tomcat.git from 6b9f84aff6 More skip.installer, remove unless in depends-only target new 9ebf845f38 Refactor TimeBucketCounter to support alternative implementations new 93c0819a42 Refactor RateLimiter FastRateLimiter for extension new f2c550898c Add an exact rate limit filter. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 java/org/apache/catalina/util/FastRateLimiter.java | 75 + java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 155 + .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 174 ++- ...> TestRateLimitFilterWithExactRateLimiter.java} | 88 +- webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 +++- 10 files changed, 450 insertions(+), 346 deletions(-) create mode 100644 java/org/apache/catalina/util/ExactRateLimiter.java create mode 100644 java/org/apache/catalina/util/RateLimiterBase.java copy java/org/apache/catalina/util/{TimeBucketCounter.java => TimeBucketCounterBase.java} (54%) copy test/org/apache/catalina/filters/{TestRateLimitFilter.java => TestRateLimitFilterWithExactRateLimiter.java} (75%) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 02/03: Refactor RateLimiter FastRateLimiter for extension
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 93c0819a4297a6f3cc4be3d9aefe99d9b536b7ad Author: Mark Thomas AuthorDate: Fri Mar 7 13:51:00 2025 + Refactor RateLimiter FastRateLimiter for extension Preparation for #794 --- java/org/apache/catalina/util/FastRateLimiter.java | 76 +-- java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 145 + 3 files changed, 154 insertions(+), 74 deletions(-) diff --git a/java/org/apache/catalina/util/FastRateLimiter.java b/java/org/apache/catalina/util/FastRateLimiter.java index 486a133a64..17544c5d28 100644 --- a/java/org/apache/catalina/util/FastRateLimiter.java +++ b/java/org/apache/catalina/util/FastRateLimiter.java @@ -14,92 +14,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.atomic.AtomicInteger; - -import jakarta.servlet.FilterConfig; - -import org.apache.tomcat.util.threads.ScheduledThreadPoolExecutor; /** * A RateLimiter that compromises accuracy for speed in order to provide maximum throughput. */ -public class FastRateLimiter implements RateLimiter { - -private static AtomicInteger index = new AtomicInteger(); - -TimeBucketCounter bucketCounter; - -int duration; - -int requests; - -int actualRequests; - -int actualDuration; - -// Initial policy name can be rewritten by setPolicyName() -private String policyName = "fast-" + index.incrementAndGet(); - -@Override -public String getPolicyName() { -return policyName; -} - -@Override -public void setPolicyName(String name) { -this.policyName = name; -} +public class FastRateLimiter extends RateLimiterBase { @Override -public int getDuration() { -return actualDuration; +protected String getDefaultPolicyName() { +return "fast"; } -@Override -public void setDuration(int duration) { -this.duration = duration; -} - -@Override -public int getRequests() { -return actualRequests; -} - -@Override -public void setRequests(int requests) { -this.requests = requests; -} @Override -public int increment(String ipAddress) { -return bucketCounter.increment(ipAddress); +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new TimeBucketCounter(duration, executorService); } -@Override -public void destroy() { -bucketCounter.destroy(); -} - -@Override -public void setFilterConfig(FilterConfig filterConfig) { - -ScheduledExecutorService executorService = (ScheduledExecutorService) filterConfig.getServletContext() -.getAttribute(ScheduledThreadPoolExecutor.class.getName()); - -if (executorService == null) { -executorService = new java.util.concurrent.ScheduledThreadPoolExecutor(1); -} - -bucketCounter = new TimeBucketCounter(duration, executorService); -actualRequests = (int) Math.round(bucketCounter.getRatio() * requests); -actualDuration = bucketCounter.getActualDuration() / 1000; -} public TimeBucketCounter getBucketCounter() { -return bucketCounter; +return (TimeBucketCounter) bucketCounter; } } diff --git a/java/org/apache/catalina/util/RateLimiter.java b/java/org/apache/catalina/util/RateLimiter.java index bdd8b27736..72effc8498 100644 --- a/java/org/apache/catalina/util/RateLimiter.java +++ b/java/org/apache/catalina/util/RateLimiter.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import jakarta.servlet.FilterConfig; @@ -46,13 +45,13 @@ public interface RateLimiter { void setRequests(int requests); /** - * Increments the number of requests by the given ipAddress in the current time window. + * Increments the number of requests by the given identifier in the current time window. * - * @param ipAddress the ip address + * @param identifier the identifier for which the number of associated requests should be incremented * * @return the new value after incrementing */ -int increment(String ipAddress); +int increment(String identifier); /** * Cleanup no longer needed resources. diff --git a/java/org/apache/catalina/util/RateLimiterBase.java b/java/org/apache/catalina/util/RateLimiterBase.java new file mode 100644 index 00..1f4c699462 --- /dev/null +++ b/java/org/apache/catalina/util/RateLimi
(tomcat) 02/03: Refactor RateLimiter FastRateLimiter for extension
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 9e1aaa5251e2990eb5fb35d1e4000ca068275236 Author: Mark Thomas AuthorDate: Fri Mar 7 13:51:00 2025 + Refactor RateLimiter FastRateLimiter for extension Preparation for #794 --- java/org/apache/catalina/util/FastRateLimiter.java | 76 +-- java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 145 + 3 files changed, 154 insertions(+), 74 deletions(-) diff --git a/java/org/apache/catalina/util/FastRateLimiter.java b/java/org/apache/catalina/util/FastRateLimiter.java index 486a133a64..17544c5d28 100644 --- a/java/org/apache/catalina/util/FastRateLimiter.java +++ b/java/org/apache/catalina/util/FastRateLimiter.java @@ -14,92 +14,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.atomic.AtomicInteger; - -import jakarta.servlet.FilterConfig; - -import org.apache.tomcat.util.threads.ScheduledThreadPoolExecutor; /** * A RateLimiter that compromises accuracy for speed in order to provide maximum throughput. */ -public class FastRateLimiter implements RateLimiter { - -private static AtomicInteger index = new AtomicInteger(); - -TimeBucketCounter bucketCounter; - -int duration; - -int requests; - -int actualRequests; - -int actualDuration; - -// Initial policy name can be rewritten by setPolicyName() -private String policyName = "fast-" + index.incrementAndGet(); - -@Override -public String getPolicyName() { -return policyName; -} - -@Override -public void setPolicyName(String name) { -this.policyName = name; -} +public class FastRateLimiter extends RateLimiterBase { @Override -public int getDuration() { -return actualDuration; +protected String getDefaultPolicyName() { +return "fast"; } -@Override -public void setDuration(int duration) { -this.duration = duration; -} - -@Override -public int getRequests() { -return actualRequests; -} - -@Override -public void setRequests(int requests) { -this.requests = requests; -} @Override -public int increment(String ipAddress) { -return bucketCounter.increment(ipAddress); +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new TimeBucketCounter(duration, executorService); } -@Override -public void destroy() { -bucketCounter.destroy(); -} - -@Override -public void setFilterConfig(FilterConfig filterConfig) { - -ScheduledExecutorService executorService = (ScheduledExecutorService) filterConfig.getServletContext() -.getAttribute(ScheduledThreadPoolExecutor.class.getName()); - -if (executorService == null) { -executorService = new java.util.concurrent.ScheduledThreadPoolExecutor(1); -} - -bucketCounter = new TimeBucketCounter(duration, executorService); -actualRequests = (int) Math.round(bucketCounter.getRatio() * requests); -actualDuration = bucketCounter.getActualDuration() / 1000; -} public TimeBucketCounter getBucketCounter() { -return bucketCounter; +return (TimeBucketCounter) bucketCounter; } } diff --git a/java/org/apache/catalina/util/RateLimiter.java b/java/org/apache/catalina/util/RateLimiter.java index bdd8b27736..72effc8498 100644 --- a/java/org/apache/catalina/util/RateLimiter.java +++ b/java/org/apache/catalina/util/RateLimiter.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import jakarta.servlet.FilterConfig; @@ -46,13 +45,13 @@ public interface RateLimiter { void setRequests(int requests); /** - * Increments the number of requests by the given ipAddress in the current time window. + * Increments the number of requests by the given identifier in the current time window. * - * @param ipAddress the ip address + * @param identifier the identifier for which the number of associated requests should be incremented * * @return the new value after incrementing */ -int increment(String ipAddress); +int increment(String identifier); /** * Cleanup no longer needed resources. diff --git a/java/org/apache/catalina/util/RateLimiterBase.java b/java/org/apache/catalina/util/RateLimiterBase.java new file mode 100644 index 00..1f4c699462 --- /dev/null +++ b/java/org/apache/catalina/util/RateLimi
(tomcat) branch 11.0.x updated (3eee70c8fb -> d16bdb35b6)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git from 3eee70c8fb More skip.installer, remove unless in depends-only target new 5ae9abb066 Refactor TimeBucketCounter to support alternative implementations new 9e1aaa5251 Refactor RateLimiter FastRateLimiter for extension new d16bdb35b6 Add an exact rate limit filter. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/catalina/filters/RateLimitFilter.java | 7 +- .../org/apache/catalina/util/ExactRateLimiter.java | 68 java/org/apache/catalina/util/FastRateLimiter.java | 75 + java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 155 + .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 174 ++- ...> TestRateLimitFilterWithExactRateLimiter.java} | 88 +- webapps/docs/changelog.xml | 7 + webapps/docs/config/filter.xml | 30 +++- 10 files changed, 450 insertions(+), 346 deletions(-) create mode 100644 java/org/apache/catalina/util/ExactRateLimiter.java create mode 100644 java/org/apache/catalina/util/RateLimiterBase.java copy java/org/apache/catalina/util/{TimeBucketCounter.java => TimeBucketCounterBase.java} (54%) copy test/org/apache/catalina/filters/{TestRateLimitFilter.java => TestRateLimitFilterWithExactRateLimiter.java} (75%) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) 01/03: Refactor TimeBucketCounter to support alternative implementations
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 9ebf845f38d1ccfd7466bd46b3c640f4b3e1f5c9 Author: Mark Thomas AuthorDate: Fri Mar 7 13:27:58 2025 + Refactor TimeBucketCounter to support alternative implementations This is preparatory work for PR #794 --- .../apache/catalina/util/TimeBucketCounter.java| 185 +++-- ...cketCounter.java => TimeBucketCounterBase.java} | 177 ++-- 2 files changed, 145 insertions(+), 217 deletions(-) diff --git a/java/org/apache/catalina/util/TimeBucketCounter.java b/java/org/apache/catalina/util/TimeBucketCounter.java index 3b4726f7ff..78951623cc 100644 --- a/java/org/apache/catalina/util/TimeBucketCounter.java +++ b/java/org/apache/catalina/util/TimeBucketCounter.java @@ -14,122 +14,91 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ExecutionException; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.ScheduledFuture; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -import org.apache.juli.logging.Log; -import org.apache.juli.logging.LogFactory; -import org.apache.tomcat.util.res.StringManager; /** - * This class maintains a thread safe hash map that has timestamp-based buckets followed by a string for a key, and a - * counter for a value. each time the increment() method is called it adds the key if it does not exist, increments its - * value and returns it. a maintenance thread cleans up keys that are prefixed by previous timestamp buckets. + * A fast counter that optimizes efficiency at the expense of approximate bucket indexing. */ -public class TimeBucketCounter { - -private static final Log log = LogFactory.getLog(TimeBucketCounter.class); -private static final StringManager sm = StringManager.getManager(TimeBucketCounter.class); - -/** - * Map to hold the buckets - */ -private final ConcurrentHashMap map = new ConcurrentHashMap<>(); +public class TimeBucketCounter extends TimeBucketCounterBase { -/** - * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute - */ +// Milliseconds bucket size as a Power of 2 for bit shift math, e.g. 16 for 65_536ms which is about 1:05 minute. private final int numBits; -/** - * Ratio of actual duration to config duration - */ +// Ratio of actual duration to config duration private final double ratio; -/** - * The future allowing control of the background processor. - */ -private ScheduledFuture maintenanceFuture; -private ScheduledFuture monitorFuture; -private final ScheduledExecutorService executorService; -private final long sleeptime; -/** - * Creates a new TimeBucketCounter with the specified lifetime. - * - * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 - * @param executorService the executor service which will be used to run the maintenance - */ public TimeBucketCounter(int bucketDuration, ScheduledExecutorService executorService) { +super(getActualDuration(bucketDuration), executorService); +this.numBits = determineShiftBitsOfDuration(bucketDuration); +this.ratio = ratioToPowerOf2(bucketDuration * 1000); +} -this.executorService = executorService; - -int durationMillis = bucketDuration * 1000; - -int bits = 0; -int pof2 = nextPowerOf2(durationMillis); -int bitCheck = pof2; -while (bitCheck > 1) { -bitCheck = pof2 >> ++bits; -} - -this.numBits = bits; -this.ratio = ratioToPowerOf2(durationMillis); +/** + * {@inheritDoc} + * + * Calculates the current time bucket index by shifting bits for fast division, e.g. shift 16 bits is the same as + * dividing by 65,536 which is about 1:05m. + */ +@Override +public long getBucketIndex(long timestamp) { +return timestamp >> this.numBits; +} -int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; -sleeptime = durationMillis / cleanupsPerBucketDuration; -// Start our thread -if (sleeptime > 0) { -monitorFuture = executorService.scheduleWithFixedDelay(new MaintenanceMonitor(), 0, 60, TimeUnit.SECONDS); -} +public int getNumBits() { +return numBits; } + /** - * Increments the counter for the passed identifier in the current time bucket and returns the new value. - * - * @param identifier an identifier for which we want to maintain count, e.g. IP Address + * The actual du
(tomcat) branch main updated: Refactor RateLimiter FastRateLimiter for extension
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new 137c9f4da9 Refactor RateLimiter FastRateLimiter for extension 137c9f4da9 is described below commit 137c9f4da9c83b35377baa9622f1a1bac1d3dcc8 Author: Mark Thomas AuthorDate: Fri Mar 7 13:51:00 2025 + Refactor RateLimiter FastRateLimiter for extension Preparation for #794 --- java/org/apache/catalina/util/FastRateLimiter.java | 76 +-- java/org/apache/catalina/util/RateLimiter.java | 7 +- java/org/apache/catalina/util/RateLimiterBase.java | 145 + 3 files changed, 154 insertions(+), 74 deletions(-) diff --git a/java/org/apache/catalina/util/FastRateLimiter.java b/java/org/apache/catalina/util/FastRateLimiter.java index 486a133a64..17544c5d28 100644 --- a/java/org/apache/catalina/util/FastRateLimiter.java +++ b/java/org/apache/catalina/util/FastRateLimiter.java @@ -14,92 +14,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.atomic.AtomicInteger; - -import jakarta.servlet.FilterConfig; - -import org.apache.tomcat.util.threads.ScheduledThreadPoolExecutor; /** * A RateLimiter that compromises accuracy for speed in order to provide maximum throughput. */ -public class FastRateLimiter implements RateLimiter { - -private static AtomicInteger index = new AtomicInteger(); - -TimeBucketCounter bucketCounter; - -int duration; - -int requests; - -int actualRequests; - -int actualDuration; - -// Initial policy name can be rewritten by setPolicyName() -private String policyName = "fast-" + index.incrementAndGet(); - -@Override -public String getPolicyName() { -return policyName; -} - -@Override -public void setPolicyName(String name) { -this.policyName = name; -} +public class FastRateLimiter extends RateLimiterBase { @Override -public int getDuration() { -return actualDuration; +protected String getDefaultPolicyName() { +return "fast"; } -@Override -public void setDuration(int duration) { -this.duration = duration; -} - -@Override -public int getRequests() { -return actualRequests; -} - -@Override -public void setRequests(int requests) { -this.requests = requests; -} @Override -public int increment(String ipAddress) { -return bucketCounter.increment(ipAddress); +protected TimeBucketCounterBase newCounterInstance(int duration, ScheduledExecutorService executorService) { +return new TimeBucketCounter(duration, executorService); } -@Override -public void destroy() { -bucketCounter.destroy(); -} - -@Override -public void setFilterConfig(FilterConfig filterConfig) { - -ScheduledExecutorService executorService = (ScheduledExecutorService) filterConfig.getServletContext() -.getAttribute(ScheduledThreadPoolExecutor.class.getName()); - -if (executorService == null) { -executorService = new java.util.concurrent.ScheduledThreadPoolExecutor(1); -} - -bucketCounter = new TimeBucketCounter(duration, executorService); -actualRequests = (int) Math.round(bucketCounter.getRatio() * requests); -actualDuration = bucketCounter.getActualDuration() / 1000; -} public TimeBucketCounter getBucketCounter() { -return bucketCounter; +return (TimeBucketCounter) bucketCounter; } } diff --git a/java/org/apache/catalina/util/RateLimiter.java b/java/org/apache/catalina/util/RateLimiter.java index bdd8b27736..72effc8498 100644 --- a/java/org/apache/catalina/util/RateLimiter.java +++ b/java/org/apache/catalina/util/RateLimiter.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.catalina.util; import jakarta.servlet.FilterConfig; @@ -46,13 +45,13 @@ public interface RateLimiter { void setRequests(int requests); /** - * Increments the number of requests by the given ipAddress in the current time window. + * Increments the number of requests by the given identifier in the current time window. * - * @param ipAddress the ip address + * @param identifier the identifier for which the number of associated requests should be incremented * * @return the new value after incrementing */ -int increment(String ipAddress); +int increment(String identifier); /** * Cleanup no longer needed resources. diff --git a/java/org/apache/catalina/util/RateLimiterBas
[Bug 69605] Possibility to clear the notFoundClassResource cache from within a running webapp
https://bz.apache.org/bugzilla/show_bug.cgi?id=69605 --- Comment #2 from so...@his.de --- Thank you for your fast response :) I know that this is not a support forum, but I am unsure what parts of tomcat are considered stable API or what parts are not. Would the following code use the stable API of tomcat? ClassLoader classLoader = Thread.currentThread().getContextClassLoader(); Class webappClassLoaderBaseClass = classLoader.getClass().getSuperclass(); Method getNotFoundClassResourceCacheSizeMethod = webappClassLoaderBaseClass.getDeclaredMethod("getNotFoundClassResourceCacheSize"); Integer cacheSize = (Integer) getNotFoundClassResourceCacheSizeMethod.invoke(classLoader); -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat) branch main updated: Expand test coverage
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new e770b48dd0 Expand test coverage e770b48dd0 is described below commit e770b48dd0064f7f21d67278e73f5c1ac30460bf Author: remm AuthorDate: Sat Mar 8 00:00:00 2025 +0100 Expand test coverage Cleanup. --- .../catalina/valves/AbstractAccessLogValve.java| 10 ++- .../catalina/valves/ExtendedAccessLogValve.java| 76 +- .../valves/TestExtendedAccessLogValve.java | 17 - .../valves/TestExtendedAccessLogValveWrap.java | 34 +++--- 4 files changed, 82 insertions(+), 55 deletions(-) diff --git a/java/org/apache/catalina/valves/AbstractAccessLogValve.java b/java/org/apache/catalina/valves/AbstractAccessLogValve.java index f6aeeb7e62..5dde798b64 100644 --- a/java/org/apache/catalina/valves/AbstractAccessLogValve.java +++ b/java/org/apache/catalina/valves/AbstractAccessLogValve.java @@ -1924,6 +1924,10 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access * encoding which may not be true for Tomcat so Tomcat uses the Java \\u encoding. */ protected static void escapeAndAppend(String input, CharArrayWriter dest) { +escapeAndAppend(input, dest, false); +} + +protected static void escapeAndAppend(String input, CharArrayWriter dest, boolean escapeQuoteAsDouble) { if (input == null || input.isEmpty()) { dest.append('-'); return; @@ -1959,7 +1963,11 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access dest.write(input, next, current - next); } next = current + 1; -dest.append("\\\""); +if (escapeQuoteAsDouble) { +dest.append("\"\""); +} else { +dest.append("\\\""); +} break; // Don't output individual unchanged chars, // write the sub string only when the first char to encode diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index ce4de03805..268a56b414 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -81,26 +81,6 @@ import org.apache.tomcat.util.ExceptionUtils; * x-H(scheme): getScheme * x-H(secure): isSecure * - * - * Log rotation can be on or off. This is dictated by the rotatable property. - * - * - * For UNIX users, another field called checkExists is also available. If set to true, the log file's - * existence will be checked before each logging. This way an external log rotator can move the file somewhere and - * Tomcat will start with a new file. - * - * - * For JMX junkies, a public method called rotate has been made available to allow you to tell this - * instance to move the existing log file to somewhere else and start writing a new log file. - * - * - * Conditional logging is also supported. This can be done with the condition property. If the value - * returned from ServletRequest.getAttribute(condition) yields a non-null value, the logging will be skipped. - * - * - * For extended attributes coming from a getAttribute() call, it is you responsibility to ensure there are no newline or - * control characters. - * * * @author Peter Rossbach */ @@ -115,15 +95,17 @@ public class ExtendedAccessLogValve extends AccessLogValve { * double quotes (""). * * @param value - The value to wrap + * @param buf the buffer to write to * * @return '-' if null. Otherwise, toString() will be called on the object and the value will be wrapped in quotes * and any quotes will be escaped with 2 sets of quotes. */ -static String wrap(Object value) { +static void wrap(Object value, CharArrayWriter buf) { String svalue; // Does the value contain a " ? If so must encode it if (value == null || "-".equals(value)) { -return "-"; +buf.append('-'); +return; } try { @@ -131,11 +113,15 @@ public class ExtendedAccessLogValve extends AccessLogValve { } catch (Throwable e) { ExceptionUtils.handleThrowable(e); /* Log error */ -return "-"; +buf.append('-'); +return; } -/* Wrap all values in double quotes. */ -return "\"" + svalue.replace("\"", "\"\"") + "\""; +buf.append('\"'); +if (!svalue.isEmpty()) { +escapeAndAppend(svalue, buf, true); +
(tomcat) branch 11.0.x updated: Expand test coverage
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 11.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/11.0.x by this push: new abf9c58911 Expand test coverage abf9c58911 is described below commit abf9c58911690306dbb2be93dd9d2b219f4ff346 Author: remm AuthorDate: Sat Mar 8 00:00:00 2025 +0100 Expand test coverage Cleanup. --- .../catalina/valves/AbstractAccessLogValve.java| 10 ++- .../catalina/valves/ExtendedAccessLogValve.java| 76 +- .../valves/TestExtendedAccessLogValve.java | 17 - .../valves/TestExtendedAccessLogValveWrap.java | 34 +++--- 4 files changed, 82 insertions(+), 55 deletions(-) diff --git a/java/org/apache/catalina/valves/AbstractAccessLogValve.java b/java/org/apache/catalina/valves/AbstractAccessLogValve.java index f6aeeb7e62..5dde798b64 100644 --- a/java/org/apache/catalina/valves/AbstractAccessLogValve.java +++ b/java/org/apache/catalina/valves/AbstractAccessLogValve.java @@ -1924,6 +1924,10 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access * encoding which may not be true for Tomcat so Tomcat uses the Java \\u encoding. */ protected static void escapeAndAppend(String input, CharArrayWriter dest) { +escapeAndAppend(input, dest, false); +} + +protected static void escapeAndAppend(String input, CharArrayWriter dest, boolean escapeQuoteAsDouble) { if (input == null || input.isEmpty()) { dest.append('-'); return; @@ -1959,7 +1963,11 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access dest.write(input, next, current - next); } next = current + 1; -dest.append("\\\""); +if (escapeQuoteAsDouble) { +dest.append("\"\""); +} else { +dest.append("\\\""); +} break; // Don't output individual unchanged chars, // write the sub string only when the first char to encode diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index ce4de03805..268a56b414 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -81,26 +81,6 @@ import org.apache.tomcat.util.ExceptionUtils; * x-H(scheme): getScheme * x-H(secure): isSecure * - * - * Log rotation can be on or off. This is dictated by the rotatable property. - * - * - * For UNIX users, another field called checkExists is also available. If set to true, the log file's - * existence will be checked before each logging. This way an external log rotator can move the file somewhere and - * Tomcat will start with a new file. - * - * - * For JMX junkies, a public method called rotate has been made available to allow you to tell this - * instance to move the existing log file to somewhere else and start writing a new log file. - * - * - * Conditional logging is also supported. This can be done with the condition property. If the value - * returned from ServletRequest.getAttribute(condition) yields a non-null value, the logging will be skipped. - * - * - * For extended attributes coming from a getAttribute() call, it is you responsibility to ensure there are no newline or - * control characters. - * * * @author Peter Rossbach */ @@ -115,15 +95,17 @@ public class ExtendedAccessLogValve extends AccessLogValve { * double quotes (""). * * @param value - The value to wrap + * @param buf the buffer to write to * * @return '-' if null. Otherwise, toString() will be called on the object and the value will be wrapped in quotes * and any quotes will be escaped with 2 sets of quotes. */ -static String wrap(Object value) { +static void wrap(Object value, CharArrayWriter buf) { String svalue; // Does the value contain a " ? If so must encode it if (value == null || "-".equals(value)) { -return "-"; +buf.append('-'); +return; } try { @@ -131,11 +113,15 @@ public class ExtendedAccessLogValve extends AccessLogValve { } catch (Throwable e) { ExceptionUtils.handleThrowable(e); /* Log error */ -return "-"; +buf.append('-'); +return; } -/* Wrap all values in double quotes. */ -return "\"" + svalue.replace("\"", "\"\"") + "\""; +buf.append('\"'); +if (!svalue.isEmpty()) { +escapeAndAppend(svalue, buf, true);
(tomcat) branch 10.1.x updated: Expand test coverage
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 10.1.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/10.1.x by this push: new 7c022789c3 Expand test coverage 7c022789c3 is described below commit 7c022789c37ce6c7968c762c5532288e5745429e Author: remm AuthorDate: Sat Mar 8 00:00:00 2025 +0100 Expand test coverage Cleanup. --- .../catalina/valves/AbstractAccessLogValve.java| 10 ++- .../catalina/valves/ExtendedAccessLogValve.java| 82 ++ .../valves/TestExtendedAccessLogValve.java | 17 - .../valves/TestExtendedAccessLogValveWrap.java | 34 ++--- 4 files changed, 88 insertions(+), 55 deletions(-) diff --git a/java/org/apache/catalina/valves/AbstractAccessLogValve.java b/java/org/apache/catalina/valves/AbstractAccessLogValve.java index b462ebc470..b704b3a1f0 100644 --- a/java/org/apache/catalina/valves/AbstractAccessLogValve.java +++ b/java/org/apache/catalina/valves/AbstractAccessLogValve.java @@ -1926,6 +1926,10 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access * encoding which may not be true for Tomcat so Tomcat uses the Java \\u encoding. */ protected static void escapeAndAppend(String input, CharArrayWriter dest) { +escapeAndAppend(input, dest, false); +} + +protected static void escapeAndAppend(String input, CharArrayWriter dest, boolean escapeQuoteAsDouble) { if (input == null || input.isEmpty()) { dest.append('-'); return; @@ -1961,7 +1965,11 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access dest.write(input, next, current - next); } next = current + 1; -dest.append("\\\""); +if (escapeQuoteAsDouble) { +dest.append("\"\""); +} else { +dest.append("\\\""); +} break; // Don't output individual unchanged chars, // write the sub string only when the first char to encode diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index 672c0c4fb8..268a56b414 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -81,26 +81,6 @@ import org.apache.tomcat.util.ExceptionUtils; * x-H(scheme): getScheme * x-H(secure): isSecure * - * - * Log rotation can be on or off. This is dictated by the rotatable property. - * - * - * For UNIX users, another field called checkExists is also available. If set to true, the log file's - * existence will be checked before each logging. This way an external log rotator can move the file somewhere and - * Tomcat will start with a new file. - * - * - * For JMX junkies, a public method called rotate has been made available to allow you to tell this - * instance to move the existing log file to somewhere else and start writing a new log file. - * - * - * Conditional logging is also supported. This can be done with the condition property. If the value - * returned from ServletRequest.getAttribute(condition) yields a non-null value, the logging will be skipped. - * - * - * For extended attributes coming from a getAttribute() call, it is you responsibility to ensure there are no newline or - * control characters. - * * * @author Peter Rossbach */ @@ -115,15 +95,17 @@ public class ExtendedAccessLogValve extends AccessLogValve { * double quotes (""). * * @param value - The value to wrap + * @param buf the buffer to write to * * @return '-' if null. Otherwise, toString() will be called on the object and the value will be wrapped in quotes * and any quotes will be escaped with 2 sets of quotes. */ -static String wrap(Object value) { +static void wrap(Object value, CharArrayWriter buf) { String svalue; // Does the value contain a " ? If so must encode it if (value == null || "-".equals(value)) { -return "-"; +buf.append('-'); +return; } try { @@ -131,11 +113,15 @@ public class ExtendedAccessLogValve extends AccessLogValve { } catch (Throwable e) { ExceptionUtils.handleThrowable(e); /* Log error */ -return "-"; +buf.append('-'); +return; } -/* Wrap all values in double quotes. */ -return "\"" + svalue.replace("\"", "\"\"") + "\""; +buf.append('\"'); +if (!svalue.isEmpty()) { +escapeAndAppend(svalue, buf, true);
(tomcat) branch 9.0.x updated: Expand test coverage
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/9.0.x by this push: new 163004d96f Expand test coverage 163004d96f is described below commit 163004d96fe6614d518ce18493c55149eadacff0 Author: remm AuthorDate: Sat Mar 8 00:00:00 2025 +0100 Expand test coverage Cleanup. --- .../catalina/valves/AbstractAccessLogValve.java| 10 ++- .../catalina/valves/ExtendedAccessLogValve.java| 80 ++ .../valves/TestExtendedAccessLogValve.java | 16 - .../valves/TestExtendedAccessLogValveWrap.java | 34 ++--- 4 files changed, 86 insertions(+), 54 deletions(-) diff --git a/java/org/apache/catalina/valves/AbstractAccessLogValve.java b/java/org/apache/catalina/valves/AbstractAccessLogValve.java index 3f39d609f0..c223e0de30 100644 --- a/java/org/apache/catalina/valves/AbstractAccessLogValve.java +++ b/java/org/apache/catalina/valves/AbstractAccessLogValve.java @@ -1816,6 +1816,10 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access * encoding which may not be true for Tomcat so Tomcat uses the Java \\u encoding. */ protected static void escapeAndAppend(String input, CharArrayWriter dest) { +escapeAndAppend(input, dest, false); +} + +protected static void escapeAndAppend(String input, CharArrayWriter dest, boolean escapeQuoteAsDouble) { if (input == null || input.isEmpty()) { dest.append('-'); return; @@ -1851,7 +1855,11 @@ public abstract class AbstractAccessLogValve extends ValveBase implements Access dest.write(input, next, current - next); } next = current + 1; -dest.append("\\\""); +if (escapeQuoteAsDouble) { +dest.append("\"\""); +} else { +dest.append("\\\""); +} break; // Don't output individual unchanged chars, // write the sub string only when the first char to encode diff --git a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java index 745ba56889..42d10ec7ae 100644 --- a/java/org/apache/catalina/valves/ExtendedAccessLogValve.java +++ b/java/org/apache/catalina/valves/ExtendedAccessLogValve.java @@ -80,26 +80,6 @@ import org.apache.tomcat.util.ExceptionUtils; * x-H(scheme): getScheme * x-H(secure): isSecure * - * - * Log rotation can be on or off. This is dictated by the rotatable property. - * - * - * For UNIX users, another field called checkExists is also available. If set to true, the log file's - * existence will be checked before each logging. This way an external log rotator can move the file somewhere and - * Tomcat will start with a new file. - * - * - * For JMX junkies, a public method called rotate has been made available to allow you to tell this - * instance to move the existing log file to somewhere else and start writing a new log file. - * - * - * Conditional logging is also supported. This can be done with the condition property. If the value - * returned from ServletRequest.getAttribute(condition) yields a non-null value, the logging will be skipped. - * - * - * For extended attributes coming from a getAttribute() call, it is you responsibility to ensure there are no newline or - * control characters. - * * * @author Peter Rossbach */ @@ -114,15 +94,17 @@ public class ExtendedAccessLogValve extends AccessLogValve { * double quotes (""). * * @param value - The value to wrap + * @param buf the buffer to write to * * @return '-' if null. Otherwise, toString() will be called on the object and the value will be wrapped in quotes * and any quotes will be escaped with 2 sets of quotes. */ -static String wrap(Object value) { +static void wrap(Object value, CharArrayWriter buf) { String svalue; // Does the value contain a " ? If so must encode it if (value == null || "-".equals(value)) { -return "-"; +buf.append('-'); +return; } try { @@ -130,11 +112,15 @@ public class ExtendedAccessLogValve extends AccessLogValve { } catch (Throwable e) { ExceptionUtils.handleThrowable(e); /* Log error */ -return "-"; +buf.append('-'); +return; } -/* Wrap all values in double quotes. */ -return "\"" + svalue.replace("\"", "\"\"") + "\""; +buf.append('\"'); +if (!svalue.isEmpty()) { +escapeAndAppend(svalue, buf, true); +
Buildbot failure in on tomcat-12.0.x
Build status: BUILD FAILED: failed compile (failure) Worker used: bb_worker2_ubuntu URL: https://ci2.apache.org/#builders/120/builds/431 Blamelist: remm Build Text: failed compile (failure) Status Detected: new failure Build Source Stamp: [branch main] 947802bb12e42b6f98ef1552cfb4c46490bf76d4 Steps: worker_preparation: 0 git: 0 shell: 0 shell_1: 0 shell_2: 0 shell_3: 0 shell_4: 0 shell_5: 0 shell_6: 0 compile: 1 shell_7: 0 shell_8: 0 shell_9: 0 shell_10: 0 Rsync docs to nightlies.apache.org: 0 shell_11: 0 Rsync RAT to nightlies.apache.org: 0 compile_1: 2 shell_12: 0 Rsync Logs to nightlies.apache.org: 0 -- ASF Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Buildbot failure in on tomcat-11.0.x
Build status: BUILD FAILED: failed compile (failure) Worker used: bb_worker2_ubuntu URL: https://ci2.apache.org/#builders/112/builds/1561 Blamelist: Mark Thomas , remm Build Text: failed compile (failure) Status Detected: new failure Build Source Stamp: [branch 11.0.x] 550c47eb3e8b0c43a93d0bf8f6221f9f13b67ed2 Steps: worker_preparation: 0 git: 0 shell: 0 shell_1: 0 shell_2: 0 shell_3: 0 shell_4: 0 shell_5: 0 shell_6: 0 compile: 1 shell_7: 0 shell_8: 0 shell_9: 0 shell_10: 0 Rsync docs to nightlies.apache.org: 0 shell_11: 0 Rsync RAT to nightlies.apache.org: 0 compile_1: 2 shell_12: 0 Rsync Logs to nightlies.apache.org: 0 -- ASF Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 10.1.39
Am 04.03.25 um 20:13 schrieb Christopher Schultz: The proposed Apache Tomcat 10.1.39 release is now available for voting. All committers and PMC members are kindly requested to provide a vote if possible. ANY TOMCAT USER MAY VOTE, though only PMC members votes are binding. We welcome non-committer votes or comments on release builds. Note that 10.1.37 and 10.1.38 were not released due to packaging errors. The notable changes compared to 10.1.36 are: - Improve the checks for exposure to and protection against CVE-2024-56337 so that reflection is not used unless required. The checks for whether the file system is case sensitive or not have been removed. - Use Transfer-Encoding for compression rather than Content-Encoding if the client submits a TE header containing gzip. - Add makensis as an option for building the Installer for Windows on non-Windows platforms. For full details, see the change log: https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 without changes. Java EE applications designed for Tomcat 9 and earlier may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will automatically convert them to Jakarta EE and copy them to the webapps directory. It can be obtained from: https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.38/ The Maven staging repo is: https://repository.apache.org/content/repositories/orgapachetomcat-1539 The tag is: https://github.com/apache/tomcat/tree/10.1.39 https://github.com/apache/tomcat/ commit/1e17b1d03591faa17a6eedeaeb3a9f9f7aac12fa Please reply with a +1 for release or +0/-0/-1 with an explanation. I am late, but for the sake of completeness: +1 Reproducibility of the build checked (including the Windows installer) on Linux Mint 22.1 (x86_64) using custom makensis. OK after setting LANG=en_US.utf8 for javadoc (fixed in main). Tested on platforms - RHEL 6, 7, 8 and 9, SLES 11, 12 and 15, Solaris 11 Sparc using - current patch versions of JDK 11, 17, 21, 23 and 24+25 (current EAs) from - Eclipse Adoptium, Azul Zulu, Amazon Corretto, Oracle, RedHat and OpenJDK (for the EAs) where available. Also tested with - tcnative 1.3.1, tcnative 2.0.8 and panama based on - OpenSSL 3.0.16, 3.1.8, 3.2.4, 3.3.3 and 3.4.1. All tests fine, except for the usual sporadic crashes with tcnative during shutdown. Thanks for RM! Best regards, Rainer - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 69607] MD5 algorithm insecure usage in tomcat-util
https://bz.apache.org/bugzilla/show_bug.cgi?id=69607 Christopher Schultz changed: What|Removed |Added Status|NEW |NEEDINFO --- Comment #2 from Christopher Schultz --- HTTP Digest Authentication [RFC 2617] practically requires the use of MD5. Yes, there is RFC 7616 but I'm not sure how widely-supported that is among web browsers. Even if web browsers support it, the migration path for applications from MD5 to another digest algorithm is probably so convoluted they are more likely to start again from scratch. The servlet spec (6.1) 13.6.2 says "Servlet containers SHOULD support HTTP_DIGEST authentication." Because of this, Tomcat SHOULD (IMHO: must) continue to support both HTTP_DIGEST and, because of the above, MD5. If you are failing a security audit because Tomcat is initializing a prohibited algorithm *but not actually using it*, then I think a patch that uses something like a system property to cause Tomcat to skip initializing an MD5 MessageDigest is reasonable. But Tomcat itself is not using MD5 for anything it's not configured by its user to actually use, so the default configuration is indeed safe. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 10.1.39
All, On 3/4/25 2:13 PM, Christopher Schultz wrote: The proposed Apache Tomcat 10.1.39 release is now available for voting. All committers and PMC members are kindly requested to provide a vote if possible. ANY TOMCAT USER MAY VOTE, though only PMC members votes are binding. We welcome non-committer votes or comments on release builds. Note that 10.1.37 and 10.1.38 were not released due to packaging errors. The notable changes compared to 10.1.36 are: - Improve the checks for exposure to and protection against CVE-2024-56337 so that reflection is not used unless required. The checks for whether the file system is case sensitive or not have been removed. - Use Transfer-Encoding for compression rather than Content-Encoding if the client submits a TE header containing gzip. - Add makensis as an option for building the Installer for Windows on non-Windows platforms. For full details, see the change log: https://nightlies.apache.org/tomcat/tomcat-10.1.x/docs/changelog.html Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 without changes. Java EE applications designed for Tomcat 9 and earlier may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat will automatically convert them to Jakarta EE and copy them to the webapps directory. It can be obtained from: https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.1.38/ The Maven staging repo is: https://repository.apache.org/content/repositories/orgapachetomcat-1539 The tag is: https://github.com/apache/tomcat/tree/10.1.39 https://github.com/apache/tomcat/ commit/1e17b1d03591faa17a6eedeaeb3a9f9f7aac12fa Please reply with a +1 for release or +0/-0/-1 with an explanation. +1 for stable release Unit tests pass on MacOS aarch64. Details: * Junit Tests: PASSED chris@Mac:~/git/apache-tomcat-stuff$ grep '^*' tomcat-10.1.39.log * Environment * Java (build):openjdk version "23.0.2" 2025-01-21 OpenJDK Runtime Environment Temurin-23.0.2+7 (build 23.0.2+7) OpenJDK 64-Bit Server VM Temurin-23.0.2+7 (build 23.0.2+7, mixed mode, sharing) * Java (test): openjdk version "23.0.2" 2025-01-21 OpenJDK Runtime Environment Temurin-23.0.2+7 (build 23.0.2+7) OpenJDK 64-Bit Server VM Temurin-23.0.2+7 (build 23.0.2+7, mixed mode, sharing) * Ant: Apache Ant(TM) version 1.10.15 compiled on August 25 2024 * OS: Darwin 24.1.0 arm64 * cc: Apple clang version 16.0.0 (clang-1600.0.26.4) * make:GNU Make 3.81 * OpenSSL: OpenSSL 3.4.0 22 Oct 2024 (Library: OpenSSL 3.4.0 22 Oct 2024) * APR: 1.7.5 * * Valid SHA-512 signature for apache-tomcat-10.1.39.zip * Valid GPG signature for apache-tomcat-10.1.39.zip * Valid SHA-512 signature for apache-tomcat-10.1.39.tar.gz * Valid GPG signature for apache-tomcat-10.1.39.tar.gz * Valid SHA-512 signature for apache-tomcat-10.1.39.exe * Valid GPG signature for apache-tomcat-10.1.39.exe * Valid Windows Digital Signature for apache-tomcat-10.1.39.exe * Valid SHA512 signature for apache-tomcat-10.1.39-src.zip * Valid GPG signature for apache-tomcat-10.1.39-src.zip * Valid SHA512 signature for apache-tomcat-10.1.39-src.tar.gz * Valid GPG signature for apache-tomcat-10.1.39-src.tar.gz * * Binary Zip and tarball: Same * Source Zip and tarball: Same * * Building dependencies returned: 0 * Tomcat builds cleanly * tcnative builds cleanly * Junit Tests: PASSED - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 69607] MD5 algorithm insecure usage in tomcat-util
https://bz.apache.org/bugzilla/show_bug.cgi?id=69607 --- Comment #4 from Christopher Schultz --- It's not as simple as dropping that call. If you drop the call, MD5 cannot be used at all, even when the environment (later) requests it. If you want to provide a patch, please ensure that init(MD5) is called at some point if any part of the environment requires it (e.g. because HTTP-Digest has been requested, and the algorithm is MD5). -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1924217 - in /tomcat/site/trunk: ./ docs/ docs/tomcat-10.1-doc/ docs/tomcat-10.1-doc/annotationapi/ docs/tomcat-10.1-doc/annotationapi/jakarta/annotation/ docs/tomcat-10.1-doc/annotationa
Author: schultz Date: Fri Mar 7 20:59:19 2025 New Revision: 1924217 URL: http://svn.apache.org/viewvc?rev=1924217&view=rev Log: Announce Tomcat v10.1.39 [This commit notification would consist of 52 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r75358 - /release/tomcat/tomcat-10/v10.1.36/
Author: schultz Date: Fri Mar 7 21:00:01 2025 New Revision: 75358 Log: Drop old release artifacts Removed: release/tomcat/tomcat-10/v10.1.36/ - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org