Re: [tomcat] branch main updated: Initialize Random during server initialization
On Thu, Apr 6, 2023 at 10:47 PM Christopher Schultz wrote: > > Rémy, > > On 4/6/23 10:11, r...@apache.org wrote: > > This is an automated email from the ASF dual-hosted git repository. > > > > remm pushed a commit to branch main > > in repository https://gitbox.apache.org/repos/asf/tomcat.git > > > > > > The following commit(s) were added to refs/heads/main by this push: > > new 0c0db9f9de Initialize Random during server initialization > > 0c0db9f9de is described below > > > > commit 0c0db9f9dea9630a41ec289576fbdddc975d2291 > > Author: remm > > AuthorDate: Thu Apr 6 16:11:09 2023 +0200 > > > > Initialize Random during server initialization > > > > BZ66554, causing possible thread creation by the JVM using the context > > of the webapp. > > --- > > .../core/JreMemoryLeakPreventionListener.java | 24 > > ++ > > webapps/docs/changelog.xml | 5 + > > webapps/docs/config/listeners.xml | 10 + > > 3 files changed, 39 insertions(+) > > > > diff --git > > a/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java > > b/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java > > index df675f6b11..babf34ad90 100644 > > --- a/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java > > +++ b/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java > > @@ -17,6 +17,7 @@ > > package org.apache.catalina.core; > > > > import java.net.URLConnection; > > +import java.security.SecureRandom; > > import java.sql.DriverManager; > > import java.util.StringTokenizer; > > > > @@ -106,6 +107,20 @@ public class JreMemoryLeakPreventionListener > > implements LifecycleListener { > > this.classesToInitialize = classesToInitialize; > > } > > > > +/** > > + * Initialize JVM seed generator. On some platforms, the JVM will > > create a thread for this task, which can get > > + * associated with a web application depending on the timing. > > + */ > > +private boolean initSeedGenerator = false; > > + > > +public boolean getInitSeedGenerator() { > > +return this.initSeedGenerator; > > +} > > + > > +public void setInitSeedGenerator(boolean initSeedGenerator) { > > +this.initSeedGenerator = initSeedGenerator; > > +} > > + > > > > @Override > > public void lifecycleEvent(LifecycleEvent event) { > > @@ -170,6 +185,15 @@ public class JreMemoryLeakPreventionListener > > implements LifecycleListener { > > URLConnection.setDefaultUseCaches("JAR", false); > > } > > > > +/* > > + * Initialize the SeedGenerator of the JVM, as some > > platforms use > > + * a thread which could end up being associated with a > > webapp rather > > + * than the container. > > + */ > > +if (initSeedGenerator) { > > +SecureRandom.getSeed(1); > > +} > > What about the various kinds of SecureRandom that you can get these days: > > SecureRandom.getInstance(String) > SecureRandom.getInstanceStrong() > > I'll be there is still a way to get the webapp's ClassLoader pinned, but > maybe this takes care of the most common situations. The idea is to use some seed generated by sun.security.provider.SeedGenerator. On some platforms this class creates a thread on demand. Rémy - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] rmaucher commented on a diff in pull request #607: Added RateLimitFilter
rmaucher commented on code in PR #607: URL: https://github.com/apache/tomcat/pull/607#discussion_r1160568819 ## java/org/apache/catalina/util/TimeBucketCounter.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.catalina.util; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * this class maintains a thread safe hash map that has timestamp-based buckets + * followed by a string for a key, and a counter for a value. each time the + * increment() method is called it adds the key if it does not exist, increments + * its value and returns it. + * + * a maintenance thread cleans up keys that are prefixed by previous timestamp + * buckets. + */ +public class TimeBucketCounter { + +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. + * 16 for 65_536ms which is about 1:05 minute + */ +private final int numBits; + +/** + * ratio of actual duration to config duration + */ +private final double ratio; + +/** + * flag for the maintenance thread + */ +volatile boolean isRunning = false; + +/** + * + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + */ +public TimeBucketCounter(int bucketDuration) { + +int durationMillis = bucketDuration * 1000; + +int bits = 0; +int pof2 = nextPowerOf2(durationMillis); +int bitCheck = pof2; +while (bitCheck > 1) { +bitCheck = pof2 >> ++bits; +} + +this.numBits = bits; + +this.ratio = ratioToPowerOf2(durationMillis); + +int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; +Thread mt = new MaintenanceThread(durationMillis / cleanupsPerBucketDuration); Review Comment: Using the background events from Tomcat would be nice, but this is not a valve. A regular filter in a regular webapp cannot use it. In EE there is https://jakarta.ee/specifications/concurrency/3.0/ but Tomcat does not provide it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] rmannibucau commented on a diff in pull request #607: Added RateLimitFilter
rmannibucau commented on code in PR #607: URL: https://github.com/apache/tomcat/pull/607#discussion_r1160576139 ## java/org/apache/catalina/util/TimeBucketCounter.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.catalina.util; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * this class maintains a thread safe hash map that has timestamp-based buckets + * followed by a string for a key, and a counter for a value. each time the + * increment() method is called it adds the key if it does not exist, increments + * its value and returns it. + * + * a maintenance thread cleans up keys that are prefixed by previous timestamp + * buckets. + */ +public class TimeBucketCounter { + +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. + * 16 for 65_536ms which is about 1:05 minute + */ +private final int numBits; + +/** + * ratio of actual duration to config duration + */ +private final double ratio; + +/** + * flag for the maintenance thread + */ +volatile boolean isRunning = false; + +/** + * + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + */ +public TimeBucketCounter(int bucketDuration) { + +int durationMillis = bucketDuration * 1000; + +int bits = 0; +int pof2 = nextPowerOf2(durationMillis); +int bitCheck = pof2; +while (bitCheck > 1) { +bitCheck = pof2 >> ++bits; +} + +this.numBits = bits; + +this.ratio = ratioToPowerOf2(durationMillis); + +int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; +Thread mt = new MaintenanceThread(durationMillis / cleanupsPerBucketDuration); Review Comment: https://jakarta.ee/specifications/concurrency/3.0/ does not bring anything to java se except some useless listeners for that case so it is fine to stay away from it but goal is to reuse some banalised pool from tomcat instead of creating multiple leaky active threads. I'm not sure why this filter wouldn't use it since the filter is an internal of tomcat as the background pool, in terms of classloading there is no strong blocker - and worse case you can still get the scheduled executor service from a servlet context attribute if so which would enable to keep the filter standard and enable tomcat to inject the background pool there if the filter is present but dont think it is needed - and the filter will not be reused in another server so guess it should be fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] rmaucher commented on a diff in pull request #607: Added RateLimitFilter
rmaucher commented on code in PR #607: URL: https://github.com/apache/tomcat/pull/607#discussion_r1160592553 ## java/org/apache/catalina/util/TimeBucketCounter.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.catalina.util; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * this class maintains a thread safe hash map that has timestamp-based buckets + * followed by a string for a key, and a counter for a value. each time the + * increment() method is called it adds the key if it does not exist, increments + * its value and returns it. + * + * a maintenance thread cleans up keys that are prefixed by previous timestamp + * buckets. + */ +public class TimeBucketCounter { + +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. + * 16 for 65_536ms which is about 1:05 minute + */ +private final int numBits; + +/** + * ratio of actual duration to config duration + */ +private final double ratio; + +/** + * flag for the maintenance thread + */ +volatile boolean isRunning = false; + +/** + * + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + */ +public TimeBucketCounter(int bucketDuration) { + +int durationMillis = bucketDuration * 1000; + +int bits = 0; +int pof2 = nextPowerOf2(durationMillis); +int bitCheck = pof2; +while (bitCheck > 1) { +bitCheck = pof2 >> ++bits; +} + +this.numBits = bits; + +this.ratio = ratioToPowerOf2(durationMillis); + +int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; +Thread mt = new MaintenanceThread(durationMillis / cleanupsPerBucketDuration); Review Comment: Sorry for liking that spec ... I like it because it provides a standard way to pass scheduling services to the app. Also I had to reimplement "robust" scheduling (logging and reschedule when something doesn't work). As a result, if given the choice, I would prefer having it than not having it. Right now, there's no way to pass Tomcat's utility executor to the filter, so this PR should keep the dedicated thread. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] rmannibucau commented on a diff in pull request #607: Added RateLimitFilter
rmannibucau commented on code in PR #607: URL: https://github.com/apache/tomcat/pull/607#discussion_r1160616435 ## java/org/apache/catalina/util/TimeBucketCounter.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.catalina.util; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * this class maintains a thread safe hash map that has timestamp-based buckets + * followed by a string for a key, and a counter for a value. each time the + * increment() method is called it adds the key if it does not exist, increments + * its value and returns it. + * + * a maintenance thread cleans up keys that are prefixed by previous timestamp + * buckets. + */ +public class TimeBucketCounter { + +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. + * 16 for 65_536ms which is about 1:05 minute + */ +private final int numBits; + +/** + * ratio of actual duration to config duration + */ +private final double ratio; + +/** + * flag for the maintenance thread + */ +volatile boolean isRunning = false; + +/** + * + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + */ +public TimeBucketCounter(int bucketDuration) { + +int durationMillis = bucketDuration * 1000; + +int bits = 0; +int pof2 = nextPowerOf2(durationMillis); +int bitCheck = pof2; +while (bitCheck > 1) { +bitCheck = pof2 >> ++bits; +} + +this.numBits = bits; + +this.ratio = ratioToPowerOf2(durationMillis); + +int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; +Thread mt = new MaintenanceThread(durationMillis / cleanupsPerBucketDuration); Review Comment: > I like it because it provides a standard way to pass scheduling services to the app. But no way to configure it, to define if it leaks between apps or not (here we want it leaks) and other common pitfalls :(. That said the way to link it to tomcat is JNDI and I guess the background scheduler could be made boundable in JNDI too quite easily so the solution can still converge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] rmaucher commented on a diff in pull request #607: Added RateLimitFilter
rmaucher commented on code in PR #607: URL: https://github.com/apache/tomcat/pull/607#discussion_r1160653462 ## java/org/apache/catalina/util/TimeBucketCounter.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.catalina.util; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * this class maintains a thread safe hash map that has timestamp-based buckets + * followed by a string for a key, and a counter for a value. each time the + * increment() method is called it adds the key if it does not exist, increments + * its value and returns it. + * + * a maintenance thread cleans up keys that are prefixed by previous timestamp + * buckets. + */ +public class TimeBucketCounter { + +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. + * 16 for 65_536ms which is about 1:05 minute + */ +private final int numBits; + +/** + * ratio of actual duration to config duration + */ +private final double ratio; + +/** + * flag for the maintenance thread + */ +volatile boolean isRunning = false; + +/** + * + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + */ +public TimeBucketCounter(int bucketDuration) { + +int durationMillis = bucketDuration * 1000; + +int bits = 0; +int pof2 = nextPowerOf2(durationMillis); +int bitCheck = pof2; +while (bitCheck > 1) { +bitCheck = pof2 >> ++bits; +} + +this.numBits = bits; + +this.ratio = ratioToPowerOf2(durationMillis); + +int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; +Thread mt = new MaintenanceThread(durationMillis / cleanupsPerBucketDuration); Review Comment: Yes, JNDI is supposed to be used, but it's not very good here (users can disable naming, also still need to add an object factory for that, plus configuration). Tomcat sets some custom Servlet context attributes in ServletContext.startInternal, this can be used for some scheduling service. I think using a thread for this filter is "ok" until it is removed once/if the feature is added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] rmannibucau commented on a diff in pull request #607: Added RateLimitFilter
rmannibucau commented on code in PR #607: URL: https://github.com/apache/tomcat/pull/607#discussion_r1160660871 ## java/org/apache/catalina/util/TimeBucketCounter.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.catalina.util; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * this class maintains a thread safe hash map that has timestamp-based buckets + * followed by a string for a key, and a counter for a value. each time the + * increment() method is called it adds the key if it does not exist, increments + * its value and returns it. + * + * a maintenance thread cleans up keys that are prefixed by previous timestamp + * buckets. + */ +public class TimeBucketCounter { + +/** + * Map to hold the buckets + */ +private final ConcurrentHashMap map = new ConcurrentHashMap<>(); + +/** + * Milliseconds bucket size as a Power of 2 for bit shift math, e.g. + * 16 for 65_536ms which is about 1:05 minute + */ +private final int numBits; + +/** + * ratio of actual duration to config duration + */ +private final double ratio; + +/** + * flag for the maintenance thread + */ +volatile boolean isRunning = false; + +/** + * + * @param bucketDuration duration in seconds, e.g. for 1 minute pass 60 + */ +public TimeBucketCounter(int bucketDuration) { + +int durationMillis = bucketDuration * 1000; + +int bits = 0; +int pof2 = nextPowerOf2(durationMillis); +int bitCheck = pof2; +while (bitCheck > 1) { +bitCheck = pof2 >> ++bits; +} + +this.numBits = bits; + +this.ratio = ratioToPowerOf2(durationMillis); + +int cleanupsPerBucketDuration = (durationMillis >= 60_000) ? 6 : 3; +Thread mt = new MaintenanceThread(durationMillis / cleanupsPerBucketDuration); Review Comment: what about this compromise: in init store a scheduledexecutorservice (one thread) in the servletcontext if it does not already exists else reuse, will at least enable to chain filters without recreating it, wdyt? acceptable compromise? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org