Question Regarding Incorrect Cache Size Calculation Issue
Hello, I am using Tomcat 9.0.78 on a machine running Windows Server 2019 Standard. I am experiencing an issue with an instance of `org.apache.catalina.webresources.Cache`, where `Cache.size` is out of sync with the actual size of `Cache.resourceCache` in memory. Based on a heap dump of the server, `Cache.size` has a value of 700,932,119 while the actual size of the `Cache` instance is merely 4.45 KB. This issue causes unnecessary and misleading logs to be outputted such as the one below, where 684,534 KB is from the value of `Cache.size` which is corrupt. > 26-Mar-2025 11:50:35.267 INFO [Catalina-utility-1] > org.apache.catalina.webresources.Cache.backgroundProcess The background cache > eviction process was unable to free [10] percent of the cache for Context > [/...] - consider increasing the maximum size of the cache. After eviction > approximately [684,534] KB of data remained in the cache. `cacheMaxSize` in `context.xml` is not explicitly set, so it defaults to 10MB. I believe commit `636017459a88befe1c5f1fd9d8f31ff2f13f74f6` by Mark Thomas should fix this issue, but I wanted to ask for opinions here just to be sure. Is it safe to assume that upgrading Tomcat to a version that includes above commit will fix this issue? Regards, Yuta - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Question Regarding Incorrect Cache Size Calculation Issue
> On 2025 Mar 27, at 16:06, Yuta Akiya > wrote: > > I am using Tomcat 9.0.78 on a machine running Windows Server 2019 Standard. > > I am experiencing an issue with an instance of > `org.apache.catalina.webresources.Cache`, where `Cache.size` is out of sync > with the actual size of `Cache.resourceCache` in memory. > Based on a heap dump of the server, `Cache.size` has a value of 700,932,119 > while the actual size of the `Cache` instance is merely 4.45 KB. > This issue causes unnecessary and misleading logs to be outputted such as the > one below, where 684,534 KB is from the value of `Cache.size` which is > corrupt. > >> 26-Mar-2025 11:50:35.267 INFO [Catalina-utility-1] >> org.apache.catalina.webresources.Cache.backgroundProcess The background >> cache eviction process was unable to free [10] percent of the cache for >> Context [/...] - consider increasing the maximum size of the cache. After >> eviction approximately [684,534] KB of data remained in the cache. > > `cacheMaxSize` in `context.xml` is not explicitly set, so it defaults to 10MB. > > I believe commit `636017459a88befe1c5f1fd9d8f31ff2f13f74f6` by Mark Thomas > should fix this issue, but I wanted to ask for opinions here just to be sure. > Is it safe to assume that upgrading Tomcat to a version that includes above > commit will fix this issue? That (and a couple of subsequent changes) should fix the problem. Not updating Tomcat in over a year and a half is rather risky by itself, so you really should upgrade to the current version regardless. - Chuck - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Additional Property File For Substitution Variables
On 27/03/2025 14:54, William Crowell wrote:
Sebastian,
Thanks for your reply. I did know about environment variables. I would be
concerned about someone doing a “ps -ef” on the box and getting the password
from the command line arguments. I will keep looking.
Write a small class that implements IntrospectionUtils.PropertySource
and reads your property file.
Configure as per:
https://tomcat.apache.org/tomcat-11.0-doc/config/systemprops.html#Property_replacements
Might be sufficiently generally useful to add to the other
implementations in org.apache.tomcat.util.digester
Note that catalina.properties is read and the properties are added as
system properties. The SystemPropertySource then finds them for the
Digester.
Mark
Regards,
William Crowell
From: Sebastian Trost
Date: Thursday, March 27, 2025 at 9:53 AM
To: users@tomcat.apache.org
Subject: Re: Additional Property File For Substitution Variables
William,
On 27.03.2025 14:39, William Crowell wrote:
Is there a way to add an additional properties file to the startup of Apache
Tomcat, so that server.xml and context.xml can reference the values like we do
with catalina.properties?
For instance:
application.properties:
my.user=foo
my.password=bar
context.xml:
…
You can use environment variables (and setenv.sh/.bat). Example:
Set your environment variable:
> export USER_NAME=user1
Modify / create setenv.sh:
JAVA_OPTS=-Dcustom_user_name=${USER_NAME}
Modify your context.xml:
[...] https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F44761831%2Ftomcat-8-context-xml-use-environment-variable-in-datasource&data=05%7C02%7CWCrowell%40perforce.com%7Ce2e937739a6740052d3608dd6d36b2e1%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C638786803907292634%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WSI4V4Kpep8wXy295CZ%2BeNseW7a1Scd9%2B5dc3YYKf9k%3D&reserved=0
Sebastian
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
CAUTION: This email originated from outside of the organization. Do not click
on links or open attachments unless you recognize the sender and know the
content is safe.
This e-mail may contain information that is privileged or confidential. If you
are not the intended recipient, please delete the e-mail and any attachments
and notify us immediately.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Additional Property File For Substitution Variables
Sebastian,
Thanks for your reply. I did know about environment variables. I would be
concerned about someone doing a “ps -ef” on the box and getting the password
from the command line arguments. I will keep looking.
Regards,
William Crowell
From: Sebastian Trost
Date: Thursday, March 27, 2025 at 9:53 AM
To: users@tomcat.apache.org
Subject: Re: Additional Property File For Substitution Variables
William,
On 27.03.2025 14:39, William Crowell wrote:
> Is there a way to add an additional properties file to the startup of Apache
> Tomcat, so that server.xml and context.xml can reference the values like we
> do with catalina.properties?
>
> For instance:
>
> application.properties:
>
> my.user=foo
> my.password=bar
>
> context.xml:
>
> …
> password="${my.password}” …
> />
> …
> [...]
You can use environment variables (and setenv.sh/.bat). Example:
Set your environment variable:
> export USER_NAME=user1
Modify / create setenv.sh:
JAVA_OPTS=-Dcustom_user_name=${USER_NAME}
Modify your context.xml:
[...] https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F44761831%2Ftomcat-8-context-xml-use-environment-variable-in-datasource&data=05%7C02%7CWCrowell%40perforce.com%7Ce2e937739a6740052d3608dd6d36b2e1%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C638786803907292634%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WSI4V4Kpep8wXy295CZ%2BeNseW7a1Scd9%2B5dc3YYKf9k%3D&reserved=0
Sebastian
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
CAUTION: This email originated from outside of the organization. Do not click
on links or open attachments unless you recognize the sender and know the
content is safe.
This e-mail may contain information that is privileged or confidential. If you
are not the intended recipient, please delete the e-mail and any attachments
and notify us immediately.
Re: Monitoring Virtual Threads via JMX / MBeans in Tomcat
On 26/03/2025 10:38, Rose Mary P T wrote: Dear Tomcat Users, I hope this message finds you well. As per your previous email, we attempted to fetch the virtual thread count from the keepAliveCount attribute in the Catalina.ThreadPool MBean. For context, here is the setup we used: * We created a sample Spring Boot application that continuously creates virtual threads in a loop. If the application is creating the threads then this won't work. The (connectionCount - keepAliveCount) approach only works for virtual threads created by Tomcat for processing requests. If the application is creating the virtual threads then I'd suggest adding tracking for the current number of virtual threads to the application. Mark * The application was deployed in the TOMCAT_LOCATION/webapps directory and started on localhost. In Apache Tomcat 10.1.36, we added the following configuration to the server.xml file to enable virtual threads: Its observed in the logs that virtual threads were being created as expected. However, we noticed that the keepAliveCount attribute in the Catalina.ThreadPool MBean is showing a value of 0, even though virtual threads are being spawned. It seems that the keepAliveCount attribute does not provide a valid value for counting the virtual threads. We were wondering if this is expected behavior, or if there is a different way to monitor the virtual threads created in Tomcat. We would also like to know if there's a way to differentiate between platform threads and virtual threads using any MBean attribute in the Catalina service or elsewhere in Tomcat's MBean architecture. We would greatly appreciate any guidance or insights you can provide regarding this issue. Best Regards, Rose Mary From: Joash Jose Date: Wednesday, 26 March 2025 at 12:48 PM To: Rose Mary P T Subject: Begin forwarded message: From: Mark Thomas Subject: [EXTERNAL] Re: Monitoring Virtual Threads via JMX / MBeans in Tomcat Date: 6 March 2025 at 2:08:43 PM IST To: Reply-To: "Tomcat Users List" On 06/03/2025 06:29, Joash Jose wrote: Dear Apache Tomcat Support Team, I hope this message finds you well. I am writing to inquire whether Apache Tomcat (tomacat version is 10.1.33 running on Java 21) exposes virtual thread metrics through JMX / MBeans. Specifically: Virtual Thread Visibility: Does Tomcat provide MBeans (e.g., under Catalina:type=Executor) to monitor virtual thread usage, such as active virtual thread counts, creation rate, or parking states? No with a few caveats - see below). If not, are there plans to add such metrics in future releases? No. Or any alternate ways with which we can monitor this from tomcat side.? (connectionCount - keepAliveCount) should be a reasonable estimate of the virtual threads currently being used. If the requests are synchronous (and there is little point using virtual threads with async requests) then the creation rate is derivable from the requestCount in the GlobalRequestProcessor. There is no information on parking states. Configuration Clarification: We have configured Tomcat 10.1.33 with, The element isn't being used. Why configure it? While this works, the existing ThreadPool MBeans show currentThreadsBusy=-1 and maxThreads=200 (incorrect in case of virtual threads). Is this expected behavior? Yes. JVM vs. Tomcat Metrics: We observe that the JVM’s java.lang.management.ThreadMXBean includes virtual threads, but this aggregates data across all applications on the JVM. Does Tomcat offer a way to isolate virtual thread metrics specific to Tomcat? No, because the JVM doesn't provide a mechanism to have multiple pools/groups/anything of virtual threads. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Begin forwarded message: From: Joash Jose Subject: Monitoring Virtual Threads via JMX / MBeans in Tomcat Date: 6 March 2025 at 11:59:29 AM IST To: , Dear Apache Tomcat Support Team, I hope this message finds you well. I am writing to inquire whether Apache Tomcat (tomacat version is 10.1.33 running on Java 21) exposes virtual thread metrics through JMX / MBeans. Specifically: Virtual Thread Visibility: Does Tomcat provide MBeans (e.g., under Catalina:type=Executor) to monitor virtual thread usage, such as active virtual thread counts, creation rate, or parking states? If not, are there plans to add such metrics in future releases? Or any alternate ways with which we can monitor this from tomcat side.? Configuration Clarification: We have configured Tomcat 10.1.33 with, While this works, the existing ThreadPool MBeans show currentThreadsBusy=-1 and maxThreads=200 (incorrect in case of virtual threads). Is this expected behavior? JVM vs. Tomcat Metrics: We observe that the JVM’s java.lang.management.ThreadMXBean includes virtual threads, but this aggre
Additional Property File For Substitution Variables
Is there a way to add an additional properties file to the startup of Apache Tomcat, so that server.xml and context.xml can reference the values like we do with catalina.properties? For instance: application.properties: my.user=foo my.password=bar context.xml: …
Re: Additional Property File For Substitution Variables
William,
On 27.03.2025 14:39, William Crowell wrote:
Is there a way to add an additional properties file to the startup of Apache
Tomcat, so that server.xml and context.xml can reference the values like we do
with catalina.properties?
For instance:
application.properties:
my.user=foo
my.password=bar
context.xml:
…
You can use environment variables (and setenv.sh/.bat). Example:
Set your environment variable:
> export USER_NAME=user1
Modify / create setenv.sh:
JAVA_OPTS=-Dcustom_user_name=${USER_NAME}
Modify your context.xml:
[...] username="${custom_user_name}" [...]
I didn't test it, but it should work like this.
Reference:
https://stackoverflow.com/questions/44761831/tomcat-8-context-xml-use-environment-variable-in-datasource
Sebastian
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
