Re: Axis Fault, Xerces sees the webapp as stopped although it is running
Hi Simon, On 22.04.2025 17:44, Simon Arame wrote: org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [org.apache.xerces.jaxp.datatype.XMLGregorianCalendarImpl$DaysInMonth]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. This is a known issue with JAXB: JAXB was part of the JRE until version 11. The default `j.x.bind.DatatypeConverter` implementation uses the thread context classloader to find an implementation of `j.x.datatype.DatatypeFactory` and binds it to a static field. Since Xerces provides this service, the result is a nice memory leak. To solve this, add: javax.xml.datatype.DatatypeFactory = com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl to your `catalina.properties` file. Piotr - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Axis Fault, Xerces sees the webapp as stopped although it is running
Hi, On 25.04.2025 15:57, Piotr P. Karwasz wrote: To solve this, add: javax.xml.datatype.DatatypeFactory = com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl A more proper solution could be to initialize `j.x.b.DatatypeConverterImpl` using the `JreMemoryLeakPreventionListener`: Piotr - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Axis Fault, Xerces sees the webapp as stopped although it is running
There is a lot of information here. Responses in-line. On 24/04/2025 21:51, Simon Arame wrote: Not sure I am interpreting the doc correctly, does this mean that the concerned classes of the xercesImpl jar in /WEB-INF/lib will be ignored when there exists the equivalent in the bootstrap class loader and/or the parent one if any ? The default search order is: - JRE - WEB-INF/classes - WEB-INF/lib - $CATALINA_BASE/lib The first class found with a matching name will be loaded. 1) GC Root from one of our class that has a static method that calls SSLContext.setDefault() with an ssl context declared inside that static method as parameter. 2) GC Root from a sun.awt.AppContext. We are not using AppContext directly so I blocked there. Does that finding of 2 ParallelWebappClassLoader was the right thing to find ? That seems reasonable at this point. Moreover, I've found that there is one of the existing org.apache.jasper.servlet.JasperLoader that contains the compiled JSP class that is faulty, could that be related somehow? Probably not. I've changed the SSLContext.setDefault call from one of our class so it's not dangling anymore Great. It isn't surprising that there are multiple memory leaks. As you fix some of them, others may appear. Are you using org.apache.catalina.core.JreMemoryLeakPreventionListener ? It is enabled by default and automatically fixes a bunch of known issues with JREs and memory leaks. I ask as sun.awt.AppContext is one of the leaks it can protect against but you have to enable it specifically with appContextProtection="true" and run the steps one more time, stopping all contexts this time except tomcat's manager and opening the HPROF dump found: 1) XMLGregorianCalendarImpl in classes of org.apache.catalina.loader.ParallelWebappClassLoader [Held by JVM] 2) XMLGregorianCalendarImpl from constant pool of org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl which is the datatypeFactory of com.sun.xml.bind.DatatypeConverterImpl which is the theConverter of javax.xml.bind.DatatypeConverter which itself is a GC Root. That is the leak. You have an instance of web application provided class set as a static field in a JVM provided class. And there is no public API provided by the JVM to enable you to clear that field. Could I for instance add a Class.forName(javax.xml.bind.DatatypeConverter) in my JSP or perhaps try to explicitly load that class differently? At this point, I do not know what to do next! A few options for you to try/consider 1. If you run on a newer version of Java (11, 17, 21, etc), the JRE no longer provides the javax.xml.bind package so that might resolve the issue provided that the libraries packaged the web application do include those classes. If not, you'll need to add the JAXB API JAR (and possibly others). If you try this option, I'd suggest using Java 11 since using later versions may create other issues due to deprecations and removals. 2. Move (don't copy) the Xerces JARs from WEB-INF/lib to $CATALINA_BASE/lib - you'll need to do this for all of your web applications. 3. Write a ServletContextListener that uses reflection to clear javax.xml.bind.DatatypeConverter.theConverter on web application stop. HTH, Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Help with Cluster Setup on Tomcat 9
Zoran,
On 4/24/25 4:21 AM, Zoran Avtarovski wrote:
Thanks Chuck,
I missed that and will implement. Bigger problem is that
PersistentManager is incompatible with Clusters so we have to migrate
back to memory based sessions and use clusters to share sessions.
I would have thought that cluster would be easier to support with DB
based sessions. Does anyone know why they aren't supported?
What is the primary problem you are trying to solve? Are you trying to
solve the problem of mixed-up sessions (which sounds very surprising to
me that it happens, with the session identifier space being so big) or
do you want to move to clustering anyway? It seems like you've decided
that clustering is the solution to your problem and so now you have a
new problem: how to implement clustering.
You can use both persistent sessions AND clustering if you want. It's
how your sessions can survive a situation where all members of the
cluster are stopped.
But it would be helpful if you would tell us your ideal situation and we
can help you get there.
-chris
On 24/4/2025 11:20 am, Chuck Caldarale wrote:
On 2025 Apr 23, at 20:02, Zoran Avtarovski
wrote:
We have a cluster of tomcat servers on AWS EC2 which operate behind
an AWS load balancer with sticky sessions.
We have our session storage on a DB using a JDBC store which for the
most part is working well, but we occasionally see duplicate session
ids which create issues where a new session with a duplicate session
id hijacks an existing session.
As you can imagine we would like to prevent this from occurring and
have been looking into the issue. It looks like using the cluster tag
might be the solution but I wanted to tap into the collective wisdom
of the group on the best way forward. We can't just add the
| |
tag as it uses ip multicast which doesn't work in EC2 as there is no
physical broadcast layer. The alternative appears to be to use the
StaticMemebrshipService and I wanted to confirm if my config ideas
are correct? I have the following setup:
className="org.apache.catalina.tribes.membership.StaticMembershipService">
className="org.apache.catalina.tribes.membership.StaticMember"
host="10.0.1.11" port="4004"
uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"/>
className="org.apache.catalina.tribes.membership.StaticMember"
host="10.0.1.12" port="4004"
uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,1}"/>
Does this look right to others and do I need a separate Member tag
for each member of the cluster?
I'd appreciate any assistance on this and other suggestions you guys
may have.
Have you specified a unique jvmRoute attribute in the element
of each Tomcat server?
https://tomcat.apache.org/tomcat-9.0-doc/config/
engine.html#Common_Attributes
- Chuck
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Axis Fault, Xerces sees the webapp as stopped although it is running
Mark, Piotr, thanks a lot for your answers. I tested both fixes proposed by Piotr: 1) adding to catalina.properties works. 2) adding to JreMemoryLeakPreventionListener.classesToInitialize did not work. Simon On Fri, Apr 25, 2025 at 10:12 AM Piotr P. Karwasz wrote: > Hi, > > On 25.04.2025 15:57, Piotr P. Karwasz wrote: > > To solve this, add: > > javax.xml.datatype.DatatypeFactory = > > com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl > > A more proper solution could be to initialize > `j.x.b.DatatypeConverterImpl` using the `JreMemoryLeakPreventionListener`: > > className="org.apache.catalina.core.JreMemoryLeakPreventionListener" >classesToInitialize="javax.xml.bind.DatatypeConverterImpl"/> > > Piotr > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: tomcat 11 tomcat-util.jar is contains a soon to be removed class
Rick, On 4/24/25 10:47 AM, Rick Noel wrote: Hello, When I upgraded to tomcat 11.0.5 and Java 24, Tomcat log gives this warning.. WARNING: A terminally deprecated method in sun.misc.Unsafe has been called WARNING: sun.misc.Unsafe::invokeCleaner has been called by org.apache.tomcat.util.buf.ByteBufferUtilsUnsafe (file:/home/web/servers/apache-tomcat-11.0.5/lib/tomcat-util.jar) WARNING: Please consider reporting this to the maintainers of class org.apache.tomcat.util.buf.ByteBufferUtilsUnsafe WARNING: sun.misc.Unsafe::invokeCleaner will be removed in a future release Thanks for the report. The code is already sensitive to the availability of the Unsafe.invokeCleaner method and will continue to work with a JVM where it has been removed. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
