https://bz.apache.org/bugzilla/show_bug.cgi?id=58586
Bug ID: 58586
Summary: Classloader memory leak on Tomcat application .war
file redeployment.
Product: Tomcat Modules
Version: unspecified
Hardware: All
OS: AIX
Status: NEW
Severity: major
Priority: P2
Component: jdbc-pool
Assignee: dev@tomcat.apache.org
Reporter: bht...@gmail.com
The leak can be observed with the use of the latest version of an IBM jdbc
driver.
IBM support have worked on the issue for us. Their resolution is to deregister
the JDBC driver with a ServletContextListener.
Additionally they recommend to include the driver jar db2jcc4.jar in the web
archive WEB-INF/lib directory.
Unfortunately while doing this as recommended, the exception below still
appears once per second in the log file:
java.lang.IllegalStateException: Illegal access: this web application instance
has been stopped already. Could not load [DB2JccConfiguration.properties]. The
following stack trace is thrown for debugging purposes as well as to attempt to
terminate the thread which caused the illegal access.
at
org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1327)
at
org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1023)
at com.ibm.db2.jcc.am.ud.run(Unknown Source)
at
java.security.AccessController.doPrivileged(AccessController.java:285)
at com.ibm.db2.jcc.am.GlobalProperties.a(Unknown Source)
at com.ibm.db2.jcc.am.GlobalProperties.d(Unknown Source)
at com.ibm.db2.jcc.am.mq.run(Unknown Source)
at java.util.TimerThread.mainLoop(Timer.java:567)
at java.util.TimerThread.run(Timer.java:517)
Environment information is as follows:
Tomcat version: 8.0
Operating system: AIX
java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build pap6470sr7-20140410_01(SR7))
IBM J9 VM (build 2.6, JRE 1.7.0 AIX ppc64-64 Compressed References
20140409_195732 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR7_20140409_1418_B195732
JIT - r11.b06_20140409_61252
GC - R26_Java726_SR7_20140409_1418_B195732_CMPRSS
J9CL - 20140409_195732)
JCL - 20140409_01 based on Oracle 7u55-b13
JDBC driver downloaded from:
http://www-01.ibm.com/support/docview.wss?uid=swg21363866
DB2 JDBC and SQLJ driver 10.5.0
v10.5fp5_jdbc_sqlj.tar.gz
Configuration is as follows:
A data source is defined in the Tomcat application context:
$CATALINA_BASE/conf/Catalina/localhost/db2test.xml as follows:
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Resource name="jdbc/db2test"
auth="Container"
type="javax.sql.DataSource"
username="username"
password="password"
driverClassName="com.ibm.db2.jcc.DB2Driver"
url="jdbc:db2://server:port/databaseName:currentSchema=schemaName;"
maxActive="8"
/>
</Context>
web.xml as follows:
<?xml version="1.0" encoding="UTF-8"?>
<web-app version="3.0" xmlns="http://java.sun.com/xml/ns/javaee";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd";>
<display-name>Leaky Application</display-name>
<session-config>
<session-timeout>
30
</session-timeout>
</session-config>
<listener>
<listener-class>
connect.listener.DatabaseGetConnectionContextListener
</listener-class>
</listener>
<listener>
<listener-class>
connect.listener.DeregisterDriverServletContextListener
</listener-class>
</listener>
</web-app>
with listeners as follows:
public class DatabaseGetConnectionContextListener implements
ServletContextListener {
private static final Logger LOGGER =
LoggerFactory.getLogger(DatabaseGetConnectionContextListener.class);
public void contextInitialized(ServletContextEvent sce) {
LOGGER.info("contextInitialized trying to get connection...");
try {
Context initCtx = new InitialContext();
Context envCtx = (Context) initCtx.lookup("java:comp/env");
DataSource ds = (DataSource) envCtx.lookup("jdbc/db2test");
Connection con = null;
try {
con = ds.getConnection();
LOGGER.info("Opened connection: " + con);
} catch (SQLException ex) {
LOGGER.error("Cannot open connection", ex);
} finally {
if (con != null) {
try {
con.close();
LOGGER.info("Closed connection: " + con);
} catch (SQLException ex) {
throw new RuntimeException(ex);
}
}
}
} catch (NamingException ex) {
throw new RuntimeException(ex);
}
}
public void contextDestroyed(ServletContextEvent sce) {}
}
public class DeregisterDriverServletContextListener implements
ServletContextListener {
private static final Logger LOGGER =
LoggerFactory.getLogger(DeregisterDriverServletContextListener.class);
public void contextInitialized(ServletContextEvent sce) {}
@Override
public void contextDestroyed(ServletContextEvent sce) {
LOGGER.info("contextDestroyed() executing ...");
final ClassLoader classLoader =
Thread.currentThread().getContextClassLoader();
// Loop through all drivers
final Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
final Driver driver = drivers.nextElement();
if (driver.getClass().getClassLoader() == classLoader) {
// This driver was registered by the webapp's ClassLoader, so
// deregister it:
try {
LOGGER.info("Deregistering JDBC driver { " + driver + "
}");
DriverManager.deregisterDriver(driver);
} catch (SQLException ex) {
LOGGER.error("Error deregistering JDBC driver { " + driver
+ " }", ex);
}
} else {
// driver was not registered by the webapp's ClassLoader and
may
// be in use elsewhere
LOGGER.error("Not deregistering JDBC driver {} as it does not
belong to this webapp's ClassLoader { " + driver + " }");
}
}
}
db2jcc4.jar is in the WEB-INF/lib directory.
When the war file is re-deployed e.g. vie UNIX touch command, the following log
entries are printed in catalina.out:
INFO: Reloading Context with name [/db2test] has started
Nov 04, 2015 3:21:57 PM org.apache.catalina.loader.WebappClassLoaderBase
clearReferencesThreads
WARNING: The web application [db2test] appears to have started a thread named
[Timer-0] but has failed to stop it. This is very likely to create a memory
leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:196)
java.util.TimerThread.mainLoop(Timer.java:564)
java.util.TimerThread.run(Timer.java:517)
Nov 04, 2015 3:21:59 PM org.apache.catalina.core.StandardContext reload
INFO: Reloading Context with name [/db2test] is completed
Nov 04, 2015 3:22:44 PM org.apache.catalina.loader.WebappClassLoaderBase
checkStateForResourceLoading
INFO: Illegal access: this web application instance has been stopped already.
Could not load [DB2JccConfiguration.properties]. The following stack trace is
thrown for debugging purposes as well as to attempt
to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance
has been stopped already. Could not load [DB2JccConfiguration.properties]. The
following stack trace is thrown for debugging pur
poses as well as to attempt to terminate the thread which caused the illegal
access.
at
org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1327)
at
org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1023)
at com.ibm.db2.jcc.am.ud.run(Unknown Source)
at
java.security.AccessController.doPrivileged(AccessController.java:285)
at com.ibm.db2.jcc.am.GlobalProperties.a(Unknown Source)
at com.ibm.db2.jcc.am.GlobalProperties.d(Unknown Source)
at com.ibm.db2.jcc.am.mq.run(Unknown Source)
at java.util.TimerThread.mainLoop(Timer.java:567)
at java.util.TimerThread.run(Timer.java:517)
java.lang.IllegalStateException: Illegal access: this web application instance
has been stopped already. Could not load [DB2JccConfiguration.properties]. The
following stack trace is thrown for debugging pur
poses as well as to attempt to terminate the thread which caused the illegal
access.
at
org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1327)
at
org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1023)
at com.ibm.db2.jcc.am.ud.run(Unknown Source)
at
java.security.AccessController.doPrivileged(AccessController.java:285)
at com.ibm.db2.jcc.am.GlobalProperties.a(Unknown Source)
at com.ibm.db2.jcc.am.GlobalProperties.d(Unknown Source)
at com.ibm.db2.jcc.am.mq.run(Unknown Source)
at java.util.TimerThread.mainLoop(Timer.java:567)
at java.util.TimerThread.run(Timer.java:517)
Nov 04, 2015 3:23:44 PM org.apache.catalina.loader.WebappClassLoaderBase
checkStateForResourceLoading
INFO: Illegal access: this web application instance has been stopped already.
Could not load [DB2JccConfiguration.properties]. The following stack trace is
thrown for debugging purposes as well as to attempt
to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance
has been stopped already. Could not load [DB2JccConfiguration.properties]. The
following stack trace is thrown for debugging pur
poses as well as to attempt to terminate the thread which caused the illegal
access.
at
org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1327)
at
org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1023)
at com.ibm.db2.jcc.am.ud.run(Unknown Source)
at
java.security.AccessController.doPrivileged(AccessController.java:285)
at com.ibm.db2.jcc.am.GlobalProperties.a(Unknown Source)
at com.ibm.db2.jcc.am.GlobalProperties.d(Unknown Source)
at com.ibm.db2.jcc.am.mq.run(Unknown Source)
at java.util.TimerThread.mainLoop(Timer.java:567)
at java.util.TimerThread.run(Timer.java:517)
....
More of the same repeats every minute.
We have tried an alternate configuration method as follows:
/conf/context.xml file as follows:
<Context>
...
<Resource auth="Container"
driverClassName="com.ibm.db2.jcc.DB2Driver"
name="jdbc/..."
password="..."
username="..."
type="javax.sql.DataSource"
url="jdbc:db2://server:.../.../>
...
</Context>
and a resource link via db2test.xml
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<ResourceLink name="jdbc/db2test" global="jdbc/db2test"
type="javax.sql.DataSource"/>
</Context>
The results are the same.
Again, alternatively, the resource can be defined in server.xml, and a
<ResourceLink, again with no changes in web.xml
We have NOT tried this as we do not prefer this solution.
We used documentation at
https://tomcat.apache.org/tomcat-8.0-doc/jndi-resources-howto.html
https://www-01.ibm.com/support/knowledgecenter/SSZH4A_5.0.5/com.ibm.worklight.help.doc/admin/t_configuring_apache_tomcat_for_db2_manually.html
https://numberformat.wordpress.com/2009/09/07/db2-datasource-configuration-in-tomcat-6/
In addition to what we see, the proposed workaround (which I cannot get to
work), from my current perspective, has 2 issues:
a) No other application can use the same data source because it is local
b) Including a specific driver file in the application is not consistent with
the concept of a data source which from the application perspective, should not
be driver specific.
The attached file contains as a fully functional test case a minimal maven
based web project where a ServletContextListener connects to the data source on
start up and deregisters the driver on un-deployment.
Configuration files are also included, please refer to configuration
directories.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
