Re: Tomcat JDBC Pool memory leak when using StatementFinalizer interceptor
Hi, (moving to developers list) any ideas on the problem below? This thing is kind of itching me :-) So I instrumented the "StatementFinalizer" class with some logging and learned that over time a few instances of the "StatementFinalizer" are created, used and destroyed. So far so good. For most of those instances, the overall number of statements that are added to the "statements" list by "createStatement" and the number of statements removed from the list by "closeInvoked" is identical and after "closeInvoked" finishes, the list is empty. But only for most instances of "StatementFinalizer". I could find that there is one instance that is used (statements are added), but the invocation of "closednvoked" stops after some minutes into the application. As a result the "statements" list starts growing. Rings any bells? Thanks Martin On Wed, Jul 11, 2018 at 4:22 PM, Martin Knoblauch wrote: > Hi, > > while analyzing some heap dump for other reasons, I found that our > application is apparently aggregating a considerable amount of memory in > "org.apache.tomcat.jdbc.pool.TrapException", which is never cleaned by > GC. Digging deeper, it seems that the entries of the "statements" linked > list in the StatementFinalizer are never removed from the list, so after > three weeks of lifetime one ends up with a list of 7 million entries, each > 80 bytes. > > Now it might be, that we are just using the StatementFinalizer in a wrong > manner. And what we see is expected behavior. Below is our pool > configuration. Maybe something is just missing :-) > > We are at Tomcat 8.0.36 (yeah, I know, but that is the version we have to > use) and Java 8 (1.8.0_171). Underlying DB is Oracle 12.1.0.2 and we are > using the latest "ojdbc7.jar" from Oracle. > > > name="jdbc/SimManagerDS" > auth="Container" > > type="javax.sql.DataSource" > description="Oracle datasource for xxx using > tomcat.jdbc.pool" > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" > jmxEnabled="true" > jdbcInterceptors="ConnectionState;StatementFinalizer; > ResetAbandonedTimer;StatementCache(prepared=true,max=200)" > > initialSize="7" > minIdle="7" > maxActive="30" > maxIdle="10" > > testWhileIdle="true" > testOnBorrow="true" > testOnConnect="false" > testOnReturn="false" > validationQuery="SELECT 1 from dual" > validationInterval="3" > > logAbandoned="true" > removeAbandoned="false" > removeAbandonedTimeout="0" > suspectTimeout="600" > > timeBetweenEvictionRunsMillis="3" > minEvictableIdleTimeMillis="6" > maxWait="6" > maxAge="0" > > connectionProperties="(defaultRowPrefetch=200)" > > driverClassName="oracle.jdbc.OracleDriver" > url="jdbc:oracle:thin:@s###" > username="" > password="" >/> > > Thanks > Martin > -- > -- > Martin Knoblauch > email: k n o b i AT knobisoft DOT de > www: http://www.knobisoft.de > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
svn commit: r1835831 - in /tomcat/trunk: java/org/apache/catalina/tribes/group/GroupChannel.java webapps/docs/changelog.xml
Author: remm
Date: Fri Jul 13 12:56:42 2018
New Revision: 1835831
URL: http://svn.apache.org/viewvc?rev=1835831&view=rev
Log:
Propagate heartbeat from channel down to membership service.
Modified:
tomcat/trunk/java/org/apache/catalina/tribes/group/GroupChannel.java
tomcat/trunk/webapps/docs/changelog.xml
Modified: tomcat/trunk/java/org/apache/catalina/tribes/group/GroupChannel.java
URL:
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/tribes/group/GroupChannel.java?rev=1835831&r1=1835830&r2=1835831&view=diff
==
--- tomcat/trunk/java/org/apache/catalina/tribes/group/GroupChannel.java
(original)
+++ tomcat/trunk/java/org/apache/catalina/tribes/group/GroupChannel.java Fri
Jul 13 12:56:42 2018
@@ -187,6 +187,11 @@ public class GroupChannel extends Channe
@Override
public void heartbeat() {
super.heartbeat();
+
+if (coordinator.getMembershipService() instanceof Heartbeat) {
+((Heartbeat) coordinator.getMembershipService()).heartbeat();
+}
+
for (MembershipListener listener : membershipListeners) {
if ( listener instanceof Heartbeat )
((Heartbeat)listener).heartbeat();
}
Modified: tomcat/trunk/webapps/docs/changelog.xml
URL:
http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/changelog.xml?rev=1835831&r1=1835830&r2=1835831&view=diff
==
--- tomcat/trunk/webapps/docs/changelog.xml (original)
+++ tomcat/trunk/webapps/docs/changelog.xml Fri Jul 13 12:56:42 2018
@@ -70,6 +70,9 @@
62527: Revert restriction of JNDI to the java:
namespace. (remm)
+
+Propagate heartbeat from channel down to membership service. (remm)
+
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1835832 - in /tomcat/tc8.5.x/trunk: ./ java/org/apache/catalina/tribes/group/GroupChannel.java webapps/docs/changelog.xml
Author: remm Date: Fri Jul 13 12:58:27 2018 New Revision: 1835832 URL: http://svn.apache.org/viewvc?rev=1835832&view=rev Log: Propagate heartbeat from channel down to membership service. Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/java/org/apache/catalina/tribes/group/GroupChannel.java tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jul 13 12:58:27 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,1762168,1762172,1762182,1762201-1762202,1762204,1762208,1762288,1762296,1762324,1762348,1762353,1762362,1762374,1762492,1762503,1762505,1762541,1762608,1762710,1762753,1762766,1762769,1762944,1762947,1762953,1763167,1763179,1763232,1763259,1763271-1763272,1763276-1763277,1763319-1763320,1763370,1763372,1763375,1763377,1763393,1763412,1763430,1763450,1763462,1763505,1763511-17
buildbot success in on tomcat-trunk
The Buildbot has detected a restored build on builder tomcat-trunk while building . Full details are available at: https://ci.apache.org/builders/tomcat-trunk/builds/3455 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: silvanus_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' triggered this build Build Source Stamp: [branch tomcat/trunk] 1835831 Blamelist: remm Build succeeded! Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Tomcat JDBC Pool memory leak when using StatementFinalizer interceptor
hi Martin, On Fri, Jul 13, 2018 at 5:48 AM, Martin Knoblauch wrote: > Hi, (moving to developers list) > > any ideas on the problem below? This thing is kind of itching me :-) > > So I instrumented the "StatementFinalizer" class with some logging and > learned that over time a few instances of the "StatementFinalizer" are > created, used and destroyed. So far so good. For most of those instances, > the overall number of statements that are added to the "statements" list by > "createStatement" and the number of statements removed from the list by > "closeInvoked" is identical and after "closeInvoked" finishes, the list is > empty. > > But only for most instances of "StatementFinalizer". I could find that > there is one instance that is used (statements are added), but the > invocation of "closednvoked" stops after some minutes into the application. > As a result the "statements" list starts growing. > Could it be that your application checks out a connection and uses it for the life time of the application? Meaning Connection.close is never called? I see that you set `removeAbandonedTimeout="0"` which yields in an infinite value. This would be done if an application never closes a connection. If you set removeAbandonedTimeout to a positive value and the system logs that your connections are not being closed, then you know this is the case. If that is the scenario you have, then you should simply remove the StatementFinalizer as it will not do anything for you. In most cases you shouldn't need StatementFinalizer, as applications (and frameworks) are pretty at closing resources properly. Filip > > Rings any bells? > > Thanks > Martin > > On Wed, Jul 11, 2018 at 4:22 PM, Martin Knoblauch > wrote: > > > Hi, > > > > while analyzing some heap dump for other reasons, I found that our > > application is apparently aggregating a considerable amount of memory in > > "org.apache.tomcat.jdbc.pool.TrapException", which is never cleaned by > > GC. Digging deeper, it seems that the entries of the "statements" linked > > list in the StatementFinalizer are never removed from the list, so after > > three weeks of lifetime one ends up with a list of 7 million entries, > each > > 80 bytes. > > > > Now it might be, that we are just using the StatementFinalizer in a > wrong > > manner. And what we see is expected behavior. Below is our pool > > configuration. Maybe something is just missing :-) > > > > We are at Tomcat 8.0.36 (yeah, I know, but that is the version we have to > > use) and Java 8 (1.8.0_171). Underlying DB is Oracle 12.1.0.2 and we are > > using the latest "ojdbc7.jar" from Oracle. > > > > > > > name="jdbc/SimManagerDS" > > auth="Container" > > > > type="javax.sql.DataSource" > > description="Oracle datasource for xxx using > > tomcat.jdbc.pool" > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" > > jmxEnabled="true" > > jdbcInterceptors="ConnectionState;StatementFinalizer; > > ResetAbandonedTimer;StatementCache(prepared=true,max=200)" > > > > initialSize="7" > > minIdle="7" > > maxActive="30" > > maxIdle="10" > > > > testWhileIdle="true" > > testOnBorrow="true" > > testOnConnect="false" > > testOnReturn="false" > > validationQuery="SELECT 1 from dual" > > validationInterval="3" > > > > logAbandoned="true" > > removeAbandoned="false" > > removeAbandonedTimeout="0" > > suspectTimeout="600" > > > > timeBetweenEvictionRunsMillis="3" > > minEvictableIdleTimeMillis="6" > > maxWait="6" > > maxAge="0" > > > > connectionProperties="(defaultRowPrefetch=200)" > > > > driverClassName="oracle.jdbc.OracleDriver" > > url="jdbc:oracle:thin:@s###" > > username="" > > password="" > >/> > > > > Thanks > > Martin > > -- > > -- > > Martin Knoblauch > > email: k n o b i AT knobisoft DOT de > > www: http://www.knobisoft.de > > > > > > -- > -- > Martin Knoblauch > email: k n o b i AT knobisoft DOT de > www: http://www.knobisoft.de >
svn commit: r1835844 - /tomcat/trunk/test/javax/el/TestImportHandlerStandardPackages.java
Author: markt
Date: Fri Jul 13 14:41:19 2018
New Revision: 1835844
URL: http://svn.apache.org/viewvc?rev=1835844&view=rev
Log:
Tray and fix test failure on Gump
Modified:
tomcat/trunk/test/javax/el/TestImportHandlerStandardPackages.java
Modified: tomcat/trunk/test/javax/el/TestImportHandlerStandardPackages.java
URL:
http://svn.apache.org/viewvc/tomcat/trunk/test/javax/el/TestImportHandlerStandardPackages.java?rev=1835844&r1=1835843&r2=1835844&view=diff
==
--- tomcat/trunk/test/javax/el/TestImportHandlerStandardPackages.java (original)
+++ tomcat/trunk/test/javax/el/TestImportHandlerStandardPackages.java Fri Jul
13 14:41:19 2018
@@ -19,6 +19,7 @@ package javax.el;
import java.io.File;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
+import java.net.URI;
import java.net.URL;
import java.util.Enumeration;
import java.util.Map;
@@ -95,7 +96,13 @@ public class TestImportHandlerStandardPa
URL resource = resources.nextElement();
// Debugging for Gump failure
System.out.println("Scanning: [" + resource + "]");
-File dir = new File(resource.toURI());
+URI uri = resource.toURI();
+// Gump includes some JARs on classpath - skip them
+if (!"file".equals(uri.getScheme())) {
+continue;
+}
+File dir = new File(uri);
+
String[] files = dir.list();
for (String file : files) {
if (!file.endsWith(".class")) {
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot failure in on tomcat-trunk
The Buildbot has detected a new failure on builder tomcat-trunk while building . Full details are available at: https://ci.apache.org/builders/tomcat-trunk/builds/3456 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: silvanus_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' triggered this build Build Source Stamp: [branch tomcat/trunk] 1835844 Blamelist: markt BUILD FAILED: failed compile_1 Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62538] New: Connector attribute "compressableMimeType" is ignored
https://bz.apache.org/bugzilla/show_bug.cgi?id=62538 Bug ID: 62538 Summary: Connector attribute "compressableMimeType" is ignored Product: Tomcat 9 Version: 9.0.10 Hardware: PC Status: NEW Severity: normal Priority: P2 Component: Catalina Assignee: dev@tomcat.apache.org Reporter: benjamenwe...@gmail.com Target Milestone: - Created attachment 36028 --> https://bz.apache.org/bugzilla/attachment.cgi?id=36028&action=edit Modified server.xml Hello. It appears that the "compressableMimeType" attribute in "server.xml" is being ignored. Versions: Tomcat 9.0.10 Java 1.8.0_171(Doesn't seem to matter what version of 8) Repro: Using the attached configuration files, and Java servlet file, startup a tomcat 9 instance. Navigate to localhost:8080/deflate/whatever Expected: "I should show up in requests as deflated text!" deflated, with headers: "Content-Encoding","deflate" and "Content-Type","text/plain" Actual: The deflated string gzipped with headers: "Content-Encoding","gzip" and "Content-Type","text/plain" Even though "text/plain" isn't one of the configured "compressableMimeTypes" it still got gzipped. Even you need any more details or if I was unclear please let me know, thanks. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62538] Connector attribute "compressableMimeType" is ignored
https://bz.apache.org/bugzilla/show_bug.cgi?id=62538 --- Comment #1 from benjamenwe...@gmail.com --- Created attachment 36029 --> https://bz.apache.org/bugzilla/attachment.cgi?id=36029&action=edit Simple web.xml -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62538] Connector attribute "compressableMimeType" is ignored
https://bz.apache.org/bugzilla/show_bug.cgi?id=62538 --- Comment #2 from benjamenwe...@gmail.com --- Created attachment 36030 --> https://bz.apache.org/bugzilla/attachment.cgi?id=36030&action=edit Simple servlet that returns a deflated string -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
