Re: Database Connection Requests Initiated but Not Sent on the Wire (Some, Not All)
On 29/05/2024 17:03, Eric Robinson wrote: One of the webapps is related to voice reminder messages that go out to people. The reminders go out sometime after 9 am, which tracks with the slowdowns. Ack. Something to try while I work on a patch is setting archiveIndexStrategy="bloom" on the resources. You'd configure that in META-INF/context.xml something like this: Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: PersistentManager and ClassNotFoundException
Hello Chris, On Tue, May 28, 2024 at 6:38 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > Jakub, > > On 5/24/24 09:31, Jakub Królikowski wrote: > > On Fri, May 24, 2024 at 11:23 AM Mark Thomas wrote: > > > >> Can you provide the simplest web application (with source) that > >> replications the problem? > >> > >> Mark > >> > >> > >> On 23/05/2024 23:45, Jakub Królikowski wrote: > >>> Hi, > >>> > >>> I'm working with Tomcat 10.1. > >>> > >>> When a user starts using the store in my web application, I save the > >>> ShopCart object on the "cart" session attribute. > >>> I want the "cart" attributes to return to the session after restarting > >> the > >>> app. > >>> > >>> > >>> To enable session persistence I added > >>> > >>> > >>> > >>> to the Context. It loads the StandardManager. > >>> > >>> And this works fine - after reload / restart the object "ShopCart" is > >> back > >>> in the session. > >>> > >>> > >>> > >>> I want to experiment with PersistentManager. Tomcat docs says: " > >>> The persistence across restarts provided by the *StandardManager* is a > >>> simpler implementation than that provided by the *PersistentManager*. > If > >>> robust, production quality persistence across restarts is required then > >> the > >>> *PersistentManager* should be used with an appropriate configuration. > >>> > >>> " > >>> > >>> I hope for a Listener of deserialization of the session attributes. > >>> > >>> The new Manager configuration looks like this: > >>> > >>> >>> maxActiveSessions="2" saveOnRestart="true"> > >>> > >>> >>> "c:\tomcat10\sessionperm"/> > >>> > >>> > >>> > >>> But it doesn't work. After restart I get this exception: > >>> > >>> > >>> java.lang.ClassNotFoundException: ShopCart > >>> > >>> at > >>> > >> > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1332) > >>> > >>> at > >>> > >> > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1144) > >>> > >>> at java.base/java.lang.Class.forName0(Native Method) > >>> > >>> at java.base/java.lang.Class.forName(Class.java:534) > >>> > >>> at java.base/java.lang.Class.forName(Class.java:513) > >>> > >>> at > >>> > >> > org.apache.catalina.util.CustomObjectInputStream.resolveClass(CustomObjectInputStream.java:158) > >>> > >>> at > >>> java.base/java.io > >> .ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2061) > >>> > >>> at > >>> java.base/java.io > >> .ObjectInputStream.readClassDesc(ObjectInputStream.java:1927) > >>> > >>> at > >>> java.base/java.io > >> .ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2252) > >>> > >>> at > >>> java.base/java.io > >> .ObjectInputStream.readObject0(ObjectInputStream.java:1762) > >>> > >>> at > >>> java.base/java.io > >> .ObjectInputStream.readObject(ObjectInputStream.java:540) > >>> > >>> at > >>> java.base/java.io > >> .ObjectInputStream.readObject(ObjectInputStream.java:498) > >>> > >>> at > >>> > >> > org.apache.catalina.session.StandardSession.doReadObject(StandardSession.java:1198) > >>> > >>> at > >>> > >> > org.apache.catalina.session.StandardSession.readObjectData(StandardSession.java:831) > >>> > >>> at org.apache.catalina.session.FileStore.load(FileStore.java:203) > >>> > >>> at > >> org.apache.catalina.session.StoreBase.processExpires(StoreBase.java:138) > >>> > >>> at > >>> > >> > org.apache.catalina.session.PersistentManagerBase.processExpires(PersistentManagerBase.java:409) > >>> > >>> at > >>> > >> > org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:587) > >>> > >>> at > >>> > >> > org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:4787) > >>> > >>> at > >>> > >> > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1172) > >>> > >>> at > >>> > >> > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1176) > >>> > >>> at > >>> > >> > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1176) > >>> > >>> at > >>> > >> > org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1154) > >>> > >>> at > >>> > >> > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) > >> > > > > > >>> > >>> at > >>> > >> > java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) > >>> > >>> at > >>> > >> > java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) > >>> > >>> at > >>> > >> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) > >>> > >>> at > >>> > >> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) > >>> > >>> at > >>> > >> > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) > >>> > >>> at java.base/java.lang.Thread.run(Thread.java:1583)
Write listener question
Hello,
I have a NIO connector with an asynchronous servlet with its write listener.
@Override
public void onWritePossible() throws IOException {
if (this.isFirst) {
this.os = this.asyncContext.getResponse().getOutputStream();
this.startIdx = 0;
this.endIdx = WRITE_BUFFER_SIZE;
}
while (this.startIdx < this.endIdx && this.os.isReady()) {
this.os.write(this.response, this.startIdx, this.endIdx - this.startIdx);
this.startIdx = this.endIdx;
this.endIdx += WRITE_BUFFER_SIZE;
if (this.endIdx > this.response.length) this.endIdx = this.response.length;
}
if (this.startIdx == this.endIdx) {
this.ac.complete();
}
}
Only when the response to return is bigger than 32K then:
1. Writing the response in chunks of WRITE_BUFFER_SIZE = 32K, sometimes our
client receives the response in wrong order (complete, all bytes written, but
in different order).
2. Setting a WRITE_BUFFER_SIZE = 8K seems to fix this issue.
But it's weird, this is only happening in this client, we have other
installations returning responses of megabytes with no issues.
So just a couple of questions:
1. Do you think the implementation of the onWritePossible method above is
correct?
2. Is it worth to provide a buffered implementation of the ServletOuputStream
to write the response?
Thanks,
Joan.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Write listener question
Sorry, this issue happens with both Tomcat 8.5.x and 10.1.x.
-Original Message-
From: joan.balagu...@ventusproxy.com
Sent: Thursday, May 30, 2024 11:57 AM
To: 'Tomcat Users List'
Subject: Write listener question
Hello,
I have a NIO connector with an asynchronous servlet with its write listener.
@Override
public void onWritePossible() throws IOException {
if (this.isFirst) {
this.os = this.asyncContext.getResponse().getOutputStream();
this.startIdx = 0;
this.endIdx = WRITE_BUFFER_SIZE;
}
while (this.startIdx < this.endIdx && this.os.isReady()) {
this.os.write(this.response, this.startIdx, this.endIdx - this.startIdx);
this.startIdx = this.endIdx;
this.endIdx += WRITE_BUFFER_SIZE;
if (this.endIdx > this.response.length) this.endIdx = this.response.length;
}
if (this.startIdx == this.endIdx) {
this.ac.complete();
}
}
Only when the response to return is bigger than 32K then:
1. Writing the response in chunks of WRITE_BUFFER_SIZE = 32K, sometimes our
client receives the response in wrong order (complete, all bytes written, but
in different order).
2. Setting a WRITE_BUFFER_SIZE = 8K seems to fix this issue.
But it's weird, this is only happening in this client, we have other
installations returning responses of megabytes with no issues.
So just a couple of questions:
1. Do you think the implementation of the onWritePossible method above is
correct?
2. Is it worth to provide a buffered implementation of the ServletOuputStream
to write the response?
Thanks,
Joan.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Database Connection Requests Initiated but Not Sent on the Wire (Some, Not All)
Eric, On 5/29/24 12:10, Eric Robinson wrote: -Original Message- From: Mark Thomas Sent: Wednesday, May 29, 2024 10:19 AM To: users@tomcat.apache.org Subject: Re: Database Connection Requests Initiated but Not Sent on the Wire (Some, Not All) On 29/05/2024 16:08, Eric Robinson wrote: I believe your assessment is correct. How hard is it to enable pooling? Can it be bolted on, so to speak, through changes to the app context, such that the webapp itself does not necessarily need to implement special code? It looks like - from the database configuration you provided earlier - there is an option to configure the database via JNDI. If you do that with Tomcat you will automatically get pooling. That might be something to follow up with the vendor. If you go that route, I'd recommend configuring the pool to remove abandoned connections to avoid any issues with connection leaks. In reviewing live threads with Visual VM, I note that there are apparently threads related to cleaning up abandoned connections, and maybe even pooling? The threads are: mysql-cj-abandoned-connection-cleanup (2 of those) This thread is started by the MySQL driver to clean-up certain resources and isn't related to connection pooling. I've had issues with these things not shutting down on application-stop in certain versions of MySQL's driver. :/ OkHttp Connection Pool (2 of those) OkHttp https://ps.pndsn.com (not sure what that is) OkHttp is a network connection pool for HTTP connections. It's not related to dB connection pooling. Most pools do not have any extra threads required, so you wouldn't find any evidence of such things running in your system. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: PersistentManager and ClassNotFoundException
Jakub, On 5/30/24 05:25, Jakub Królikowski wrote: Where is your configuration located? It *should* be inside your located in META-INF/context.xml in your web application. If it's in there, then everything it does should be in the context (and ClassLoader) of your web application -- where your classes should be locatable. If you have it anywhere else, it probably won't work the way you expect it to work. Yes, you are right! Thank you for this hint! I have configured in /conf/server.xml. And indeed - interestingly, StandardManager, works correctly even there! PersistentManager, however, does not. After moving the configuration to /META-INF/context.xml, both managers work fine. It may be worth mentioning this in the documentation: https://tomcat.apache.org/tomcat-10.1-doc/config/manager.html#Persistence_Across_Restarts Would you like to provide a documentation patch/PR that works for you? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Write listener question
Joan,
Please don't hijack threads. Start a new message to the list without
replying to an existing one.
-chris
On 5/30/24 06:03, joan.balagu...@ventusproxy.com wrote:
Sorry, this issue happens with both Tomcat 8.5.x and 10.1.x.
-Original Message-
From: joan.balagu...@ventusproxy.com
Sent: Thursday, May 30, 2024 11:57 AM
To: 'Tomcat Users List'
Subject: Write listener question
Hello,
I have a NIO connector with an asynchronous servlet with its write listener.
@Override
public void onWritePossible() throws IOException {
if (this.isFirst) {
this.os = this.asyncContext.getResponse().getOutputStream();
this.startIdx = 0;
this.endIdx = WRITE_BUFFER_SIZE;
}
while (this.startIdx < this.endIdx && this.os.isReady()) {
this.os.write(this.response, this.startIdx, this.endIdx - this.startIdx);
this.startIdx = this.endIdx;
this.endIdx += WRITE_BUFFER_SIZE;
if (this.endIdx > this.response.length) this.endIdx =
this.response.length;
}
if (this.startIdx == this.endIdx) {
this.ac.complete();
}
}
Only when the response to return is bigger than 32K then:
1. Writing the response in chunks of WRITE_BUFFER_SIZE = 32K, sometimes our
client receives the response in wrong order (complete, all bytes written, but
in different order).
2. Setting a WRITE_BUFFER_SIZE = 8K seems to fix this issue.
But it's weird, this is only happening in this client, we have other
installations returning responses of megabytes with no issues.
So just a couple of questions:
1. Do you think the implementation of the onWritePossible method above is
correct?
2. Is it worth to provide a buffered implementation of the ServletOuputStream
to write the response?
Thanks,
Joan.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: PersistentManager and ClassNotFoundException
Hi Christopher, On Thu, May 30, 2024 at 1:40 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > Jakub, > > On 5/30/24 05:25, Jakub Królikowski wrote: > >> Where is your configuration located? It *should* be inside > >> your located in META-INF/context.xml in your web application. > >> If it's in there, then everything it does should be in the context (and > >> ClassLoader) of your web application -- where your classes should be > >> locatable. > >> > >> If you have it anywhere else, it probably won't work the way you expect > >> it to work. > > > > Yes, you are right! Thank you for this hint! > > I have configured in /conf/server.xml. > > And indeed - interestingly, StandardManager, works correctly even there! > > PersistentManager, however, does not. > > After moving the configuration to /META-INF/context.xml, both > > managers work fine. > > It may be worth mentioning this in the documentation: > > > https://tomcat.apache.org/tomcat-10.1-doc/config/manager.html#Persistence_Across_Restarts > > Would you like to provide a documentation patch/PR that works for you? > > -chris > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > I can write a paragraph about how to properly prepare the web service for serialisation and deserialisation, but I need some more time for that, because I see that the PersistentManager behaves differently from the StandardManager and I think it tries to serialise also session attributes, which are not instances of classes that implement java.io.Serializable. I need more time to experiment. Jakub
Need help installing SSL certificate in tomcat keystore
I am running a java web app on windows 2019 server and need some help getting the SSL certificate installed into my keystore. I am running tomcat 9.x and java 11 I am able to generate a certificate request using both keytool and/or openssl For both the CSR file looks like this, but the openssl also generates a private key xxx. -BEGIN NEW CERTIFICATE REQUEST- MIIC2TCCAcECAQAwZDELMAkGA1UEBhMCdXMxCzAJBgNVBAgTAm1hMRAwDgYDVQQH -END NEW CERTIFICATE REQUEST- Private key from OPENSSL -BEGIN PRIVATE KEY- MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC5EqmuGM9nRQ5n -END PRIVATE KEY- I use the CSR to submit a request to my company's certificate server and I am able to download 2 files in DER format [cid:image002.png@01DAB26D.E0B15B70] The downloaded certificate has a name certnew.cer, the downloaded chain certificate has a name cernew.p7b and both appear to be binary because when I open them in notepad++ they are unreadable Not sure how I go about importing converting and importing these into my keystore using keytool. The documenation is confusing to me as to what needs to be done. https://tomcat.apache.org/tomcat-9.0-doc/ssl-howto.html the section on importing the certificate does nto go into how to convert or merge the certificate or the certificate chain and also does not say anyting about a private keyfile Has anyone out there done this consistenly and successfully. Thanks for the help mark
context.xml file location
I am trying to setup JNDI for tomcat with a java webapp. I am using the official tomcat docker image (version 10.1.24-jdk21-temurin-jammy). However, I'm a bit confused about where to put the context.xml file. I tried putting it in /usr/local/tomcat/conf/Catalina/localhost/context.xml but tomcat refuses to start with the error "The main resource set specified [/usr/local/tomcat/webapps/context] is not a directory or war file, or is not readable'. I put my war file in /usr/local/tomcat/webapps/ROOT.war Are these paths correct or should I have put the context.xml or war file in a different directory? My hosting provider does not want me to edit the server.xml file (I have found google articles suggesting defining the JNDI tag in the server.xml and then using a in the context.xml, but my hosting provider wants me to define the tag in the context.xml file instead of the server.xml file). Thanks for your help!
Re: context.xml file location
I don't know about any docker-related differences, but I think that if you put a context config file under Catalina/localhost you need to name the .xml file the same as your .war file. So if you have foo.war, then you'd have Catalina/localhost/foo.xml You can also put a file named context.xml into the .war file itself, under META-INF directory. David On Thu, May 30, 2024 at 7:35 AM firstName lastName wrote: > I am trying to setup JNDI for tomcat with a java webapp. I am using the > official tomcat docker image (version 10.1.24-jdk21-temurin-jammy). > However, I'm a bit confused about where to put the context.xml file. I > tried putting it in /usr/local/tomcat/conf/Catalina/localhost/context.xml > but tomcat refuses to start with the error "The main resource set specified > [/usr/local/tomcat/webapps/context] is not a directory or war file, or is > not readable'. I put my war file in /usr/local/tomcat/webapps/ROOT.war > > Are these paths correct or should I have put the context.xml or war file in > a different directory? My hosting provider does not want me to edit the > server.xml file (I have found google articles suggesting defining the JNDI > tag in the server.xml and then using a in the > context.xml, but my hosting provider wants me to define the tag > in the context.xml file instead of the server.xml file). > > Thanks for your help! > -- E-Mail to and from me, in connection with the transaction of public business, is subject to the Wyoming Public Records Act and may be disclosed to third parties.
Write listener question
Hello,
Sorry for the previous mail ...
I have a NIO connector with an asynchronous servlet with its write listener
(working in both tomcat 8.5 and tomcat 10.1.20).
@Override
public void onWritePossible() throws IOException {
if (this.isFirst) {
this.os = this.asyncContext.getResponse().getOutputStream();
this.startIdx = 0;
this.endIdx = WRITE_BUFFER_SIZE;
}
while (this.startIdx < this.endIdx && this.os.isReady()) {
this.os.write(this.response, this.startIdx, this.endIdx - this.startIdx);
this.startIdx = this.endIdx;
this.endIdx += WRITE_BUFFER_SIZE;
if (this.endIdx > this.response.length) this.endIdx = this.response.length;
}
if (this.startIdx == this.endIdx) {
this.ac.complete();
}
}
Only when the response to return is bigger than 32K then:
1. Writing the response in chunks of WRITE_BUFFER_SIZE = 32K, sometimes our
client receives the response in wrong order (complete, all bytes written, but
in different order).
2. Setting a WRITE_BUFFER_SIZE = 8K seems to fix this issue.
But it's weird, this is only happening in this client, we have other
installations returning responses of megabytes with no issues.
So just a couple of questions:
1. Do you think the implementation of the onWritePossible method above is
correct?
2. Is it worth to provide a buffered implementation of the ServletOuputStream
to write the response?
Thanks,
Joan.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: context.xml file location
Renaming my context.xml to ROOT.xml (without changing the folder) fixed the problem. Thanks for the help! On Thu, May 30, 2024 at 8:42 AM David Rush wrote: > I don't know about any docker-related differences, but > > I think that if you put a context config file under Catalina/localhost you > need to name the .xml file the same as your .war file. So if you have > foo.war, then you'd have Catalina/localhost/foo.xml > > You can also put a file named context.xml into the .war file itself, under > META-INF directory. > > David > > On Thu, May 30, 2024 at 7:35 AM firstName lastName < > nouser...@gmail.com> > wrote: > > > I am trying to setup JNDI for tomcat with a java webapp. I am using the > > official tomcat docker image (version 10.1.24-jdk21-temurin-jammy). > > However, I'm a bit confused about where to put the context.xml file. I > > tried putting it in /usr/local/tomcat/conf/Catalina/localhost/context.xml > > but tomcat refuses to start with the error "The main resource set > specified > > [/usr/local/tomcat/webapps/context] is not a directory or war file, or is > > not readable'. I put my war file in /usr/local/tomcat/webapps/ROOT.war > > > > Are these paths correct or should I have put the context.xml or war file > in > > a different directory? My hosting provider does not want me to edit the > > server.xml file (I have found google articles suggesting defining the > JNDI > > tag in the server.xml and then using a in the > > context.xml, but my hosting provider wants me to define the > tag > > in the context.xml file instead of the server.xml file). > > > > Thanks for your help! > > > > -- > > E-Mail to and from me, in connection with the transaction > of public > business, is subject to the Wyoming Public Records > Act and may be > disclosed to third parties. >
Re: context.xml file location
Hello, On 5/30/24 10:12, firstName lastName wrote: Renaming my context.xml to ROOT.xml (without changing the folder) fixed the problem. Thanks for the help! The best practice would be to put your context.xml file into your WAR file's META-INF/context.xml path. That way, it will be deployed from the WAR file and you don't have to keep a separate file around. If you need to override the contents of the WAR-packaged file, then you will need to use conf/[Engine]/[Host]/[App].xml https://tomcat.apache.org/tomcat-10.1-doc/config/context.html#Defining_a_context -chris On Thu, May 30, 2024 at 8:42 AM David Rush wrote: I don't know about any docker-related differences, but I think that if you put a context config file under Catalina/localhost you need to name the .xml file the same as your .war file. So if you have foo.war, then you'd have Catalina/localhost/foo.xml You can also put a file named context.xml into the .war file itself, under META-INF directory. David On Thu, May 30, 2024 at 7:35 AM firstName lastName < nouser...@gmail.com> wrote: I am trying to setup JNDI for tomcat with a java webapp. I am using the official tomcat docker image (version 10.1.24-jdk21-temurin-jammy). However, I'm a bit confused about where to put the context.xml file. I tried putting it in /usr/local/tomcat/conf/Catalina/localhost/context.xml but tomcat refuses to start with the error "The main resource set specified [/usr/local/tomcat/webapps/context] is not a directory or war file, or is not readable'. I put my war file in /usr/local/tomcat/webapps/ROOT.war Are these paths correct or should I have put the context.xml or war file in a different directory? My hosting provider does not want me to edit the server.xml file (I have found google articles suggesting defining the JNDI tag in the server.xml and then using a in the context.xml, but my hosting provider wants me to define the tag in the context.xml file instead of the server.xml file). Thanks for your help! -- E-Mail to and from me, in connection with the transaction of public business, is subject to the Wyoming Public Records Act and may be disclosed to third parties. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Database Connection Requests Initiated but Not Sent on the Wire (Some, Not All)
OK. This is an interim binary patch for 9.0.80 only. The purpose is to: - confirm the proposed change fixes the problem - provide you with a workaround in the short term This is the binary patch: https://people.apache.org/~markt/dev/classloader-not-found-cache-9.0.80-v1.zip Extract the contents into $CATALINA_HOME/lib You should end up with: $CATALINA_HOME/lib/org/apache/... Usual caveats apply. This is not an official release. Use it at your own risk. Don't blame either me or the ASF it is results in alien invasion, a tax bill, the server catching fire or anything else unexpected and/or unwanted. Longer term, I'm not sure this is exactly how I want to fix it in Tomcat. I am convinced of the need to cache classes that don't exist but exactly where / how to do that and what degree of control the user should have is very much TBD. I suspect this will be a topic of discussion at Community Over Code at Bratislava next week. I am expecting that any fix won't be in the June release round but should be in the July release round. Let us know how you get on and good luck. Mark On 30/05/2024 10:16, Mark Thomas wrote: On 29/05/2024 17:03, Eric Robinson wrote: One of the webapps is related to voice reminder messages that go out to people. The reminders go out sometime after 9 am, which tracks with the slowdowns. Ack. Something to try while I work on a patch is setting archiveIndexStrategy="bloom" on the resources. You'd configure that in META-INF/context.xml something like this: Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Database Connection Requests Initiated but Not Sent on the Wire (Some, Not All)
ср, 29 мая 2024 г. в 13:34, Mark Thomas : > > > > It is also problem number 3. The reason it is expensive is that class > loaders don't cache misses so if a web application has a large number of > JARs, they all get scanned every time the DriverManager tries to create > a new connection. > > > The slowness occurs in the web application that depends on the second > JDBC driver in DriverManager's list. When a request that requires a > database connection is received, there is a short delay while the web > application tries, and fails, to load the first JDBC driver in the list. > Class loading is synchronized on class name being loaded so if any other > requests also need a database connection, they have to wait for this > request to finish the search for the JDBC driver before they can > continue. This creates a bottleneck. Requests are essentially rate > limited to 1 request that requires a database connection per however > long it takes to scan every JAR in the web application for a class that > isn't there. If the average rate of requests exceeds this rate limit > then a queue is going to build up and it won't subside until the average > rate of requests falls below this rate limit. > > [...] > > Problem number 3 is a Tomcat issue. It should be relatively easy to > start caching misses (i.e. this class loader cannot load this class) and > save the time spent repeatedly scanning JARs for a class that isn't there. > (I wonder if unpacking all JARs into the WEB-INF/classes directory will help.) > Something to try while I work on a patch is setting > archiveIndexStrategy="bloom" on the resources. > > You'd configure that in META-INF/context.xml something like this: > > > > +1 for archiveIndexStrategy="bloom". https://tomcat.apache.org/tomcat-9.0-doc/config/resources.html The option is available since Tomcat 9.0.69 and thus should be OK here. (Earlier versions had the feature since 9.0.39, but it was configured through an attribute of Context https://bz.apache.org/bugzilla/show_bug.cgi?id=66209 Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Write listener question
> On May 30, 2024, at 08:47,
> wrote:
>
> I have a NIO connector with an asynchronous servlet with its write listener
> (working in both tomcat 8.5 and tomcat 10.1.20).
>
> @Override
> public void onWritePossible() throws IOException {
>
> if (this.isFirst) {
>this.os = this.asyncContext.getResponse().getOutputStream();
>this.startIdx = 0;
>this.endIdx = WRITE_BUFFER_SIZE;
> }
Is there any logic somewhere to clear isFirst? If not, the start and end
indices will be reset every time this method is called should os.isReady()
returns false.
> while (this.startIdx < this.endIdx && this.os.isReady()) {
> this.os.write(this.response, this.startIdx, this.endIdx - this.startIdx);
>
> this.startIdx = this.endIdx;
> this.endIdx += WRITE_BUFFER_SIZE;
> if (this.endIdx > this.response.length) this.endIdx =
> this.response.length;
> }
>
> if (this.startIdx == this.endIdx) {
>this.ac.complete();
> }
> }
>
> Only when the response to return is bigger than 32K then:
> 1. Writing the response in chunks of WRITE_BUFFER_SIZE = 32K, sometimes our
> client receives the response in wrong order (complete, all bytes written, but
> in different order).
> 2. Setting a WRITE_BUFFER_SIZE = 8K seems to fix this issue.
>
> But it's weird, this is only happening in this client, we have other
> installations returning responses of megabytes with no issues.
Perhaps this one client has a slow network, so isReady() returns false in that
environment, but not in other ones.
- Chuck
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Write listener question
Hi Chuck,
isFirst is initialized to 'true' when the class is instantiated, so that
piece of code is just executed the first time the execution enters the '
onWritePossible' method. Later, within the same 'if', 'isFirst' is set to
false, (not shown in the code, sorry)
Perhaps this one client has a slow network, so isReady() returns false in
that environment, but not in other ones.
--> And how can this be solved? I mean, no one wants to have a piece of
code that works fine just depending on the network speed. The point is the
whole response is returned back (until the last byte), but not in the
correct order. The responses are xml, so some documents cannot be parsed
correctly, that 's why the alert was thrown on the client side.
This is just a sample, but it's like instead of receiving this:
.
...
The client is receiving this:
...
el>.
It's like a chunk is received in wrong order.
Thanks!
Joan.
-Original Message-
From: Chuck Caldarale
Sent: Thursday, May 30, 2024 7:27 PM
To: Tomcat Users List
Subject: Re: Write listener question
> On May 30, 2024, at 08:47,
wrote:
>
> I have a NIO connector with an asynchronous servlet with its write
listener (working in both tomcat 8.5 and tomcat 10.1.20).
>
> @Override
> public void onWritePossible() throws IOException {
>
> if (this.isFirst) {
>this.os = this.asyncContext.getResponse().getOutputStream();
>this.startIdx = 0;
>this.endIdx = WRITE_BUFFER_SIZE;
> }
Is there any logic somewhere to clear isFirst? If not, the start and end
indices will be reset every time this method is called should os.isReady()
returns false.
> while (this.startIdx < this.endIdx && this.os.isReady()) {
> this.os.write(this.response, this.startIdx, this.endIdx -
> this.startIdx);
>
> this.startIdx = this.endIdx;
> this.endIdx += WRITE_BUFFER_SIZE;
> if (this.endIdx > this.response.length) this.endIdx =
> this.response.length; }
>
> if (this.startIdx == this.endIdx) {
>this.ac.complete();
> }
> }
>
> Only when the response to return is bigger than 32K then:
> 1. Writing the response in chunks of WRITE_BUFFER_SIZE = 32K, sometimes
our client receives the response in wrong order (complete, all bytes
written, but in different order).
> 2. Setting a WRITE_BUFFER_SIZE = 8K seems to fix this issue.
>
> But it's weird, this is only happening in this client, we have other
installations returning responses of megabytes with no issues.
Perhaps this one client has a slow network, so isReady() returns false in
that environment, but not in other ones.
- Chuck
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Write listener question
> On May 30, 2024, at 12:53, > wrote: > > isFirst is initialized to 'true' when the class is instantiated, so that > piece of code is just executed the first time the execution enters the ' > onWritePossible' method. Later, within the same 'if', 'isFirst' is set to > false, (not shown in the code, sorry) If you don’t show us all of the code, then it’s rather difficult to answer your question about the implementation being correct. > Perhaps this one client has a slow network, so isReady() returns false in > that environment, but not in other ones. > --> And how can this be solved? The slow network possibility was based on the assumption that isFirst was never cleared, since that’s what you presented. - Chuck - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Write listener question
Trying to simplify the code was a mistake ...
private static final int WRITE_BUFFER_SIZE = 8 * 1024; <-- with 32K fails on
this client
private final AsyncContext ac;
private final ServletResponse sr;
private ServletOutputStream os;
private boolean firstTime = true;
private byte[] response;
private int responseSize, startIdx, endIdx;
// Constructor
public WriteListenerImpl(AsyncContext ac, byte[] response) {
this.ac = ac;
this.sr = this.ac.getResponse();
this.response = response;
}
@Override
public void onWritePossible() throws IOException
{
// If this is the first time we call 'onWritePossible' ...
if (this.firstTime) {
this.firstTime = false;
this.os = this.sr.getOutputStream();
this.responseSize = this.response.length;
this.startIdx = 0;
this.endIdx = Math.min(this.responseSize, WRITE_BUFFER_SIZE);
}
// And write the 'response'.
while (this.startIdx < this.endIdx && this.os.isReady()) {
this.os.write(this.response, this.startIdx, this.endIdx - this.startIdx);
this.startIdx = this.endIdx;
this.endIdx += WRITE_BUFFER_SIZE;
if (this.endIdx > this.responseSize) this.endIdx = this.responseSize;
}
// Just perform the 'postProcess' when response writing is finished (start
== end)
if (this.startIdx == this.endIdx) {
this.postProcess(null);
}
}
@Override
public void onError(Throwable t) {
this.postProcess(t);
}
private void postProcess(Throwable t) {
this.ac.complete();
if (t != null) {
t.printStackeTrace();
}
}
-Original Message-
From: Chuck Caldarale
Sent: Thursday, May 30, 2024 8:01 PM
To: Tomcat Users List
Subject: Re: Write listener question
> On May 30, 2024, at 12:53,
> wrote:
>
> isFirst is initialized to 'true' when the class is instantiated, so
> that piece of code is just executed the first time the execution enters the '
> onWritePossible' method. Later, within the same 'if', 'isFirst' is set
> to false, (not shown in the code, sorry)
If you don’t show us all of the code, then it’s rather difficult to answer your
question about the implementation being correct.
> Perhaps this one client has a slow network, so isReady() returns false
> in that environment, but not in other ones.
> --> And how can this be solved?
The slow network possibility was based on the assumption that isFirst was never
cleared, since that’s what you presented.
- Chuck
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Database Connection Requests Initiated but Not Sent on the Wire (Some, Not All)
Hi Mark, > -Original Message- > From: Mark Thomas > Sent: Thursday, May 30, 2024 9:30 AM > To: users@tomcat.apache.org > Subject: Re: Database Connection Requests Initiated but Not Sent on the Wire > (Some, Not All) > > OK. > > This is an interim binary patch for 9.0.80 only. > > The purpose is to: > - confirm the proposed change fixes the problem > - provide you with a workaround in the short term > > This is the binary patch: > > https://people.apache.org/~markt/dev/classloader-not-found-cache-9.0.80- > v1.zip > > Extract the contents into $CATALINA_HOME/lib > > You should end up with: > > $CATALINA_HOME/lib/org/apache/... > I'll get on this right away. > Usual caveats apply. This is not an official release. Use it at your own > risk. Don't > blame either me or the ASF it is results in alien invasion, a tax bill, the > server > catching fire or anything else unexpected and/or unwanted. > Okay, but if we're invaded by alien tax collectors riding flaming servers, THEN I'm coming after you. > Longer term, I'm not sure this is exactly how I want to fix it in Tomcat. I am > convinced of the need to cache classes that don't exist but exactly where / > how > to do that and what degree of control the user should have is very much TBD. > > I suspect this will be a topic of discussion at Community Over Code at > Bratislava > next week. > > I am expecting that any fix won't be in the June release round but should be > in > the July release round. > > Let us know how you get on and good luck. > Will do! > Mark > > > On 30/05/2024 10:16, Mark Thomas wrote: > > On 29/05/2024 17:03, Eric Robinson wrote: > > > > > > > >> One of the webapps is related to voice reminder messages that go out > >> to people. The reminders go out sometime after 9 am, which tracks > >> with the slowdowns. > > > > Ack. > > > > Something to try while I work on a patch is setting > > archiveIndexStrategy="bloom" on the resources. > > > > You'd configure that in META-INF/context.xml something like this: > > > > > > > > > > Mark > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
