Re: Reg: tomcat CPU spikes
Jalaj, On 7/15/24 18:18, Jalaj Asher wrote: We ran into 2 issues 1. We needed to allocate significant amount of -XMX for heap space, if we allowed caching, since increasing memory by a few hundred MB as well was not enough. Interesting. How much static content do you have? This seems like a good use-case for a reverse-proxy to handle your static content for you. 2. Also with the setting being enabled, it generated logs stating "could not add a resource as there wasn’t enough space". Which was very quickly filling up our disk space as well as increasing disk IO causing latency concerns. Also interesting. Can you post one of those messages here? Was there a stack trace shown or just the warning? -chris -Original Message- From: Christopher Schultz Sent: Monday, July 15, 2024 4:19 PM To: users@tomcat.apache.org Subject: Re: Reg: tomcat CPU spikes Attention! - This email has originated from an External Source outside of eClinicalWorks. Always use caution when opening attachments, clicking links, or when responding to this email. If you feel this is a phishing scam, please use the Phish Alert Report button in Outlook. Jalaj, On 7/15/24 15:03, Jalaj Asher wrote: Yeah I was wondering the same as this has been in place since a few years now atleast 4 years since cachingAllowed had some changes in tomcat 8 which was resulting in it caching all static content as well as jsps and jars and our though process was if we have static content being cached on the client end and jsps in the work folder each time on access we don’t need the cache. Does the cache actively hurt you? Is there a way to cache just the jars and not every thing else in memory ? I think the short answer is "no there is not a way to do this" but I may be wrong. The long answer might be "maybe, but you will have to play games with and and maybe some other things to get it working. I would save yourself some complexity and simply enable caching. -chris -Original Message- From: Christopher Schultz Sent: Friday, July 12, 2024 4:02 PM To: users@tomcat.apache.org Subject: Re: Reg: tomcat CPU spikes Attention! - This email has originated from an External Source outside of eClinicalWorks. Always use caution when opening attachments, clicking links, or when responding to this email. If you feel this is a phishing scam, please use the Phish Alert Report button in Outlook. Jalaj, On 7/12/24 10:19, Jalaj Asher wrote: Thank you Chuck and John for the responses. Just a few points from the things you highlighted and wanted me to check 1. unpackwar is set to true. I checked and was informed that we need that to be true for a specific war file. 2. cachingAllowed=false. We keep it as false across the board. Well... that'll do it. In order to locate resources, Tomcat needs to sift through all of those JAR files every time. Scanning ZIP files is expensive. You might want to reconsider this particular setting in your environment. Also the reason I shared 2 different stacks is to highlight that the problem does not occur post restart or with any specific part of the application like the parser going in a loop but it kicks of at random times impacting multiple tomcats. But all having the same stack waiting on archiveresourceset and java.util.zip. My only question would be "why is this only now coming to your attention? Things should have been behaving this way .. for a long time. I am working on getting both types of stacks to share here as well. Had a question we just have one war file if unpack war is triggered why should it impact loading jars from the entire webapp lib ? Because WEB-INF/lib is full of JAR files that need to be scanned every tie you try to load ... anything. Since you have disabled caching, it has to re-check every file for every resource-load request. -chris -Original Message- From: Chuck Caldarale Sent: Wednesday, July 10, 2024 6:19 PM To: Tomcat Users List Subject: Re: Reg: tomcat CPU spikes Attention! - This email has originated from an External Source outside of eClinicalWorks. Always use caution when opening attachments, clicking links, or when responding to this email. If you feel this is a phishing scam, please use the Phish Alert Report button in Outlook. On Jul 10, 2024, at 17:02, Jalaj Asher wrote: Sharing another stack to see if this can give any more insights.this thread is the tomcat main thread was loading about 65MB of data. "main" #1 prio=5 os_prio=0 java.lang.Thread.State: RUNNABLE at java.util.zip.ZipFile.getEntry(Native Method) at java.util.zip.ZipFile.getEntry(ZipFile.java:328) - locked <0xa1b04418> (a java.util.jar.JarFile) at java.util.jar.JarFile.getEntry(JarFile.java:253) at java.util.jar.JarFile.getJarEntry(JarFile.java:236) at org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(Abstra
RE: Reg: tomcat CPU spikes
> space". Which was very quickly filling up our disk space as well as > increasing disk IO causing latency concerns. 1. Also interesting. Can you post one of those messages here? Was there a stack trace shown or just the warning? It is just the warning. No stack trace. I will work on recreating this since all our environments has this disabled. 2. Interesting. How much static content do you have? This seems like a good use-case for a reverse-proxy to handle your staticcontent for you. We have not collated the complete size of it. But are reasons we cannot do that. Also I was reviewing some older heap dumps and I could see that the jars are getting cached in tomcat even with cachingAllowed=false. Also this is not a consistent issue once it happens it takes sometime for the stack to go away as well as post tomcat reboots the problem goes away with the same settings and we do see that the wars are getting deployed during tomcat startup as well. Regards Jalaj P Asher -Original Message- From: Christopher Schultz Sent: Tuesday, July 16, 2024 10:05 AM To: users@tomcat.apache.org Subject: Re: Reg: tomcat CPU spikes Attention! - This email has originated from an External Source outside of eClinicalWorks. Always use caution when opening attachments, clicking links, or when responding to this email. If you feel this is a phishing scam, please use the Phish Alert Report button in Outlook. Jalaj, On 7/15/24 18:18, Jalaj Asher wrote: > We ran into 2 issues > 1. We needed to allocate significant amount of -XMX for heap space, > if we allowed caching, since increasing memory by a few hundred MB as > well was not enough. Interesting. How much static content do you have? This seems like a good use-case for a reverse-proxy to handle your static content for you. > 2. Also with the setting being enabled, it generated logs stating > "could not add a resource as there wasn’t enough > space". Which was very quickly filling up our disk space as well as > increasing disk IO causing latency concerns. Also interesting. Can you post one of those messages here? Was there a stack trace shown or just the warning? -chris > -Original Message- > From: Christopher Schultz > Sent: Monday, July 15, 2024 4:19 PM > To: users@tomcat.apache.org > Subject: Re: Reg: tomcat CPU spikes > > Attention! - This email has originated from an External Source outside of > eClinicalWorks. Always use caution when opening attachments, clicking links, > or when responding to this email. If you feel this is a phishing scam, please > use the Phish Alert Report button in Outlook. > > > Jalaj, > > On 7/15/24 15:03, Jalaj Asher wrote: >> Yeah I was wondering the same as this has been in place since a few >> years now atleast 4 years since cachingAllowed had some changes in >> tomcat 8 which was resulting in it caching all static content as well >> as jsps and jars and our though process was if we have static content >> being cached on the client end and jsps in the work folder each time >> on access we don’t need the cache. > Does the cache actively hurt you? > >> Is there a way to cache just the jars and not every thing else in memory ? > > I think the short answer is "no there is not a way to do this" but I may be > wrong. > > The long answer might be "maybe, but you will have to play games with > and and maybe some other things to get it > working. > > I would save yourself some complexity and simply enable caching. > > -chris > >> -Original Message- >> From: Christopher Schultz >> Sent: Friday, July 12, 2024 4:02 PM >> To: users@tomcat.apache.org >> Subject: Re: Reg: tomcat CPU spikes >> >> Attention! - This email has originated from an External Source outside of >> eClinicalWorks. Always use caution when opening attachments, clicking links, >> or when responding to this email. If you feel this is a phishing scam, >> please use the Phish Alert Report button in Outlook. >> >> >> Jalaj, >> >> On 7/12/24 10:19, Jalaj Asher wrote: >>> Thank you Chuck and John for the responses. >>> >>> Just a few points from the things you highlighted and wanted me to >>> check 1. unpackwar is set to true. I checked and was informed that we need >>> that to be true for a specific war file. >>> 2. cachingAllowed=false. We keep it as false across the board. >> >> Well... that'll do it. In order to locate resources, Tomcat needs to sift >> through all of those JAR files every time. Scanning ZIP files is expensive. >> >> You might want to reconsider this particular setting in your environment. >> >>> Also the reason I shared 2 different stacks is to highlight that the >>> problem does not occur post restart or with any specific part of the >>> application like the parser going in a loop but it kicks of at >>> random times impacting multiple tomcats. But all having the same >>> stack waiting on archiveresourceset and java.util.zip. >> My only question would be "why is this only now coming to your attention?
Re: Win10 installation progress
Dear Christopher, Thanks for all your advice.Progress here.The port regards hck app (for medical diagnosis) with 6.1Mb source ofwhich 4.6Mb java code.Have been able to compile without using j2ee.jar (with a dos .batscript tackling 19 directories, etc.)Used instead in CLASSPATH files in lib: servlet-api.jar,javax.mail.jar,javax.activation.jar. There IS a web.xml file in WEB-INF with, among the others, forcom.ZAjax: com.ZAjax com.ZAjax com.ZAjax com.ZAjax Starting up tomcat 9.0.88 still hiccups when hck is encountered.The catalina log is attached. Seems to be close ... :-)))dennis de champeaux [[ Next week will present a paper 'Medical Diagnosis Algorithm at a conference. :--]] -- Home page: rs6.risingnet.net/~ddccMarketing site: www . OntoOO.comKindle books: Side Effects: Impacts on the 21st Century::https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09MHJ5W48 Even if not true::https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09Y4WR9J7 9/11:: They had the Sun in their Eyeshttps://www.amazon.com/11-They-had-their-Eyes-ebook/dp/B0CMQ2WSK9https://www.amazon.nl/dp/B0CMQ2WSK9 On Tuesday, July 9, 2024 at 06:29:41 AM PDT, Christopher Schultz wrote: Dennis, On 7/8/24 17:05, DdC wrote: > Still struggling launching my hck app on Win10 & tomcat 9.0.88.It > runs (dont laugh) on: XP/Win7 & tomcat 4.0.4 Linux-gnu & tomcat > 6.0.32 Tomcat 9 ought to be able to run a Servlet-2.0-era application just fine unless you use one or two things that have been deprecated and won't work anymore (20 years later). > Tomcat 9.0.88 displays fine with localhost:8080. Do you mean that the ROOT application ("Welcome to Tomcat") runs? I wouldn't expect anything less. What version of Java are you using, now? Hopefully it's at least 17 or 21. > App hck compiles OK (dont laugh) with a script > using:CLASSPATH=.;c:\tomcat4\j2ee.jar;c:\tomcat4\webapps\hck\WEB-INF\classes;c:\tomcat4\lib\ This isn't a script. Can you post the whole thing? > Starting subsequently tomcat4 hiccups with: JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED > --add-opens=java.base/java.io=ALL-UNNAMED > --add-opens=java.base/java.util=ALL-UNNAMED > --add-opens=java.base/java.util.concurrent=ALL-UNNAMED > --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED08-Jul-2024 12:00:24.731 > INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server > version name: Apache Tomcat/9.0.8808-Jul-2024 12:00:24.748 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Server built: > Apr 9 2024 13:22:30 UTC08-Jul-2024 12:00:24.748 INFO [main] > org.apache.catalina.startup.VersionLoggerListener.log Server version number: > 9.0.88.008-Jul-2024 12:00:24.748 INFO > [main]org.apache.catalina.startup.VersionLoggerListener.log OS Name:Windows > 10..08-Jul-2024 12:00:27.313 INFO [main] > org.apache.catalina.startup.HostConfig.deployDirectory Deploying web > application directory [C:\tomcat4\webapps\hck]08-Jul-2024 12:00:30.041 SEVERE > [main] org.apache.catalina.startup.HostConfig.deployDirectory Error deploying > web application directory [C:\tomcat4\webapps\hck] > java.lang.IllegalStateException: Error starting child at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:690) > at > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:659) > > atorg.apache.catalina.core.StandardHost.addChild(StandardHost.java:661)at... > ... ...> > Why, oh why does this tomcat not accept what has been compiled withj2ee.jar?? This copy-pasta is very difficult to read. Can you sent it without ... re-formatting or whatever else is going on with your email sender? Your error does not contain the "real error". If there are 50 lines of stack trace, please post them ALL. Otherwise it will not be possible to tell what the root problem is. You have a runtime error above and a compiler error below. It's tough to follow your journey, here. Definitely do not put j2ee.jar into your web application's WEB-INF/lib directory. > Using the classpath without j2ee.jar: > CLASSPATH=.;c:\tomcat4\webapps\hck\WEB-INF\classes;c:\tomcat4\lib\the > compilation of the hck app fails early > with:javac > *.javaZAjax.java:9: error: cannot find symbolpublic class ZAjax extends > HttpServlet { ^ symbol: class > HttpServletZAjax.java:12: error: cannot find symbol public void doGet > (HttpServletRequest req, ^ symbol: class > HttpServletRequest location: class ZAjax > > Why, oh why is the servlet infrastructure not found? This is what I'm seeing (roughly) C:> SET CLASSPATH=.;c:\tomcat4\webapps\hck\WEB-INF\classes;c:\tomcat4\lib\ C:> CD c:\tomcat4\webapps\hck\WEB-INF\classes\com C:> javac *.java This is an unusual way to compile
Re: Win10 installation progress
> On Jul 16, 2024, at 15:57, DdC wrote: > > Dear Christopher, > > Thanks for all your advice. > Progress here. > The port regards hck app (for medical diagnosis) with 6.1Mb source of > which 4.6Mb java code. > Have been able to compile without using j2ee.jar (with a dos .bat > script tackling 19 directories, etc.) > Used instead in CLASSPATH files in lib: > servlet-api.jar, > javax.mail.jar, > javax.activation.jar. > There IS a web.xml file in WEB-INF with, among the others, for > com.ZAjax: > > > com.ZAjax > com.ZAjax > > > > com.ZAjax > com.ZAjax > > > Starting up tomcat 9.0.88 still hiccups when hck is encountered. > The catalina log is attached. Attachments are stripped by the mailing list handler; could you post the log directly into the message (if not too big), or at some publicly accessible location? - Chuck > Seems to be close ... :-))) > dennis de champeaux > > [[ Next week will present a paper 'Medical Diagnosis Algorithm at a > conference. :--]] > > -- > > Home page: rs6.risingnet.net/~ddcc > Marketing site: www . OntoOO.com > Kindle books: >Side Effects: Impacts on the 21st Century:: > https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09MHJ5W48 >Even if not true:: > https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09Y4WR9J7 >9/11:: They had the Sun in their Eyes > https://www.amazon.com/11-They-had-their-Eyes-ebook/dp/B0CMQ2WSK9 > https://www.amazon.nl/dp/B0CMQ2WSK9 > > On Tuesday, July 9, 2024 at 06:29:41 AM PDT, Christopher Schultz > wrote: > > > Dennis, > > On 7/8/24 17:05, DdC wrote: > > Still struggling launching my hck app on Win10 & tomcat 9.0.88.It > > runs (dont laugh) on: XP/Win7 & tomcat 4.0.4 Linux-gnu & tomcat > > 6.0.32 > Tomcat 9 ought to be able to run a Servlet-2.0-era application just fine > unless you use one or two things that have been deprecated and won't > work anymore (20 years later). > > > Tomcat 9.0.88 displays fine with localhost:8080. > > Do you mean that the ROOT application ("Welcome to Tomcat") runs? I > wouldn't expect anything less. > > What version of Java are you using, now? Hopefully it's at least 17 or 21. > > > App hck compiles OK (dont laugh) with a script > > using:CLASSPATH=.;c:\tomcat4\j2ee.jar;c:\tomcat4\webapps\hck\WEB-INF\classes;c:\tomcat4\lib\ > > This isn't a script. Can you post the whole thing? > > > Starting subsequently tomcat4 hiccups with: > JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED > > --add-opens=java.base/java.io=ALL-UNNAMED > > --add-opens=java.base/java.util=ALL-UNNAMED > > --add-opens=java.base/java.util.concurrent=ALL-UNNAMED > > --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED08-Jul-2024 12:00:24.731 > > INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server > > version name: Apache Tomcat/9.0.8808-Jul-2024 12:00:24.748 INFO [main] > > org.apache.catalina.startup.VersionLoggerListener.log Server built: > > Apr 9 2024 13:22:30 UTC08-Jul-2024 12:00:24.748 INFO [main] > > org.apache.catalina.startup.VersionLoggerListener.log Server version > > number: 9.0.88.008-Jul-2024 12:00:24.748 INFO > > [main]org.apache.catalina.startup.VersionLoggerListener.log OS Name:Windows > > 10..08-Jul-2024 12:00:27.313 INFO [main] > > org.apache.catalina.startup.HostConfig.deployDirectory Deploying web > > application directory [C:\tomcat4\webapps\hck]08-Jul-2024 12:00:30.041 > > SEVERE [main] org.apache.catalina.startup.HostConfig.deployDirectory Error > > deploying web application directory [C:\tomcat4\webapps\hck] > > java.lang.IllegalStateException: Error starting childat > > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:690) > > at > > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:659) > > > > atorg.apache.catalina.core.StandardHost.addChild(StandardHost.java:661)at... > > ... ...> > > Why, oh why does this tomcat not accept what has been compiled > > withj2ee.jar?? > > This copy-pasta is very difficult to read. Can you sent it without ... > re-formatting or whatever else is going on with your email sender? > > Your error does not contain the "real error". If there are 50 lines of > stack trace, please post them ALL. Otherwise it will not be possible to > tell what the root problem is. > > You have a runtime error above and a compiler error below. It's tough to > follow your journey, here. > > Definitely do not put j2ee.jar into your web application's WEB-INF/lib > directory. > > > Using the classpath without j2ee.jar: > > CLASSPATH=.;c:\tomcat4\webapps\hck\WEB-INF\classes;c:\tomcat4\lib\the > > compilation of the hck app fails early > > with:javac > > *.javaZAjax.java:9: error: cannot find symbolpublic class ZAjax extends > > HttpServlet { ^
Re: Win10 installation progress
Dear Christopher, Link to catalina log is below Thanks for all your advice.Progress here.The port regards hck app (for medical diagnosis) with 6.1Mb source ofwhich 4.6Mb java code.Have been able to compile without using j2ee.jar (with a dos .batscript tackling 19 directories, etc.)Used instead in CLASSPATH files in lib: servlet-api.jar,javax.mail.jar,javax.activation.jar. There IS a web.xml file in WEB-INF with, among the others, forcom.ZAjax: com.ZAjax com.ZAjax com.ZAjax com.ZAjax Starting up tomcat 9.0.88 still hiccups when hck is encountered.The catalina log is at: https://www.ontooo.com/catalina.log Seems to be close ... :-)))dennis de champeaux [[ Next week will present a paper 'Medical Diagnosis Algorithm at a conference. :--]] -- Home page: rs6.risingnet.net/~ddccMarketing site: www . OntoOO.comKindle books: Side Effects: Impacts on the 21st Century::https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09MHJ5W48 Even if not true::https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09Y4WR9J7 9/11:: They had the Sun in their Eyeshttps://www.amazon.com/11-They-had-their-Eyes-ebook/dp/B0CMQ2WSK9https://www.amazon.nl/dp/B0CMQ2WSK9 On Tuesday, July 16, 2024 at 02:18:17 PM PDT, Chuck Caldarale wrote: > On Jul 16, 2024, at 15:57, DdC wrote: > > Dear Christopher, > > Thanks for all your advice. > Progress here. > The port regards hck app (for medical diagnosis) with 6.1Mb source of > which 4.6Mb java code. > Have been able to compile without using j2ee.jar (with a dos .bat > script tackling 19 directories, etc.) > Used instead in CLASSPATH files in lib: > servlet-api.jar, > javax.mail.jar, > javax.activation.jar. > There IS a web.xml file in WEB-INF with, among the others, for > com.ZAjax: > > > com.ZAjax > com.ZAjax > > > > com.ZAjax > com.ZAjax > > > Starting up tomcat 9.0.88 still hiccups when hck is encountered. > The catalina log is attached. Attachments are stripped by the mailing list handler; could you post the log directly into the message (if not too big), or at some publicly accessible location? - Chuck > Seems to be close ... :-))) > dennis de champeaux > > [[ Next week will present a paper 'Medical Diagnosis Algorithm at a > conference. :--]] > > -- > > Home page: rs6.risingnet.net/~ddcc > Marketing site: www . OntoOO.com > Kindle books: > Side Effects: Impacts on the 21st Century:: > https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09MHJ5W48 > Even if not true:: > https://www.amazon.com/-/es/Dennis-Champeaux-ebook/dp/B09Y4WR9J7 > 9/11:: They had the Sun in their Eyes > https://www.amazon.com/11-They-had-their-Eyes-ebook/dp/B0CMQ2WSK9 > https://www.amazon.nl/dp/B0CMQ2WSK9 > > On Tuesday, July 9, 2024 at 06:29:41 AM PDT, Christopher Schultz > wrote: > > > Dennis, > > On 7/8/24 17:05, DdC wrote: > > Still struggling launching my hck app on Win10 & tomcat 9.0.88.It > > runs (dont laugh) on: XP/Win7 & tomcat 4.0.4 Linux-gnu & tomcat > > 6.0.32 > Tomcat 9 ought to be able to run a Servlet-2.0-era application just fine > unless you use one or two things that have been deprecated and won't > work anymore (20 years later). > > > Tomcat 9.0.88 displays fine with localhost:8080. > > Do you mean that the ROOT application ("Welcome to Tomcat") runs? I > wouldn't expect anything less. > > What version of Java are you using, now? Hopefully it's at least 17 or 21. > > > App hck compiles OK (dont laugh) with a script > > using:CLASSPATH=.;c:\tomcat4\j2ee.jar;c:\tomcat4\webapps\hck\WEB-INF\classes;c:\tomcat4\lib\ > > This isn't a script. Can you post the whole thing? > > > Starting subsequently tomcat4 hiccups with: > JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED > > --add-opens=java.base/java.io=ALL-UNNAMED > > --add-opens=java.base/java.util=ALL-UNNAMED > > --add-opens=java.base/java.util.concurrent=ALL-UNNAMED > > --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED08-Jul-2024 12:00:24.731 > > INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server > > version name: Apache Tomcat/9.0.8808-Jul-2024 12:00:24.748 INFO [main] > > org.apache.catalina.startup.VersionLoggerListener.log Server built: > > Apr 9 2024 13:22:30 UTC08-Jul-2024 12:00:24.748 INFO [main] > > org.apache.catalina.startup.VersionLoggerListener.log Server version > > number: 9.0.88.008-Jul-2024 12:00:24.748 INFO > > [main]org.apache.catalina.startup.VersionLoggerListener.log OS Name:Windows > > 10..08-Jul-2024 12:00:27.313 INFO [main] > > org.apache.catalina.startup.HostConfig.deployDirectory Deploying web > > application directory [C:\tomcat4\webapps\hck]08-Jul-2024 12:00:30.041 > > SEVERE [main] org.apache.catalina.startup.HostConfig.deployDirectory Error > > deploying web applica