Hello. Recently we received complaints from a customer about slow response times (45 seconds) every morning. Every morning starting a fresh session in the browser would require a long wait time for the first user, but once this hurdle had been taken everything seemed to work fine. We could reproduce this in house, and doing a thread dump during this time span showed the class loader trying to open a jar file. Immediately we suspected a virus scanner slowing access to jar files (and since signatures are updated daily, this happened daily). Blocking the virus scanner from the tomcat folder helped but this is just alleviating the symptoms, not the cure. Doing a full trace log of the class loader package showed the problem: For any unknown resource or class the class loader seems to populate all known jars, and either finds it, then loads it, and keeps a cache entry (with the class for classes), or if it can’t find it, asks the parent class loader (this is the spec order of things). The log was filled classes and resources that couldn’t be found and were delegated to the parent. Over and over again. Each and every one of theses requests scans all jar files. Clear the OS cache and the problem appears, too.
For classes one would believe this cannot be, once a class is loaded, it’s loaded. But with dynamic script languages, they often use Class.forName() and for example Rhino even does that for packages (that might not even exist!). We fixed at least Rhino with an application wide global top level , so for classes this problem did go away (there might be other, unless they implement their own caching, there is a risk). Resources are still a problem, our log is filled with 24-Sep-2019 17:22:55.482 FINE [http-nio-8080-exec-6] org.apache.catalina.loader.WebappClassLoaderBase.findResources findResources(META-INF/services/javax.xml.parsers.DocumentBuilderFactory) 24-Sep-2019 17:22:55.484 FINE [http-nio-8080-exec-6] org.apache.catalina.loader.WebappClassLoaderBase.getResourceAsStream getResourceAsStream(META-INF/services/org.apache.xerces.xni.parser.XMLParserConfiguration) 24-Sep-2019 17:22:55.484 FINE [http-nio-8080-exec-6] org.apache.catalina.loader.WebappClassLoaderBase.getResourceAsStream Searching local repositories 24-Sep-2019 17:22:55.485 FINE [http-nio-8080-exec-6] org.apache.catalina.loader.WebappClassLoaderBase.getResourceAsStream Delegating to parent classloader unconditionally java.net.URLClassLoader@578486a3 24-Sep-2019 17:22:55.485 FINE [http-nio-8080-exec-6] org.apache.catalina.loader.WebappClassLoaderBase.getResourceAsStream --> Resource not found, returning null If there is only a single repeated “Searching local repositories” for a certain URL, the problem is there (doesn’t matter if the resource or class exists or not). It’s obviously a bad idea to scan the filesystem or jars every time. The WebappClassLoaderBase keeps information about every resource and class it finds, so repeated calls for these do not cost that much time, especially for classes that are stored in the cache. I believe it might work to keep a negative entry for classes it doesn’t find but the parent class loader knows and immediately ask the parent class loader. Maybe also for resources. But I don’t know what is supposed to happen if somebody overwrites a resource (or class) later. Even worse if the class/resource does not exist yet. Maybe populating the whole resource tree in memory is the only solution (with background update?!) . I mainly wanted to raise awareness of this problem, I am not well versed with class loading, so I can’t really help there. Sincerely, Gunnar Brand Gunnar Brand | Softwareentwickler interface projects GmbH | www.intergator.de Zwinglistraße 11/13 | 01277 Dresden | Deutschland Tel: +49-351-31809-41 | Fax: +49-351-31809-33 Folgen Sie intergator auf: Twitter : https://twitter.com/intergator xing : https://www.xing.com/companies/interfaceprojectsgmbh LinkedIn: http://www.linkedin.com/company/interface-projects-gmbh YouTube : http://www.youtube.com/user/TheEnterpriseSearch?feature=watch Facebook: https://www.facebook.com/intergator/ RSS : https://www.intergator.de/feed/ Geschäftsführer: Dr. Uwe Crenze, Eduard Daoud Amtsgericht Dresden HRB 8740 Haftungsausschluss / Disclaimer http://www.intergator.de/email-disclaimer.shtml
