My experience with "OutOfMemoryError: unable to create new native thread" as follows: it occurs on envs where devs refuse to use threadpools in favor of old good new Thread(). Then, it turns rather interesting: If there are plenty of heap, GC doesn't sweep Thread instances. Since they are native in Java, every of them hold some ram for native stack. That exceeds stack space at some point of time. So, check how many thread JVM hold after this particular OOME occurs by jstack; you can even force GC to release that native stack space. Then, rewrite the app, or reduce heap to enforce GC.
On Tue, Dec 10, 2019 at 9:44 AM Shawn Heisey <apa...@elyograg.org> wrote: > On 12/9/2019 2:23 PM, Joe Obernberger wrote: > > Getting this error on some of the nodes in a solr cloud during heavy > > indexing: > > <snip> > > > Caused by: java.lang.OutOfMemoryError: unable to create new native thread > > Java was not able to start a new thread. Most likely this is caused by > the operating system imposing limits on the number of processes or > threads that a user is allowed to start. > > On Linux, the default limit is usually 1024 processes. It doesn't take > much for a Solr install to need more threads than that. > > How to increase the limit will depend on what OS you're running on. > Typically on Linux, this is controlled by /etc/security/limits.conf. If > you're not on Linux, then you'll need to research how to increase the > process limit. > > As long as you're fiddling with limits, you'll probably also want to > increase the open file limit. > > Thanks, > Shawn > -- Sincerely yours Mikhail Khludnev
