Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-17 Thread Christopher Schultz
Konstantin, On 7/17/14, 7:58 AM, Konstantin Kolinko wrote: > 2014-07-15 23:43 GMT+04:00 Christopher Schultz : >> Mark, >> >> (...) >> >> We have had situations /in production/ where a "single thread" has >> triggered an OOME .. in a particular case, that thread was trying to >> pull an unbounded s

Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-17 Thread Konstantin Kolinko
2014-07-15 23:43 GMT+04:00 Christopher Schultz : > Mark, > > (...) > > We have had situations /in production/ where a "single thread" has > triggered an OOME .. in a particular case, that thread was trying to > pull an unbounded set of results back from a database query and cache > them in a local

Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-15 Thread Christopher Schultz
Mark, On 7/15/14, 12:44 PM, Mark Thomas wrote: > On 14 July 2014 21:58:22 CEST, Christopher Schultz > wrote: >> On 7/14/14, 12:18 PM, Mark Thomas wrote: > 2. If the exception is VirtualMachineError, it gets re-thrown with no log. This skips the "log" call in the above code and so the

Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-15 Thread Mark Thomas
On 14 July 2014 21:58:22 CEST, Christopher Schultz wrote: >On 7/14/14, 12:18 PM, Mark Thomas wrote: >>> 2. If the exception is VirtualMachineError, it gets re-thrown with >>> no log. This skips the "log" call in the above code and so the only >>> log will come from the VM's "unhandled exception"

Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-15 Thread Konstantin Kolinko
2014-07-14 23:58 GMT+04:00 Christopher Schultz : > Mark, > > On 7/14/14, 12:18 PM, Mark Thomas wrote: >>> ExceptionUtils.handleException checks for two things that I think >>> we might want to change: >>> >>> 1. If the exception is StackOverflowError, it silently ignores the >>> error and continues

Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-14 Thread Christopher Schultz
Mark, On 7/14/14, 12:18 PM, Mark Thomas wrote: >> ExceptionUtils.handleException checks for two things that I think >> we might want to change: >> >> 1. If the exception is StackOverflowError, it silently ignores the >> error and continues. I think we should at least log something, >> probably at

Re: ContainerBackgroundProcessor and compounding OOMEs

2014-07-14 Thread Mark Thomas
On 14/07/2014 17:01, Christopher Schultz wrote: > All, > > It seems that ContainerBackgroundProcessor can die and end up > silently destabilizing things. It is only silent if the system admin has done something stupid like redirecting stderr to /dev/null. Frankly, if they do that they deserve eve