Re: Optimize fails with OutOfMemory Exception - sun.nio.ch.FileChannelImpl.map involved

2011-09-22 Thread Michael McCandless
Unfortunately I really don't know ;) Every time I set forth to figure things like this out I seem to learn some new way... Maybe someone else knows? Mike McCandless http://blog.mikemccandless.com On Thu, Sep 22, 2011 at 2:15 PM, Shawn Heisey wrote: > Michael, > > What is the best central plac

Re: Optimize fails with OutOfMemory Exception - sun.nio.ch.FileChannelImpl.map involved

2011-09-22 Thread Shawn Heisey
Michael, What is the best central place on an rpm-based distro (CentOS 6 in my case) to raise the vmem limit for specific user(s), assuming it's not already correct? I'm using /etc/security/limits.conf to raise the open file limit for the user that runs Solr: ncindex hardnofile

Re: Optimize fails with OutOfMemory Exception - sun.nio.ch.FileChannelImpl.map involved

2011-09-22 Thread Michael McCandless
OK, excellent. Thanks for bringing closure, Mike McCandless http://blog.mikemccandless.com On Thu, Sep 22, 2011 at 9:00 AM, Ralf Matulat wrote: > Dear Mike, > thanks for your your reply. > Just a couple of minutes we found a solution or - to be honest - where we > went wrong. > Our failure was

Re: Optimize fails with OutOfMemory Exception - sun.nio.ch.FileChannelImpl.map involved

2011-09-22 Thread Ralf Matulat
Dear Mike, thanks for your your reply. Just a couple of minutes we found a solution or - to be honest - where we went wrong. Our failure was the use of ulimit. We missed, that ulimit sets the vmem for each shell seperatly. So we set 'ulimit -v unlimited' on a shell, thinking that we've done the

Re: Optimize fails with OutOfMemory Exception - sun.nio.ch.FileChannelImpl.map involved

2011-09-22 Thread Michael McCandless
Are you sure you are using a 64 bit JVM? Are you sure you really changed your vmem limit to unlimited? That should have resolved the OOME from mmap. Or: can you run "cat /proc/sys/vm/max_map_count"? This is a limit on the total number of maps in a single process, that Linux imposes. But the de

Optimize fails with OutOfMemory Exception - sun.nio.ch.FileChannelImpl.map involved

2011-09-22 Thread Ralf Matulat
Good morning! Recently we slipped into an OOME by optimizing our index. It looks like it's regarding to the nio class and the memory-handling. I'll try to describe the environment, the error and what we did to solve the problem. Nevertheless, none of our approaches was successful. The environm