Re: OOM during indexing nested docs

2014-06-25 Thread Tang, Rebecca
How big is your request size from client to server? I ran into OOM problems too. For me the reason was that I was sending big requests (1+ docs) at too fast a pace. So I put a throttle on the client to control the throughput of the request it sends to the server, and that got rid of the OOM e

Re: OOM during indexing nested docs

2014-06-25 Thread adfel70
I made two tests, one with MaxRamBuffer=128 and the second with MaxRamBuffer=256. In both i got OOM. I also made two tests on autocommit: one with commit every 5 min, and the second with commit every 100,000 docs. (disabled softcommit) In both i got OOM. merge policy - Tiered (max segment size o

Re: OOM during indexing nested docs

2014-06-24 Thread Mikhail Khludnev
enable heap dump on OOME, and build the histogram by jhat. Did you try to reduce MaxRamBuffer or max buffered docs? or enable autocommit? On Tue, Jun 24, 2014 at 7:43 PM, adfel70 wrote: > Hi, > > I am getting OOM during indexing 400 million docs (nested 7-20 children). > The memory usage gets h