Re: hadoop inserts blow out heap

2012-09-14 Thread aaron morton
Hi Brian did you see my follow up questions here http://www.mail-archive.com/user@cassandra.apache.org/msg24840.html Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 12/09/2012, at 11:52 PM, Brian Jeltema wrote: > I'm a fairly novice Cass

Re: hadoop inserts blow out heap

2012-09-13 Thread aaron morton
What version of Cassandra are you using ? > 2.9G of the heap is consumed by a JMXConfigurableThreadPoolExecutor that > appears to be full of batch mutations. That sounds like writes were backing up and/or blocked for some reason. Check the logs for errors. Notetool tpstats will show you if muta

Re: hadoop inserts blow out heap

2012-09-13 Thread Brian Jeltema
I didn't get a response to this, so I'll give it another shot. I tweaked some parameters and cleaned up my schema. My Hadoop/Cassandra job got further, but still dies with an OOM error. This time, the heap dump displays a JMXConfigurableThradPoolExecutor with a retained heap of 7.5G. I presume th

hadoop inserts blow out heap

2012-09-12 Thread Brian Jeltema
I'm a fairly novice Cassandra/Hadoop guy. I have written a Hadoop job (using the Cassandra/Hadoop integration API) that performs a full table scan and attempts to populate a new table from the results of the map/reduce. The read works fine and is fast, but the table insertion is failing with OOM