Re: min_compaction_threshold/max_compaction_threshold

2012-11-03 Thread Vivek Mishra
" ,"IS_EXCEPTIONAL" ,"AGE" ,"SEMESTER" ,"DIGITAL_SIGNATURE" ,"CGPA" ,"PERCENTAGE" ,"HEIGHT" ,"ENROLMENT_DATE" ,"ENROLMENT_TIME" ,"JOINING_DATE_TIME" ,"YEARS_SPENT" ,"ROLL_NUMBER&

Re: max_compaction_threshold

2011-11-09 Thread Radim Kolar
i consulted with hadoop expert and he told me that he is using value 100 for merging segments. I will rerun tests with 100 to check.

max_compaction_threshold

2011-11-09 Thread Radim Kolar
I found in stress tests that default setting this to 32 is way too high. Hadoop guys are using value 10 during merge sorts to not stress IO that much. I also discovered that filesystems like ZFS are using default io queue size of 10 per drive. I tried run tests with 10, 15 and 32 and there is