RE: JVM heap constraints and garbage collection

2014-02-03 Thread Michael Della Bitta
> i2.xlarge looks vastly better than m2.2xlarge at about the same price, so I must be missing something: Is it the 120 IPs that explains why anyone would choose m2.2xlarge? i2.xlarge is a relatively new instance type (December 2013). In our case, we're partway through a yearlong reservation of m2.

RE: JVM heap constraints and garbage collection

2014-02-01 Thread Toke Eskildsen
Michael Della Bitta [michael.della.bi...@appinions.com] wrote: > Here at Appinions, we use mostly m2.2xlarges, but the new i2.xlarges look > pretty tasty primarily because of the SSD, and I'll probably push for a > switch to those when our reservations run out. > http://www.ec2instances.info/ i2.

Re: JVM heap constraints and garbage collection

2014-01-31 Thread Erick Erickson
Be a little careful when looking at on-disk index sizes. The *.fdt and *.fdx files are pretty irrelevant for the in-memory requirements. They are just read to assemble the response (usually 10-20 docs). That said, you can _make_ them more relevant by specifying very large document cache sizes. Bes

Re: JVM heap constraints and garbage collection

2014-01-31 Thread Michael Della Bitta
Joesph: Not so much after using some of the settings available on Shawn's Solr Wiki page: https://wiki.apache.org/solr/ShawnHeisey This is what we're running with right now: -Xmx6g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80 Michael Della Bitta Applications Developer o: +1

Re: JVM heap constraints and garbage collection

2014-01-31 Thread Joseph Hagerty
Thanks, Shawn. This information is actually not all that shocking to me. It's always been in the back of my mind that I was "getting away with something" in serving from the m1.large. Remarkably, however, it has served me well for nearly two years; also, although the index has not always been 30GB,

Re: JVM heap constraints and garbage collection

2014-01-31 Thread Michael Della Bitta
Here at Appinions, we use mostly m2.2xlarges, but the new i2.xlarges look pretty tasty primarily because of the SSD, and I'll probably push for a switch to those when our reservations run out. http://www.ec2instances.info/ Michael Della Bitta Applications Developer o: +1 646 532 3062 appinions

Re: JVM heap constraints and garbage collection

2014-01-30 Thread Shawn Heisey
On 1/30/2014 3:20 PM, Joseph Hagerty wrote: I'm using Solr 3.5 over Tomcat 6. My index has reached 30G. - The box is an m1.large on AWS EC2. 2 virtual CPUs, 4 ECU, 7.5 GiB RAM One detail that you did not provide was how much of your 7.5GB RAM you are allocating to the Java heap for Solr,

JVM heap constraints and garbage collection

2014-01-30 Thread Joseph Hagerty
Greetings esteemed Solr-ites, I'm using Solr 3.5 over Tomcat 6. My index has reached 30G. Since my average load during peak hours is becoming quite high, and since I'm finally starting to notice a little bit of performance degradation and intermittent errors (e.g. "Solr returned response 0" on pe