On 2/8/2014 1:40 AM, samarth s wrote: > Yes it is amazon ec2 indeed. > > To expqnd on that, > This solr deployment was working fine, handling the same load, on a 34 GB > instance on ebs storage for quite some time. To reduce the time taken by a > commit, I shifted this to a 30 GB SSD instance. It performed better in > writes and commits for sure. But, since the last week I started facing this > problem of infinite back to back commits. Not being able to resolve this, I > have finally switched back to a 34 GB machine with ebs storage, and now the > commits are working fine, though slow.
The extra 4GB of RAM is almost guaranteed to be the difference. If your index continues to grow, you'll probably be having problems very soon even with 34GB of RAM. If you could put it on a box with 128 to 256GB of RAM, you'd likely see your performance increase dramatically. Can you share your solrconfig.xml file? I may be able to confirm a couple of things I suspect, and depending on what's there, may be able to offer some ideas to help a little bit. It's best if you use a file sharing site like dropbox - the list doesn't deal with attachments very well. Sometimes they work, but most of the time they don't. I will reiterate my main point -- you really need a LOT more memory. Another option is to shard your index across multiple servers. This doesn't actually reduce the TOTAL memory requirement, but it is sometimes easier to get management to agree to buy more servers than it is to get them to agree to buy really large servers. It's a paradox that doesn't make any sense to me, but I've seen it over and over. Thanks, Shawn
