On 7/30/2013 11:22 PM, Dotan Cohen wrote: > I see, thanks. I thought that 'disk cache' was something on disk, such > as swap space. The server is already maxed out on RAM: > $ free -m > total used free shared buffers > cached > Mem: 14980 14906 73 0 167 > 5293 > -/+ buffers/cache: 9444 5535 > Swap: 0 0 0
This shows a total RAM of 15GB. You have about 5GB of memory in your disk cache (the 5293 value). The rest is used by programs, most likely the Solr jvm heap. This means that if your index is considerably larger than about 10GB, you don't have enough RAM for good Solr performance. Looking back over the email thread, you've said your index is 95GB. This means that you'll want at least 40GB for your disk cache, and more is always better. If you doubled the overall system RAM (to 30GB), performance would be much better, though it might not be stellar. If you are happy with your current performance levels for queries other than your dupe facet, then you'd be absolutely amazed at the performance if you had enough RAM. A large Solr install like this on Amazon can get very expensive, because it's such a RAM-hungry beast. If you take into account the need for redundancy, the cost really skyrockets. I just read read your latest message where you talk about performance slowdown going from 4.1 to 4.3. It is likely that you were on the verge of having those performance problems on 4.1 as well, you just hadn't pushed over that line yet. A common phenomenon for performance issues is that everything seems fine, then when you cross some invisible threshold, performance goes WAY down. Thanks, Shawn
