On 9/24/2014 2:18 AM, Toke Eskildsen wrote: > Norgorn [lsunnyd...@mail.ru] wrote: >> I have CLOUD with 3 nodes and 16 MB RAM on each. >> My index is about 1 TB and search speed is awfully bad. > > We all have different standard with regards to search performance. What is > "awfully bad" and what is "good enough" for you? > > Related to this: How many documents are in your index, how do you query > (faceting, sorting, special searches) and how often is an index performed? > >> I've read, that one needs at least 50% of index size in RAM, > > That is the common advice, yes. The advice is not bad for some use cases. The > problem is that it has become gospel. > > I am guessing that you are using spinning drives? Solr needs fast random > access reads and spinning drives are very slow for that. You can either > compensate by buying enough RAM or you can change to a faster underlying > storage technology. The obvious choice these days are Solid State Drives (we > bought Samsung 840 EVO's last time and would probably buy those again). They > will not give you RAM speed, but they do give a lot more bang for the buck > and depending on your performance requirements they can be enough.
I am guilty of spreading the "gospel" that you need 50-100% of your index to fit in the OS disk cache, as Toke mentioned. This wiki page is my creation: http://wiki.apache.org/solr/SolrPerformanceProblems I've seen decent performance out of systems with standard hard disks that only had enough RAM to fit about 25% of the index into the disk cache, but I've also seen systems with 50% that can't complete a simple query in less than 10 seconds. With a terabyte of index on the system (assuming that's how much is on each one), 25% is still at least 256GB of RAM. With only 16GB, there's simply no way you'll ever get good performance. I've heard quite a lot of anecdotal evidence that if you put the index on SSD, you only need 10% of the index to fit in RAM. I'm a little bit skeptical that this would be true as a general rule, but I do not doubt that it's been done successfully. For a terabyte index, that's still 100GB of RAM, so 128GB would be the absolute minimum that you'll want to consider. The more RAM you can throw at this problem, the better your performance will be. Thanks, Shawn
