From: Tim Terlegård [tim.terleg...@gmail.com] > If the index size is more than you can have in RAM, do you recommend > to split the index to several servers so it can all be in RAM? > > I do expect phrase queries. Total index size is 107 GB. *prx files are > total 65GB and *frq files 38GB. It's probably worth buying more RAM.
Have you considered throwing one or more SSD's at the problem? Intel X25-M G2 (or X25-E if you're dictated by your organization to buy enterprise level) is my personal favorite right now. They are, compared to RAM or even high-end spinning harddrives, often quite cost-effective. Most SSD's has random access time for reads at about 0.1ms. For us that meant that we moved the bottleneck for a 70GB index (10 million documents) from IO to CPU on a quad-core machine. We tried testing SSD vs. RAMDirectory and found it to perform at about 75% speed for a 14GB subset of the index. - Toke Eskildsen - http://statsbiblioteket.dk
