On 09/29/2013 04:03 PM, adfel70 wrote:
How can I configure the disk storage so that disk access is optimized? I'm considering having RAID-10 and I think I'll have arround 4-8 disks per machine. Should I run each solr jvm to point on a datadir on differnet disks, or is there some other way to optimize this?
Best way to deal with this, is trial and error. There are many factors that can contribute to your hardware decisions. Will there be concurrent access on all solr instances? Will some be used more than others?
If you have a couple of highly used and many seldom used instances, then there's no problem in running in each in a different JVM. If you have more highly used instances/JVMs than CPU cores...you're in trouble.
Are you doing real time search? Or is the data mostly static? If the data doesn't change much, then good warming-up queries will be a lot more useful than trying to tie solr to specific disks.
If you're doing real time on a 5TB index then you'll probably want to throw your money at the fastest storage you can afford (SSDs vs spinning rust made a huge difference in our benchmarks) and the fastest CPUs you can get your hands on. Memory is important too, but in our benchmarks that didn't have as much impact as the other factors. Keeping a 5TB index in memory is going to be tricky, so in my opinion you'd be better off investing in faster disks instead.
- Bram
