On 01/09/2013 08:27 AM, Vincent Diepeveen wrote: > What would be a rather interesting thought for building a single box > dirt cheap with huge 'RAM' > is the idea of having 1 fast RAID array of SSD's function as the 'RAM'.
This may be a more inexpensive route, but let's all note that the raw latency differences between DDR2/3 RAM and /any/ SSD is multiple orders of magnitude. So for a single threaded application that has been asked to run on all RAM, I have a strong suspicion that RAM latencies are what it really does need -- not just reasonable latency and high throughput. But we should await Jorg's response on the application nature to better flesh that out. > You can get a bandwidth of 2 GB/s then to the SSD's "RAM" pretty > easily, yet for some calculations that bandwidth might be enough > given the fact you can then parallelize a few cores. I am at a loss as to how you can achieve that high of bandwidth "pretty easily." In the /absolute/ best case a single SATA SSD can serve reads at close to 400-500MB/s, and software RAIDing them will definitely not get you the 4x you speak of. Moreover, most hardware RAID cards aren't built for that level of IOP/s or bandwidth since they usually have HDDs strapped to them, so you are going to have to get a very pricey HW RAID controller to try and achieve this. And whether or not you are bottlenecked at the end of the day after the RAID controller is beyond my expertise. Does anyone have ideas/suggestions on good RAID controllers for SSD specific workloads? To get anywhere near the aforementioned number, you are likely going to have to drop a pretty penny on a Fusion-IO or Virident PCI-Express Flash Device -- not buy just any SSD and expect to take their quoted numbers and 4x them (much less get the quoted numbers). I suspect going the RAM route may be comparably costly at this point. > The random latency to such SSD raid might be 70 microseconds on > average, maybe even faster with specific SSD's. Random read maybe -- certainly not random write latency. Again, it's probably best to wait on Jorg to comment on the nature of the application to decide whether we should care about read or write latency more. > One might need to optimize which file system gets used in such case > and the way how to access files - as one probably wants to > avoid global locks that avoid several requests pending simultaneously > to the SSD's. You could instead just mmap the flash device directly if you really just want to use it truly like 'RAM.' Optimizing filesystems is entirely nontrivial. Best, ellis _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
