On Tue, Nov 27, 2012 at 03:01:57PM +0100, Jonathan Aquilina wrote: > One thing that really caught my eye was seamicros new storage servers up to > 5 petabytes of storage
Please do not top-post and please trim your replies. Message below unchanged for illustration. The Seamicro is not the same thing, since I would be looking at least for an Exynos 5 with 1-2 GByte RAM and some GByte flash for each SATA spindle. You would not just run a distributed filesystem or even a database, but to process code locally that uses 1-3 TByte worth of data on said spindle, with some SSD/RAM caching. If you're moving a sliding window across a 3d voxelset where you do some processing, then you would likely to be able to stream at 100-200 MByte/s, and process your TByte/node data within nongeological time spans. There would be very little traffic on the GBit port, as you would only query adjacent nodes when you need to get a slice at the surface of the cube. In principle this would map very well to a 3d torus, but GBit Ethernet for many 10 k nodes is reasonably cheap. > On Tue, Nov 27, 2012 at 2:59 PM, Eugen Leitl <eu...@leitl.org> wrote: > > > On Tue, Nov 27, 2012 at 09:10:32AM +0100, Jonathan Aquilina wrote: > > > Hey guys I was looking at the hadoop page and it got me wondering. is it > > > possible to cluster together storage servers? If so how efficient would a > > > cluster of them be? > > > > An interesting problem would be to use reasonably powerful but > > cheap ARM SoCs in few GBytes onboard RAM and some flash > > for hybrid filesystems for each hard drive, and cluster > > them via GBit Ethernet on a very large scale. > > > > That would be a custom Beowulf for more storage-related > > tasks. E.g. an application I have in mind are volumetric > > datasets with e.g. 8 nm - voxels for biological systems, > > which are way too large to process in memory. > > _______________________________________________ > > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > > To change your subscription (digest mode or unsubscribe) visit > > http://www.beowulf.org/mailman/listinfo/beowulf > > > > > > -- > Jonathan Aquilina -- Eugen* Leitl <a href="http://leitl.org";>leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
