On 2/9/11 1:21 PM, xingqiu yuan wrote: > > > On Wed, Feb 9, 2011 at 2:37 PM, Prentice Bisbal <prent...@ias.edu > <mailto:prent...@ias.edu>> wrote: > > Craig Tierney wrote: > > On 2/9/11 7:38 AM, Prentice Bisbal wrote: > > > I don't think Numascale/ScaleMP has much of a cost advantage anymore. > > > > > > About 6 months ago, I purchased a couple of Dell PowerEdge R815s with > > > 128 GB of RAM and 32 cores. We looked at similar RAM configurations a > > > couple of years ago. and the cost premium for that much RAM was > > > prohibitive (the price for additional RAM seemed to go up > exponentially > > > with the amount of RAM) so we stayed at 32 GB. This time around, the > > > premium for the additional RAM seemed marginal. Not sure how > large you > > > can go and keep that "marginal" relationship, but it's much > larger than > > > it was a couple of years ago. > > > > > > > There is only so much memory you can put in a box, regardless of the > > cost. ScaleMP/Numascale let you get around that issue. What if I > need > > a TB of RAM? Yes, I might be able to convert my algorithms, but > if I can > > add $3k per node to get cache-coherent memory, why not go that route? > > >Agreed. I was going to add the caveat that there are limits to how much > >RAM you can add to a single box, or there eventually be a point the > >amount of RAM in a single box vs. price is no longer nonlinear, but then > >I got lazy. ;) > > > > Don't understand why you need TB of RAM? Don't forget the fact that $3K > can almost buy one-node!!! > >
Reasons for a TB of RAM include databases, a lot of users that need 10's of GB, but also having the additional cores provides some flexibility. I can't say for certain a setup like this would be beneficial, but at $3k per node, I would be willing to try it out! You are right about the node price. Well, I would put the cost much higher. You could put 48GB in a node without much of a price penalty, and then you are probably paying $5-6K per node. Also, the $3k/node is list and who ever pays list? :-) Craig > > > > > > > > And don't forget to figure in the cost of the interconnect. I > don't know > > > what Numascale requires, but ScaleMP require IB, which can add > > > significant costs if you don't have an existing IB fabric to use. > > > > > > > ScaleMP is a software solution, Numascale is a cache-coherent > hardware solution. > > The list cost of the 3D torus version Numascale card with 4GB Ram > (highest end card). > > Cables are similar to IB, about $120 each. > > > > It would be really cool to build SMP system to sit next to my > cluster. When users > > need large memory, or want to do things with UPC or other similar > languages, > > an SMP system like this would be handy. > > > > - I am not advocating for Numascale, or any other products here. > I am just > > trying to provide information to the community. > > > > Craig > > > > > > > > > Prentice > > > > > > > > > Douglas Eadline wrote: > > >> Although I wrote about the SMP and the Numascale hardware, > > >> I have not yet had a chance to use it. > > >> > > >> To me there are two issues worth considering. First, > > >> if you need a big SMP this might be a low > > >> cost solution. Second, should your application > > >> not scale best in a node-based ccNuma system (ScaleMP > > >> or Numascale), MPI is still an option. Indeed, no need > > >> to rewrite the codes. And, management is probably > > >> much easier. > > >> > > >> Of course for large systems, clusters work best. > > >> > > >> -- > > >> Doug > > >> > > >> > > >>> On Thu, 2011-02-03 at 09:45 +0000, Hearns, John wrote: > > >>>> Also look at ScaleMP and Numascale > > >>>> Here's a damn good article from Doug Eadline: > > >>>> http://www.linux-mag.com/id/7947 > > >>>> > > >>>> I must admit though I don't know how far a budget of 30K takes you > > >>>> there! > > >>> My understanding is that the Numascale cards cost "a bit more" > than IB > > >>> cards but you don't need a switch (it's a 3D taurus I think) so > it would > > >>> be worth looking into. > > >>> > > >>> Rob > > >>> > > > _______________________________________________ > > > Beowulf mailing list, Beowulf@beowulf.org > <mailto:Beowulf@beowulf.org> sponsored by Penguin Computing > > > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > _______________________________________________ > > Beowulf mailing list, Beowulf@beowulf.org > <mailto:Beowulf@beowulf.org> sponsored by Penguin Computing > > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > > -- > Prentice Bisbal > Linux Software Support Specialist/System Administrator > School of Natural Sciences > Institute for Advanced Study > Princeton, NJ > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > <mailto:Beowulf@beowulf.org> sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
