On 1/22/24 11:38 AM, Scott Atchley wrote:
On Mon, Jan 22, 2024 at 11:16 AM Prentice Bisbal <pbis...@pppl.gov> wrote:

    <snip>

        > Another interesting topic is that nodes are becoming
        many-core - any
        > thoughts?

        Core counts are getting too high to be of use in HPC. High
        core-count
        processors sound great until you realize that all those cores
        are now
        competing for same memory bandwidth and network bandwidth,
        neither of
        which increase with core-count.

        Last April we were evaluating test systems from different
        vendors for a
        cluster purchase. One of our test users does a lot of CFD
        simulations
        that are very sensitive to mem bandwidth. While he was
        getting a 50%
        speed up in AMD compared to Intel (which makes sense since
        AMDs require
        12 DIMM slots to be filled instead of Intel's 8), he asked us
        consider
        servers with LESS cores. Even with the AMDs, he was
        saturating the
        memory bandwidth before scaling to all the cores, causing his
        performance to plateau. For him, buying cheaper processors
        with lower
        core-counts was better for him, since the savings would allow
        us to by
        additional nodes, which would be more beneficial to him.


    We see this as well in DOE especially when GPUs are doing a
    significant amount of the work.

    Yeah, I noticed that Frontier and Aurora will actually be
    single-socket systems w/ "only" 64 cores.

 Yes, Frontier is a *single* *CPU* socket and *four GPUs* (actually eight GPUs from the user's perspective). It works out to eight cores per Graphics Compute Die (GCD). The FLOPS ratio is roughly 1:100 between the CPU and GPUs.

Note, Aurora is a dual CPU and six GPU. I am not sure if the user sees six or more GPUs. The Aurora node is similar to our Summit node but with more connectivity between the GPUs.

Thanks for clarfying! I thought it was a  single-CPU system like Frontier. Not only is the FLOPS ratio much higher on GPUs, so if the FLOPS/W ratio. Even though CPUs have gotten much more efficient lately, it's practically stagnant compared to GPU-based clusters, based on my analysis of the Top500 and Green500 trends.

Prentice
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit 
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf

Reply via email to