From: Beowulf <beowulf-boun...@beowulf.org> on behalf of Oddo Da <oddodao...@gmail.com> Date: Wednesday, October 14, 2020 at 11:09 AM To: Michael Di Domenico <mdidomeni...@gmail.com> Cc: "beowulf@beowulf.org" <beowulf@beowulf.org> Subject: [EXTERNAL] Re: [Beowulf] ***UNCHECKED*** Re: Spark, Julia, OpenMPI etc. - all in one place
On Wed, Oct 14, 2020 at 1:24 PM Michael Di Domenico <mdidomeni...@gmail.com<mailto:mdidomeni...@gmail.com>> wrote: On Wed, Oct 14, 2020 at 11:53 AM Oddo Da <oddodao...@gmail.com<mailto:oddodao...@gmail.com>> wrote: i'll agree that in some respects software engineering has gotten better in the last 20yrs, but it's subjective. there are a lot of things that have gotten better and there are a lot of things that are much worse. but i'm not sure you can apply that statement to HPC. HPC code doesn't churn like business code or even more volatile cloud code. HPC code is usually written to solve something specific and gets incremental updates over time. usually that something specific hasn't changed the last 20yrs (think physics/chemistry) the models we use to describe or solve the problems likely have, but the underlying code is probably basically the same with tweaks along the way to fit the new model. I disagree. I think yes, there is old code that does not churn but there are always new people/grad students coming into the field. They too are being pointed in the same direction of how to do things, which is what we are discussing here ;-) Yes, there are new people coming in. But they’re not developing new modeling codes from scratch – they’re typically “improving” the existing codes. And as Michael pointed out, there’s significant resistance to change when you’ve got a code base that is known, debugged, and has known warts. Big changes occur when a modeling paradigm shift occurs. And those are not real common. an evolution to MPI. but it goes back to technical debt. to re-write something in chapel is non-trivial and may not be worth the time. writing something new and choosing chapel is really left up to the developer. i have some chapel users here and there, but they're a minority. and since chapel is largely only found on cray machines its exposure is low It seems that in your world nothing new ever gets written? You are talking only about re-writes ;). For many HPC science applications this is true – Physics models change very slowly. Once you have a gridded finite element model, it “just works” and there’s not huge demand for new modeling approaches. Changes in language usage usually occur because of a technical problem with the existing code that means it just cannot be modified. More than one ambitious person has said “let’s re-do program X from Fortran to C++ or Java or Ada or Python” and found it a bigger challenge than expected. There is a definite preference for the devil you know.
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
