Looking at my question I posted earlier how is it that java is so high performing when it comes to large data sets?
Regards, Jonathan From: Beowulf <beowulf-boun...@beowulf.org> On Behalf Of Oddo Da Sent: 13 October 2020 14:38 To: Michael Di Domenico <mdidomeni...@gmail.com> Cc: Beowulf Mailing List <beowulf@beowulf.org> Subject: Re: [Beowulf] [External] Spark, Julia, OpenMPI etc. - all in one place On Tue, Oct 13, 2020 at 8:33 AM Michael Di Domenico <mdidomeni...@gmail.com<mailto:mdidomeni...@gmail.com>> wrote: i can't speak from a general industry sense, but i've had everything run through my center over the past 11 years. Hadoop seemed like something that was going to take off. it didn't with my group of users. we aren't counting clicks nor parsing text from huge files, so its utility to us faded. my understanding is the group behind hadoop also made several industry missteps when trying to commercialize, i'm not sure what happened after that. i think a lot people realized that hadoop made things easier, but the overhead was too high given the limited functionality most people wanted to use it for Michael, thank you for the insight. I think Hadoop in general is mostly dying, Spark is really the derivative that took off. Basically, what you are saying is that there is no demand on your infra for this kind of work. Do you have any insights as to why not? Do the AI/DS/ML guys just know that they cannot use your resources to run standard loads and go straight to the cloud or local ethernet clusters? In your estimate, how many of your users write code in Julia vs MPI vs Python?
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
