Re: [Beowulf] Jupyter and EP HPC

2018-07-27 Thread Fred Youhanaie
Jim I'm not a jupyter user, yet, however, out of curiosity I just googled for what I think you're looking for. Is this any good? https://ipyparallel.readthedocs.io/en/stable/ I have now bookmarked it for my own future use! Cheers, Fred On 27/07/18 21:56, Lux, Jim (337K) wrote: -Origina

Re: [Beowulf] Jupyter and EP HPC

2018-07-27 Thread Lux, Jim (337K)
-Original Message- From: Beowulf [mailto:beowulf-boun...@beowulf.org] On Behalf Of Joe Landman Sent: Friday, July 27, 2018 11:54 AM To: beowulf@beowulf.org Subject: Re: [Beowulf] Jupyter and EP HPC On 07/27/2018 02:47 PM, Lux, Jim (337K) wrote: > > I’ve just started using Jupyter to org

Re: [Beowulf] Jupyter and EP HPC

2018-07-27 Thread Joe Landman
On 07/27/2018 02:47 PM, Lux, Jim (337K) wrote: I’ve just started using Jupyter to organize my Pythonic ramblings.. What would be kind of cool is to have a high level way to do some embarrassingly parallel python stuff, and I’m sure it’s been done, but my google skills appear to be lacking (

[Beowulf] Jupyter and EP HPC

2018-07-27 Thread Lux, Jim (337K)
I've just started using Jupyter to organize my Pythonic ramblings.. What would be kind of cool is to have a high level way to do some embarrassingly parallel python stuff, and I'm sure it's been done, but my google skills appear to be lacking (for all I know there's someone at JPL who is doing

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Prentice Bisbal
'Top Secret Rosies" is a good documentary on the women computers of yesteryear: https://en.wikipedia.org/wiki/Top_Secret_Rosies:_The_Female_%22Computers%22_of_WWII On 07/27/2018 09:06 AM, Lux, Jim (337K) wrote: William Henry Dana – “Two Years Before the Mast” – An excellent book describing w

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Lux, Jim (337K)
If you need more pictures of the early JPL cluster computers, using acoustic and optical interconnects, let me know.. They’ve recently reorganized the photo archives here, and it’s a lot easier to find stuff (like pictures of the foundation of the building my office is in, from the 1950s, when t

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread John Hearns via Beowulf
Jim, thankyou for that link. It is quite helpful! I have a poster accepted for the Julia Conference in two weeks time. My proposal is to discuss computers just like that - on the Manhattan project etc. Then to show how Julia can easily be used to solve the equation for critical mass from the Los Al

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Lux, Jim (337K)
William Henry Dana – “Two Years Before the Mast” – An excellent book describing what it was like to be a sailor in the days just before steam, there’s plenty of climbing the rigging in a sleet storm, while trying to round Cape Horn. – but when they got to California, the weather was a lot nicer.

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Lux, Jim (337K)
Indeed.. Interestingly, we were having this discussion (in a similar form) with respect to a radio telescope at work - modern arrays like SKA, LOFAR, MeerKAT, etc. have *lots of data* being pushed around. I'm working on one that flies in space (so we're not going to have a fiber to the ground)

Re: [Beowulf] Fwd: SSD performance

2018-07-27 Thread Joshua Mora
Buy server grade not consumer grade. Also use trim. Joshua -- Original Message -- Received: 11:30 PM PDT, 07/26/2018 From: Jonathan Engwall  To: Beowulf Mailing List  Subject: [Beowulf] Fwd: SSD performance > While tarring files after cloning the drive became hopeless, I realized all >

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Fred Youhanaie
On 27/07/18 09:01, John Hearns via Beowulf wrote: I can just see HPC types being made to climb the rigging in a gale... ... and they would be called DevOpSailors Happy SysAdmin / DevOpSailor Day :-) http://sysadminday.com/ ___ Beowulf maili

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Fred Youhanaie
They do mention up to 1Tb/s transfer rate between the truck and the data centre, which, for 100PB, would take 10 days to transfer into the truck and 10 days out of it into the AWS storage. I think the main purpose of the service, snowmobile and snowball, is the initial transfer of high volume

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Jörg Saßmannshausen
Hi John, good idea! Specially as the ship has been restaurated and is in my neighbourhood. The only flip side here might be that some tourists might not like the idea and it might be a wee bit difficult to get it back into the Thames. :-) All the best Jörg Am Freitag, 27. Juli 2018, 10:01:53

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread John Hearns via Beowulf
Jörg, then the days of the Tea Clipper Races should be revived. We have just the ship for it already. Powered by green energy, and built in Scotland of course. https://en.wikipedia.org/wiki/Cutty_Sark Just fill her hold with hard drives and set sail. Aaar me hearties. I can just see HPC types bei

Re: [Beowulf] Lustre Upgrades

2018-07-27 Thread Jörg Saßmannshausen
Hi all, Jim: the flip side of the cable is: once it is installed you still can use it, whereas with the snow mobile you have to pay for every use. So in the long run the cable is cheaper, specially as we do need fast connection for scientific purposes. I was at a talk here in London not so lo