Re: [Beowulf] NFS over IPoIB

On Friday, 12 June 2020 2:36:26 PM PDT John McCulloch wrote: > It is my understanding that setting MTU to 9000 is recommended but that > seems to be applicable for 10GbE. That depends how you're running your IB fabric. In datagram mode (which I think is the default these days) you're limited to

Re: [Beowulf] Neocortex unreal supercomputer

I honestly cannot imagine the cost involved here for such a system. Regards, Jonathan Aquilina EagleEyeT Phone: +356 2033 0099 Moblie + 356 7995 7942 Email: sa...@eagleeyet.net Website: https://eagleeyet.net From: Beowulf On Behalf Of Peter St. John Sent: 12 June 202

Re: [Beowulf] NFS over IPoIB

We mount home via NFS over IPoIB. Other than bumping the nfs, mountd daemons we left everything default. We left the IB in datagram mode and mtu at 2044. We alhave over 400 nodes and it works ok. Raj On Fri, Jun 12, 2020, 5:37 PM John McCulloch wrote: > Thanks for the quick response Alex... Th

Re: [Beowulf] NFS over IPoIB

Thanks for the quick response Alex... There's no problem yet... It's a new install with 100GbE Mellanox IB and currently at defaults except I bumped the number of NFS daemons to 128. Just researching ways to optimize performance for 36 nodes concurrently accessing the storage server. It is my un

Re: [Beowulf] NFS over IPoIB

I think you should start with all defaults and then describe the problem you're having with those settings. IIRC last time I ran NFS over IPoIB I didn't tune anything and it was fine. On Fri, Jun 12, 2020 at 12:10 PM John McCulloch wrote: > Can anyone comment on experience with compute nodes mo

[Beowulf] NFS over IPoIB

Can anyone comment on experience with compute nodes mounting NFS v4.1 shares over IPoIB... i.e., tuning parameters that are likely to be most effective... We looked at NFS over RDMA but doing that would require a kernel upgrade... https://www.admin-magazine.com/HPC/Articles/Useful-NFS-Options-f

Re: [Beowulf] Neocortex unreal supercomputer

I'm imaging the 8"-on-a-side thing as an Array Coprocessor; the way NVIDIA cards can be used. Video cards fill an ISA card slot (in my day an uber gaming rig could use two in tandem in adjacent slots). Mine was a box about the length of my hand that filled the card, it had appreciable memory, ma

Re: [Beowulf] Neocortex unreal supercomputer

Yes, it may. It is designed for natural language processing and convoluted neural networks. Here is a second article. https://insidehpc.com/2020/06/ai-supercomputer-at-psc-to-combine-cerebras-wafer-scale-chips-and-hpe-superdome-flex/ On Fri, Jun 12, 2020, 1:19 AM John Hearns wrote: > Will it dre

Re: [Beowulf] Neocortex unreal supercomputer

Will it dream of electric sheep when they turn out the lights and let it sleep? https://www.lanl.gov/discover/news-release-archive/2020/June/0608-artificial-brains.php On Fri, 12 Jun 2020 at 01:16, Jonathan Engwall < engwalljonathanther...@gmail.com> wrote: > This machine is planned, or possibl