Michael, the Mellanox gateway devices do exactly what you want. What I implemented was a connection between an Infiniband HPC cluster and a Data Lake system connected solely by Ethernet.
High level overview https://community.mellanox.com/docs/DOC-2301 http://www.mellanox.com/related-docs/prod_gateway_systems/PB_SX6036G.pdf Look at this more up to date implementation though http://www.mellanox.com/page/ib_fabricit_bxm_management Drop me a mail off list if necessary. On Tue, 11 Dec 2018 at 23:40, John Hearns <hear...@googlemail.com> wrote: > Michael, yes. Panasas engineered IPOIB to Ethernet routers for their > storage platform. Remember that until the latest generation of their kit > they ran on BSD, which had no Infiniband capability. Panasas IB routers > booted from an onboard SATA DOM, which was quite a neat solution. > I know this as I had to revive one from the dead after the technical staff > at a company I worked with deleted the partition table during a test cycle > (long war story there). > > > More relevantly, when working at ASML I configured the Mellanox gateway > devices which connect between IB networks and Ethernet networks. > Technically these are bridges which implement the proxy arp function. I > installed four of these devices, and got to know them pretty well. > I think they do the job which you need. > > > > > > > > > On Tue, 11 Dec 2018 at 17:03, Michael Di Domenico <mdidomeni...@gmail.com> > wrote: > >> does anyone use linux based routers to get from an ipoib network out >> onto a regular ethernet network? if so, do you have multiple routers >> or just one? in the case of multiple routers how to you handle the >> traffic coming back from the ethernet network? (ie how does the >> ethernet network know which router to send the data back through) >> >> --- >> >> to expand further on what i'm trying to do (sanity/success is still to >> be determine)... >> >> imagine a big cluster of machines all connected via infiniband running >> ipoib >> there are 4 linux routers on the ipoib network that also have 10Gb >> connections to an ethernet network >> the ethernet network contains a router to other resources >> >> what i want to accomplish is to have the compute nodes spray their >> traffic across the 4 linux routers and end up on the ethernet network >> i'm sure someone's done something like this before, but my google foo >> is failing me >> _______________________________________________ >> Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> >
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
