Re: [Beowulf] What is rdma, ofed, verbs, psm etc?

2017-09-21 Thread Brice Goglin
Le 21/09/2017 17:28, Douglas Eadline a écrit : >> What about RoCE? Is this something that is commonly used (I would guess >> no since I have not found much)? Are there other protocols that are >> worth considering (like "gamma" which doesn't seem to be developed >> anymore)? > Gamma has not been

Re: [Beowulf] What is rdma, ofed, verbs, psm etc?

2017-09-21 Thread Alex Chekholko
I don't know about RoCE but here is plain 10G ICMP ECHO round trip: # ping n0030 PING n0030.localdomain (192.168.0.204) 56(84) bytes of data. 64 bytes from n0030.localdomain (192.168.0.204): icmp_seq=1 ttl=64 time=0.141 ms 64 bytes from n0030.localdomain (192.168.0.204): icmp_seq=2 ttl=64 time=0.1

Re: [Beowulf] What is rdma, ofed, verbs, psm etc?

2017-09-21 Thread Jon Tegner
What kind of latency can one expect using RoCE/10G? On 09/21/2017 05:28 PM, Douglas Eadline wrote: I think the main reason it stopped was that IB is the choice of most clusters and most 10G nics provide low latency with default tcp/ip (less than 10us in most cases). ___

Re: [Beowulf] What is rdma, ofed, verbs, psm etc?

2017-09-21 Thread Douglas Eadline
> What about RoCE? Is this something that is commonly used (I would guess > no since I have not found much)? Are there other protocols that are > worth considering (like "gamma" which doesn't seem to be developed > anymore)? Gamma has not been around for years. There was open-mx http://open-mx