On Wed, 2007-01-10 at 13:47, Or Gerlitz wrote: > On 1/10/07, Sean Hefty <[EMAIL PROTECTED]> wrote: > >> How about adding sendonly param to the ABI and having the ucma kernel > >> code returning -EINVAL if someone tries to set it to true. Such code can > >> be pushed to 2.6.21 and when you have the time to complete the > >> implementation you can complete this? > > > I don't think adding this is a huge deal; I just haven't gotten to it yet. > > However, I'd like to make sure there's enough time once the change is made > > to > > verify that we have the right result before pushing it upstream. > > Compared to the amount of functionality (ib_sa ref mcast counting and > rdma cm u/k mcast API) provided by this code, i really think it can be > done in two phases, first push a code that does not support sendonly > and later once its implemented push a patch that adds this. > > Anyway, per the best of my knowledge you would not be able to fully > verify the "right" result , i understand the opensm does not really > support a sendonly join (*) in the sense of that it does not support > configuring "one-way" MFTs for a subset of the branches/leaves of the > multicast group spanning tree, but this is orthogonal to your code.
Yes, this is orthogonal and only affects strict conformance vis a vis the reception of packets for this group which would be discarded anyhow. This is an optimization provided by the spec. > (*) there are some more issues here which need to be addressed, see > for example the "Some SMs don't support send-only yet" weird comment > at ipoib_mcast_sendonly_join() It's more likely an SA issue but I'm only guessing... It may also be historical... -- Hal > > Woody has also seen this issue. And of course, I can't reproduce it on my > > systems, but I'm actively looking into the problem. It looks like some > > sort of > > issue with ipoib trying to join a non-existent multicast group. > > mmm, weird, are there active rdmacm multicast consumers involved in > this settings? > > >> Looking on the code, i understand that if an multicast consumer attempts > >> to join a group for which another consumer is already joined then it > >> just gets the group params, that is the mgid is your discriminator (with > >> the exception of an all zeros mgid which has a different treatment) > >> which makes much sense to me. > > > Not exactly. The rdma_cm consumer gets the group parameters for the ipoib > > broadcast group. It uses this information as a template for joining new > > groups. > > OK, got you at last (sorry but i have somehow ignored the call to > ib_addr_get_mgid() at the rdmacm code). So to achieve interop with > IPoIB all we need to do is remove the rdmacm signature bit and not to > over-write the rdmacm qkey on the the qkey of the ipoib ipv4 broadcast > group, are you ok with that? > > > One issue is that an rdma_cm consumer can first allocate a UD QP to use > > with UD > > traffic. When it later joins a multicast group, the qkey must be the same. > > How > > does ipoib handle this? > > Well, ipoib first sets a zero qkey into its qp in ipoib_init_qp() and > later in ipoib_mcast_attach() does another qp modify providing > priv->qkey which is the ipv4 broadcast group one, the rdma_cm can > mimic this behaviour for qps created with rdma_create_qp and also for > those created by the user. > > >> Since for our apps needs we do intend to join the 224.0.0.1 group, > >> resolving a) above is fine for us --> we will join 224.0.0.1 above, > >> provide the qkey to the rdma cm and it will join to the other group (eg > >> 224.5.5.5) with this qkey. > > > I'm not completely following you on this yet. > > forget this, there is no need in such tricks since the rdmacm has the > ipv4 broadcast group qkey. > > >> can you remind me what the idea/trick here, aren't you supposed to > >> generate an mgid for this case? > > > This either returns an existing MCMemberRecord that this node has joined, > > or it > > fills out an MCMemberRecord that can be used to join a new group. If the > > mgid > > is zero, the SA will assign one. > > thanks > > Or. > > _______________________________________________ > openib-general mailing list > [email protected] > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
