At 09:10 PM 2/11/2007, Devesh Sharma wrote: >On 2/10/07, Tang, Changqing <[EMAIL PROTECTED]> wrote: >> > > >> > >Not for the receiver, but the sender will be severely slowed down by >> > >having to wait for the RNR timeouts. >> > >> > RNR = Receiver Not Ready so by definition, the data flow >> > isn't going to >> > progress until the receiver is ready to receive data. If a >> > receive QP >> > enters RNR for a RC, then it is likely not progressing as >> > desired. RNR >> > was initially put in place to enable a receiver to create >> > back pressure to the sender without causing a fatal error >> > condition. It should rarely be entered and therefore should >> > have negligible impact on overall performance however when a >> > RNR occurs, no forward progress will occur so performance is >> > essentially zero. >> >>Mike: >> I still do not quite understand this issue. I have two >>situations that have RNR triggered. >> >>1. process A and process B is connected with QP. A first post a send to >>B, B does not post receive. Then A and B are doing a long time >>RDMA_WRITE each other, A and B just check memory for the RDMA_WRITE >>message. Finally B will post a receive. Does the first pending send in A >>block all the later RDMA_WRITE ? >According to IBTA spec HCA will process WR entries in strict order in >which they are posted so the send will block all WR posted after this >send, Until-unless HCA has multiple processing elements, I think even >then processing order will be maintained by HCA >If not, since RNR is triggered
The source HCA is responsible for processing work requests in the order they are posted. If the SEND cannot proceed and receives a RNR, then the subsequent RDMA Write should not proceed, i.e. the sequence numbers that define the valid window will not progress and given IB requires strong ordering within the fabric, nothing sent subsequently should be made visible at the sink HCA. In your example, if A is sending a SEND followed by a RDMA Write, the first check should have been that B had provided an ACK with a credit indicating that a SEND is allowed. If B subsequently removed access to the buffer that had to be posted to provide that credit, then it should trigger a RNR NAK and the subsequent RDMA Writes should not be visible at B since there is no an effective hole in the transmission stream. >>periodically till B post receive, does it affect the RDMA_WRITE >>performance between A and B ? >> >>2. extend above to three processes, A connect to B, B connect to C, so B >>has two QPs, but one CQ. A posts a send to B, B does not post receive, >>rather B and C are doing a long time RDMA_WRITE, or send/recv. But B >>must sends RNR periodically to A, right?. So does the pending message >>from A affects B's overall performance between B and C ? Neither IB nor iWARP provide any ordering guarantees between different data flows. This is strictly under application control. Hence, if a RNR NAK or whatever occurs on a RC between A and B, then it has no impact on what occurs between A and C or B and C. It is simply outside the scope of either technology to address. Mike _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
