Hi Matt, It seems that my reply was lost in the pile of mails during holiday period?
Is there any update on this? Regards, Nitesh On Tue, Dec 30, 2014 at 5:54 PM, Nitesh Bansal <[email protected]> wrote: > Hi Matt, > > The scenario which i have in my mind is as follows: > > WebRTC browser -----------> (Inbound) Asterisk --------------> SIP phone > (VP.8 video call) > > Since this is a webRTC call, RTP stream will be encrypted between Web > Browser and Asterisk (call leg 1) and in our case and call leg between > Asterisk and SIP phone(call leg 2) can be secured/unsecured depending on > the config. Since one leg of the call is encrypted, there can't be a P2P > bridge. > But the fact is that asterisk isn't the source of the video, it is just > forwarding the RTP stream. > In this case, i believe it makes sense to forward RTCP FB messages like > NACK, PLI and FIR. > NACKs -- Asterisk can request the retransmission of video frames > PLI/FIR -- Asterisk can request image refresh from either endpoint > > To avoid the RTCP FB forwarding in other scenarios, we can probably make > it as config enabled feature. > > > Regards, > Nitesh > > > > > > On Mon, Dec 29, 2014 at 4:46 PM, Matthew Jordan <[email protected]> > wrote: > >> >> >> On Tue, Dec 23, 2014 at 11:01 AM, Nitesh Bansal <[email protected]> >> wrote: >> > Hi Matt, >> > >> > Thanks for this nice explanation. >> > >> > Well, we still don't know if we are going to do it or not, it is still >> a >> > pending decision. >> > >> > But i do have some comments ref your response, responding inline. >> > >> > General comment: By adding the support for NACK, i meant just adding the >> > ability >> > to forward the RTCP FB messages, i do agree that adding the capability >> in >> > asterisk >> > to generate NACKs on its own will make it much more complicated? >> >> Let's assume you aren't in a packet-to-packet bridge provided by >> bridge_native_rtp, as in that case, the RTCP messages should be >> auto-magically forwarded anyway. >> >> In a core bridge, forwarding may be more complicated than generating the >> RTCP FB messages. >> >> Remember that Asterisk is a B2BUA. As such, the general placement of the >> RTP stack in the architecture will look something like this: >> >> ----------------- >> -------------| Bridging Core |------------- >> | ----------------- | >> ------------------ ------------------ >> | Channel Core | | Channel Core | >> ------------------ ------------------ >> | Channel Driver | | Channel Driver | >> ------------------ ------------------ >> | RTP instance A | | RTP instance B | >> ------------------ ------------------ >> >> >> Let's say RTP instance A wanted to forward RTCP FB messages to whatever >> it is talking to on the other side of the bridge. There are a few issues >> with doing this: >> (1) The channel on the other side of the bridge may - or may not - be an >> RTP capable channel driver. >> (2) You may have more than one channel on the other side of the bridge >> (such as is the case in a conference). >> (3) Reaching across a bridge from the RTP instance requires: >> - Grabbing the channel through the driver/core (and dealing with the >> locking situation between the channel and the driver pvt) >> - Grabbing the bridge_channel structure safely, which mediates >> between the bridge and the channel >> - Grabbing the bridge structure safely >> - Grabbing the other channel on the other side of the bridge safely >> - (You can see where this is going) >> Suffice to say, there are a lot of locks you have to grab safely to >> "own" the whole picture of the world and pass the information over. >> Alternatively, you can pass a control frame (which is the right way to do >> this), but does mean more custom control frames, and all the channel >> drivers have to be updated to understand said control frame. >> (4) Forwarding a FB message across a bridge - particularly in a >> multi-party scenario - may result in the wrong information being requested >> (and more confusion besides). Say, for example, we have three devices >> talking to Asterisk, one of which - say device A - is suffering from >> significant packet loss. As a result, device A tells Asterisk via a FB >> message that it is missing some packets. However, who does Asterisk forward >> that to? The answer has to be both parties: the FB message itself was not >> destined for a single party on the other side of a bridge, but for Asterisk >> itself. However, both parties are not providing the actual video stream >> that is sent to device A - Asterisk chooses which stream of video is being >> sent to device A. As such - particularly if the source of the stream >> updated during a period of packet loss - we won't know what set of RTP >> packets should be re-transmitted to device A. Even if we could make that >> determination, if both devices do re-transmit their RTP, then the RTP >> implementation for each of those devices will probably drop the packets, as >> the sequence number/timestamp of received RTP from those devices will >> already be passed the received packets. >> >> Asterisk fares poorly as a proxy. >> >> Asterisk is a B2BUA: the device talking to Asterisk through RTP instance >> A is talking to Asterisk, not directly to the device talking to RTP >> instance B. Ideally, Asterisk would be generating the feedback RTCP >> reports, as it is the thing that is actually receiving, decoding, and >> generating the RTP packets being sent to the device(s) on the other side of >> the bridge. >> >> >> > (1) You'll need to modify rtp_engine and res_rtp_asterisk to >> > understand compound RTCP packets properly. Currently, we always assume >> > that the RTCP packets contain a single SR/RR: that means modifications >> > to the parsing in ast_rtcp_read, as well as the conversion of received >> > RTCP reports in rtp_engine into JSON for delivery to RTP Stasis topic >> > consumers. Note that this would be an opportune time to make the >> > parsing in ast_rtcp_read not be quite so terrible as it is today. >> > >> > (2) Going along with #1, you'll need to add additional RTCP report >> > types - both when reading and when writing. Currently, Asterisk only >> > supports SR/RR. The RFC mandates FR reports, along with SDES packets. >> > You'll also need to change the algorithm for when reports are >> > scheduled. Since RFC 4585 modifies the scheduling algorithm, you'll >> > pretty much have to abandon how scheduling is set up currently in >> > ast_rtp_read. There are different modes of operation, each with slight >> > variations to the algorithm for how/when reports should be sent. A >> > state machine would probably be handy for this. >> > >> > Nitesh -- since asterisk is just going to forward the RTCP FB reports, >> won't >> > it >> > be sufficient to parse the RTCP FB messages, adapt the packet header >> like >> > SSRC >> > and other stuff and just forward it, without modifying the current >> > algorithm? >> >> See previous comments on why I think forwarding the FB messages is >> probably wrong for this scenario. >> >> > (3) NACK mode. I would be very careful here: as a B2BUA, Asterisk >> > *always* assumes there are only two participants in an RTP session - >> > and making Asterisk realize that this isn't the case is going to be >> > more challenging than you may think. Asterisk has a lot of >> > implementation 'shortcuts' it takes in RTCP statistic management that >> > it takes because it knows that there is only one other RTP participant >> > other than itself: if you go down the road of wanting to maintain >> > additional statistics beyond the RTP instance and whoever it is >> > talking to, the scope of this would expand dramatically. >> > Nitesh -- I am not sure if i understood your comment, why does it need >> to >> > maintain >> > addiitonal info beyond RTP instance, in my opinion, for forwarding RTCP >> FB >> > reports, >> > all we need is the SSRC and sequence numbers? >> > >> > (4) If I'm reading the RFC correctly, you _may_ not need to buffer the >> > RTP packets themselves; rather, you need to inform the other side of >> > the RTP packets you've lost. Generally, this means keeping a limited >> > history of which RTP sequence numbers you've received, and which ones >> > are missed. When constructing the NACK message (in 5a), you'd see if >> > you had a missed RTP sequence number, then look to see if any of the >> > 16 RTP packets after that one were also missed. After reporting this, >> > the buffer could be flushed - how long the buffer is probably depends >> > on the frequency of report sending, which is variable as per (2). >> > >> > Nitesh -- I have made this assumption considering that many traditional >> > SIP phones don't support NACK and even if they do, they don't send the >> > retransmission as specified in the RFC 4588. >> > This draft http://tools.ietf.org/pdf/draft-ietf-straw-b2bua-rtcp-02.pdf >> > talks >> > about the issue of handling the NACK messages in B2BUA environment. >> > Here is a piece of text from Section 4 of this draft: >> > >> > It is worth noting that some additional care may be needed, with >> > respect to RTCP, when acting as an intermediary between two secure >> > sessions. Specifically, issues may arise when relaying NACK feedback >> > originated by a user who failed to receive some RTP packets, that >> > were instead received by the B2BUA: such an issue might occur when >> > the packet loss happens between the user and the B2BUA itself, and >> > not between the RTP packet sender and the B2BUA. Such a situation >> > might result in the RTP media sender retransmitting the encrypted >> > packet, which would then be rejected by the B2BUA as a replayed one. >> > However, this issue is well known and addressed in [ >> > RFC4588 >> > ], which >> > both the B2BUA and the involved participants in the communication >> > SHOULD make use of to prevent it from happening. This mechanism >> > allows for a Redundancy RTP Stream to be used for the purpose, which >> > would prevent the replay error. Of course, all recommendations given >> > previously with respect to managing SSRCs across the B2BUA still >> > apply here as well: in fact, such a redundant RTP stream would make >> > use of a different SSRC, that would need to be taken care of both at >> > the RTP and the SDP level. If [ >> > RFC4588 >> > ] is not supported, the B2BUA >> > SHOULD handle NACK packets directly, and only forward feedback on >> > lost packets it has not access to >> > >> >> I think this is more of an indictment against merely relaying NACK >> feedback messages then anything else. If Asterisk forwarded FB messages >> then I can see this scenario arising. If, however, Asterisk generates the >> feedback messages itself, I'm not sure how this would apply. Asterisk >> already completely decodes and re-encodes encrypted media streams between >> two endpoint. If Asterisk generates the feedback messages itself as well, >> then an RTP packet sender would not request a re-send of packets from >> Asterisk that Asterisk was unaware of. >> >> The scheduling issues would - I think - still be a problem. How much, in >> practice, the scheduling issues really matter is another concern. It may be >> that maintaining the existing scheduling is sufficient - I'd be curious to >> know what the real impact of the existing scheduling on FB reports would be. >> >> > (5) RTCP feedback messages. The transport layer feedback messages >> > would obviously be provided by the RTP engine and/or res_rtp_asterisk. >> > However, the other layers - payload/application - should have >> > corresponding APIs in rtp_engine. In particular, the codec layer >> > (payload) would be interesting, as it would require the ability for >> > codec modules (or something else in the media 'section' of Asterisk) >> > to take an RTP instance and register a handler for providing feedback >> > messages at opportune times. The application layer would be similar, >> > except that currently, we probably wouldn't have an implementation. >> > You may want to consider that as 'future work', as it is probably >> > sufficient to be a project in its own right. >> > >> > Nitesh -- Do we require to register any handlers if we just care about >> > forwarding >> > the RTCP FB messages? >> >> I think if you don't concern yourself with application or codec level >> feedback mechanisms, then this part of the RFC could be ignored and the >> need for registered handlers (or a framework to support it) could go away. >> >> > (a) Generic NACK: these messages should be implemented in such a >> > fashion that res_rtp_asterisk can have one be constructed based on the >> > statistics in (4). A part of me wishes all of this were implemented in >> > RTP engine, so that other RTP implementations could take advantage of >> > the RTCP work. That's a consideration that should be given some >> > thought, given how much of the RTCP implementation would have to be >> > ripped apart by this work anyway. >> > (b) PLI/SLI/RPSI: I'm not sure these apply to Asterisk very well. We >> > don't decode video, so it's kind of immaterial. You may want to have >> > the ability for Asterisk to pass through these types of RTCP messages >> > from a video decoder to a video encoder; that would entail >> > understanding the RTCP messages, enqueuing a control frame onto the >> > channel, and - in the far end - taking that control frame and turning >> > it into the appropriate information on the other side of the bridge. >> > There's bridging considerations to think through here on how the >> > control frame is propagated, particularly for multi-party bridges. >> > >> > If you're still interested in doing this work, please do let me us >> > know! I think the first major hurdle here is to hammer out how all of >> > this can be effectively introduced into the RTP engine/implementation >> > in Asterisk - as a large part of what makes this so difficult is the >> > non-extensible nature of Asterisk's RTP implementation. It may be that >> > we can help out some with that part, such that the actual act of >> > implementing the RFC and its various feedback mechanisms is a bit >> > easier. >> > >> >> While I am leaning towards "don't forward the FB messages", I think a >> concrete example of feedback messages that you think should be forwarded >> might help here. Right now, I'm having a hard time envisioning the specific >> scenario where you have two or more endpoints providing video to Asterisk >> and where Asterisk can simply relay the entire feedback message and not >> generate the message itself. I could be mistaken, but it feels like while >> this could work in a simple two-party core bridge, there are a lot of other >> scenarios where this goes wrong quickly. >> >> Matt >> >> -- >> Matthew Jordan >> Digium, Inc. | Engineering Manager >> 445 Jan Davis Drive NW - Huntsville, AL 35806 - USA >> Check us out at: http://digium.com & http://asterisk.org >> >> -- >> _____________________________________________________________________ >> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >> >> asterisk-dev mailing list >> To UNSUBSCRIBE or update options visit: >> http://lists.digium.com/mailman/listinfo/asterisk-dev >> > >
-- _____________________________________________________________________ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
