[asterisk-dev] Adding the support for NACK in asterisk
Nitesh Bansal
nitesh.bansal at gmail.com
Tue Dec 30 10:54:42 CST 2014
Hi Matt,
The scenario which i have in my mind is as follows:
WebRTC browser -----------> (Inbound) Asterisk --------------> SIP phone
(VP.8 video call)
Since this is a webRTC call, RTP stream will be encrypted between Web
Browser and Asterisk (call leg 1) and in our case and call leg between
Asterisk and SIP phone(call leg 2) can be secured/unsecured depending on
the config. Since one leg of the call is encrypted, there can't be a P2P
bridge.
But the fact is that asterisk isn't the source of the video, it is just
forwarding the RTP stream.
In this case, i believe it makes sense to forward RTCP FB messages like
NACK, PLI and FIR.
NACKs -- Asterisk can request the retransmission of video frames
PLI/FIR -- Asterisk can request image refresh from either endpoint
To avoid the RTCP FB forwarding in other scenarios, we can probably make it
as config enabled feature.
Regards,
Nitesh
On Mon, Dec 29, 2014 at 4:46 PM, Matthew Jordan <mjordan at digium.com> wrote:
>
>
> On Tue, Dec 23, 2014 at 11:01 AM, Nitesh Bansal <nitesh.bansal at gmail.com>
> wrote:
> > Hi Matt,
> >
> > Thanks for this nice explanation.
> >
> > Well, we still don't know if we are going to do it or not, it is still a
> > pending decision.
> >
> > But i do have some comments ref your response, responding inline.
> >
> > General comment: By adding the support for NACK, i meant just adding the
> > ability
> > to forward the RTCP FB messages, i do agree that adding the capability in
> > asterisk
> > to generate NACKs on its own will make it much more complicated?
>
> Let's assume you aren't in a packet-to-packet bridge provided by
> bridge_native_rtp, as in that case, the RTCP messages should be
> auto-magically forwarded anyway.
>
> In a core bridge, forwarding may be more complicated than generating the
> RTCP FB messages.
>
> Remember that Asterisk is a B2BUA. As such, the general placement of the
> RTP stack in the architecture will look something like this:
>
> -----------------
> -------------| Bridging Core |-------------
> | ----------------- |
> ------------------ ------------------
> | Channel Core | | Channel Core |
> ------------------ ------------------
> | Channel Driver | | Channel Driver |
> ------------------ ------------------
> | RTP instance A | | RTP instance B |
> ------------------ ------------------
>
>
> Let's say RTP instance A wanted to forward RTCP FB messages to whatever it
> is talking to on the other side of the bridge. There are a few issues with
> doing this:
> (1) The channel on the other side of the bridge may - or may not - be an
> RTP capable channel driver.
> (2) You may have more than one channel on the other side of the bridge
> (such as is the case in a conference).
> (3) Reaching across a bridge from the RTP instance requires:
> - Grabbing the channel through the driver/core (and dealing with the
> locking situation between the channel and the driver pvt)
> - Grabbing the bridge_channel structure safely, which mediates
> between the bridge and the channel
> - Grabbing the bridge structure safely
> - Grabbing the other channel on the other side of the bridge safely
> - (You can see where this is going)
> Suffice to say, there are a lot of locks you have to grab safely to
> "own" the whole picture of the world and pass the information over.
> Alternatively, you can pass a control frame (which is the right way to do
> this), but does mean more custom control frames, and all the channel
> drivers have to be updated to understand said control frame.
> (4) Forwarding a FB message across a bridge - particularly in a
> multi-party scenario - may result in the wrong information being requested
> (and more confusion besides). Say, for example, we have three devices
> talking to Asterisk, one of which - say device A - is suffering from
> significant packet loss. As a result, device A tells Asterisk via a FB
> message that it is missing some packets. However, who does Asterisk forward
> that to? The answer has to be both parties: the FB message itself was not
> destined for a single party on the other side of a bridge, but for Asterisk
> itself. However, both parties are not providing the actual video stream
> that is sent to device A - Asterisk chooses which stream of video is being
> sent to device A. As such - particularly if the source of the stream
> updated during a period of packet loss - we won't know what set of RTP
> packets should be re-transmitted to device A. Even if we could make that
> determination, if both devices do re-transmit their RTP, then the RTP
> implementation for each of those devices will probably drop the packets, as
> the sequence number/timestamp of received RTP from those devices will
> already be passed the received packets.
>
> Asterisk fares poorly as a proxy.
>
> Asterisk is a B2BUA: the device talking to Asterisk through RTP instance A
> is talking to Asterisk, not directly to the device talking to RTP instance
> B. Ideally, Asterisk would be generating the feedback RTCP reports, as it
> is the thing that is actually receiving, decoding, and generating the RTP
> packets being sent to the device(s) on the other side of the bridge.
>
>
> > (1) You'll need to modify rtp_engine and res_rtp_asterisk to
> > understand compound RTCP packets properly. Currently, we always assume
> > that the RTCP packets contain a single SR/RR: that means modifications
> > to the parsing in ast_rtcp_read, as well as the conversion of received
> > RTCP reports in rtp_engine into JSON for delivery to RTP Stasis topic
> > consumers. Note that this would be an opportune time to make the
> > parsing in ast_rtcp_read not be quite so terrible as it is today.
> >
> > (2) Going along with #1, you'll need to add additional RTCP report
> > types - both when reading and when writing. Currently, Asterisk only
> > supports SR/RR. The RFC mandates FR reports, along with SDES packets.
> > You'll also need to change the algorithm for when reports are
> > scheduled. Since RFC 4585 modifies the scheduling algorithm, you'll
> > pretty much have to abandon how scheduling is set up currently in
> > ast_rtp_read. There are different modes of operation, each with slight
> > variations to the algorithm for how/when reports should be sent. A
> > state machine would probably be handy for this.
> >
> > Nitesh -- since asterisk is just going to forward the RTCP FB reports,
> won't
> > it
> > be sufficient to parse the RTCP FB messages, adapt the packet header like
> > SSRC
> > and other stuff and just forward it, without modifying the current
> > algorithm?
>
> See previous comments on why I think forwarding the FB messages is
> probably wrong for this scenario.
>
> > (3) NACK mode. I would be very careful here: as a B2BUA, Asterisk
> > *always* assumes there are only two participants in an RTP session -
> > and making Asterisk realize that this isn't the case is going to be
> > more challenging than you may think. Asterisk has a lot of
> > implementation 'shortcuts' it takes in RTCP statistic management that
> > it takes because it knows that there is only one other RTP participant
> > other than itself: if you go down the road of wanting to maintain
> > additional statistics beyond the RTP instance and whoever it is
> > talking to, the scope of this would expand dramatically.
> > Nitesh -- I am not sure if i understood your comment, why does it need to
> > maintain
> > addiitonal info beyond RTP instance, in my opinion, for forwarding RTCP
> FB
> > reports,
> > all we need is the SSRC and sequence numbers?
> >
> > (4) If I'm reading the RFC correctly, you _may_ not need to buffer the
> > RTP packets themselves; rather, you need to inform the other side of
> > the RTP packets you've lost. Generally, this means keeping a limited
> > history of which RTP sequence numbers you've received, and which ones
> > are missed. When constructing the NACK message (in 5a), you'd see if
> > you had a missed RTP sequence number, then look to see if any of the
> > 16 RTP packets after that one were also missed. After reporting this,
> > the buffer could be flushed - how long the buffer is probably depends
> > on the frequency of report sending, which is variable as per (2).
> >
> > Nitesh -- I have made this assumption considering that many traditional
> > SIP phones don't support NACK and even if they do, they don't send the
> > retransmission as specified in the RFC 4588.
> > This draft http://tools.ietf.org/pdf/draft-ietf-straw-b2bua-rtcp-02.pdf
> > talks
> > about the issue of handling the NACK messages in B2BUA environment.
> > Here is a piece of text from Section 4 of this draft:
> >
> > It is worth noting that some additional care may be needed, with
> > respect to RTCP, when acting as an intermediary between two secure
> > sessions. Specifically, issues may arise when relaying NACK feedback
> > originated by a user who failed to receive some RTP packets, that
> > were instead received by the B2BUA: such an issue might occur when
> > the packet loss happens between the user and the B2BUA itself, and
> > not between the RTP packet sender and the B2BUA. Such a situation
> > might result in the RTP media sender retransmitting the encrypted
> > packet, which would then be rejected by the B2BUA as a replayed one.
> > However, this issue is well known and addressed in [
> > RFC4588
> > ], which
> > both the B2BUA and the involved participants in the communication
> > SHOULD make use of to prevent it from happening. This mechanism
> > allows for a Redundancy RTP Stream to be used for the purpose, which
> > would prevent the replay error. Of course, all recommendations given
> > previously with respect to managing SSRCs across the B2BUA still
> > apply here as well: in fact, such a redundant RTP stream would make
> > use of a different SSRC, that would need to be taken care of both at
> > the RTP and the SDP level. If [
> > RFC4588
> > ] is not supported, the B2BUA
> > SHOULD handle NACK packets directly, and only forward feedback on
> > lost packets it has not access to
> >
>
> I think this is more of an indictment against merely relaying NACK
> feedback messages then anything else. If Asterisk forwarded FB messages
> then I can see this scenario arising. If, however, Asterisk generates the
> feedback messages itself, I'm not sure how this would apply. Asterisk
> already completely decodes and re-encodes encrypted media streams between
> two endpoint. If Asterisk generates the feedback messages itself as well,
> then an RTP packet sender would not request a re-send of packets from
> Asterisk that Asterisk was unaware of.
>
> The scheduling issues would - I think - still be a problem. How much, in
> practice, the scheduling issues really matter is another concern. It may be
> that maintaining the existing scheduling is sufficient - I'd be curious to
> know what the real impact of the existing scheduling on FB reports would be.
>
> > (5) RTCP feedback messages. The transport layer feedback messages
> > would obviously be provided by the RTP engine and/or res_rtp_asterisk.
> > However, the other layers - payload/application - should have
> > corresponding APIs in rtp_engine. In particular, the codec layer
> > (payload) would be interesting, as it would require the ability for
> > codec modules (or something else in the media 'section' of Asterisk)
> > to take an RTP instance and register a handler for providing feedback
> > messages at opportune times. The application layer would be similar,
> > except that currently, we probably wouldn't have an implementation.
> > You may want to consider that as 'future work', as it is probably
> > sufficient to be a project in its own right.
> >
> > Nitesh -- Do we require to register any handlers if we just care about
> > forwarding
> > the RTCP FB messages?
>
> I think if you don't concern yourself with application or codec level
> feedback mechanisms, then this part of the RFC could be ignored and the
> need for registered handlers (or a framework to support it) could go away.
>
> > (a) Generic NACK: these messages should be implemented in such a
> > fashion that res_rtp_asterisk can have one be constructed based on the
> > statistics in (4). A part of me wishes all of this were implemented in
> > RTP engine, so that other RTP implementations could take advantage of
> > the RTCP work. That's a consideration that should be given some
> > thought, given how much of the RTCP implementation would have to be
> > ripped apart by this work anyway.
> > (b) PLI/SLI/RPSI: I'm not sure these apply to Asterisk very well. We
> > don't decode video, so it's kind of immaterial. You may want to have
> > the ability for Asterisk to pass through these types of RTCP messages
> > from a video decoder to a video encoder; that would entail
> > understanding the RTCP messages, enqueuing a control frame onto the
> > channel, and - in the far end - taking that control frame and turning
> > it into the appropriate information on the other side of the bridge.
> > There's bridging considerations to think through here on how the
> > control frame is propagated, particularly for multi-party bridges.
> >
> > If you're still interested in doing this work, please do let me us
> > know! I think the first major hurdle here is to hammer out how all of
> > this can be effectively introduced into the RTP engine/implementation
> > in Asterisk - as a large part of what makes this so difficult is the
> > non-extensible nature of Asterisk's RTP implementation. It may be that
> > we can help out some with that part, such that the actual act of
> > implementing the RFC and its various feedback mechanisms is a bit
> > easier.
> >
>
> While I am leaning towards "don't forward the FB messages", I think a
> concrete example of feedback messages that you think should be forwarded
> might help here. Right now, I'm having a hard time envisioning the specific
> scenario where you have two or more endpoints providing video to Asterisk
> and where Asterisk can simply relay the entire feedback message and not
> generate the message itself. I could be mistaken, but it feels like while
> this could work in a simple two-party core bridge, there are a lot of other
> scenarios where this goes wrong quickly.
>
> Matt
>
> --
> Matthew Jordan
> Digium, Inc. | Engineering Manager
> 445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
> Check us out at: http://digium.com & http://asterisk.org
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-dev mailing list
> To UNSUBSCRIBE or update options visit:
> http://lists.digium.com/mailman/listinfo/asterisk-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20141230/347c6846/attachment-0001.html>
More information about the asterisk-dev
mailing list