[asterisk-dev] Adding the support for NACK in asterisk

[Nitesh Bansal]

Hi Matt,

Thanks for this nice explanation.

Well,  we still don't know if we are going to do it or not, it is still a
pending decision.

But i do have some comments ref your response, responding inline.

General comment: By adding the support for NACK, i meant just adding the
ability
to forward the RTCP FB messages, i do agree that adding the capability in
asterisk
to generate NACKs on its own will make it much more complicated?

(1) You'll need to modify rtp_engine and res_rtp_asterisk to
understand compound RTCP packets properly. Currently, we always assume
that the RTCP packets contain a single SR/RR: that means modifications
to the parsing in ast_rtcp_read, as well as the conversion of received
RTCP reports in rtp_engine into JSON for delivery to RTP Stasis topic
consumers. Note that this would be an opportune time to make the
parsing in ast_rtcp_read not be quite so terrible as it is today.

(2) Going along with #1, you'll need to add additional RTCP report
types - both when reading and when writing. Currently, Asterisk only
supports SR/RR. The RFC mandates FR reports, along with SDES packets.
You'll also need to change the algorithm for when reports are
scheduled. Since RFC 4585 modifies the scheduling algorithm, you'll
pretty much have to abandon how scheduling is set up currently in
ast_rtp_read. There are different modes of operation, each with slight
variations to the algorithm for how/when reports should be sent. A
state machine would probably be handy for this.

Nitesh -- since asterisk is just going to forward the RTCP FB reports,
won't it
be sufficient to parse the RTCP FB messages, adapt the packet header like
SSRC
and other stuff and just forward it, without modifying the current
algorithm?

(3) NACK mode. I would be very careful here: as a B2BUA, Asterisk
*always* assumes there are only two participants in an RTP session -
and making Asterisk realize that this isn't the case is going to be
more challenging than you may think. Asterisk has a lot of
implementation 'shortcuts' it takes in RTCP statistic management that
it takes because it knows that there is only one other RTP participant
other than itself: if you go down the road of wanting to maintain
additional statistics beyond the RTP instance and whoever it is
talking to, the scope of this would expand dramatically.
Nitesh -- I am not sure if i understood your comment, why does it need to
maintain
addiitonal info beyond RTP instance, in my opinion, for forwarding RTCP FB
reports,
all we need is the SSRC and sequence numbers?

(4) If I'm reading the RFC correctly, you _may_ not need to buffer the
RTP packets themselves; rather, you need to inform the other side of
the RTP packets you've lost. Generally, this means keeping a limited
history of which RTP sequence numbers you've received, and which ones
are missed. When constructing the NACK message (in 5a), you'd see if
you had a missed RTP sequence number, then look to see if any of the
16 RTP packets after that one were also missed. After reporting this,
the buffer could be flushed - how long the buffer is probably depends
on the frequency of report sending, which is variable as per (2).

Nitesh -- I have made this assumption considering that many traditional
SIP phones don't support NACK and even if they do, they don't send the
retransmission as specified in the RFC 4588.
This draft http://tools.ietf.org/pdf/draft-ietf-straw-b2bua-rtcp-02.pdf
talks
about the issue of handling the NACK messages in B2BUA environment.
Here is a piece of text from Section 4 of this draft:

*It is worth noting that some additional care may be needed, with*
* respect to RTCP, when acting as an intermediary between two secure*
* sessions. Specifically, issues may arise when relaying NACK feedback*
* originated by a user who failed to receive some RTP packets, that*
* were instead received by the B2BUA: such an issue might occur when*
* the packet loss happens between the user and the B2BUA itself, and*
* not between the RTP packet sender and the B2BUA. Such a situation*
* might result in the RTP media sender retransmitting the encrypted*
* packet, which would then be rejected by the B2BUA as a replayed one.*
* However, this issue is well known and addressed in [*
*RFC4588*
*], which*
* both the B2BUA and the involved participants in the communication*
* SHOULD make use of to prevent it from happening. This mechanism*
* allows for a Redundancy RTP Stream to be used for the purpose, which*
* would prevent the replay error. Of course, all recommendations given*
* previously with respect to managing SSRCs across the B2BUA still*
* apply here as well: in fact, such a redundant RTP stream would make*
* use of a different SSRC, that would need to be taken care of both at*
* the RTP and the SDP level. If [*
*RFC4588*
*] is not supported, the B2BUA*
* SHOULD handle NACK packets directly, and only forward feedback on*

* lost packets it has not access to*



(5) RTCP feedback messages. The transport layer feedback messages
would obviously be provided by the RTP engine and/or res_rtp_asterisk.
However, the other layers - payload/application - should have
corresponding APIs in rtp_engine. In particular, the codec layer
(payload) would be interesting, as it would require the ability for
codec modules (or something else in the media 'section' of Asterisk)
to take an RTP instance and register a handler for providing feedback
messages at opportune times. The application layer would be similar,
except that currently, we probably wouldn't have an implementation.
You may want to consider that as 'future work', as it is probably
sufficient to be a project in its own right.

Nitesh -- Do we require to register any handlers if we just care about
forwarding
the RTCP FB messages?

  (a) Generic NACK: these messages should be implemented in such a
fashion that res_rtp_asterisk can have one be constructed based on the
statistics in (4). A part of me wishes all of this were implemented in
RTP engine, so that other RTP implementations could take advantage of
the RTCP work. That's a consideration that should be given some
thought, given how much of the RTCP implementation would have to be
ripped apart by this work anyway.
  (b) PLI/SLI/RPSI: I'm not sure these apply to Asterisk very well. We
don't decode video, so it's kind of immaterial. You may want to have
the ability for Asterisk to pass through these types of RTCP messages
from a video decoder to a video encoder; that would entail
understanding the RTCP messages, enqueuing a control frame onto the
channel, and - in the far end - taking that control frame and turning
it into the appropriate information on the other side of the bridge.
There's bridging considerations to think through here on how the
control frame is propagated, particularly for multi-party bridges.

If you're still interested in doing this work, please do let me us
know! I think the first major hurdle here is to hammer out how all of
this can be effectively introduced into the RTP engine/implementation
in Asterisk - as a large part of what makes this so difficult is the
non-extensible nature of Asterisk's RTP implementation. It may be that
we can help out some with that part, such that the actual act of
implementing the RFC and its various feedback mechanisms is a bit
easier.

On Fri, Dec 19, 2014 at 4:36 PM, Matthew Jordan <mjordan at digium.com> wrote:

> On Thu, Dec 18, 2014 at 5:06 AM, Nitesh Bansal <nitesh.bansal at gmail.com>
> wrote:
> >
> > Hello,
> >
> > I am talking about NACK in the context of RTCP feedback (a=rtcp-fb:*
> nack) to request retransmission of lost frames.
> > I would like to advertise in my SDP that asterisk supports NACK, so
> WebRTC browsers can
> > request for lost packets, lost FIR or PLI messages via RTCP feedback.
> >
> > Since asterisk isn't source of the video, it will have to buffer the RTP
> messages received from the remote peer and retransmit
> > them when requested by WebRTC browsers.
>
>
> Short answer: complicated and hard. :-)
>
> For the sake of the explanation, I'm going to assume you want to
> implement RFC 4585 (https://tools.ietf.org/html/rfc4585)
>
> Part of this will be because you will be fighting inside
> res_rtp_asterisk - which is a module that is long overdue for a
> serious overhaul. Much like chan_sip, it has grown organically over
> the years, and suffers from a lack of structure in certain parts (and
> some dead code, and strange hacks for equipment that probably not
> longer exists, etc.) Unlike chan_sip, it's not nearly so large that
> it's impossible to work inside of - but it is challenging.
>
> That isn't a reason _not_ to do this work, but it should be a
> consideration - you'll want to think through how you can modify
> res_rtp_asterisk such that the feedback mechanisms don't interfere
> with the 'normal' operation of res_rtp_asterisk. A decent portion of
> the work may just be simply restructuring res_rtp_asterisk to easily
> handle the addition.
>
> As for the work itself:
>
> (1) You'll need to modify rtp_engine and res_rtp_asterisk to
> understand compound RTCP packets properly. Currently, we always assume
> that the RTCP packets contain a single SR/RR: that means modifications
> to the parsing in ast_rtcp_read, as well as the conversion of received
> RTCP reports in rtp_engine into JSON for delivery to RTP Stasis topic
> consumers. Note that this would be an opportune time to make the
> parsing in ast_rtcp_read not be quite so terrible as it is today.
>
> (2) Going along with #1, you'll need to add additional RTCP report
> types - both when reading and when writing. Currently, Asterisk only
> supports SR/RR. The RFC mandates FR reports, along with SDES packets.
> You'll also need to change the algorithm for when reports are
> scheduled. Since RFC 4585 modifies the scheduling algorithm, you'll
> pretty much have to abandon how scheduling is set up currently in
> ast_rtp_read. There are different modes of operation, each with slight
> variations to the algorithm for how/when reports should be sent. A
> state machine would probably be handy for this.
>
> (3) NACK mode. I would be very careful here: as a B2BUA, Asterisk
> *always* assumes there are only two participants in an RTP session -
> and making Asterisk realize that this isn't the case is going to be
> more challenging than you may think. Asterisk has a lot of
> implementation 'shortcuts' it takes in RTCP statistic management that
> it takes because it knows that there is only one other RTP participant
> other than itself: if you go down the road of wanting to maintain
> additional statistics beyond the RTP instance and whoever it is
> talking to, the scope of this would expand dramatically.
>
> (4) If I'm reading the RFC correctly, you _may_ not need to buffer the
> RTP packets themselves; rather, you need to inform the other side of
> the RTP packets you've lost. Generally, this means keeping a limited
> history of which RTP sequence numbers you've received, and which ones
> are missed. When constructing the NACK message (in 5a), you'd see if
> you had a missed RTP sequence number, then look to see if any of the
> 16 RTP packets after that one were also missed. After reporting this,
> the buffer could be flushed - how long the buffer is probably depends
> on the frequency of report sending, which is variable as per (2).
>
> (5) RTCP feedback messages. The transport layer feedback messages
> would obviously be provided by the RTP engine and/or res_rtp_asterisk.
> However, the other layers - payload/application - should have
> corresponding APIs in rtp_engine. In particular, the codec layer
> (payload) would be interesting, as it would require the ability for
> codec modules (or something else in the media 'section' of Asterisk)
> to take an RTP instance and register a handler for providing feedback
> messages at opportune times. The application layer would be similar,
> except that currently, we probably wouldn't have an implementation.
> You may want to consider that as 'future work', as it is probably
> sufficient to be a project in its own right.
>   (a) Generic NACK: these messages should be implemented in such a
> fashion that res_rtp_asterisk can have one be constructed based on the
> statistics in (4). A part of me wishes all of this were implemented in
> RTP engine, so that other RTP implementations could take advantage of
> the RTCP work. That's a consideration that should be given some
> thought, given how much of the RTCP implementation would have to be
> ripped apart by this work anyway.
>   (b) PLI/SLI/RPSI: I'm not sure these apply to Asterisk very well. We
> don't decode video, so it's kind of immaterial. You may want to have
> the ability for Asterisk to pass through these types of RTCP messages
> from a video decoder to a video encoder; that would entail
> understanding the RTCP messages, enqueuing a control frame onto the
> channel, and - in the far end - taking that control frame and turning
> it into the appropriate information on the other side of the bridge.
> There's bridging considerations to think through here on how the
> control frame is propagated, particularly for multi-party bridges.
>
> If you're still interested in doing this work, please do let me us
> know! I think the first major hurdle here is to hammer out how all of
> this can be effectively introduced into the RTP engine/implementation
> in Asterisk - as a large part of what makes this so difficult is the
> non-extensible nature of Asterisk's RTP implementation. It may be that
> we can help out some with that part, such that the actual act of
> implementing the RFC and its various feedback mechanisms is a bit
> easier.
>
> (Also: have you given any thought on how you'd test this in an
> automated fashion? :-)
>
> Matt
>
> --
> Matthew Jordan
> Digium, Inc. | Engineering Manager
> 445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
> Check us out at: http://digium.com & http://asterisk.org
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-dev mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20141223/3f86df04/attachment-0001.html>