[asterisk-dev] ASTERISK-25897 - RTCP feedback broken for video streams

[Jacek Konieczny]

On 2016-04-14 16:16, Matt Fredrickson wrote:
> Negotiation is a problem - how can we advertise to an endpoint support
> for messages that don't exist if we don't terminate video stream.

That might be not that bad, see below.

> In the past, I translated sequence numbers and other things to improve
> RTCP experience.

Can you point me to any tool or algorithm to do that? I don't know
Asterisk code enough to figure it out quickly myself.

> NACKs and other feedback types would be good to be able to pass
> through end to end (translated, of course) - or Asterisk can try to
> generate them independently.

Generating NACKs in Asterisk would work only for the half of the
stream's way (from the source to Asterisk). It could fix some packet
loss errors faster, but would not help at all with errors on the other
half of the connection. And the receiver could confused even more by the
out-of-order retransmitted packets.

Forwarding NACKs from the media receiver would, on the other hand, work for
all lost packets and not much worse.

> On Mon, Apr 11, 2016 at 1:23 AM, Jacek Konieczny <jajcus at jajcus.net> wrote:
>> Raw RTCP packages cannot be forwarded, as SSRC and sequence numbers are
>> different on the two legs of the stream, the packets would have to be
>> recreated. What should be the format for the data in the forwarded frames?
>
> Presumably a sequence number mapped version of the RTCP NACK, or PLI,
> or SLI request - mapped to the Asterisk internal sequence number of
> the frame.

For NACK we need the sequence number (which needs to be mapped), for PLI
we need nothing, but for SLI we need the picture and slice
identification from the receiver (no mapping needed here).

So we need:
- feedback message type
- reference to the media stream this is feedback for
- message-type-specific data:
   - sequence number for NACK and similar messages
   - payload specific data for SLI and similar messages

All these come in the RTCP messages, but asterisk can store this in a
custom structure.

I was thinking if we might pass around the original RCTP packets in
asterisk frames and only modify the parts which need to be modified.
There would be no need to define a new data type or to actually parse
incoming messages, but the code to access and alter the message content
would be more complicated.

>> How should the SDP negotiation be handled for rtcp-fb parameters? Should
>> asterisk announce every message type it can forward? It is not known in
>> advance
>> what the other party (where the stream is to be forwarded) can handle.
>
> This is the part I'm unsure of, especially if you can't do end to end
> negotiation up front.  Most of this only works if we are working with
> two endpoints that support the same packet loss resiliency mechanism.
> There isn't much that can be done as far as translation goes.
> Asterisk right now doesn't terminate video in such a way that it can,
> itself, utilize or respond to RTCP FB mechanisms when it's the one
> terminating/generating media.

First we should note, that we already have no real negotiation for video
streams. The codec configured for an endpoints must match what the
endpoints will use and must be the same for all endpoints that will be
bridged together.  RTCP FB mechanisms could be configured / hardcoded in
a similar way. Though, I don't like the need for adding all those
details to endpoint configuration.

Fortunately, RFC 4585 gives us a bit of freedom:
> The answer is binding for the media session, and both
> offerer and answerer MUST only use feedback mechanisms negotiated in
> this way.  Both offerer and answerer MAY independently decide to send
> RTCP FB messages of only a subset of the negotiated feedback
> mechanisms, but they SHOULD react properly to all types of the
> negotiated FB messages when received.

> RTP senders MUST be prepared to receive any kind of RTCP FB messages
> and MUST silently discard all those RTCP FB messages that they do not
> understand.

So, Asterisk can announce any feedback message type it can forward and
it will be ok even if the other end of the connection doesn't support
that. It should just keep track of the message types _not_ negotiated
for a given media leg and not forward such messages to the media source.

That should work quite well for negative feedback messages (NACK, PLI)
– if the feedback doesn't come (was not negotiated and/or is not
generated by receiver) then a problem won't be corrected – like there is
no feedback done.

This won't work for positive feedback (ACK) – media source would keep
retransmitting even when the receiver has no problem with reception,
just doesn't do the ACK feedback.

So, always announcing NACK, PLI, SLI and FIR in SDP should be quite
safe.

I have mixed feelings with RPSI it is generally a positive feedback
message, but it might be needed for efficient handling of SLI messages
for VP8 codec, so it would be good to forward it too. Though, I am not
sure if negotiating RPSI when the receiver might not understand it,
won't cause too much unnecessary traffic.

Jacek