[asterisk-dev] ASTERISK-25897 - RTCP feedback broken for video streams

[Matt Fredrickson]

Response inline.

Lovely email though, by the way.  It's a problem that I've worked on
in the past and is near and dear to my heart.

Negotiation is a problem - how can we advertise to an endpoint support
for messages that don't exist if we don't terminate video stream.

In the past, I translated sequence numbers and other things to improve
RTCP experience.

NACKs and other feedback types would be good to be able to pass
through end to end (translated, of course) - or Asterisk can try to
generate them independently.

On Mon, Apr 11, 2016 at 1:23 AM, Jacek Konieczny <jajcus at jajcus.net> wrote:
> Hi,
>
> Recently I have been working hard to make WebRTC video call working with
> Asterisk 13 and PJSIP. After fixing a few problems I made it work, but
> some things still seem wrong.
>
> One of this is the handling of RTCP feedback for video pass-through. I
> described my thoughts in
> https://issues.asterisk.org/jira/browse/ASTERISK-25897
> but Joshua Colp suggested the discussion should be moved here.
>
> Here is the issue description for reply convenience:
>
>
> Video streaming over RTP relies on RTCP feedback messages for error
> connection
> and stream integrity. The messages used to be defined for a specific payload
> (RFC 2032 for H.261), but now more generic protocol is available: AVPF (RFC
> 4585 – transport layer NACK and payload specific: PLI, SLI, RPSI) and RFC
> 5104
> adding more on top of AVPF, including FIR.
>
> Of all those asterisk has some minimal support for RFC 2032 FIR for H.261
> and
> RFC 5104 FIR for VP8. In fact, every other RFC 4585 payload specific
> feedback
> message is treated like it is FIR.
>
> This kind of works, but is inefficient and wrong, as RFC 5104 explicitly
> states:
>
>> Using the FIR command to recover from errors is explicitly
>> disallowed, and instead the PLI message defined in AVPF [RFC4585]
>> should be used. The PLI message reports lost pictures and has been
>> included in AVPF for precisely that purpose.
>
>
> If AVPF PLI or NACK is received due to a lost packet, Asterisk would
> translate
> it to FIR and request a whole key frame. The video source might comply and
> waste bandwidth or ignore too frequent FIR messages and provide no
> correction
> for the error.
>
> On the other hand, neither PLI or NACK should be sent to Asterisk if
> Asterisk
> negotiated only ccm fir for a=rtcp-fb in the SDP.
>
> Asterisk doesn't handle video directly (only passes it through), so it
> cannot
> react to payload-specific feedback directly. It doesn't keep forwarded
> frames,
> so it won't be able to react to generic NACKs either. What is left is
> forwarding the feedback to the video source, but this should not be limited
> to
> FIR messages.
>
> How should it be handled? A new frame type? Data added to the
> AST_CONTROL_VIDUPDATE frames (currently used for FIR messages)? A new
> AST_FRAME_CONTROL subclass?

I've worked a bit on this problem in the past, and those are the
options that I'd come up with as well.  I'm not sure which would be
best though.

> Raw RTCP packages cannot be forwarded, as SSRC and sequence numbers are
> different on the two legs of the stream, the packets would have to be
> recreated. What should be the format for the data in the forwarded frames?

Presumably a sequence number mapped version of the RTCP NACK, or PLI,
or SLI request - mapped to the Asterisk internal sequence number of
the frame.

> Note, that a single incoming RTCP packed may be a compound packet and
> contain
> multiple feedback messages together with RR/SR and SDES and currently
> ast_rtcp_read() can return only a single frame.
>
> How can we compute original RTP packet sequence number for a forwarded
> feedback
> message (which referenced sequence number generated by asterisk, in the
> forwarded RTP stream)?

There should be a fixed and calculable mapping between RTP sequence
numbers and internal Asterisk frame numbers that can be used to
determine what the mapped gap should be.

> How should the SDP negotiation be handled for rtcp-fb parameters? Should
> asterisk announce every message type it can forward? It is not known in
> advance
> what the other party (where the stream is to be forwarded) can handle.

This is the part I'm unsure of, especially if you can't do end to end
negotiation up front.  Most of this only works if we are working with
two endpoints that support the same packet loss resiliency mechanism.
There isn't much that can be done as far as translation goes.
Asterisk right now doesn't terminate video in such a way that it can,
itself, utilize or respond to RTCP FB mechanisms when it's the one
terminating/generating media.

-- 
Matthew Fredrickson
Digium, Inc. | Engineering Manager
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA