[asterisk-dev] ASTERISK-25897 - RTCP feedback broken for video streams

[Jacek Konieczny]

Hi,

Recently I have been working hard to make WebRTC video call working with
Asterisk 13 and PJSIP. After fixing a few problems I made it work, but
some things still seem wrong.

One of this is the handling of RTCP feedback for video pass-through. I
described my thoughts in
https://issues.asterisk.org/jira/browse/ASTERISK-25897
but Joshua Colp suggested the discussion should be moved here.

Here is the issue description for reply convenience:


Video streaming over RTP relies on RTCP feedback messages for error 
connection
and stream integrity. The messages used to be defined for a specific payload
(RFC 2032 for H.261), but now more generic protocol is available: AVPF (RFC
4585 – transport layer NACK and payload specific: PLI, SLI, RPSI) and 
RFC 5104
adding more on top of AVPF, including FIR.

Of all those asterisk has some minimal support for RFC 2032 FIR for 
H.261 and
RFC 5104 FIR for VP8. In fact, every other RFC 4585 payload specific 
feedback
message is treated like it is FIR.

This kind of works, but is inefficient and wrong, as RFC 5104 explicitly
states:

> Using the FIR command to recover from errors is explicitly
> disallowed, and instead the PLI message defined in AVPF [RFC4585]
> should be used. The PLI message reports lost pictures and has been
> included in AVPF for precisely that purpose.

If AVPF PLI or NACK is received due to a lost packet, Asterisk would 
translate
it to FIR and request a whole key frame. The video source might comply and
waste bandwidth or ignore too frequent FIR messages and provide no 
correction
for the error.

On the other hand, neither PLI or NACK should be sent to Asterisk if 
Asterisk
negotiated only ccm fir for a=rtcp-fb in the SDP.

Asterisk doesn't handle video directly (only passes it through), so it 
cannot
react to payload-specific feedback directly. It doesn't keep forwarded 
frames,
so it won't be able to react to generic NACKs either. What is left is
forwarding the feedback to the video source, but this should not be 
limited to
FIR messages.

How should it be handled? A new frame type? Data added to the
AST_CONTROL_VIDUPDATE frames (currently used for FIR messages)? A new
AST_FRAME_CONTROL subclass?

Raw RTCP packages cannot be forwarded, as SSRC and sequence numbers are
different on the two legs of the stream, the packets would have to be
recreated. What should be the format for the data in the forwarded frames?

Note, that a single incoming RTCP packed may be a compound packet and 
contain
multiple feedback messages together with RR/SR and SDES and currently
ast_rtcp_read() can return only a single frame.

How can we compute original RTP packet sequence number for a forwarded 
feedback
message (which referenced sequence number generated by asterisk, in the
forwarded RTP stream)?

How should the SDP negotiation be handled for rtcp-fb parameters? Should
asterisk announce every message type it can forward? It is not known in 
advance
what the other party (where the stream is to be forwarded) can handle.


Jacek