[asterisk-dev] ASTERISK-25897 - RTCP feedback broken for video streams
jajcus at jajcus.net
Mon Apr 11 01:23:33 CDT 2016
Recently I have been working hard to make WebRTC video call working with
Asterisk 13 and PJSIP. After fixing a few problems I made it work, but
some things still seem wrong.
One of this is the handling of RTCP feedback for video pass-through. I
described my thoughts in
but Joshua Colp suggested the discussion should be moved here.
Here is the issue description for reply convenience:
Video streaming over RTP relies on RTCP feedback messages for error
and stream integrity. The messages used to be defined for a specific payload
(RFC 2032 for H.261), but now more generic protocol is available: AVPF (RFC
4585 – transport layer NACK and payload specific: PLI, SLI, RPSI) and
adding more on top of AVPF, including FIR.
Of all those asterisk has some minimal support for RFC 2032 FIR for
RFC 5104 FIR for VP8. In fact, every other RFC 4585 payload specific
message is treated like it is FIR.
This kind of works, but is inefficient and wrong, as RFC 5104 explicitly
> Using the FIR command to recover from errors is explicitly
> disallowed, and instead the PLI message defined in AVPF [RFC4585]
> should be used. The PLI message reports lost pictures and has been
> included in AVPF for precisely that purpose.
If AVPF PLI or NACK is received due to a lost packet, Asterisk would
it to FIR and request a whole key frame. The video source might comply and
waste bandwidth or ignore too frequent FIR messages and provide no
for the error.
On the other hand, neither PLI or NACK should be sent to Asterisk if
negotiated only ccm fir for a=rtcp-fb in the SDP.
Asterisk doesn't handle video directly (only passes it through), so it
react to payload-specific feedback directly. It doesn't keep forwarded
so it won't be able to react to generic NACKs either. What is left is
forwarding the feedback to the video source, but this should not be
How should it be handled? A new frame type? Data added to the
AST_CONTROL_VIDUPDATE frames (currently used for FIR messages)? A new
Raw RTCP packages cannot be forwarded, as SSRC and sequence numbers are
different on the two legs of the stream, the packets would have to be
recreated. What should be the format for the data in the forwarded frames?
Note, that a single incoming RTCP packed may be a compound packet and
multiple feedback messages together with RR/SR and SDES and currently
ast_rtcp_read() can return only a single frame.
How can we compute original RTP packet sequence number for a forwarded
message (which referenced sequence number generated by asterisk, in the
forwarded RTP stream)?
How should the SDP negotiation be handled for rtcp-fb parameters? Should
asterisk announce every message type it can forward? It is not known in
what the other party (where the stream is to be forwarded) can handle.
More information about the asterisk-dev