[asterisk-dev] Strategies for handling RTCP feedback in codec modules

[Lorenzo Miniero]

2016-11-04 11:28 GMT+01:00 Lorenzo Miniero <lminiero at gmail.com>:

> Hi all,
>
> apologies if this has been discussed before, but I couldn't find anything
> in the recent months on this group so I thought I'd write anyway.
>
> As a few others, I believe, I have been trying to find a way to make codec
> modules more aware of what's happening on the wire. In particular, the
> motivation for that comes from an attempt to make the open source Opus
> codec module more responsive and adaptive to changes in the network, taking
> advantage of the functionality the library provides in that respect (e.g.,
> dynamic bitrate adaptation). The best approach to do that would obviously
> be providing codec modules with info on the RTCP feedback loop, e.g., in
> terms of losses the recipient has experienced, so that you can, for
> instance, change the bitrate in the encoder. Unfortunately, as of now there
> doesn't seem to be any way to make this possible, at least not in an easy
> way, in Asterisk out of the box.
> I've been investigating a few ways to do that, and have come up with some
> possible approaches, that I first wanted to discuss with you guys though,
> first of all to make sure I'm actually on the right path, and then to
> evaluate whether or not any of those can actually be integrated within the
> Asterisk code base (as I do believe such a feedback loop would be
> beneficial to other codec modules as well, and not only Opus). If you're
> interested in some more motivation behind this, you can read my discussion
> with Alexander Traud in a comment to his fork to the asterisk-opus repo
> here: https://github.com/traud/asterisk-opus/issues/3
>
>
> For the sake of completeness, Alexander himself thought of a possible
> approach for integrating this feedback in a comment to another post:
> https://github.com/seanbright/asterisk-opus/issues/25#
> issuecomment-249420010
> where the idea is to pass a reference of the ast_rtp_instance into the
> codec module itself. While this could possibly do the trick, I don't
> believe this would be a viable option, as it would break the architecture
> and module relationships, but I thought I'd mention it anyway.
>
> One possible option that I had thought about was extending ast_frame to
> convey RTCP feedback to modules, along with media to translate. This would
> allow such feedback to take the same "path" as media packets, meaning codec
> modules wouldn't need to be aware of any additional core-related feature,
> but only that sometimes they might receive control data instead of media to
> translate. Anyway, this could probably be problematic to integrate with the
> translator's architecture, and would probably need "cooperation" from
> channel modules as well, so may be a bit overkill and bug-prone.
>
> Another possible approach, and possibly the way to go, is to make use of
> the Stasis message bus, something I was not aware of until I watched Matt's
> excellent presentation at a conference recently. I saw how the RTP engine
> in Asterisk does publish RTCP feecback on the bus, and that you can
> subscribe to that as other events (as the HEP integration does, for
> instance) from other parts of the code. I tried doing the same within the
> Opus codec implementation, and apart from some quirks (e.g., weird fields
> in report blocks, like negative source_ssrc) it seemed to do exactly what I
> was looking for. The only problem, though, is that while a Stasis event
> contains a whole lot of info, codec modules are pretty much clueless and
> have no way of matching a specific event related to a specific call to a
> translator context they're handling. In fact, AFAIK codec modules have no
> visibility at all of the channel a translator is associated with, or of
> other identifiers it could rely on. Thinking about this I did find a way to
> implement some kind of loose mapping by extending the ast_frame structure
> with two new properties, "ssrc" and "themssrc": basically, anytime an RTP
> packet is received, the RTP engine copies the local and remote SSRC to the
> frame before passing it to the core. When the first packet gets to the
> codec module, it can keep track of them and save them locally to its own
> internal struct. So, when a new event comes later from Stasis (e.g., an
> RTCP RR), it can look at the SSRC it relates to and match it with the SSRCs
> it is aware of, so that it knows it's related to a specific translator
> context and react accordingly. While this seems effective, it has a few
> issues, though. For instance, there's no way to assume remote SSRCs will be
> unique, which means this could result in either missing or misleading
> feedback in some cases, or even that they'll stay the same for the whole
> call. Another aspect I haven't considered is the possible overhead,
> although I don't think crawling a JSON object should take much resources.
> Besides, I still haven't understood how Asterisk hashtables work, so this
> part is still just theory :-)
>
>
> That's basically it. Do you have any feeling/feedback on this? Is this
> actually worth investigating, and do you believe I'm on the right track?
> Any suggestion on how to make the mapping more effective in the codec
> module? I'm of course willing to contribute to such an effort, if it's
> deemed worthwhile.
>
> Thanks,
> Lorenzo
>


Just as an update to this, I eventually managed to get the Asterisk
hashtables working, and so I'm able to associate incoming RTCP feedback
(RTCP RR only at the moment, as anticipated in my previous mail) to a
specific encoder in the Opus module, which means in theory other codec
modules can do the same. As a proof of concept, I'm using the "fraction
lost" field to dynamically update the "OPUS_SET_PACKET_LOSS_PERC" control
of the encoder: next step would be doing using the feedback to also update
the bitrate used by the encoder accordingly to see if it does indeed help
achieve a better audio quality in troublesome connectivity scenarios.

Anyway, as I said I'm relying on the remote SSRC as a key for the mapping,
which isn't a reliable way of doing that. I guess I might use the
combination of local and remote SSRC for this (an RTCP RR contains both) to
avoid or at least greatly limit the risk of collisions, but from what I've
understood Asterisk seems to use different SSRCs from time to time (e.g.,
when playing an announcement before joining a bridge, and possibly in other
circumstances as well), which might make the local SSRC troublesome to pin
down.

Any feedback on how the mapping could be made more robust, e.g., via some
sort of new internal mechanism like the ones I discussed about in my
previous mail? Or is such a feature of no interest at all?

Thanks,
Lorenzo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20161107/1590c7cf/attachment.html>