[asterisk-dev] Packet Loss Concealment in confbridge

Wed Oct 20 17:40:49 CDT 2021

I can't speak much directly about the confbridge packet loss scenario, but
can talk a bit about OPUS. No guarantees it'll be helpful to this
situation, or not information you don't already know :-)

Most folks will want to leave codec OPUS alone, and let it do its thing.
Meaning, OPUS is pretty good at automatically adjusting itself to a given
situation. However, it does have some options, mostly encoder side, that
one can configure if needed, or desired. Note though, the setting of some
these options is more along the lines of making a recommendation to the
encoder/decoder vs telling it exactly what to do. Also, it can be a
balancing act when doing so. A gain in one direction may be a loss in
another. For example, setting a higher 'complexity' value will probably
result in more CPU utilization.

Now on to OPUS and packet loss. With nothing configured OPUS _should_
default to doing native PLC when packet loss is detected. FEC can also help
in a packet loss situation, but it's a little more complicated when, and
how that occurs. Enabling FEC does not necessarily mean it's being
utilized. For instance, in order for FEC data to be decoded by Sangoma's
codec_opus module for Asterisk several things must occur:

1) it must be enabled through configuration
2) both sides of a call must negotiate for it (via SDP)
3) packet loss must be perceived by the codec_opus module
4) a frame containing FEC data is received

Similarly, several conditions apply as to whether or not FEC data is
encoded:

1) it must be enabled through configuration
2) both sides of a call must negotiate for it (via SDP)
3) the encoder must be told to expect packet loss
4) the codec must be operating in a mode conducive to lower bandwidths.

More details below ...

On Wed, Oct 20, 2021 at 1:27 PM Pascal Cadotte <pcm at wazo.io> wrote:

> Thank you for taking the time to look into this and reply. It is very
> appreciated.
>
> I've done some more testing to get as few variables as possible from the
> tests I'm doing. This time around I've disable genericplc in codecs.conf to
> check the results from the opus fec option. With two users using the opus
> codec. When using the Dial application a 10% packet loss yields a decent
> audio quality. When using the Confbridge application the audio is VERY
> degraded.
>

In the non Confbridge scenario is transcoding occuring? If not then
Asterisk is simply passing through the audio frames, which may account for
some of the improved audio quality. Try forcing a transcode scenario
between two legs of a call and see if the audio sounds better or worse. Any
options set for codec_opus are only utilized while encoding or decoding.

>
> I've added JITTERBUFFER to all channels and added the option to the
> confbridge bridge.
> I've configured opus with fec=yes and have tried packet_loss=0 and
> packet_loss=50 with no difference in sound quality.
>

The 'packet_loss' option is an encoder side only option used in conjunction
with the 'fec' option. Setting the 'packet_loss' option adjusts an internal
threshold as to when the OPUS encoder should include FEC data in an encoded
frame. The higher the expected 'packet_loss' value the lower the threshold.

> I've used the following command on the laptop running one of the softphone
> to simulate packet loss
> sudo tc qdisc add dev enp0s31f6 root netem delay 2000ms 100ms 10%
> distribution normal
>
> Would I be able to dig deeper into this problem with the Asterisk sources
> only or does it sound like a problem in the codec implementation?
>

 It's hard to tell at this point. If you haven't already done so I'd try
the following:

* Use a different codec in the Confbridge scenario, and see how it
compares. For instance, if you swap opus for ulaw does it sound worse,
better, about the same?
* Next using only the default configuration forcodec_opus. Does it sound
better than the ulaw scenario?
* Enable FEC for codec_opus. Does it sound better, worse, the same?
* If worse or the same, then ensure both sides negotiated for it.
Ensure/force the encoder into a lower bandwidth mode by setting the
'max_bandwidth' option to narrow or medium, or set the 'max_playback_rate'
option to 16khz or less. Ensure the 'packet_loss' option is also
appropriately set (might just try 100, and if things improve then work
down).
* Enable the jitter buffer (Can try this option before enabling FEC as well
if you want)

If after all of that you're still having issues then try to find out if the
problem is occuring on the encoding or decoding side of things. Not sure if
this would work, but to check this you could try recording the Confbridge,
or perhaps "spy" on the incoming (decoding) side. I believe in the
recording scenario the Confbridge should write to the file after decoding,
but before encoding. So you should be able to listen to the recording to
see how the decoded audio sounds.

As far as looking at Asterisk sources, and if you're interested in how it
might all work, or not work then I'd start in main/translate.c (Note, not
the easiest code to understand!). Specifically, the ast_translate, and
framein functions. In the framein function for example you can see if a
codec implementation has the 'native_plc' flag set it will pass things on
to the codec implementation to be handled there. But this only happens if a
given frame's datalen is 0. So might want to check, and look for the
situations where Asterisk sets the length to 0. Generic PLC appears to only
be available for 8khz slin (see generate_interpolated_slin in
main/translate.c). A comment there says the interpolated frame may be
handled later by other resources. You'd have to track down what those are
if any are still around.

> Thanks again for your time.
>
> - Pascal Cadotte Michaud
>
>
>
>
> Le mer. 20 oct. 2021 à 01:43, Olle E. Johansson <oej at edvina.net> a écrit :
>
>>
>>
>> > On 19 Oct 2021, at 21:56, Matt Fredrickson <mfredrickson at sangoma.com>
>> wrote:
>> >
>> > On Thu, Oct 14, 2021 at 2:37 PM Pascal Cadotte <pcm at wazo.io> wrote:
>> >>
>> >> Hello everyone,
>> >>
>> >> We've been trying to improve the quality of our video conferences
>> using confbridge. We've been able to figure out how to get the video usable
>> for all participants even for users using bad internet connections using
>> the REMB configuration options.
>> >>
>> >> However, we are still having problems getting decent audio when
>> there's packet loss. We think this is because PLC is not used for channels
>> in confbridge. We found that information in this page
>> https://wiki.asterisk.org/wiki/display/AST/PLC+Restrictions+and+Caveats
>> "In addition, MeetMe and ConfBridge calls will not use PLC."
>> >>
>> >> We are looking into ways to improve this situation and would like to
>> gather some information about this restriction before working on it.
>> >>
>> >> 1. Does anyone know why PLC has not been implemented for meetme and
>> confbridge?
>> >> 2. Is there a known workaround to allow a channel to have PLC enabled
>> while being in confbridge?
>> >> 3. Is it a problem many people have? I've only seen this question that
>> seems to be related
>> https://community.asterisk.org/t/jitterbuffer-plc-fec-etc-how-do-i-know-they-are-working/85146
>> >> 4. Is there something in progress that we could contribute to?
>> >>
>> >> The tests have been made using the opus codec, PJSIP on a WSS
>> transport on Asterisk 18.6.0.
>> >>
>> >> Given two users Alice and Bob
>> >> Given a 10% paquet loss on Alice
>> >> When Alice and Bob are talking through confbridge Bob hears a lot of
>> cracking
>> >> When Alice and Bob are talking to each other using the Dial
>> application to call Bob's phone, the call quality is almost flawless
>> >
>> > I'll take a quick stab at an answer here.  I went looking at
>> > translate.c, plc.c, and channel.c, and here are some notes that I
>> > wrote as I was trying to remember how this works:
>> >
>> > The generic plc code in Asterisk was written a long time ago and can
>> > be only used for 8 KHz SLINEAR, not for other sample rates.  I'd
>> > presume that this means a codec such as OPUS needs to output to 8KHz
>> > SLINEAR somehow to even start using the generic PLC code.
>> >
>> > It appears that generic plc is actually calling the concealment
>> > functions when making a write to an ast_channel (so after a frame is
>> > produced from a mixing bridge presumably and goes out to a channel
>> > driver) - not on the reception side, reading a frame from a channel.
>> > (channel.c)  Unless someone else wants to go look at and correct me, I
>> > would assume that that means concealment comes after a bridge mixes
>> > frames together and presumably all input information about packet loss
>> > is wiped clean (meaning no generic PLC is happening with a mixing
>> > bridge).
>> >
>> > Looking at the codecs that support native PLC (such iLBC, speex, etc)
>> > it looks like they implement this on the conversion to SLINEAR (and
>> > correspondingly on an ast_read on the channel in quesiton), which
>> > should work for a case where you have a confbridge or meetme involved.
>> >
>> > For Sangoma's codec_opus, we have support for its native PLC and FEC
>> > functionality also, so I'd make sure that you have fec=yes enabled for
>> > its entry in codecs.conf.
>> >
>> > I'd also make sure you have jitterbuffer enabled on any
>> > channels/conferences in question for good measure.
>> >
>> > So, to sum it up:
>> > If you're using OPUS, use the opus native fec option (I think PLC
>> > should be happening also automatically as well) and make sure that the
>> > jitterbuffer is enabled on the channels/conferences in question.
>> >
>> > If you'd like to use a codec with non-native PLC, you're going to need
>> > to figure out how to get the generic PLC code working on the
>> > ast_read() instead of the ast_write() to the channel in question.
>> > (hoping my assumptions about some of this is correct too :-) )
>> >
>> > Hope that helps,
>> > Matthew Fredrickson
>>
>> I had this problem with pure RTP channels using ALAW. To fix it, since
>> G.711 is a simple codec, I wrote “Poor mans PLC” where the RTP channel just
>> copied the previoius packet up to a maximum number of packets. This fixed
>> our problem, but it was very specific and very crude.
>> There was settings in rtp.conf and possibly sip.conf to enable this.
>>
>> The code was published in one of my branches but never integrated into
>> asterisk as far as I remember.
>>
>> /O
>> --
>> _____________________________________________________________________
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> asterisk-dev mailing list
>> To UNSUBSCRIBE or update options visit:
>>    http://lists.digium.com/mailman/listinfo/asterisk-dev
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-dev mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20211020/860b69cd/attachment-0001.html>