[asterisk-dev] Packet Loss Concealment in confbridge

Tue Oct 26 09:20:42 CDT 2021

Le mer. 20 oct. 2021 à 18:42, Kevin Harwell <kharwell at sangoma.com> a écrit :

> I can't speak much directly about the confbridge packet loss scenario, but
> can talk a bit about OPUS. No guarantees it'll be helpful to this
> situation, or not information you don't already know :-)
>
> Most folks will want to leave codec OPUS alone, and let it do its thing.
> Meaning, OPUS is pretty good at automatically adjusting itself to a given
> situation. However, it does have some options, mostly encoder side, that
> one can configure if needed, or desired. Note though, the setting of some
> these options is more along the lines of making a recommendation to the
> encoder/decoder vs telling it exactly what to do. Also, it can be a
> balancing act when doing so. A gain in one direction may be a loss in
> another. For example, setting a higher 'complexity' value will probably
> result in more CPU utilization.
>
> Now on to OPUS and packet loss. With nothing configured OPUS _should_
> default to doing native PLC when packet loss is detected. FEC can also help
> in a packet loss situation, but it's a little more complicated when, and
> how that occurs. Enabling FEC does not necessarily mean it's being
> utilized. For instance, in order for FEC data to be decoded by Sangoma's
> codec_opus module for Asterisk several things must occur:
>
> 1) it must be enabled through configuration
> 2) both sides of a call must negotiate for it (via SDP)
> 3) packet loss must be perceived by the codec_opus module
> 4) a frame containing FEC data is received
>
> Similarly, several conditions apply as to whether or not FEC data is
> encoded:
>
> 1) it must be enabled through configuration
> 2) both sides of a call must negotiate for it (via SDP)
> 3) the encoder must be told to expect packet loss
> 4) the codec must be operating in a mode conducive to lower bandwidths.
>

I'll make sure all those prerequisite are met

>
> More details below ...
>
> On Wed, Oct 20, 2021 at 1:27 PM Pascal Cadotte <pcm at wazo.io> wrote:
>
>> Thank you for taking the time to look into this and reply. It is very
>> appreciated.
>>
>> I've done some more testing to get as few variables as possible from the
>> tests I'm doing. This time around I've disable genericplc in codecs.conf to
>> check the results from the opus fec option. With two users using the opus
>> codec. When using the Dial application a 10% packet loss yields a decent
>> audio quality. When using the Confbridge application the audio is VERY
>> degraded.
>>
>
> In the non Confbridge scenario is transcoding occuring? If not then
> Asterisk is simply passing through the audio frames, which may account for
> some of the improved audio quality. Try forcing a transcode scenario
> between two legs of a call and see if the audio sounds better or worse. Any
> options set for codec_opus are only utilized while encoding or decoding.
>
>
>>
>> I've added JITTERBUFFER to all channels and added the option to the
>> confbridge bridge.
>> I've configured opus with fec=yes and have tried packet_loss=0 and
>> packet_loss=50 with no difference in sound quality.
>>
>
> The 'packet_loss' option is an encoder side only option used in
> conjunction with the 'fec' option. Setting the 'packet_loss' option adjusts
> an internal threshold as to when the OPUS encoder should include FEC data
> in an encoded frame. The higher the expected 'packet_loss' value the lower
> the threshold.
>
>
>> I've used the following command on the laptop running one of the
>> softphone to simulate packet loss
>> sudo tc qdisc add dev enp0s31f6 root netem delay 2000ms 100ms 10%
>> distribution normal
>>
>> Would I be able to dig deeper into this problem with the Asterisk sources
>> only or does it sound like a problem in the codec implementation?
>>
>
>  It's hard to tell at this point. If you haven't already done so I'd try
> the following:
>
> * Use a different codec in the Confbridge scenario, and see how it
> compares. For instance, if you swap opus for ulaw does it sound worse,
> better, about the same?
> * Next using only the default configuration forcodec_opus. Does it sound
> better than the ulaw scenario?
> * Enable FEC for codec_opus. Does it sound better, worse, the same?
> * If worse or the same, then ensure both sides negotiated for it.
> Ensure/force the encoder into a lower bandwidth mode by setting the
> 'max_bandwidth' option to narrow or medium, or set the 'max_playback_rate'
> option to 16khz or less. Ensure the 'packet_loss' option is also
> appropriately set (might just try 100, and if things improve then work
> down).
> * Enable the jitter buffer (Can try this option before enabling FEC as
> well if you want)
>

I'll try that

>
> If after all of that you're still having issues then try to find out if
> the problem is occuring on the encoding or decoding side of things. Not
> sure if this would work, but to check this you could try recording the
> Confbridge, or perhaps "spy" on the incoming (decoding) side. I believe in
> the recording scenario the Confbridge should write to the file after
> decoding, but before encoding. So you should be able to listen to the
> recording to see how the decoded audio sounds.
>
> As far as looking at Asterisk sources, and if you're interested in how it
> might all work, or not work then I'd start in main/translate.c (Note, not
> the easiest code to understand!). Specifically, the ast_translate, and
> framein functions. In the framein function for example you can see if a
> codec implementation has the 'native_plc' flag set it will pass things on
> to the codec implementation to be handled there. But this only happens if a
> given frame's datalen is 0. So might want to check, and look for the
> situations where Asterisk sets the length to 0. Generic PLC appears to only
> be available for 8khz slin (see generate_interpolated_slin in
> main/translate.c). A comment there says the interpolated frame may be
> handled later by other resources. You'd have to track down what those are
> if any are still around.
>
>
>
>> Thanks again for your time.
>>
>> - Pascal Cadotte Michaud
>>
>>
>>
>>
>> Le mer. 20 oct. 2021 à 01:43, Olle E. Johansson <oej at edvina.net> a
>> écrit :
>>
>>>
>>>
>>> > On 19 Oct 2021, at 21:56, Matt Fredrickson <mfredrickson at sangoma.com>
>>> wrote:
>>> >
>>> > On Thu, Oct 14, 2021 at 2:37 PM Pascal Cadotte <pcm at wazo.io> wrote:
>>> >>
>>> >> Hello everyone,
>>> >>
>>> >> We've been trying to improve the quality of our video conferences
>>> using confbridge. We've been able to figure out how to get the video usable
>>> for all participants even for users using bad internet connections using
>>> the REMB configuration options.
>>> >>
>>> >> However, we are still having problems getting decent audio when
>>> there's packet loss. We think this is because PLC is not used for channels
>>> in confbridge. We found that information in this page
>>> https://wiki.asterisk.org/wiki/display/AST/PLC+Restrictions+and+Caveats
>>> "In addition, MeetMe and ConfBridge calls will not use PLC."
>>> >>
>>> >> We are looking into ways to improve this situation and would like to
>>> gather some information about this restriction before working on it.
>>> >>
>>> >> 1. Does anyone know why PLC has not been implemented for meetme and
>>> confbridge?
>>> >> 2. Is there a known workaround to allow a channel to have PLC enabled
>>> while being in confbridge?
>>> >> 3. Is it a problem many people have? I've only seen this question
>>> that seems to be related
>>> https://community.asterisk.org/t/jitterbuffer-plc-fec-etc-how-do-i-know-they-are-working/85146
>>> >> 4. Is there something in progress that we could contribute to?
>>> >>
>>> >> The tests have been made using the opus codec, PJSIP on a WSS
>>> transport on Asterisk 18.6.0.
>>> >>
>>> >> Given two users Alice and Bob
>>> >> Given a 10% paquet loss on Alice
>>> >> When Alice and Bob are talking through confbridge Bob hears a lot of
>>> cracking
>>> >> When Alice and Bob are talking to each other using the Dial
>>> application to call Bob's phone, the call quality is almost flawless
>>> >
>>> > I'll take a quick stab at an answer here.  I went looking at
>>> > translate.c, plc.c, and channel.c, and here are some notes that I
>>> > wrote as I was trying to remember how this works:
>>> >
>>> > The generic plc code in Asterisk was written a long time ago and can
>>> > be only used for 8 KHz SLINEAR, not for other sample rates.  I'd
>>> > presume that this means a codec such as OPUS needs to output to 8KHz
>>> > SLINEAR somehow to even start using the generic PLC code.
>>> >
>>> > It appears that generic plc is actually calling the concealment
>>> > functions when making a write to an ast_channel (so after a frame is
>>> > produced from a mixing bridge presumably and goes out to a channel
>>> > driver) - not on the reception side, reading a frame from a channel.
>>> > (channel.c)  Unless someone else wants to go look at and correct me, I
>>> > would assume that that means concealment comes after a bridge mixes
>>> > frames together and presumably all input information about packet loss
>>> > is wiped clean (meaning no generic PLC is happening with a mixing
>>> > bridge).
>>> >
>>> > Looking at the codecs that support native PLC (such iLBC, speex, etc)
>>> > it looks like they implement this on the conversion to SLINEAR (and
>>> > correspondingly on an ast_read on the channel in quesiton), which
>>> > should work for a case where you have a confbridge or meetme involved.
>>> >
>>> > For Sangoma's codec_opus, we have support for its native PLC and FEC
>>> > functionality also, so I'd make sure that you have fec=yes enabled for
>>> > its entry in codecs.conf.
>>> >
>>> > I'd also make sure you have jitterbuffer enabled on any
>>> > channels/conferences in question for good measure.
>>> >
>>> > So, to sum it up:
>>> > If you're using OPUS, use the opus native fec option (I think PLC
>>> > should be happening also automatically as well) and make sure that the
>>> > jitterbuffer is enabled on the channels/conferences in question.
>>> >
>>> > If you'd like to use a codec with non-native PLC, you're going to need
>>> > to figure out how to get the generic PLC code working on the
>>> > ast_read() instead of the ast_write() to the channel in question.
>>> > (hoping my assumptions about some of this is correct too :-) )
>>> >
>>> > Hope that helps,
>>> > Matthew Fredrickson
>>>
>>> I had this problem with pure RTP channels using ALAW. To fix it, since
>>> G.711 is a simple codec, I wrote “Poor mans PLC” where the RTP channel just
>>> copied the previoius packet up to a maximum number of packets. This fixed
>>> our problem, but it was very specific and very crude.
>>> There was settings in rtp.conf and possibly sip.conf to enable this.
>>>
>>> The code was published in one of my branches but never integrated into
>>> asterisk as far as I remember.
>>>
>>> /O
>>> --
>>> _____________________________________________________________________
>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>>
>>> asterisk-dev mailing list
>>> To UNSUBSCRIBE or update options visit:
>>>    http://lists.digium.com/mailman/listinfo/asterisk-dev
>>
>> --
>> _____________________________________________________________________
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> asterisk-dev mailing list
>> To UNSUBSCRIBE or update options visit:
>>    http://lists.digium.com/mailman/listinfo/asterisk-dev
>
> --
> _____________________________________________________________________
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> asterisk-dev mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-dev

Thank you for the detailed answer, it's really helpful.

- Pascal Cadotte Michaud
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20211026/9ef662ba/attachment-0001.html>