[asterisk-bugs] [JIRA] (ASTERISK-26924) Codec OPUS bad quality

Fri Apr 7 11:14:57 CDT 2017

    [ https://issues.asterisk.org/jira/browse/ASTERISK-26924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=236437#comment-236437 ] 

Kevin Harwell commented on ASTERISK-26924:
------------------------------------------

I ran some tests with the same criteria using two host machines. Unfortunately though I am unable to use the PESQ tool at this time as it requires a license. So all my testing was done by ear. However, there were significant enough differences in audio quality when testing various scenarios that made it easy enough to know if something was working or not. I used the following settings:

ubuntu 16.04
libopus 1.1.4 and opusfile 0.7 compiled from source
Asterisk 13.14.0 using chan_sip
Digium Opus codec: 1.1.0
[Open source Asterisk codec|http://github.com/traud/asterisk-opus/archive/asterisk-13.7.tar.gz]

host1's _sip.conf_:
{noformat}
[general]
sipdebug=yes
nat=force_rport
;jbenable=yes
;jbforce=yes

[endpoint_t](!)
type=friend
context=default
nat=force_rport
directmedia=no

[host2](endpoint_t)
host=<host2's IP address>
fromuser=host1
allow=!all,opus
{noformat}
When applicable I would un/comment the "jbenable" and "jbforce" settings.

host1's _extensions.conf_:
{noformat}
[default]
exten => 997,1,Noop()
      same => n,Monitor(wav,/tmp/degraded-ID-${RAND(1,1000)},bi)
      same => n,Dial(SIP/host2/999)
      same => n,Hangup()
{noformat}
host2's _sip.conf_:
{noformat}
[general]
sipdebug=yes
nat=force_rport

[endpoint_t](!)
type=friend
context=default
nat=force_rport
directmedia=no

[host1](endpoint_t)
host=<host1's IP address>
fromuser=host2
allow=!all,opus
{noformat}
host2's _extensions.conf_:
{noformat}
exten => 999,1,Answer()
      same => n,Playback(demo-instruct)
      same => n,Hangup()
{noformat}
host1's and host2's _codec.conf_ (note these settings only apply for the digium opus codec):
{noformat}
[opus]
type=opus
packet_loss=40
max_playback_rate=48000
bitrate=24000
cbr=0
fec=1
;max_bandwidth=medium
{noformat}
Below is a table of the testing I did. For each test I would initiate a call from an endpoint/phone that called extension 997 on host1. The table is a map of which codec implementation the a host was running. All combinations sounded good without delay and loss applied. On host2 I issued the following command to simulate packet loss with delay:
{noformat}
tc qdisc add dev eth0 root netem loss 20% delay 100ms 20ms distribution normal
{noformat}
OK = Intelligible with some audio anomalies, but what one would expect from 20% packet loss
Bad = Unintelligible, poor audio quality.
jb = Jitter buffer

||open source opus codec||open source opus codec with plc patch||digum opus codec||digum opus codec with jb||Result||
|host2 & host1| | | |Bad|
|host2|host1| | |Bad|
|host2| |host1| |Bad|
|host2| | |host1|Very Bad|
|host1| |host2| |Bad|
|host1 (with jb)| |host2| |Bad|
| |host1|host2| |Bad|
| |host1 (with jb)|host2| |OK|
| | |host1 & host2| |Bad|
| | |host2|host1|OK|

However, using the above _codec_.conf settings it is doubtful the digium codec was actually encoding FEC. This is because along with setting the "fec" and "packet_loss" options the "max_bandwidth" must also be set to medium or narrow. That being said I saw the best results when using a jitter buffer and I uncommented/set the max_bandwidth using the digium opus codec. While there were still noticeable audio anomalies (at 20% packet loss you are going to get some) the audio did sound better than without FEC.

> Codec OPUS bad quality
> ----------------------
>
>                 Key: ASTERISK-26924
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-26924
>             Project: Asterisk
>          Issue Type: Improvement
>      Security Level: None
>          Components: Codecs/codec_opus
>         Environment: Ubuntu 16.04 i386/x86_64
>            Reporter: TSAREGORODTSEV Yury
>            Assignee: Kevin Harwell
>
> Codec OPUS developed by Digium doesn't provide same good quality as open source version in case of packet losses, delays.
> After comparing recorded degraded media with original sample using PESQ results shows average predicted MOS of Digium Opus almost twice less then open source.
> How to reproduce issue:
> 1. Environment: ubuntu 16.04
> 2. Libopus latest stable release 1.1.4 (compiled from sources).
> 3. Asterisk 13.14.0
> 4. Channel driver: chan_sip
> 5. Configure 2 asterisk hosts, where 1st host origination side, 2nd host termination side.
> 6. Configure dialplan: on 1st host Dialplan with AppMonitor
> {noformat}
> [opus-test]
> exten => _X.,1,Monitor(wav,/records/degraded-ID-${RAND(1,1000)},bi)
> exten => _X.,n,Dial(SIP/asterisk_host2/${EXTEN})
> {noformat}
> 7. Configure dialplan: on 2nd host Playback any wave file (8000, 16bit):
> {noformat}
> [opus-test]
> exten => _X.,1,Answer()
> exten => _X.,n,Playback(demo-instruct)
> {noformat}
> 8. Opus config on both:
> {noformat}
> [opus]
> type=opus
> packet_loss=40
> max_playback_rate=48000
> bitrate=24000
> cbr=0
> fec=1
> {noformat}
> 9. Compile ITU-T PESQ utility from
> https://github.com/dennisguse/ITU-T_pesq
> 10. Simulate packet loss/delays on 2nd host with
>     tc qdisc add dev eth0 root netem loss 20% delay 100ms 20ms distribution normal
> 11. Compare recorded degraded media (only incoming channel of course) with original:
> ./itu-t-pesq2005 +8000 original.wav recorded.wav
> P.S. Please keep in mind without applying patch ASTERISK-25629 asterisk doesn't ignore lately arrived RTP no matter of enabled, forced jitterbuffer. Which is make quality worst.

--
This message was sent by Atlassian JIRA
(v6.2#6252)