[asterisk-bugs] [JIRA] (ASTERISK-30201) Repeat uses of Manager command PlayDTMF on a channel with inband DTMF mode may cause infinite duration DTMF, possibly the DTMF generator will continue to use free'd memory when this happens

[Jonathan Rose (JIRA)]

     [ https://issues.asterisk.org/jira/browse/ASTERISK-30201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Rose updated ASTERISK-30201:
-------------------------------------

    Description: 
The PlayDTMF manager command results in a sticking tone intermittently. I haven't seen this happen the first time this command is used on a channel, but frequently (not always) if I use PlayDTMF several times on a channel during a call the tone will persist until I send another PlayDTMF command at which point it will stop. I'm generally able to reproduce this every other call, but depending on chance it may be more or less frequent. Presently I think this only happens for channels that use Inband DTMF modes, but I haven't fully investigated the effects in other instances since they are harder to discern (an endless noise is easier to notice than a strange behavior with an info message)

The bug was discovered on a test system in the lab. We have a browser 'softphone' that uses playdtmf to inject DTMF into a call... there are other circumstances where we use playdtmf, but that's not super important. Our prod/test systems use a heavily modified version of Asterisk anyway (plus they are still using chan_sip).

So I did a fresh clone of Asterisk 18 (commit bd821549af3bccb000c809121094adb5b84fec7f) tonight on a relatively clean Ubuntu 22.04 system and reproduced it with a very minimal setup while making sure everything is super recent and still supported.

PJSIP config:

[general]
allowoverlap = no

[transport-udp]
type = transport
protocol = udp
bind = 0.0.0.0

[testuser1]
type = aor
max_contacts = 1

[testuser1]
type = endpoint
context = testtext
disallow = all
allow = ulaw
allow = alaw
allow = t140
allow = red
aors = testuser1
asymmetric_rtp_codec = true
direct_media = false
dtmf_mode = inband

extensions.conf

[testtext]
exten => 100,1,Answer()
exten => 100,n,Echo()

manager.conf
[general]
enabled = yes

[adminpizzapasta]
secret=itsasecrettoeveryone
deny=0.0.0.0/0.0.0.0
permit=127.0.0.1
read=system,call,log
write=system,call,log

Step 1:
Start a call using the phone/softphone of your choice (specifically a Polycom VVX 410 was the one it was reproduced on in the lab while I reproduced it with TipCon1 softphone at home -- doesn't really matter, I have also prodded deeply enough to figure out that the DTMF is being repeatedly generated by Asterisk even after the generator detach function was called) to extension 100 using the testuser1 account. This will play an echo which is mostly just used so we have some media.

Step 2:
Login to manager using adminpizzapasta

Step 3:
Start spamming the call with PlayDTMF messages until you get a sticky tone

Some examples of manager commands I sent:

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: 4

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: 5

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: 6

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: #

Note the lack of a specified duration. It makes no difference if I include a duration, but it will use the default lowest possible value as a result.

If you don't get it within 20 tones or so, I recommend starting over and trying again. Usually if I can get it to happen once on a call I can get it to happen again quite frequently.

It's technically a regression, but it started happening after we moved from 1.6.2 to 16.3, so I can't really help to isolate specifically when it started right now. That said, there was a patch by Richard Mudgett back in 2019 that may have been involved. It changed some channel API stuff around so that there is a specific 'ast_senddigit_external' function intended for use when invoking DTMF outside the main media thread of a channel.

Now onto the scary part...

In the event of these tones sticking, I noticed ast_playtones_stop is invoked on the channel which in turn invokes ast_deactivate_generator, which I think is then going to call the generator release function in ast_generator_playtones (indications.c) which will then also free the playtones_state which is used byt he playtones_generator function which is still running after the fact. I don't know with certainy right now if anything is going to use this free'd memory and I haven't seen any errors coming out of Asterisk during these times just yet, so it might be insignificant, but I still found it a little spooky.

One other thing to note I suppose -- it might be better to move to schedule the stops in a task processor or something rather than having it stop for the duration of the DTMF. Probably not super important and it isn't exactly hurting our use any, but I can imagine that would get annoying with long duration DTMF.

I'll be trying to isolate this more over the next week and hopefully get a patch written, I just figured I'd go ahead and let you guys know about it.

  was:
The PlayDTMF manager command results in a sticking tone intermittently. I haven't seen this happen the first time this command is used on a channel, but frequently (not always) if I use PlayDTMF several times on a channel during a call the tone will persist until I send another PlayDTMF command at which point it will stop. I'm generally able to reproduce this every other call, but depending on chance it may be more or less frequent. Presently I think this only happens for channels that use Inband DTMF modes, but I haven't fully investigated the effects in other instances since they are harder to discern (an endless noise is easier to notice than a strange behavior with an info message)

The bug was discovered on a test system in the lab. We have a browser 'softphone' that uses playdtmf to inject DTMF into a call... there are other circumstances where we use playdtmf, but that's not super important. Our prod/test systems use a heavily modified version of Asterisk anyway (plus they are still using chan_sip).

So I did a fresh clone of Asterisk 18 (commit bd821549af3bccb000c809121094adb5b84fec7f) tonight on a relatively clean Ubuntu 22.04 system and reproduced it with a very minimal setup while making sure everything is super recent and still supported.

PJSIP config:

[general]
allowoverlap = no

[transport-udp]
type = transport
protocol = udp
bind = 0.0.0.0

[testuser1]
type = aor
max_contacts = 1

[testuser1]
type = endpoint
context = testtext
disallow = all
allow = ulaw
allow = alaw
allow = t140
allow = red
aors = testuser1
asymmetric_rtp_codec = true
direct_media = false
dtmf_mode = inband

extensions.conf

[testtext]
exten => 100,1,Answer()
exten => 100,n,Echo()

manager.conf
[general]
enabled = yes

[adminpizzapasta]
secret=itsasecrettoeveryone
deny=0.0.0.0/0.0.0.0
permit=127.0.0.1
read=system,call,log
write=system,call,log

Step 1:
Start a call using the phone/softphone of your choice (specifically a Polycom VVX 410 was the one it was reproduced on in the lab while I reproduced it with TipCon1 softphone at home -- doesn't really matter, I have also prodded deeply enough to figure out that the DTMF is being repeatedly generated by Asterisk even after the generator detach function was called)

Step 2:
Login to manager using adminpizzapasta

Step 3:
Start spamming the call with PlayDTMF messages until you get a sticky tone

Some examples of manager commands I sent:

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: 4

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: 5

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: 6

Action: PlayDTMF
Channel: PJSIP/testuser1-00000000
Digit: #

Note the lack of a specified duration. It makes no difference if I include a duration, but it will use the default lowest possible value as a result.

If you don't get it within 20 tones or so, I recommend starting over and trying again. Usually if I can get it to happen once on a call I can get it to happen again quite frequently.

It's technically a regression, but it started happening after we moved from 1.6.2 to 16.3, so I can't really help to isolate specifically when it started right now. That said, there was a patch by Richard Mudgett back in 2019 that may have been involved. It changed some channel API stuff around so that there is a specific 'ast_senddigit_external' function intended for use when invoking DTMF outside the main media thread of a channel.

Now onto the scary part...

In the event of these tones sticking, I noticed ast_playtones_stop is invoked on the channel which in turn invokes ast_deactivate_generator, which I think is then going to call the generator release function in ast_generator_playtones (indications.c) which will then also free the playtones_state which is used byt he playtones_generator function which is still running after the fact. I don't know with certainy right now if anything is going to use this free'd memory and I haven't seen any errors coming out of Asterisk during these times just yet, so it might be insignificant, but I still found it a little spooky.

One other thing to note I suppose -- it might be better to move to schedule the stops in a task processor or something rather than having it stop for the duration of the DTMF. Probably not super important and it isn't exactly hurting our use any, but I can imagine that would get annoying with long duration DTMF.

I'll be trying to isolate this more over the next week and hopefully get a patch written, I just figured I'd go ahead and let you guys know about it.


> Repeat uses of Manager command PlayDTMF on a channel with inband DTMF mode may cause infinite duration DTMF, possibly the DTMF generator will continue to use free'd memory when this happens
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: ASTERISK-30201
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-30201
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Applications/app_senddtmf
>    Affects Versions: 16.3.0, 18.9.0, 18.12.1
>         Environment: System 1 - Ubuntu 22.04 - Intel core i7 6th generation NUC
> System 2 - Centos 7 - some dell server blade thing with server grade hardware, it's not important, I'm just including it for the purpose of indicating that I've reproduced this on multiple systems
>            Reporter: Jonathan Rose
>            Severity: Major
>
> The PlayDTMF manager command results in a sticking tone intermittently. I haven't seen this happen the first time this command is used on a channel, but frequently (not always) if I use PlayDTMF several times on a channel during a call the tone will persist until I send another PlayDTMF command at which point it will stop. I'm generally able to reproduce this every other call, but depending on chance it may be more or less frequent. Presently I think this only happens for channels that use Inband DTMF modes, but I haven't fully investigated the effects in other instances since they are harder to discern (an endless noise is easier to notice than a strange behavior with an info message)
> The bug was discovered on a test system in the lab. We have a browser 'softphone' that uses playdtmf to inject DTMF into a call... there are other circumstances where we use playdtmf, but that's not super important. Our prod/test systems use a heavily modified version of Asterisk anyway (plus they are still using chan_sip).
> So I did a fresh clone of Asterisk 18 (commit bd821549af3bccb000c809121094adb5b84fec7f) tonight on a relatively clean Ubuntu 22.04 system and reproduced it with a very minimal setup while making sure everything is super recent and still supported.
> PJSIP config:
> [general]
> allowoverlap = no
> [transport-udp]
> type = transport
> protocol = udp
> bind = 0.0.0.0
> [testuser1]
> type = aor
> max_contacts = 1
> [testuser1]
> type = endpoint
> context = testtext
> disallow = all
> allow = ulaw
> allow = alaw
> allow = t140
> allow = red
> aors = testuser1
> asymmetric_rtp_codec = true
> direct_media = false
> dtmf_mode = inband
> extensions.conf
> [testtext]
> exten => 100,1,Answer()
> exten => 100,n,Echo()
> manager.conf
> [general]
> enabled = yes
> [adminpizzapasta]
> secret=itsasecrettoeveryone
> deny=0.0.0.0/0.0.0.0
> permit=127.0.0.1
> read=system,call,log
> write=system,call,log
> Step 1:
> Start a call using the phone/softphone of your choice (specifically a Polycom VVX 410 was the one it was reproduced on in the lab while I reproduced it with TipCon1 softphone at home -- doesn't really matter, I have also prodded deeply enough to figure out that the DTMF is being repeatedly generated by Asterisk even after the generator detach function was called) to extension 100 using the testuser1 account. This will play an echo which is mostly just used so we have some media.
> Step 2:
> Login to manager using adminpizzapasta
> Step 3:
> Start spamming the call with PlayDTMF messages until you get a sticky tone
> Some examples of manager commands I sent:
> Action: PlayDTMF
> Channel: PJSIP/testuser1-00000000
> Digit: 4
> Action: PlayDTMF
> Channel: PJSIP/testuser1-00000000
> Digit: 5
> Action: PlayDTMF
> Channel: PJSIP/testuser1-00000000
> Digit: 6
> Action: PlayDTMF
> Channel: PJSIP/testuser1-00000000
> Digit: #
> Note the lack of a specified duration. It makes no difference if I include a duration, but it will use the default lowest possible value as a result.
> If you don't get it within 20 tones or so, I recommend starting over and trying again. Usually if I can get it to happen once on a call I can get it to happen again quite frequently.
> It's technically a regression, but it started happening after we moved from 1.6.2 to 16.3, so I can't really help to isolate specifically when it started right now. That said, there was a patch by Richard Mudgett back in 2019 that may have been involved. It changed some channel API stuff around so that there is a specific 'ast_senddigit_external' function intended for use when invoking DTMF outside the main media thread of a channel.
> Now onto the scary part...
> In the event of these tones sticking, I noticed ast_playtones_stop is invoked on the channel which in turn invokes ast_deactivate_generator, which I think is then going to call the generator release function in ast_generator_playtones (indications.c) which will then also free the playtones_state which is used byt he playtones_generator function which is still running after the fact. I don't know with certainy right now if anything is going to use this free'd memory and I haven't seen any errors coming out of Asterisk during these times just yet, so it might be insignificant, but I still found it a little spooky.
> One other thing to note I suppose -- it might be better to move to schedule the stops in a task processor or something rather than having it stop for the duration of the DTMF. Probably not super important and it isn't exactly hurting our use any, but I can imagine that would get annoying with long duration DTMF.
> I'll be trying to isolate this more over the next week and hopefully get a patch written, I just figured I'd go ahead and let you guys know about it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)