[asterisk-bugs] [JIRA] (ASTERISK-30432) res_speech_aeap: Tight loop; high CPU usage

Tue Feb 21 16:01:03 CST 2023

    [ https://issues.asterisk.org/jira/browse/ASTERISK-30432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=261416#comment-261416 ] 

Joshua C. Colp edited comment on ASTERISK-30432 at 2/21/23 3:59 PM:
--------------------------------------------------------------------

This is assuming AEAP is the source of the issue. I'm not 100% certain of that, because of this thread:

{code}
Thread 67 (Thread 0x7f3656b19700 (LWP 26451)):
#0  __ast_frdup (f=0x7f3648001970, file=0x6632f2 "file.c", line=921, func=0x663c77 <__PRETTY_FUNCTION__.18977> "read_frame") at frame.c:329
#1  0x00000000004f4c75 in __ast_frisolate (fr=fr at entry=0x7f3648001970, file=file at entry=0x6632f2 "file.c", line=line at entry=921, func=func at entry=0x663c77 <__PRETTY_FUNCTION__.18977> "read_frame") at frame.c:206
#2  0x00000000004e6f4d in read_frame (whennext=0x7f3656b157b0, s=0x7f3648001910) at file.c:921
#3  ast_readaudio_callback (s=0x7f3648001910) at file.c:960
#4  0x00000000004ec785 in ast_playstream (s=0x7f3648001910) at file.c:1066
#5  ast_streamfile (chan=chan at entry=0x7f36500414e0, filename=filename at entry=0x7f3656b15860 "custom/WellSaidLabs/Ava/1b8391878eeefd208e106401bc4795a1", preflang=0x7f365001517a "en-US") at file.c:1337
#6  0x00007f3694edf80e in playback_exec (chan=0x7f36500414e0, data=0x7f3656b16ab0 "custom/WellSaidLabs/Ava/1b8391878eeefd208e106401bc4795a1") at app_playback.c:506
#7  0x0000000000535136 in pbx_exec (c=c at entry=0x7f36500414e0, app=app at entry=0x3464120, data=data at entry=0x7f3656b16ab0 "custom/WellSaidLabs/Ava/1b8391878eeefd208e106401bc4795a1") at pbx_app.c:492
#8  0x0000000000527f69 in pbx_extension_helper (c=c at entry=0x7f36500414e0, context=0x7f3650041ea0 "yr-play-speech", exten=exten at entry=0x7f3650041ef0 "s", priority=priority at entry=4, label=label at entry=0x0, callerid=callerid at entry=0x7f3648003fa0 "17197890799", action=action at entry=E_SPAWN, found=found at entry=0x7f3656b18b10, combined_find_spawn=combined_find_spawn at entry=1, con=0x0) at pbx.c:2948
#9  0x0000000000529d5b in ast_spawn_extension (combined_find_spawn=1, found=0x7f3656b18b10, callerid=0x7f3648003fa0 "17197890799", priority=4, exten=0x7f3650041ef0 "s", context=<optimized out>, c=0x7f36500414e0) at pbx.c:4203
#10 ast_pbx_h_exten_run (chan=chan at entry=0x7f36500414e0, context=<optimized out>) at pbx.c:4233
#11 0x000000000052b2f0 in __ast_pbx_run (c=c at entry=0x7f36500414e0, args=args at entry=0x0) at pbx.c:4594
#12 0x000000000052b653 in pbx_thread (data=data at entry=0x7f36500414e0) at pbx.c:4701
#13 0x00000000005b26f8 in dummy_start (data=<optimized out>) at utils.c:1574
#14 0x00007f369871cea5 in start_thread () from /lib64/libpthread.so.0
#15 0x00007f36977bab0d in clone () from /lib64/libc.so.6
{code}

Which shows the 'h' extension executing and the Playback() dialplan application being run. So, is your dialplan constructed in such a way that the 'h' extension could lead to Playback() occurring or as well other things?

That would most likely cause issues, and could cause looping. Lots of applications are not written to run in 'h' extension or hangup handler and could cause undefined behavior. They should not be run in the 'h' extension or hangup handler.

I also don't think a hangup handler would execute before this.

was (Author: jcolp):
This is assuming AEAP is the source of the issue. I'm not 100% certain of that, because of this thread:

{code}
Thread 67 (Thread 0x7f3656b19700 (LWP 26451)):
#0  __ast_frdup (f=0x7f3648001970, file=0x6632f2 "file.c", line=921, func=0x663c77 <__PRETTY_FUNCTION__.18977> "read_frame") at frame.c:329
#1  0x00000000004f4c75 in __ast_frisolate (fr=fr at entry=0x7f3648001970, file=file at entry=0x6632f2 "file.c", line=line at entry=921, func=func at entry=0x663c77 <__PRETTY_FUNCTION__.18977> "read_frame") at frame.c:206
#2  0x00000000004e6f4d in read_frame (whennext=0x7f3656b157b0, s=0x7f3648001910) at file.c:921
#3  ast_readaudio_callback (s=0x7f3648001910) at file.c:960
#4  0x00000000004ec785 in ast_playstream (s=0x7f3648001910) at file.c:1066
#5  ast_streamfile (chan=chan at entry=0x7f36500414e0, filename=filename at entry=0x7f3656b15860 "custom/WellSaidLabs/Ava/1b8391878eeefd208e106401bc4795a1", preflang=0x7f365001517a "en-US") at file.c:1337
#6  0x00007f3694edf80e in playback_exec (chan=0x7f36500414e0, data=0x7f3656b16ab0 "custom/WellSaidLabs/Ava/1b8391878eeefd208e106401bc4795a1") at app_playback.c:506
#7  0x0000000000535136 in pbx_exec (c=c at entry=0x7f36500414e0, app=app at entry=0x3464120, data=data at entry=0x7f3656b16ab0 "custom/WellSaidLabs/Ava/1b8391878eeefd208e106401bc4795a1") at pbx_app.c:492
#8  0x0000000000527f69 in pbx_extension_helper (c=c at entry=0x7f36500414e0, context=0x7f3650041ea0 "yr-play-speech", exten=exten at entry=0x7f3650041ef0 "s", priority=priority at entry=4, label=label at entry=0x0, callerid=callerid at entry=0x7f3648003fa0 "17197890799", action=action at entry=E_SPAWN, found=found at entry=0x7f3656b18b10, combined_find_spawn=combined_find_spawn at entry=1, con=0x0) at pbx.c:2948
#9  0x0000000000529d5b in ast_spawn_extension (combined_find_spawn=1, found=0x7f3656b18b10, callerid=0x7f3648003fa0 "17197890799", priority=4, exten=0x7f3650041ef0 "s", context=<optimized out>, c=0x7f36500414e0) at pbx.c:4203
#10 ast_pbx_h_exten_run (chan=chan at entry=0x7f36500414e0, context=<optimized out>) at pbx.c:4233
#11 0x000000000052b2f0 in __ast_pbx_run (c=c at entry=0x7f36500414e0, args=args at entry=0x0) at pbx.c:4594
#12 0x000000000052b653 in pbx_thread (data=data at entry=0x7f36500414e0) at pbx.c:4701
#13 0x00000000005b26f8 in dummy_start (data=<optimized out>) at utils.c:1574
#14 0x00007f369871cea5 in start_thread () from /lib64/libpthread.so.0
#15 0x00007f36977bab0d in clone () from /lib64/libc.so.6
{code}

Which shows the 'h' extension executing and the Playback() dialplan application being run. So, is your dialplan constructed in such a way that the 'h' extension could lead to Playback() occurring or as well other things?

That would most likely cause issues, and could cause looping.

> res_speech_aeap: Tight loop; high CPU usage
> -------------------------------------------
>
>                 Key: ASTERISK-30432
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-30432
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Resources/NewFeature
>    Affects Versions: 18.16.0
>         Environment: CentOS-7 3.10.0-1160.76.1.el7.x86_64
>            Reporter: Peter Howell
>            Assignee: Unassigned
>         Attachments: asterisk-open-files.txt, core-asterisk-running-2023-02-20T17-18-13Z-brief.txt, core-asterisk-running-2023-02-20T17-18-13Z-full.txt, core-asterisk-running-2023-02-20T17-18-13Z-info.txt, core-asterisk-running-2023-02-20T17-18-13Z-locks.txt, core-asterisk-running-2023-02-20T17-18-13Z-thread1.txt, dump.pcap
>
>
> We see that on occasion closed AEAP connections to Asterisk are not closing.  We are sending an OP_CLOSE before closing the connection, but it seems under certain conditions, the connection does not close.  At this point, Asterisk goes into a tight loop on these zombie connections and the result is high CPU usage.

--
This message was sent by Atlassian JIRA
(v6.2#6252)