[asterisk-bugs] [JIRA] (ASTERISK-28093) pbx: Deadlock from holding channel lock when it shouldn't be

[Cirillo Ferreira (JIRA)]

    [ https://issues.asterisk.org/jira/browse/ASTERISK-28093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=245288#comment-245288 ] 

Cirillo Ferreira edited comment on ASTERISK-28093 at 10/29/18 12:42 PM:
------------------------------------------------------------------------

Apparently the deadlock occurs when the maximum load limit is reached.

ps: the load reached 25.26 in a few seconds.

{code:title=backtrace-threads.txt}
Thread 796 (Thread 0x7f20945b3700 (LWP 59199)):
...
#15 0x00000000005841cf in pbx_outgoing_attempt (type=0x7f20cc780a90 "Local", cap=0x7f20cf571798, addr=0x7f20cc780a98 "0111143370692 at agentes", timeout=27000, context=0x7f20cc780ab0 "agentes", exten=0x7f20cc780aba "10516", priority=1, app=0x0, appdata=0x0, reason=0x7f20945b2d6c, synchronous=1, cid_num=0x0, cid_name=0x0, vars=0x0, account=0x8e007a <__ast_string_field_empty_buffer+2> "", locked_channel=0x7f20945b2d60, early_media=0, assignedids=0x7f20945b2cf0) at pbx.c:7612
#16 0x0000000000584265 in ast_pbx_outgoing_exten (type=0x7f20cc780a90 "Local", cap=0x7f20cf571798, addr=0x7f20cc780a98 "0111143370692 at agentes", timeout=27000, context=0x7f20cc780ab0 "agentes", exten=0x7f20cc780aba "10516", priority=1, reason=0x7f20945b2d6c, synchronous=1, cid_num=0x0, cid_name=0x0, vars=0x0, account=0x8e007a <__ast_string_field_empty_buffer+2> "", locked_channel=0x7f20945b2d60, early_media=0, assignedids=0x7f20945b2cf0) at pbx.c:7792
#17 0x000000000054eff6 in fast_originate (data=0x7f20cd6b6680) at manager.c:5252
#18 0x0000000000600ed3 in dummy_start (data=0x7f20cfab1cb0) at utils.c:1239
#19 0x00007f21be162064 in start_thread (arg=0x7f20945b3700) at pthread_create.c:309
#20 0x00007f21bd10562d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
{code}

{code:title=messages}
[2018-10-05 12:23:02.214] WARNING[59202] pbx.c: Maximum loadavg limit of 10.000000 load exceeded by 'Local/0111143370692 at agentes-0002aa80;1' (currently 25.260000)!
[2018-10-05 12:23:02.214] ERROR[59202] pbx.c: Failed to start PBX on Local/0111143370692 at agentes-0002aa80;1
{code}


was (Author: cirillor at lbv.org.br):
Apparently the deadlock occurs when the maximum load limit is reached.

The load reached 25.26 in a few seconds.

{code:title=backtrace-threads.txt}
Thread 796 (Thread 0x7f20945b3700 (LWP 59199)):
...
#15 0x00000000005841cf in pbx_outgoing_attempt (type=0x7f20cc780a90 "Local", cap=0x7f20cf571798, addr=0x7f20cc780a98 "0111143370692 at agentes", timeout=27000, context=0x7f20cc780ab0 "agentes", exten=0x7f20cc780aba "10516", priority=1, app=0x0, appdata=0x0, reason=0x7f20945b2d6c, synchronous=1, cid_num=0x0, cid_name=0x0, vars=0x0, account=0x8e007a <__ast_string_field_empty_buffer+2> "", locked_channel=0x7f20945b2d60, early_media=0, assignedids=0x7f20945b2cf0) at pbx.c:7612
#16 0x0000000000584265 in ast_pbx_outgoing_exten (type=0x7f20cc780a90 "Local", cap=0x7f20cf571798, addr=0x7f20cc780a98 "0111143370692 at agentes", timeout=27000, context=0x7f20cc780ab0 "agentes", exten=0x7f20cc780aba "10516", priority=1, reason=0x7f20945b2d6c, synchronous=1, cid_num=0x0, cid_name=0x0, vars=0x0, account=0x8e007a <__ast_string_field_empty_buffer+2> "", locked_channel=0x7f20945b2d60, early_media=0, assignedids=0x7f20945b2cf0) at pbx.c:7792
#17 0x000000000054eff6 in fast_originate (data=0x7f20cd6b6680) at manager.c:5252
#18 0x0000000000600ed3 in dummy_start (data=0x7f20cfab1cb0) at utils.c:1239
#19 0x00007f21be162064 in start_thread (arg=0x7f20945b3700) at pthread_create.c:309
#20 0x00007f21bd10562d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
{code}

{code:title=messages}
[2018-10-05 12:23:02.214] WARNING[59202] pbx.c: Maximum loadavg limit of 10.000000 load exceeded by 'Local/0111143370692 at agentes-0002aa80;1' (currently 25.260000)!
[2018-10-05 12:23:02.214] ERROR[59202] pbx.c: Failed to start PBX on Local/0111143370692 at agentes-0002aa80;1
{code}

> pbx: Deadlock from holding channel lock when it shouldn't be
> ------------------------------------------------------------
>
>                 Key: ASTERISK-28093
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-28093
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>          Components: Core/PBX
>    Affects Versions: 13.18.5, 13.22.0, 13.23.1
>         Environment: Debian 8.11 64 bits
> Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz - 8 cores
> 32GB RAM
>            Reporter: Nicolas Tanski
>            Assignee: Unassigned
>              Labels: crash, deadlock
>         Attachments: backtrace-threads.txt, messages-20181005
>
>
> Hi Team,
> we are experiencing frequent crash issues in the Asterisk instance, we are using our own dialer solution with AMI protocol.
> The Asterisk service operates on average with 250 simultaneous calls, 40 record calls, the load is between 1 and 3, in some cases the load quickly rises to 10, which causes the rejection of new calls and the instance of Asterisk does not respond.
> We were only able to restart the service by killing the Asterisk service with the Kill PID command.
> It was enabled the flags DONT_OPTIMIZE, BETTER_BACKTRACES and MALLOC_DEBUG to try identify the problem.
> Summary
> Average simultaneous calls: 347
> Average record calls: 40
> Load Average: > 1 and < 3
> High Load Average: 25.26
> After the crash in Asterisk, we execute the commands and the service returns to normal operation
> \# kill -9 PID_ASTERISK
> \# /etc/init.d/asterisk start



--
This message was sent by Atlassian JIRA
(v6.2#6252)