[asterisk-bugs] [JIRA] (ASTERISK-24968) Deadlock

Wed Apr 15 17:13:32 CDT 2015

    [ https://issues.asterisk.org/jira/browse/ASTERISK-24968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=225912#comment-225912 ] 

Matt Jordan commented on ASTERISK-24968:
----------------------------------------

I don't think this is a deadlock. Nothing in the backtrace suggests a circular waiting condition.

What you do have is something waiting on a STUN packet and never receiving it:

{code}
Thread 7 (Thread 0x7f75e783f700 (LWP 5693)):
#0  0x0000003e7740e264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x0000003e77409523 in _L_lock_892 () from /lib64/libpthread.so.0
#2  0x0000003e77409407 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007f76028f1e0b in pj_mutex_lock () from /usr/lib/asterisk/modules/res_rtp_asterisk.so
#4  0x00007f76028f8cc9 in pj_lock_acquire () from /usr/lib/asterisk/modules/res_rtp_asterisk.so
#5  0x00007f76028db677 in pj_stun_session_on_rx_pkt () from /usr/lib/asterisk/modules/res_rtp_asterisk.so
#6  0x00007f76028d4a2f in pj_ice_sess_on_rx_pkt () from /usr/lib/asterisk/modules/res_rtp_asterisk.so
#7  0x00007f76028c9b17 in __rtp_recvfrom (instance=<value optimized out>, buf=0x7f75e7833100, sa=0x7f75e7835190, rtcp=1, flags=0, size=8192) at res_rtp_asterisk.c:2064
#8  0x00007f76028ca1bb in rtcp_recvfrom (instance=0x7f750001c2c8) at res_rtp_asterisk.c:2091
#9  ast_rtcp_read (instance=0x7f750001c2c8) at res_rtp_asterisk.c:3737
#10 0x00007f76028cba87 in ast_rtp_read (instance=0x7f750001c2c8, rtcp=1) at res_rtp_asterisk.c:4118
#11 0x00007f760fff4621 in sip_rtp_read (ast=0x7f7500008eb8) at chan_sip.c:8199
#12 sip_read (ast=0x7f7500008eb8) at chan_sip.c:8293
#13 0x0000000000485ee8 in __ast_read (chan=0x7f7500008eb8, dropaudio=0) at channel.c:4054
#14 0x000000000053c2a2 in remote_bridge_loop (c0=0x7f754012ad98, c1=0x7f7540123618, flags=271136480, fo=0x7f75e7837658, rc=0x7f75e7837650, timeoutms=-1) at rtp_engine.c:1319
#15 ast_rtp_instance_bridge (c0=0x7f754012ad98, c1=0x7f7540123618, flags=271136480, fo=0x7f75e7837658, rc=0x7f75e7837650, timeoutms=-1) at rtp_engine.c:1611
#16 0x000000000048b25e in ast_channel_bridge (c0=0x7f754012ad98, c1=0x7f7500008eb8, config=0x7f75e78380b0, fo=0x7f75e7837658, rc=0x7f75e7837650) at channel.c:8049
#17 0x00000000004c4f36 in ast_bridge_call (chan=0x7f754012ad98, peer=0x7f7500008eb8, config=0x7f75e78380b0) at features.c:4489
#18 0x00007f7604957efa in dial_exec_full (chan=<value optimized out>, data=<value optimized out>, peerflags=0x7f75e7838400, continue_exec=0x0) at app_dial.c:3047
#19 0x00007f7604958f66 in dial_exec (chan=<value optimized out>, data=<value optimized out>) at app_dial.c:3130
#20 0x000000000051fda4 in pbx_exec (c=0x7f754012ad98, app=0x32d2c10, data=0x7f75e7838a4a "SIP/421221299251 at bisip3,,gS(1437)") at pbx.c:1622
#21 0x00007f7617beed57 in handle_exec (chan=0x7f754012ad98, agi=0x7f75e783a4d0, argc=3, argv=0x7f75e78385b0) at res_agi.c:2562
#22 0x00007f7617bf14c4 in agi_handle_command (chan=0x7f754012ad98, agi=0x7f75e783a4d0, buf=0x7f75e7838a40 "EXEC", dead=0) at res_agi.c:3459
#23 0x00007f7617bf1c24 in run_agi (chan=0x7f754012ad98, request=0x7f75e78392c0 "ws_call.php", agi=0x7f75e783a4d0, pid=5695, status=0x7f75e783a538, dead=<value optimized out>, argc=1, argv=0x7f75e7839b08) at res_agi.c:3657
#24 0x00007f7617bf38d5 in agi_exec_full (chan=0x7f754012ad98, data=<value optimized out>, enhanced=<value optimized out>, dead=0) at res_agi.c:3942
#25 0x000000000051fda4 in pbx_exec (c=0x7f754012ad98, app=0x2b34770, data=0x7f75e783c680 "ws_call.php") at pbx.c:1622
#26 0x000000000052d489 in pbx_extension_helper (c=0x7f754012ad98, con=0x0, context=<value optimized out>, exten=<value optimized out>, priority=7, label=0x0, callerid=0x7f75402bf780 "919952573163", action=E_SPAWN, found=0x7f75e783ecfc, combined_find_spawn=1) at pbx.c:4915
#27 0x0000000000531885 in ast_spawn_extension (c=0x7f754012ad98, args=0x0) at pbx.c:6037
#28 __ast_pbx_run (c=0x7f754012ad98, args=0x0) at pbx.c:6512
#29 0x000000000053308b in pbx_thread (data=<value optimized out>) at pbx.c:6842
#30 0x0000000000574f2b in dummy_start (data=<value optimized out>) at utils.c:1223
#31 0x0000003e774079d1 in start_thread () from /lib64/libpthread.so.0
#32 0x0000003e770e88fd in clone () from /lib64/libc.so.6
{code}

Because that is holding a SIP pvt, and you have run 'sip show channels', that locks the SIP dialog container. Every channel that then needs to grab the SIP dialogs ends up getting blocked, which ends up blocking the channels container.

I'd find out why your STUN server stopped responding, or what is going on with that particular call.

> Deadlock 
> ---------
>
>                 Key: ASTERISK-24968
>                 URL: https://issues.asterisk.org/jira/browse/ASTERISK-24968
>             Project: Asterisk
>          Issue Type: Bug
>      Security Level: None
>    Affects Versions: 11.17.0
>         Environment: CentOS release 6.6 (Final)
> Linux localhost 2.6.32-504.12.2.el6.x86_64 #1 SMP Wed Mar 11 22:03:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
>            Reporter: Eldad Ran
>         Attachments: backtrace-threads.txt
>
>
> This is a new setup, the system handled 5.3K calls for the last 2 days with no issues, today I wanted to load more traffic on it and it got stuck, running the GDB showed me that there are few locks hanging there.
> The CLI gives you an answer only on the first command, the following commands are just ignored (LFCR). all other servers connected over SIP shows this server as unreachable.

--
This message was sent by Atlassian JIRA
(v6.2#6252)